November 27, 2025¶
Generated: 2025-11-27 04:29 UTC
Total Duration: 10h 30m 9s
Iterations: 1
Judge (classifier) model: gpt-4.1
About this Benchmark¶
HolmesGPT is continuously evaluated against real-world Kubernetes and cloud troubleshooting scenarios.
If you find scenarios that HolmesGPT does not perform well on, please consider adding them as evals to the benchmark.
Model Accuracy Comparison¶
| Model | Pass | Fail | Skip/Error | Total | Success Rate |
|---|---|---|---|---|---|
| deepseek-3.1 | 62 | 32 | 22 | 116 | 🟡 66% (62/94) |
| gpt-5 | 47 | 49 | 20 | 116 | 🟡 49% (47/96) |
| gpt-5.1 | 64 | 31 | 21 | 116 | 🟡 67% (64/95) |
| haiku-4.5 | 66 | 29 | 21 | 116 | 🟡 69% (66/95) |
| sonnet-4.5 | 77 | 17 | 22 | 116 | 🟡 82% (77/94) |
Model Cost Comparison¶
| Model | Tests | Avg Cost | Min Cost | Max Cost | Total Cost |
|---|---|---|---|---|---|
| gpt-5 | 87 | $0.05 | $0.01 | $0.18 | $3.97 |
| gpt-5.1 | 86 | $0.10 | $0.01 | $0.31 | $8.61 |
| haiku-4.5 | 88 | $0.05 | $0.02 | $0.13 | $4.24 |
| sonnet-4.5 | 87 | $0.15 | $0.05 | $0.62 | $13.22 |
Model Latency Comparison¶
| Model | Avg (s) | Min (s) | Max (s) | P50 (s) | P95 (s) |
|---|---|---|---|---|---|
| deepseek-3.1 | 56.7 | 6.6 | 183.1 | 46.6 | 140.8 |
| gpt-5 | 28.4 | 3.8 | 613.5 | 17.5 | 46.8 |
| gpt-5.1 | 127.7 | 7.2 | 868.9 | 100.2 | 301.1 |
| haiku-4.5 | 31.3 | 0.0 | 207.0 | 26.2 | 77.4 |
| sonnet-4.5 | 41.9 | 0.0 | 211.9 | 38.2 | 80.0 |
Performance by Tag¶
Success rate by test category and model:
| Tag | deepseek-3.1 | gpt-5 | gpt-5.1 | haiku-4.5 | sonnet-4.5 | Warnings |
|---|---|---|---|---|---|---|
| chain-of-causation | 🔴 0% (0/1) | 🔴 0% (0/1) | 🔴 0% (0/1) | 🔴 0% (0/1) | 🔴 0% (0/1) | ⚠️ 45 skipped |
| compaction | 🟡 71% (5/7) | 🟢 100% (7/7) | 🔴 0% (0/7) | 🟡 29% (2/7) | 🟡 43% (3/7) | |
| context_window | 🟡 50% (3/6) | 🟡 50% (3/6) | 🟡 50% (3/6) | 🟡 50% (3/6) | 🟡 67% (4/6) | ⚠️ 5 skipped |
| counting | 🟡 50% (2/4) | 🟢 100% (4/4) | 🟢 100% (4/4) | 🟡 50% (2/4) | 🟢 100% (4/4) | |
| database | 🟢 100% (1/1) | 🔴 0% (0/1) | 🟢 100% (1/1) | 🟢 100% (1/1) | 🟢 100% (1/1) | ⚠️ 15 skipped |
| datadog | 🟡 50% (2/4) | 🟡 75% (¾) | 🟡 75% (¾) | 🟡 75% (¾) | 🟡 75% (¾) | |
| datetime | 🟡 75% (¾) | 🟡 75% (¾) | 🟡 75% (¾) | 🟡 75% (¾) | 🟢 100% (4/4) | ⚠️ 10 skipped |
| easy | 🟡 78% (32/41) | 🟡 56% (23/41) | 🟡 68% (28/41) | 🟡 76% (31/41) | 🟡 90% (37/41) | ⚠️ 5 skipped |
| hard | 🟡 40% (4/10) | 🟡 20% (2/10) | 🟡 40% (4/10) | 🟡 40% (4/10) | 🟡 70% (7/10) | ⚠️ 65 skipped |
| kafka | ⚪️ - | ⚪️ - | ⚪️ - | ⚪️ - | ⚪️ - | ⚠️ 10 skipped |
| kubernetes | 🟡 65% (28/43) | 🟡 40% (17/43) | 🟡 67% (29/43) | 🟡 72% (31/43) | 🟡 81% (35/43) | ⚠️ 50 skipped |
| logs | 🟡 58% (15/26) | 🟡 50% (13/26) | 🟡 77% (20/26) | 🟡 69% (18/26) | 🟡 77% (20/26) | ⚠️ 35 skipped |
| medium | 🟡 60% (26/43) | 🟡 49% (22/45) | 🟡 73% (32/44) | 🟡 70% (31/44) | 🟡 77% (33/43) | ⚠️ 36 skipped |
| network | 🟡 50% (2/4) | 🟡 25% (¼) | 🟡 50% (2/4) | 🟡 75% (¾) | 🟡 75% (¾) | |
| no-cicd | ⚪️ - | ⚪️ - | ⚪️ - | ⚪️ - | ⚪️ - | ⚠️ 5 skipped |
| numerical | 🟢 100% (1/1) | 🔴 0% (0/1) | 🟢 100% (1/1) | 🟢 100% (1/1) | 🟢 100% (1/1) | |
| one-test | 🟢 100% (1/1) | 🔴 0% (0/1) | 🟢 100% (1/1) | 🟢 100% (1/1) | 🟢 100% (1/1) | |
| port-forward | 🔴 0% (0/5) | 🔴 0% (0/5) | 🟡 20% (⅕) | 🔴 0% (0/5) | 🟡 20% (⅕) | ⚠️ 35 skipped |
| prometheus | 🔴 0% (0/2) | 🔴 0% (0/2) | 🔴 0% (0/2) | 🔴 0% (0/2) | 🔴 0% (0/2) | ⚠️ 25 skipped |
| question-answer | 🟡 75% (¾) | 🟡 50% (2/4) | 🟢 100% (4/4) | 🟢 100% (4/4) | 🟢 100% (4/4) | |
| runbooks | 🟡 67% (4/6) | 🟡 50% (3/6) | 🟡 67% (4/6) | 🟢 100% (6/6) | 🟢 100% (6/6) | ⚠️ 5 skipped |
| slackbot | ⚪️ - | 🔴 0% (0/1) | ⚪️ - | ⚪️ - | ⚪️ - | ⚠️ 4 skipped |
| traces | ⚪️ - | ⚪️ - | ⚪️ - | ⚪️ - | ⚪️ - | ⚠️ 25 skipped |
| transparency | 🟡 79% (11/14) | 🟡 71% (10/14) | 🟡 86% (12/14) | 🟡 79% (11/14) | 🟡 93% (13/14) | ⚠️ 5 skipped |
| Overall | 🟡 66% (62/94) | 🟡 49% (47/96) | 🟡 67% (64/95) | 🟡 69% (66/95) | 🟡 82% (77/94) | ⚠️ 106 skipped |
Raw Results¶
Status of all evaluations across models. Color coding:
- 🟢 Passing 100% (stable)
- 🟡 Passing 1-99%
- 🔴 Passing 0% (failing)
- 🔧 Mock data failure (missing or invalid test data)
- ⚠️ Setup failure (environment/infrastructure issue)
- ⏱️ Timeout or rate limit error
- ⏭️ Test skipped (e.g., known issue or precondition not met)
Detailed Raw Results¶
| Eval ID | deepseek-3.1 | gpt-5 | gpt-5.1 | haiku-4.5 | sonnet-4.5 |
|---|---|---|---|---|---|
| 001_compaction 🔗 | 🟢 100% (1/1) / ⏱️ 94.1s | 🟢 100% (1/1) / ⏱️ 72.4s | 🔴 0% (0/1) / ⏱️ 181.2s | 🟢 100% (1/1) / ⏱️ 55.1s | 🟢 100% (1/1) / ⏱️ 78.4s |
| 002_buried_exception 🔗 | 🟢 100% (1/1) / ⏱️ 30.3s | 🟢 100% (1/1) / ⏱️ 28.3s | 🔴 0% (0/1) / ⏱️ 156.4s | 🟢 100% (1/1) / ⏱️ 29.6s | 🟢 100% (1/1) / ⏱️ 47.3s |
| 003_cascading_failure 🔗 | 🔴 0% (0/1) / ⏱️ 53.1s | 🟢 100% (1/1) / ⏱️ 37.2s | 🔴 0% (0/1) / ⏱️ 172.2s | 🔴 0% (0/1) / ⏱️ 0.0s | 🔴 0% (0/1) / ⏱️ 0.1s |
| 004_multiple_root_causes 🔗 | 🔴 0% (0/1) / ⏱️ 23.1s | 🟢 100% (1/1) / ⏱️ 232.3s | 🔴 0% (0/1) / ⏱️ 234.9s | 🔴 0% (0/1) / ⏱️ 0.1s | 🔴 0% (0/1) / ⏱️ 0.0s |
| 005_configuration_change 🔗 | 🟢 100% (1/1) / ⏱️ 34.4s | 🟢 100% (1/1) / ⏱️ 27.4s | 🔴 0% (0/1) / ⏱️ 159.1s | 🔴 0% (0/1) / ⏱️ 0.0s | 🔴 0% (0/1) / ⏱️ 0.1s |
| 007_negative_findings 🔗 | 🟢 100% (1/1) / ⏱️ 20.9s | 🟢 100% (1/1) / ⏱️ 31.0s | 🔴 0% (0/1) / ⏱️ 193.5s | 🔴 0% (0/1) / ⏱️ 0.0s | 🔴 0% (0/1) / ⏱️ 0.1s |
| 008_very_long_conversation 🔗 | 🟢 100% (1/1) / ⏱️ 33.1s | 🟢 100% (1/1) / ⏱️ 32.6s | 🔴 0% (0/1) / ⏱️ 163.6s | 🔴 0% (0/1) / ⏱️ 55.2s | 🟢 100% (1/1) / ⏱️ 75.9s |
| 01_how_many_pods 🔗 | 🔴 0% (0/1) / ⏱️ 16.6s | 🟢 100% (1/1) / ⏱️ 15.8s / 💰 $0.03 | 🟢 100% (1/1) / ⏱️ 27.3s / 💰 $0.03 | 🟢 100% (1/1) / ⏱️ 15.5s / 💰 $0.03 | 🟢 100% (1/1) / ⏱️ 17.3s / 💰 $0.08 |
| 02_what_is_wrong_with_pod 🔗 | 🟢 100% (1/1) / ⏱️ 50.6s | 🟢 100% (1/1) / ⏱️ 28.1s / 💰 $0.08 | 🟢 100% (1/1) / ⏱️ 88.4s / 💰 $0.08 | 🟢 100% (1/1) / ⏱️ 21.9s / 💰 $0.04 | 🟢 100% (1/1) / ⏱️ 32.5s / 💰 $0.13 |
| 03_what_is_the_command_to_port_forward 🔗 | 🟢 100% (1/1) / ⏱️ 19.1s | 🔴 0% (0/1) / ⏱️ 5.5s / 💰 $0.01 | 🟢 100% (1/1) / ⏱️ 33.3s / 💰 $0.03 | 🟢 100% (1/1) / ⏱️ 13.0s / 💰 $0.02 | 🟢 100% (1/1) / ⏱️ 24.7s / 💰 $0.07 |
| 04_related_k8s_events 🔗 | 🟢 100% (1/1) / ⏱️ 39.0s | 🔴 0% (0/1) / ⏱️ 5.0s | 🟢 100% (1/1) / ⏱️ 45.0s / 💰 $0.05 | 🟢 100% (1/1) / ⏱️ 19.1s / 💰 $0.04 | 🟢 100% (1/1) / ⏱️ 25.0s / 💰 $0.10 |
| 05_image_version 🔗 | 🟢 100% (1/1) / ⏱️ 33.1s | 🟢 100% (1/1) / ⏱️ 24.0s / 💰 $0.05 | 🟢 100% (1/1) / ⏱️ 32.9s / 💰 $0.05 | 🟢 100% (1/1) / ⏱️ 20.3s / 💰 $0.03 | 🟢 100% (1/1) / ⏱️ 25.9s / 💰 $0.10 |
| 08_sock_shop_frontend 🔗 | ⚪️ - | ⚪️ - | ⚪️ - | ⚪️ - | ⚪️ - |
| 09_crashpod 🔗 | 🟢 100% (1/1) / ⏱️ 107.4s | 🔴 0% (0/1) / ⏱️ 4.1s / 💰 $0.01 | 🟢 100% (1/1) / ⏱️ 94.8s / 💰 $0.09 | 🟢 100% (1/1) / ⏱️ 44.2s / 💰 $0.07 | 🟢 100% (1/1) / ⏱️ 43.9s / 💰 $0.17 |
| 100a_historical_logs 🔗 | 🔴 0% (0/1) / ⏱️ 53.2s | 🔴 0% (0/1) / ⏱️ 14.9s / 💰 $0.02 | 🟢 100% (1/1) / ⏱️ 170.5s / 💰 $0.12 | 🔴 0% (0/1) / ⏱️ 207.0s / 💰 $0.13 | 🟢 100% (1/1) / ⏱️ 54.2s / 💰 $0.15 |
| 100b_historical_logs_nonstandard_label 🔗 | 🔴 0% (0/1) / ⏱️ 46.6s | 🔴 0% (0/1) / ⏱️ 17.5s / 💰 $0.03 | 🔴 0% (0/1) / ⏱️ 301.1s / 💰 $0.21 | 🔴 0% (0/1) / ⏱️ 79.4s / 💰 $0.10 | 🔴 0% (0/1) / ⏱️ 52.8s / 💰 $0.20 |
| 101_historical_logs_pod_deleted 🔗 | 🔴 0% (0/1) / ⏱️ 70.4s | 🔴 0% (0/1) / ⏱️ 9.7s / 💰 $0.01 | 🔴 0% (0/1) / ⏱️ 149.5s / 💰 $0.12 | 🔴 0% (0/1) / ⏱️ 42.9s / 💰 $0.04 | 🔴 0% (0/1) / ⏱️ 57.8s / 💰 $0.12 |
| 103_logs_transparency_default_limit 🔗 | 🔴 0% (0/1) / ⏱️ 25.0s | 🟢 100% (1/1) / ⏱️ 28.0s / 💰 $0.07 | 🟢 100% (1/1) / ⏱️ 100.2s / 💰 $0.11 | 🟢 100% (1/1) / ⏱️ 36.4s / 💰 $0.05 | 🟢 100% (1/1) / ⏱️ 37.1s / 💰 $0.12 |
| 104a_postgres_root_issue 🔗 | 🟢 100% (1/1) / ⏱️ 95.6s | 🔴 0% (0/1) / ⏱️ 41.2s / 💰 $0.11 | 🟢 100% (1/1) / ⏱️ 136.3s / 💰 $0.13 | 🟢 100% (1/1) / ⏱️ 44.8s / 💰 $0.07 | 🟢 100% (1/1) / ⏱️ 66.5s / 💰 $0.18 |
| 104b_postgres_missing_index_pgstat 🔗 | ⚪️ - | ⚪️ - | ⚪️ - | ⚪️ - | ⚪️ - |
| 104c_postgres_minimal_missing_index 🔗 | ⚪️ - | ⚪️ - | ⚪️ - | ⚪️ - | ⚪️ - |
| 105_redis_wrong_data_structure 🔗 | ⚪️ - | ⚪️ - | ⚪️ - | ⚪️ - | ⚪️ - |
| 107_log_filter_http_status_code 🔗 | 🔴 0% (0/1) / ⏱️ 118.1s | 🟢 100% (1/1) / ⏱️ 35.7s / 💰 $0.09 | 🟢 100% (1/1) / ⏱️ 251.9s / 💰 $0.20 | 🟢 100% (1/1) / ⏱️ 55.4s / 💰 $0.08 | 🟢 100% (1/1) / ⏱️ 59.6s / 💰 $0.21 |
| 108_logs_nearby_lines 🔗 | 🔴 0% (0/1) / ⏱️ 75.2s | 🔴 0% (0/1) / ⏱️ 28.3s / 💰 $0.07 | 🔴 0% (0/1) / ⏱️ 133.0s | 🔴 0% (0/1) / ⏱️ 77.4s / 💰 $0.11 | 🔴 0% (0/1) / ⏱️ 76.6s / 💰 $0.24 |
| 109_logs_transparency_not_found 🔗 | 🟢 100% (1/1) / ⏱️ 49.0s | 🟢 100% (1/1) / ⏱️ 25.7s / 💰 $0.10 | 🟢 100% (1/1) / ⏱️ 70.5s / 💰 $0.07 | 🟢 100% (1/1) / ⏱️ 31.0s / 💰 $0.04 | 🟢 100% (1/1) / ⏱️ 23.1s / 💰 $0.10 |
| 10_image_pull_backoff 🔗 | 🟢 100% (1/1) / ⏱️ 140.8s | 🔴 0% (0/1) / ⏱️ 5.0s / 💰 $0.01 | 🟢 100% (1/1) / ⏱️ 80.5s / 💰 $0.10 | 🟢 100% (1/1) / ⏱️ 36.5s / 💰 $0.06 | 🟢 100% (1/1) / ⏱️ 41.0s / 💰 $0.15 |
| 110_k8s_events_image_pull 🔗 | 🟢 100% (1/1) / ⏱️ 127.1s | 🟢 100% (1/1) / ⏱️ 21.3s / 💰 $0.05 | 🟢 100% (1/1) / ⏱️ 103.8s / 💰 $0.11 | 🟢 100% (1/1) / ⏱️ 22.1s / 💰 $0.04 | 🟢 100% (1/1) / ⏱️ 25.9s / 💰 $0.10 |
| 111_disabled_datadog_traces 🔗 | 🔴 0% (0/1) / ⏱️ 13.8s | 🟢 100% (1/1) / ⏱️ 7.6s / 💰 $0.01 | 🟢 100% (1/1) / ⏱️ 49.8s / 💰 $0.04 | 🟢 100% (1/1) / ⏱️ 10.0s / 💰 $0.02 | 🟢 100% (1/1) / ⏱️ 12.2s / 💰 $0.05 |
| 111_pod_names_contain_service 🔗 | 🔴 0% (0/1) / ⏱️ 7.0s | 🔴 0% (0/1) / ⏱️ 5.0s / 💰 $0.01 | 🔴 0% (0/1) / ⏱️ 7.2s / 💰 $0.01 | 🟢 100% (1/1) / ⏱️ 47.6s / 💰 $0.06 | 🟢 100% (1/1) / ⏱️ 45.5s / 💰 $0.28 |
| 112_find_pvcs_by_uuid 🔗 | 🟢 100% (1/1) / ⏱️ 91.5s | 🔴 0% (0/1) / ⏱️ 8.9s / 💰 $0.01 | 🟢 100% (1/1) / ⏱️ 52.9s / 💰 $0.04 | 🟢 100% (1/1) / ⏱️ 23.6s / 💰 $0.04 | 🟢 100% (1/1) / ⏱️ 35.5s / 💰 $0.15 |
| 114_checkout_latency_tracing_rebuild[0] 🔗 | ⚪️ - | ⚪️ - | ⚪️ - | ⚪️ - | ⚪️ - |
| 115_checkout_errors_tracing[0] 🔗 | ⚪️ - | ⚪️ - | ⚪️ - | ⚪️ - | ⚪️ - |
| 11_init_containers 🔗 | 🟢 100% (1/1) / ⏱️ 89.4s | 🔴 0% (0/1) / ⏱️ 4.3s / 💰 $0.01 | 🟢 100% (1/1) / ⏱️ 84.4s / 💰 $0.11 | 🟢 100% (1/1) / ⏱️ 36.1s / 💰 $0.05 | 🟢 100% (1/1) / ⏱️ 37.6s / 💰 $0.15 |
| 121_new_relic_checkout_errors_tracing[0] 🔗 | ⚪️ - | ⚪️ - | ⚪️ - | ⚪️ - | ⚪️ - |
| 122_new_relic_checkout_latency_tracing_rebuild[0] 🔗 | ⚪️ - | ⚪️ - | ⚪️ - | ⚪️ - | ⚪️ - |
| 123_new_relic_checkout_errors_tracing[0] 🔗 | ⚪️ - | ⚪️ - | ⚪️ - | ⚪️ - | ⚪️ - |
| 124_checkout_latency_prometheus[0] 🔗 | ⚪️ - | ⚪️ - | ⚪️ - | ⚪️ - | ⚪️ - |
| 12_job_crashing 🔗 | 🔴 0% (0/1) / ⏱️ 78.3s | 🔴 0% (0/1) / ⏱️ 37.5s / 💰 $0.10 | 🔴 0% (0/1) / ⏱️ 63.5s / 💰 $0.07 | 🟢 100% (1/1) / ⏱️ 31.8s / 💰 $0.05 | 🟢 100% (1/1) / ⏱️ 56.5s / 💰 $0.18 |
| 13a_pending_node_selector_basic 🔗 | 🟢 100% (1/1) / ⏱️ 45.1s | 🔴 0% (0/1) / ⏱️ 5.2s / 💰 $0.01 | 🔴 0% (0/1) / ⏱️ 15.7s / 💰 $0.01 | 🟢 100% (1/1) / ⏱️ 32.3s / 💰 $0.05 | 🟢 100% (1/1) / ⏱️ 39.2s / 💰 $0.16 |
| 13b_pending_node_selector_detailed 🔗 | 🟢 100% (1/1) / ⏱️ 66.4s | 🔴 0% (0/1) / ⏱️ 5.0s / 💰 $0.01 | 🟢 100% (1/1) / ⏱️ 211.1s / 💰 $0.19 | 🟢 100% (1/1) / ⏱️ 33.4s / 💰 $0.05 | 🟢 100% (1/1) / ⏱️ 52.8s / 💰 $0.17 |
| 14_pending_resources 🔗 | 🟢 100% (1/1) / ⏱️ 46.6s | 🔴 0% (0/1) / ⏱️ 4.7s / 💰 $0.02 | 🟢 100% (1/1) / ⏱️ 100.2s / 💰 $0.09 | 🔴 0% (0/1) / ⏱️ 8.9s / 💰 $0.02 | 🟢 100% (1/1) / ⏱️ 47.1s / 💰 $0.17 |
| 156_kafka_opensearch_latency 🔗 | ⚪️ - | ⚪️ - | ⚪️ - | ⚪️ - | ⚪️ - |
| 159_prometheus_high_cardinality_cpu[0] 🔗 | ⚪️ - | ⚪️ - | ⚪️ - | ⚪️ - | ⚪️ - |
| 159_prometheus_high_cardinality_cpu[1] 🔗 | 🔴 0% (0/1) / ⏱️ 32.2s | 🔴 0% (0/1) / ⏱️ 16.1s / 💰 $0.03 | 🔴 0% (0/1) / ⏱️ 165.9s / 💰 $0.10 | 🔴 0% (0/1) / ⏱️ 13.9s / 💰 $0.03 | 🔴 0% (0/1) / ⏱️ 20.6s / 💰 $0.09 |
| 159_prometheus_high_cardinality_cpu[2] 🔗 | 🔴 0% (0/1) / ⏱️ 32.3s | 🔴 0% (0/1) / ⏱️ 16.6s / 💰 $0.05 | 🔴 0% (0/1) / ⏱️ 115.0s / 💰 $0.09 | 🔴 0% (0/1) / ⏱️ 15.9s / 💰 $0.04 | 🔴 0% (0/1) / ⏱️ 23.6s / 💰 $0.10 |
| 15_failed_readiness_probe 🔗 | 🔴 0% (0/1) / ⏱️ 7.3s | 🔴 0% (0/1) / ⏱️ 4.3s / 💰 $0.01 | 🟢 100% (1/1) / ⏱️ 58.7s / 💰 $0.08 | 🟢 100% (1/1) / ⏱️ 33.6s / 💰 $0.05 | 🟢 100% (1/1) / ⏱️ 47.9s / 💰 $0.22 |
| 160_electricity_market_bidding_bug[0] 🔗 | ⚪️ - | ⚪️ - | ⚪️ - | ⚪️ - | ⚪️ - |
| 161_bidding_version_performance[0] 🔗 | ⚪️ - | ⚪️ - | ⚪️ - | ⚪️ - | ⚪️ - |
| 16_failed_no_toolset_found 🔗 | 🔴 0% (0/1) / ⏱️ 8.7s | 🔴 0% (0/1) / ⏱️ 9.1s / 💰 $0.01 | 🔴 0% (0/1) / ⏱️ 10.9s / 💰 $0.01 | 🔴 0% (0/1) / ⏱️ 10.9s / 💰 $0.02 | 🔴 0% (0/1) / ⏱️ 15.5s / 💰 $0.07 |
| 17_oom_kill 🔗 | 🔴 0% (0/1) / ⏱️ 73.6s | 🔴 0% (0/1) / ⏱️ 5.4s / 💰 $0.01 | 🔴 0% (0/1) / ⏱️ 113.9s / 💰 $0.12 | 🔴 0% (0/1) / ⏱️ 39.6s / 💰 $0.06 | 🔴 0% (0/1) / ⏱️ 55.0s / 💰 $0.20 |
| 18_oom_kill_from_issues_history 🔗 | 🔴 0% (0/1) / ⏱️ 37.5s | 🟢 100% (1/1) / ⏱️ 27.8s / 💰 $0.06 | 🔴 0% (0/1) / ⏱️ 298.2s / 💰 $0.26 | 🟢 100% (1/1) / ⏱️ 32.7s / 💰 $0.05 | 🟢 100% (1/1) / ⏱️ 60.1s / 💰 $0.17 |
| 19_detect_missing_app_details 🔗 | 🟢 100% (1/1) / ⏱️ 57.0s | 🔴 0% (0/1) / ⏱️ 24.4s / 💰 $0.07 | 🟢 100% (1/1) / ⏱️ 359.0s / 💰 $0.27 | 🟢 100% (1/1) / ⏱️ 34.6s / 💰 $0.04 | 🟢 100% (1/1) / ⏱️ 80.0s / 💰 $0.31 |
| 20_long_log_file_search 🔗 | 🟢 100% (1/1) / ⏱️ 91.6s | 🟢 100% (1/1) / ⏱️ 28.1s / 💰 $0.05 | 🟢 100% (1/1) / ⏱️ 105.8s / 💰 $0.08 | 🔴 0% (0/1) / ⏱️ 26.2s / 💰 $0.04 | 🟢 100% (1/1) / ⏱️ 39.3s / 💰 $0.12 |
| 21_job_fail_curl_no_svc_account 🔗 | 🔴 0% (0/1) / ⏱️ 25.2s | 🔴 0% (0/1) / ⏱️ 613.5s / 💰 $0.04 | 🟢 100% (1/1) / ⏱️ 868.9s / 💰 $0.23 | 🔴 0% (0/1) / ⏱️ 7.8s / 💰 $0.02 | 🟢 100% (1/1) / ⏱️ 47.8s / 💰 $0.15 |
| 22_high_latency_dbi_down 🔗 | ⚪️ - | ⚪️ - | ⚪️ - | ⚪️ - | ⚪️ - |
| 23_app_error_in_current_logs 🔗 | 🔴 0% (0/1) / ⏱️ 73.0s | 🔴 0% (0/1) / ⏱️ 35.0s / 💰 $0.09 | 🔴 0% (0/1) / ⏱️ 125.0s / 💰 $0.14 | 🔴 0% (0/1) / ⏱️ 51.4s / 💰 $0.07 | 🔴 0% (0/1) / ⏱️ 66.0s / 💰 $0.21 |
| 24_misconfigured_pvc 🔗 | 🟢 100% (1/1) / ⏱️ 65.1s | 🔴 0% (0/1) / ⏱️ 4.2s / 💰 $0.01 | 🔴 0% (0/1) / ⏱️ 16.8s / 💰 $0.02 | 🟢 100% (1/1) / ⏱️ 48.4s / 💰 $0.07 | 🟢 100% (1/1) / ⏱️ 41.5s / 💰 $0.17 |
| 24a_misconfigured_pvc_basic 🔗 | 🔴 0% (0/1) / ⏱️ 7.0s | 🔴 0% (0/1) / ⏱️ 3.8s / 💰 $0.01 | 🔴 0% (0/1) / ⏱️ 14.3s / 💰 $0.01 | 🔴 0% (0/1) / ⏱️ 8.5s / 💰 $0.02 | 🟢 100% (1/1) / ⏱️ 52.4s / 💰 $0.18 |
| 24b_misconfigured_pvc_detailed 🔗 | 🔴 0% (0/1) / ⏱️ 111.9s | 🔴 0% (0/1) / ⏱️ 4.7s / 💰 $0.01 | 🔴 0% (0/1) / ⏱️ 14.5s / 💰 $0.01 | 🔴 0% (0/1) / ⏱️ 7.4s / 💰 $0.02 | 🟢 100% (1/1) / ⏱️ 73.2s / 💰 $0.24 |
| 25_misconfigured_ingress_class 🔗 | 🔴 0% (0/1) / ⏱️ 73.2s | 🔴 0% (0/1) / ⏱️ 12.1s / 💰 $0.02 | 🔴 0% (0/1) / ⏱️ 33.3s / 💰 $0.03 | 🔴 0% (0/1) / ⏱️ 11.4s / 💰 $0.02 | 🔴 0% (0/1) / ⏱️ 13.4s / 💰 $0.05 |
| 26_page_render_times 🔗 | 🟢 100% (1/1) / ⏱️ 57.7s | 🔴 0% (0/1) / ⏱️ 6.4s | 🟢 100% (1/1) / ⏱️ 271.9s / 💰 $0.23 | 🟢 100% (1/1) / ⏱️ 23.9s / 💰 $0.06 | 🟢 100% (1/1) / ⏱️ 33.2s / 💰 $0.13 |
| 27a_multi_container_logs 🔗 | 🟢 100% (1/1) / ⏱️ 46.6s | 🟢 100% (1/1) / ⏱️ 39.0s / 💰 $0.10 | 🟢 100% (1/1) / ⏱️ 116.7s / 💰 $0.13 | 🟢 100% (1/1) / ⏱️ 23.3s / 💰 $0.05 | 🟢 100% (1/1) / ⏱️ 44.7s / 💰 $0.17 |
| 27b_multi_container_logs 🔗 | 🟢 100% (1/1) / ⏱️ 18.5s | 🟢 100% (1/1) / ⏱️ 39.3s / 💰 $0.10 | 🟢 100% (1/1) / ⏱️ 160.0s / 💰 $0.17 | 🟢 100% (1/1) / ⏱️ 20.1s / 💰 $0.04 | 🟢 100% (1/1) / ⏱️ 29.9s / 💰 $0.13 |
| 28_permissions_error 🔗 | 🟢 100% (1/1) / ⏱️ 23.8s | 🔴 0% (0/1) / ⏱️ 19.8s / 💰 $0.05 | 🔴 0% (0/1) / ⏱️ 96.6s / 💰 $0.09 | 🔴 0% (0/1) / ⏱️ 11.2s / 💰 $0.03 | 🟢 100% (1/1) / ⏱️ 19.7s / 💰 $0.09 |
| 33_cpu_metrics_discovery 🔗 | ⚪️ - | ⚪️ - | ⚪️ - | ⚪️ - | ⚪️ - |
| 39_failed_toolset 🔗 | 🟢 100% (1/1) / ⏱️ 50.2s | 🔴 0% (0/1) / ⏱️ 8.4s / 💰 $0.01 | 🟢 100% (1/1) / ⏱️ 84.5s / 💰 $0.06 | 🟢 100% (1/1) / ⏱️ 27.7s / 💰 $0.03 | 🟢 100% (1/1) / ⏱️ 33.5s / 💰 $0.09 |
| 41_setup_argo 🔗 | 🟢 100% (1/1) / ⏱️ 6.6s | 🟢 100% (1/1) / ⏱️ 10.1s / 💰 $0.01 | 🟢 100% (1/1) / ⏱️ 33.7s / 💰 $0.02 | 🟢 100% (1/1) / ⏱️ 7.0s / 💰 $0.02 | 🟢 100% (1/1) / ⏱️ 10.6s / 💰 $0.05 |
| 42_dns_issues_result_new_tools_no_runbook 🔗 | 🔴 0% (0/1) / ⏱️ 114.4s | 🔴 0% (0/1) / ⏱️ 7.1s / 💰 $0.02 | 🔴 0% (0/1) / ⏱️ 239.7s / 💰 $0.12 | 🟢 100% (1/1) / ⏱️ 30.3s / 💰 $0.04 | 🟢 100% (1/1) / ⏱️ 92.4s / 💰 $0.11 |
| 42_dns_issues_steps_new_tools 🔗 | 🟢 100% (1/1) / ⏱️ 182.9s | 🔴 0% (0/1) / ⏱️ 11.8s / 💰 $0.03 | 🟢 100% (1/1) / ⏱️ 212.1s / 💰 $0.18 | 🟢 100% (1/1) / ⏱️ 135.3s / 💰 $0.08 | 🟢 100% (1/1) / ⏱️ 211.9s / 💰 $0.25 |
| 43_current_datetime_from_prompt 🔗 | 🟢 100% (1/1) / ⏱️ 6.6s | 🔴 0% (0/1) / ⏱️ 9.6s / 💰 $0.02 | 🟢 100% (1/1) / ⏱️ 34.7s / 💰 $0.03 | 🟢 100% (1/1) / ⏱️ 5.8s / 💰 $0.02 | 🟢 100% (1/1) / ⏱️ 7.0s / 💰 $0.06 |
| 43_slack_deployment_logs 🔗 | ⚪️ - | ⚪️ - | ⚪️ - | ⚪️ - | ⚪️ - |
| 44_slack_statefulset_logs 🔗 | ⚪️ - | 🔴 0% (0/1) / ⏱️ 4.7s / 💰 $0.01 | ⚪️ - | ⚪️ - | ⚪️ - |
| 45_fetch_deployment_logs_simple 🔗 | 🟢 100% (1/1) / ⏱️ 55.0s | 🟢 100% (1/1) / ⏱️ 28.9s / 💰 $0.10 | 🔴 0% (0/1) / ⏱️ 15.2s / 💰 $0.01 | 🟢 100% (1/1) / ⏱️ 24.7s / 💰 $0.04 | 🟢 100% (1/1) / ⏱️ 28.6s / 💰 $0.12 |
| 48_logs_since_thursday 🔗 | ⚪️ - | ⚪️ - | ⚪️ - | ⚪️ - | ⚪️ - |
| 50_logs_since_specific_date 🔗 | 🟢 100% (1/1) / ⏱️ 25.4s | 🔴 0% (0/1) / ⏱️ 9.1s / 💰 $0.01 | 🟢 100% (1/1) / ⏱️ 55.5s / 💰 $0.05 | 🟢 100% (1/1) / ⏱️ 11.0s / 💰 $0.02 | 🟢 100% (1/1) / ⏱️ 20.4s / 💰 $0.06 |
| 50a_logs_since_last_specific_month 🔗 | 🔴 0% (0/1) / ⏱️ 8.7s | 🔴 0% (0/1) / ⏱️ 12.2s / 💰 $0.02 | 🟢 100% (1/1) / ⏱️ 89.8s / 💰 $0.04 | 🟢 100% (1/1) / ⏱️ 16.4s / 💰 $0.02 | 🟢 100% (1/1) / ⏱️ 18.7s / 💰 $0.07 |
| 51_logs_summarize_errors 🔗 | 🔴 0% (0/1) / ⏱️ 17.9s | 🔴 0% (0/1) / ⏱️ 5.6s / 💰 $0.01 | 🟢 100% (1/1) / ⏱️ 90.1s / 💰 $0.05 | 🟢 100% (1/1) / ⏱️ 15.9s / 💰 $0.03 | 🟢 100% (1/1) / ⏱️ 26.8s / 💰 $0.08 |
| 52_logs_login_issues 🔗 | 🟢 100% (1/1) / ⏱️ 30.3s | 🔴 0% (0/1) / ⏱️ 4.0s / 💰 $0.01 | 🟢 100% (1/1) / ⏱️ 93.9s / 💰 $0.09 | 🟢 100% (1/1) / ⏱️ 24.6s / 💰 $0.03 | 🟢 100% (1/1) / ⏱️ 29.6s / 💰 $0.10 |
| 53_logs_find_term 🔗 | 🟢 100% (1/1) / ⏱️ 20.8s | 🔴 0% (0/1) / ⏱️ 4.5s / 💰 $0.01 | 🟢 100% (1/1) / ⏱️ 62.0s / 💰 $0.04 | 🟢 100% (1/1) / ⏱️ 15.2s / 💰 $0.04 | 🟢 100% (1/1) / ⏱️ 22.0s / 💰 $0.11 |
| 54_not_truncated_when_getting_pods 🔗 | 🟢 100% (1/1) / ⏱️ 33.9s | 🔴 0% (0/1) / ⏱️ 6.3s / 💰 $0.01 | 🟢 100% (1/1) / ⏱️ 70.6s / 💰 $0.06 | 🟢 100% (1/1) / ⏱️ 20.1s / 💰 $0.03 | 🟢 100% (1/1) / ⏱️ 23.9s / 💰 $0.08 |
| 55_kafka_runbook 🔗 | ⚪️ - | ⚪️ - | ⚪️ - | ⚪️ - | ⚪️ - |
| 57_wrong_namespace 🔗 | 🟢 100% (1/1) / ⏱️ 20.5s | 🔴 0% (0/1) / ⏱️ 6.8s / 💰 $0.02 | 🟢 100% (1/1) / ⏱️ 65.0s / 💰 $0.06 | 🟢 100% (1/1) / ⏱️ 24.8s / 💰 $0.03 | 🔴 0% (0/1) / ⏱️ 29.6s / 💰 $0.10 |
| 59_label_based_counting 🔗 | 🟢 100% (1/1) / ⏱️ 20.9s | 🟢 100% (1/1) / ⏱️ 19.4s / 💰 $0.03 | 🟢 100% (1/1) / ⏱️ 38.6s / 💰 $0.03 | 🔴 0% (0/1) / ⏱️ 15.6s / 💰 $0.03 | 🟢 100% (1/1) / ⏱️ 16.4s / 💰 $0.09 |
| 60_count_less_than 🔗 | 🔴 0% (0/1) / ⏱️ 23.1s | 🟢 100% (1/1) / ⏱️ 15.0s / 💰 $0.04 | 🟢 100% (1/1) / ⏱️ 29.0s / 💰 $0.04 | 🟢 100% (1/1) / ⏱️ 20.2s / 💰 $0.03 | 🟢 100% (1/1) / ⏱️ 23.4s / 💰 $0.10 |
| 61_exact_match_counting 🔗 | 🟢 100% (1/1) / ⏱️ 19.2s | 🟢 100% (1/1) / ⏱️ 15.4s / 💰 $0.04 | 🟢 100% (1/1) / ⏱️ 27.2s / 💰 $0.03 | 🔴 0% (0/1) / ⏱️ 16.4s / 💰 $0.03 | 🟢 100% (1/1) / ⏱️ 17.5s / 💰 $0.09 |
| 62_fetch_error_logs_with_errors 🔗 | 🟢 100% (1/1) / ⏱️ 25.9s | 🟢 100% (1/1) / ⏱️ 18.4s / 💰 $0.04 | 🟢 100% (1/1) / ⏱️ 38.1s / 💰 $0.04 | 🟢 100% (1/1) / ⏱️ 20.9s / 💰 $0.03 | 🟢 100% (1/1) / ⏱️ 24.3s / 💰 $0.10 |
| 63_fetch_error_logs_no_errors 🔗 | 🟢 100% (1/1) / ⏱️ 30.5s | 🟢 100% (1/1) / ⏱️ 23.6s / 💰 $0.05 | 🟢 100% (1/1) / ⏱️ 102.8s / 💰 $0.07 | 🟢 100% (1/1) / ⏱️ 33.8s / 💰 $0.05 | 🟢 100% (1/1) / ⏱️ 32.5s / 💰 $0.12 |
| 64_keda_vs_hpa_confusion 🔗 | 🟢 100% (1/1) / ⏱️ 183.1s | 🔴 0% (0/1) / ⏱️ 6.3s / 💰 $0.01 | 🟢 100% (1/1) / ⏱️ 287.8s / 💰 $0.22 | 🔴 0% (0/1) / ⏱️ 38.5s / 💰 $0.06 | 🔴 0% (0/1) / ⏱️ 37.2s / 💰 $0.14 |
| 65_health_check_followup 🔗 | 🟢 100% (1/1) / ⏱️ 69.4s | 🟢 100% (1/1) / ⏱️ 41.6s / 💰 $0.12 | 🟢 100% (1/1) / ⏱️ 136.5s / 💰 $0.13 | 🔴 0% (0/1) / ⏱️ 43.5s / 💰 $0.07 | 🟢 100% (1/1) / ⏱️ 52.7s / 💰 $0.19 |
| 71_connection_pool_starvation 🔗 | 🟢 100% (1/1) / ⏱️ 95.2s | 🟢 100% (1/1) / ⏱️ 29.7s / 💰 $0.07 | 🟢 100% (1/1) / ⏱️ 206.3s / 💰 $0.22 | 🟢 100% (1/1) / ⏱️ 39.5s / 💰 $0.07 | 🟢 100% (1/1) / ⏱️ 39.4s / 💰 $0.19 |
| 73a_time_window_anomaly 🔗 | 🟢 100% (1/1) / ⏱️ 68.7s | 🟢 100% (1/1) / ⏱️ 36.0s / 💰 $0.09 | 🟢 100% (1/1) / ⏱️ 90.7s / 💰 $0.09 | 🟢 100% (1/1) / ⏱️ 42.9s / 💰 $0.07 | 🟢 100% (1/1) / ⏱️ 54.0s / 💰 $0.19 |
| 73b_time_window_anomaly 🔗 | 🔴 0% (0/1) / ⏱️ 107.0s | 🟢 100% (1/1) / ⏱️ 29.2s / 💰 $0.08 | 🔴 0% (0/1) / ⏱️ 115.4s | 🔴 0% (0/1) / ⏱️ 55.9s / 💰 $0.05 | 🟢 100% (1/1) / ⏱️ 70.5s / 💰 $0.34 |
| 76_service_discovery_issue 🔗 | 🟢 100% (1/1) / ⏱️ 127.0s | 🟢 100% (1/1) / ⏱️ 29.5s / 💰 $0.07 | 🟢 100% (1/1) / ⏱️ 82.8s / 💰 $0.13 | 🟢 100% (1/1) / ⏱️ 45.9s / 💰 $0.07 | 🟢 100% (1/1) / ⏱️ 52.1s / 💰 $0.17 |
| 77_liveness_probe_misconfiguration 🔗 | 🟢 100% (1/1) / ⏱️ 46.5s | 🟢 100% (1/1) / ⏱️ 33.2s / 💰 $0.08 | 🟢 100% (1/1) / ⏱️ 90.4s / 💰 $0.11 | 🟢 100% (1/1) / ⏱️ 31.7s / 💰 $0.05 | 🟢 100% (1/1) / ⏱️ 57.0s / 💰 $0.19 |
| 78a_missing_cpu_limits 🔗 | 🟢 100% (1/1) / ⏱️ 126.7s | 🟢 100% (1/1) / ⏱️ 29.6s / 💰 $0.08 | 🟢 100% (1/1) / ⏱️ 288.9s / 💰 $0.24 | 🟢 100% (1/1) / ⏱️ 39.0s / 💰 $0.05 | 🟢 100% (1/1) / ⏱️ 54.8s / 💰 $0.16 |
| 78b_cpu_quota_exceeded 🔗 | 🔴 0% (0/1) / ⏱️ 95.0s | 🔴 0% (0/1) / ⏱️ 30.8s / 💰 $0.05 | 🟢 100% (1/1) / ⏱️ 124.5s / 💰 $0.14 | 🟢 100% (1/1) / ⏱️ 38.1s / 💰 $0.06 | 🟢 100% (1/1) / ⏱️ 59.4s / 💰 $0.17 |
| 79_configmap_mount_issue 🔗 | 🟢 100% (1/1) / ⏱️ 40.1s | 🟢 100% (1/1) / ⏱️ 33.5s / 💰 $0.08 | 🟢 100% (1/1) / ⏱️ 161.3s / 💰 $0.12 | 🟢 100% (1/1) / ⏱️ 28.8s / 💰 $0.04 | 🟢 100% (1/1) / ⏱️ 38.2s / 💰 $0.13 |
| 80_pvc_storage_class_mismatch 🔗 | 🟢 100% (1/1) / ⏱️ 53.9s | 🔴 0% (0/1) / ⏱️ 35.2s / 💰 $0.09 | 🟢 100% (1/1) / ⏱️ 105.2s / 💰 $0.11 | 🟢 100% (1/1) / ⏱️ 38.0s / 💰 $0.06 | 🟢 100% (1/1) / ⏱️ 49.8s / 💰 $0.18 |
| 81_service_account_permission_denied 🔗 | 🟢 100% (1/1) / ⏱️ 62.0s | 🟢 100% (1/1) / ⏱️ 32.6s / 💰 $0.08 | 🟢 100% (1/1) / ⏱️ 100.1s / 💰 $0.12 | 🟢 100% (1/1) / ⏱️ 41.1s / 💰 $0.09 | 🟢 100% (1/1) / ⏱️ 71.4s / 💰 $0.26 |
| 82_pod_anti_affinity_conflict 🔗 | 🟢 100% (1/1) / ⏱️ 139.8s | 🔴 0% (0/1) / ⏱️ 39.0s / 💰 $0.10 | 🟢 100% (1/1) / ⏱️ 167.7s / 💰 $0.16 | 🔴 0% (0/1) / ⏱️ 46.9s / 💰 $0.08 | 🔴 0% (0/1) / ⏱️ 42.0s / 💰 $0.17 |
| 83_secret_not_found 🔗 | 🟢 100% (1/1) / ⏱️ 48.9s | 🟢 100% (1/1) / ⏱️ 29.8s / 💰 $0.08 | 🟢 100% (1/1) / ⏱️ 173.5s / 💰 $0.13 | 🟢 100% (1/1) / ⏱️ 29.6s / 💰 $0.05 | 🟢 100% (1/1) / ⏱️ 47.6s / 💰 $0.17 |
| 84_network_policy_blocking_traffic 🔗 | 🟢 100% (1/1) / ⏱️ 178.7s | 🟢 100% (1/1) / ⏱️ 42.6s / 💰 $0.09 | 🟢 100% (1/1) / ⏱️ 199.8s / 💰 $0.17 | 🟢 100% (1/1) / ⏱️ 54.1s / 💰 $0.08 | 🟢 100% (1/1) / ⏱️ 74.4s / 💰 $0.26 |
| 85_hpa_not_scaling 🔗 | 🟢 100% (1/1) / ⏱️ 56.4s | 🟢 100% (1/1) / ⏱️ 46.8s / 💰 $0.08 | 🟢 100% (1/1) / ⏱️ 126.6s / 💰 $0.13 | 🟢 100% (1/1) / ⏱️ 33.2s / 💰 $0.05 | 🟢 100% (1/1) / ⏱️ 39.9s / 💰 $0.15 |
| 86_configmap_like_but_secret 🔗 | 🟢 100% (1/1) / ⏱️ 66.2s | 🟢 100% (1/1) / ⏱️ 37.9s / 💰 $0.11 | 🔴 0% (0/1) / ⏱️ 14.0s / 💰 $0.01 | 🟢 100% (1/1) / ⏱️ 43.6s / 💰 $0.06 | 🟢 100% (1/1) / ⏱️ 46.6s / 💰 $0.15 |
| 89_runbook_missing_cloudwatch 🔗 | 🟢 100% (1/1) / ⏱️ 22.5s | 🟢 100% (1/1) / ⏱️ 14.0s / 💰 $0.02 | 🟢 100% (1/1) / ⏱️ 60.0s / 💰 $0.05 | 🟢 100% (1/1) / ⏱️ 17.7s / 💰 $0.02 | 🟢 100% (1/1) / ⏱️ 39.8s / 💰 $0.10 |
| 90_runbook_basic_selection 🔗 | 🟢 100% (1/1) / ⏱️ 115.2s | 🟢 100% (1/1) / ⏱️ 29.5s / 💰 $0.06 | 🟢 100% (1/1) / ⏱️ 292.6s / 💰 $0.26 | 🟢 100% (1/1) / ⏱️ 84.4s / 💰 $0.11 | 🟢 100% (1/1) / ⏱️ 147.5s / 💰 $0.62 |
| 91f_datadog_logs_historical_pod 🔗 | 🔴 0% (0/1) / ⏱️ 19.8s | 🔴 0% (0/1) / ⏱️ 7.8s / 💰 $0.01 | 🔴 0% (0/1) / ⏱️ 202.7s / 💰 $0.13 | 🔴 0% (0/1) / ⏱️ 11.1s / 💰 $0.02 | 🔴 0% (0/1) / ⏱️ 12.3s / 💰 $0.05 |
| 93_calling_datadog[0] 🔗 | 🟢 100% (1/1) / ⏱️ 20.3s | 🟢 100% (1/1) / ⏱️ 9.9s / 💰 $0.06 | 🟢 100% (1/1) / ⏱️ 36.8s / 💰 $0.06 | 🟢 100% (1/1) / ⏱️ 9.0s / 💰 $0.04 | 🟢 100% (1/1) / ⏱️ 19.2s / 💰 $0.13 |
| 93_calling_datadog[1] 🔗 | 🔴 0% (0/1) / ⏱️ 10.9s | 🟢 100% (1/1) / ⏱️ 17.5s / 💰 $0.05 | 🟢 100% (1/1) / ⏱️ 56.1s / 💰 $0.08 | 🟢 100% (1/1) / ⏱️ 8.8s / 💰 $0.04 | 🟢 100% (1/1) / ⏱️ 14.9s / 💰 $0.13 |
| 93_calling_datadog[2] 🔗 | 🟢 100% (1/1) / ⏱️ 16.0s | 🟢 100% (1/1) / ⏱️ 10.8s / 💰 $0.04 | 🟢 100% (1/1) / ⏱️ 29.1s / 💰 $0.05 | 🟢 100% (1/1) / ⏱️ 9.5s / 💰 $0.04 | 🟢 100% (1/1) / ⏱️ 15.7s / 💰 $0.13 |
| 93_events_since_specific_date 🔗 | ⚪️ - | 🔴 0% (0/1) / ⏱️ 4.4s / 💰 $0.01 | 🔴 0% (0/1) / ⏱️ 14.5s / 💰 $0.01 | 🔴 0% (0/1) / ⏱️ 7.5s / 💰 $0.02 | ⚪️ - |
| 94_runbook_transparency 🔗 | 🟢 100% (1/1) / ⏱️ 52.3s | 🟢 100% (1/1) / ⏱️ 33.7s / 💰 $0.07 | 🟢 100% (1/1) / ⏱️ 348.9s / 💰 $0.10 | 🟢 100% (1/1) / ⏱️ 67.5s / 💰 $0.10 | 🟢 100% (1/1) / ⏱️ 123.3s / 💰 $0.30 |
| 96_no_matching_runbook 🔗 | 🔴 0% (0/1) / ⏱️ 165.1s | 🔴 0% (0/1) / ⏱️ 51.4s / 💰 $0.18 | 🔴 0% (0/1) / ⏱️ 485.3s / 💰 $0.31 | 🟢 100% (1/1) / ⏱️ 52.8s / 💰 $0.07 | 🟢 100% (1/1) / ⏱️ 70.7s / 💰 $0.26 |
| 97_logs_clarification_needed 🔗 | 🟢 100% (1/1) / ⏱️ 8.0s | 🟢 100% (1/1) / ⏱️ 5.5s / 💰 $0.01 | 🟢 100% (1/1) / ⏱️ 7.2s / 💰 $0.01 | 🟢 100% (1/1) / ⏱️ 6.4s / 💰 $0.02 | 🟢 100% (1/1) / ⏱️ 9.0s / 💰 $0.06 |
| 98_logs_transparency_default_time 🔗 | ⚪️ - | ⚪️ - | ⚪️ - | ⚪️ - | ⚪️ - |
| 99_logs_transparency_custom_time 🔗 | 🟢 100% (1/1) / ⏱️ 74.5s | 🟢 100% (1/1) / ⏱️ 20.2s / 💰 $0.05 | 🟢 100% (1/1) / ⏱️ 99.7s / 💰 $0.10 | 🟢 100% (1/1) / ⏱️ 24.4s / 💰 $0.04 | 🟢 100% (1/1) / ⏱️ 35.0s / 💰 $0.12 |
Results are automatically generated and updated weekly. View full traces and detailed analysis in Braintrust experiment: local-benchmark-20251126-175926.