OG T
|
f045506abd
|
fix(flywheel): P2 Approval 逾期不結案 → KM 學習鏈斷鏈修復
CD Pipeline / build-and-deploy (push) Failing after 12m11s
問題根因:
PENDING approval 無人處置超過 48h 後應自動 EXPIRED,
但 get_pending_approvals() 只在用戶開 UI 時觸發,
若無人開 UI → Incident 永遠 PENDING → KM 永遠不寫入
→ Phase 6 SLO human_override_rate 低估,EWMA 缺少負向樣本。
修復:
1. anomaly_counter.py: 新增 "timeout_ignored" disposition 類型,
與 auto_repair / human_approved / manual_resolved 區分
2. incident_service.py: resolve_incident() 新增 resolution_type 參數,
resolution_type="timeout" 時記錄 "timeout_ignored" 而非 "manual_resolved"
3. jobs/approval_timeout_resolver.py (新): 每小時掃描逾期 PENDING approval,
批次標記 EXPIRED,對每筆有 incident_id 的記錄呼叫 resolve_incident("timeout")
4. main.py: startup 掛載 approval_timeout_resolver 排程(interval=3600s)
效果:
- 告警無人處置 48h → Incident 自動結案 → KM 寫入 → EWMA 取得樣本
- disposition="timeout_ignored" 讓 SLO 計算正確區分「AI 建議被忽略」
- 飛輪學習鏈對「無人處置告警」閉環
2026-04-15 ogt + Claude Sonnet 4.6(亞太)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
2026-04-15 19:21:21 +08:00 |
|
OG T
|
b7ea362efc
|
fix(api): Review #2 技術債清理 — I1/S1/S2/S3 全數修正
CD Pipeline / build-and-deploy (push) Successful in 12m13s
I1: error_type 欄位補全
- AnomalyCounter.derive_key_from_incident() 新增
從 signal.labels 提取 reason/error_type,確保四欄位完整
S1: 三處 signature 建構邏輯統一
- auto_repair_service._derive_anomaly_key() → 委託 derive_key_from_incident()
- approval_execution._get_anomaly_key_from_approval() → 同上
- incident_service.resolve_incident() B4 → 同上
- 消除 3 處重複的 signature 建構程式碼
S2: Redis Pipeline 批次查詢
- get_all_disposition_stats() 從 N+1 hgetall 改為 2 次 Pipeline
- Pipeline 1: 批次 hgetall 所有 disposition key
- Pipeline 2: 批次 hget metadata (alert_name)
- 效能從 O(2N) Redis round-trip 降至 O(2)
S3: auto_repair.py get_incident AttributeError 修復
- get_incident() → get_from_working_memory() (pre-existing bug)
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
|
2026-04-07 13:13:42 +08:00 |
|
OG T
|
561bcb638b
|
fix(api): Sprint 4 首席架構師 Review P0 修正 — hash 統一 + 積木化合規
P0-1: anomaly_key hash 推導統一
- B1: 新增 _derive_anomaly_key() 使用 AnomalyCounter.hash_signature()
取代 symptoms.compute_hash()
- B3/B4: namespace 改用 signal.labels.get("namespace", "")
修正 getattr(signal, "namespace", "") 永遠回傳空字串
P0-2: Router 層積木化合規
- C1/C2: 封裝 get_all_disposition_stats() 到 AnomalyCounter
- Router 不再直接存取 counter.redis
- stats.py 移除未使用的 days/stats 參數
P1: get_frequency() 填充 disposition 欄位
- 與 _record_anomaly_impl() 一致,回傳完整處置統計
首席架構師評分: 82/100 → P0 全數修正
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
|
2026-04-07 12:53:12 +08:00 |
|
OG T
|
9253281d46
|
feat(api): Sprint 4 Phase A+B — 告警處置統計資料層+寫入層
Phase A: 資料層
- A1: IncidentFrequencyStats 新增 4 欄位 (human_approved/manual_resolved/cold_start_trust/total_resolution)
- A2: AnomalyCounter.record_disposition() — Redis HINCRBY 原子遞增
- A3: get_disposition_stats() — HGETALL 回傳處置分佈
- AnomalyFrequency dataclass 擴充 + to_dict() 同步
- _record_anomaly_impl() 整合 disposition stats
Phase B: 寫入層觸發點接線
- B1: 自動修復成功 → record_disposition("auto_repair")
- B2: 冷啟動信任成功 → record_disposition("cold_start_trust")
- AutoRepairDecision 新增 is_cold_start flag
- execute_auto_repair() 接收並區分處置類型
- B3: 人工批准執行成功 → record_disposition("human_approved")
- 新增 _get_anomaly_key_from_approval() helper
- B4: 手動處理推斷 → resolve_incident() 排除法判定
- 若 resolved 且無 auto/human/cold_start 紀錄 → manual_resolved
安全設計: 所有 disposition 記錄走 try/except,失敗不阻塞主流程
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
|
2026-04-07 11:54:46 +08:00 |
|
OG T
|
89f0bae3f2
|
feat(safety-net): complete wave 1 atomicity (adr-038, adr-039, debounce, graceful degrade, xclaim)
E2E Health Check / e2e-health (push) Has been cancelled
CD Pipeline / build-and-deploy (push) Has been cancelled
|
2026-03-29 23:55:38 +08:00 |
|
OG T
|
89a2339796
|
feat(api): ADR-038 Circuit Breaker 整合 + Graceful Degradation
sentry_webhook.py:
- 整合 OpenClawGuard (Circuit Breaker + Semaphore)
- 斷路狀態快速失敗,不呼叫 OpenClaw
- 並發控制: 最多 3 個同時 LLM 推理
anomaly_counter.py:
- record_anomaly() Redis 故障 Graceful Degradation
- 失敗時返回預設 AnomalyFrequency (count=0)
- 不中斷主流程
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
|
2026-03-29 15:55:51 +08:00 |
|
OG T
|
d89f0520f9
|
fix(api): 修復 34 個 Ruff lint 錯誤
- 自動修復 import 排序、unused imports
- 手動修復 raise from、isinstance union、unused variable
- scripts/ 暫時保留 (非 CI 阻擋)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
|
2026-03-29 15:27:49 +08:00 |
|