Commit Graph

7 Commits

Author SHA1 Message Date
OG T
f045506abd fix(flywheel): P2 Approval 逾期不結案 → KM 學習鏈斷鏈修復
Some checks failed
CD Pipeline / build-and-deploy (push) Failing after 12m11s
問題根因:
  PENDING approval 無人處置超過 48h 後應自動 EXPIRED,
  但 get_pending_approvals() 只在用戶開 UI 時觸發,
  若無人開 UI → Incident 永遠 PENDING → KM 永遠不寫入
  → Phase 6 SLO human_override_rate 低估,EWMA 缺少負向樣本。

修復:
  1. anomaly_counter.py: 新增 "timeout_ignored" disposition 類型,
     與 auto_repair / human_approved / manual_resolved 區分
  2. incident_service.py: resolve_incident() 新增 resolution_type 參數,
     resolution_type="timeout" 時記錄 "timeout_ignored" 而非 "manual_resolved"
  3. jobs/approval_timeout_resolver.py (新): 每小時掃描逾期 PENDING approval,
     批次標記 EXPIRED,對每筆有 incident_id 的記錄呼叫 resolve_incident("timeout")
  4. main.py: startup 掛載 approval_timeout_resolver 排程(interval=3600s)

效果:
  - 告警無人處置 48h → Incident 自動結案 → KM 寫入 → EWMA 取得樣本
  - disposition="timeout_ignored" 讓 SLO 計算正確區分「AI 建議被忽略」
  - 飛輪學習鏈對「無人處置告警」閉環

2026-04-15 ogt + Claude Sonnet 4.6(亞太)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 19:21:21 +08:00
OG T
b7ea362efc fix(api): Review #2 技術債清理 — I1/S1/S2/S3 全數修正
All checks were successful
CD Pipeline / build-and-deploy (push) Successful in 12m13s
I1: error_type 欄位補全
- AnomalyCounter.derive_key_from_incident() 新增
  從 signal.labels 提取 reason/error_type,確保四欄位完整

S1: 三處 signature 建構邏輯統一
- auto_repair_service._derive_anomaly_key() → 委託 derive_key_from_incident()
- approval_execution._get_anomaly_key_from_approval() → 同上
- incident_service.resolve_incident() B4 → 同上
- 消除 3 處重複的 signature 建構程式碼

S2: Redis Pipeline 批次查詢
- get_all_disposition_stats() 從 N+1 hgetall 改為 2 次 Pipeline
- Pipeline 1: 批次 hgetall 所有 disposition key
- Pipeline 2: 批次 hget metadata (alert_name)
- 效能從 O(2N) Redis round-trip 降至 O(2)

S3: auto_repair.py get_incident AttributeError 修復
- get_incident() → get_from_working_memory() (pre-existing bug)

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
2026-04-07 13:13:42 +08:00
OG T
561bcb638b fix(api): Sprint 4 首席架構師 Review P0 修正 — hash 統一 + 積木化合規
P0-1: anomaly_key hash 推導統一
- B1: 新增 _derive_anomaly_key() 使用 AnomalyCounter.hash_signature()
  取代 symptoms.compute_hash()
- B3/B4: namespace 改用 signal.labels.get("namespace", "")
  修正 getattr(signal, "namespace", "") 永遠回傳空字串

P0-2: Router 層積木化合規
- C1/C2: 封裝 get_all_disposition_stats() 到 AnomalyCounter
- Router 不再直接存取 counter.redis
- stats.py 移除未使用的 days/stats 參數

P1: get_frequency() 填充 disposition 欄位
- 與 _record_anomaly_impl() 一致,回傳完整處置統計

首席架構師評分: 82/100 → P0 全數修正

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
2026-04-07 12:53:12 +08:00
OG T
9253281d46 feat(api): Sprint 4 Phase A+B — 告警處置統計資料層+寫入層
Phase A: 資料層
- A1: IncidentFrequencyStats 新增 4 欄位 (human_approved/manual_resolved/cold_start_trust/total_resolution)
- A2: AnomalyCounter.record_disposition() — Redis HINCRBY 原子遞增
- A3: get_disposition_stats() — HGETALL 回傳處置分佈
- AnomalyFrequency dataclass 擴充 + to_dict() 同步
- _record_anomaly_impl() 整合 disposition stats

Phase B: 寫入層觸發點接線
- B1: 自動修復成功 → record_disposition("auto_repair")
- B2: 冷啟動信任成功 → record_disposition("cold_start_trust")
  - AutoRepairDecision 新增 is_cold_start flag
  - execute_auto_repair() 接收並區分處置類型
- B3: 人工批准執行成功 → record_disposition("human_approved")
  - 新增 _get_anomaly_key_from_approval() helper
- B4: 手動處理推斷 → resolve_incident() 排除法判定
  - 若 resolved 且無 auto/human/cold_start 紀錄 → manual_resolved

安全設計: 所有 disposition 記錄走 try/except,失敗不阻塞主流程

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
2026-04-07 11:54:46 +08:00
OG T
89f0bae3f2 feat(safety-net): complete wave 1 atomicity (adr-038, adr-039, debounce, graceful degrade, xclaim)
Some checks failed
E2E Health Check / e2e-health (push) Has been cancelled
CD Pipeline / build-and-deploy (push) Has been cancelled
2026-03-29 23:55:38 +08:00
OG T
89a2339796 feat(api): ADR-038 Circuit Breaker 整合 + Graceful Degradation
sentry_webhook.py:
- 整合 OpenClawGuard (Circuit Breaker + Semaphore)
- 斷路狀態快速失敗,不呼叫 OpenClaw
- 並發控制: 最多 3 個同時 LLM 推理

anomaly_counter.py:
- record_anomaly() Redis 故障 Graceful Degradation
- 失敗時返回預設 AnomalyFrequency (count=0)
- 不中斷主流程

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-29 15:55:51 +08:00
OG T
d89f0520f9 fix(api): 修復 34 個 Ruff lint 錯誤
- 自動修復 import 排序、unused imports
- 手動修復 raise from、isinstance union、unused variable
- scripts/ 暫時保留 (非 CI 阻擋)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-29 15:27:49 +08:00