fix(alerts): guard approval actions and wire playbook learning
This commit is contained in:
@@ -3423,3 +3423,52 @@ curl -fsS 'https://awoooi.wooo.work/api/v1/platform/tenants' | jq '{total, tenan
|
||||
# awoooi = legacy_awoooi_default
|
||||
# ewoooc = shadow
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 2026-05-06(台北)— Alert Approval Guard + Playbook Trust 接線
|
||||
|
||||
**觸發**:Telegram 告警仍出現 Gemini/LLM 產生的錯域 `kubectl` 指令,例如把 Sentry Docker container 當成 K8s deployment,或把 `FlywheelExecutionRateMissing` 當 deployment scale;同時 production 24h `approval_records.matched_playbook_id` 為 0,導致 learning service 無法更新 Playbook trust。
|
||||
|
||||
### 已修正
|
||||
|
||||
| 範圍 | 結果 |
|
||||
|------|------|
|
||||
| Approval 前置閘門 | 新增 `alert_approval_guard.py`,AI/rule action 寫入 `ApprovalRecord` 前先檢查 kubectl grammar、namespace 與 K8s resource target |
|
||||
| 錯域動作處理 | `default` / `production` / 不存在 deployment 會降級為 `NO_ACTION - INVALID_TARGET`,避免錯誤命令進入批准與執行 |
|
||||
| Playbook 真 ID | 新增 `playbook_match_resolver.py`,將 YAML `rule_id` / alertname 解析成真正 `PB-...`,不再把 rule id 偽裝成 playbook id |
|
||||
| Alertmanager 入口 | CS1 / CS2 / CS3 建立 approval 時填入 `matched_playbook_id`,auto-execute 也沿用同一個 PB ID |
|
||||
| Telegram 顯示 | 被 guard 擋下的建議動作顯示為明確 `INVALID_TARGET`,不再把幻覺 kubectl 當成可執行建議 |
|
||||
|
||||
### 驗證
|
||||
|
||||
```bash
|
||||
DATABASE_URL=postgresql+asyncpg://awoooi:awoooi_test_2026@localhost:5432/awoooi_test \
|
||||
/Users/ogt/awoooi/apps/api/.venv/bin/python -m pytest \
|
||||
apps/api/tests/test_alert_approval_guard.py \
|
||||
apps/api/tests/test_action_parser_safety.py \
|
||||
apps/api/tests/test_rule_engine_auto_execute.py \
|
||||
apps/api/tests/test_matched_playbook_id_e2e.py \
|
||||
apps/api/tests/test_learning_chain_e2e.py -q
|
||||
# 59 passed
|
||||
|
||||
/Users/ogt/awoooi/apps/api/.venv/bin/python -m ruff check --select E9,F401,F821,F841 \
|
||||
apps/api/src/services/alert_approval_guard.py \
|
||||
apps/api/src/services/playbook_match_resolver.py \
|
||||
apps/api/src/api/v1/webhooks.py \
|
||||
apps/api/tests/test_alert_approval_guard.py
|
||||
# All checks passed
|
||||
```
|
||||
|
||||
線上只讀 resolver spot check:
|
||||
|
||||
```text
|
||||
HostHighCpuLoad -> PB-20260427-C29FE4
|
||||
NodeExporterDown -> PB-20260420-282F79
|
||||
DockerContainerCpuSustainedHigh -> PB-20260505-F4197B
|
||||
```
|
||||
|
||||
### 後續
|
||||
|
||||
- 部署後觀察 24h:`approval_records.matched_playbook_id IS NOT NULL` 必須從 0 開始增加。
|
||||
- 若 guard 擋下大量 LLM 動作,下一步不是放寬 guard,而是讓 PreDecision/MCP 先收 evidence,再產生 domain-correct SSH/K8s action。
|
||||
|
||||
Reference in New Issue
Block a user