fix(agent99): dispatch actionable alerts with receipts
All checks were successful
CD Pipeline / workflow-shape (push) Successful in 1s
CD Pipeline / cancel-stale-cd (push) Has been skipped
CD Pipeline / tests (push) Successful in 2m27s
CD Pipeline / build-and-deploy (push) Successful in 6m39s
CD Pipeline / post-deploy-checks (push) Successful in 1m55s
All checks were successful
CD Pipeline / workflow-shape (push) Successful in 1s
CD Pipeline / cancel-stale-cd (push) Has been skipped
CD Pipeline / tests (push) Successful in 2m27s
CD Pipeline / build-and-deploy (push) Successful in 6m39s
CD Pipeline / post-deploy-checks (push) Successful in 1m55s
This commit is contained in:
@@ -1,6 +1,6 @@
|
||||
# Agent99 全域 AI 自動化主計畫
|
||||
|
||||
> 建立時間:2026-07-10 15:05 CST;最近更新:2026-07-10 19:40 CST
|
||||
> 建立時間:2026-07-10 15:05 CST;最近更新:2026-07-11 10:12 CST
|
||||
> 狀態:P0 執行中,尚未達成全域 AI 自動化
|
||||
> 機器可讀總帳:`docs/operations/agent99-enterprise-ai-automation-work-items.snapshot.json`
|
||||
> 歷史需求來源:`docs/workplans/2026-07-02-commander-inserted-requirements-priority-ledger.md`
|
||||
@@ -92,7 +92,7 @@
|
||||
| 順序 | ID | 工作 | 狀態 | 驗收 |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| 1 | AG99-P0-001 | 修正 `exitCode=0 => resolved` 假綠,對齊 99 runtime script 與 Gitea source SHA | **完成** | Production 99 outcome self-test 通過;runtime manifest `7/7` matched、drift `0`;operator replay 為 verified resolved,alert replay 缺 source-event resolved 時維持 verifying |
|
||||
| 2 | AG99-P0-002 | 修正 alert taxonomy、wrong-domain routing、bridge-before-grouping 與重複事件 | **進行中,current P0;本機 package/contract 綠,99 runtime 待部署** | `suggestedMode`/kind 優先於 source 字串;5 分鐘先 group;同 fingerprint 只建立一個 active incident |
|
||||
| 2 | AG99-P0-002 | 修正 alert taxonomy、wrong-domain routing、bridge-before-grouping、actionable Telegram card 漏接與重複事件 | **進行中,current P0;source/test 綠,99 runtime 待部署** | Alertmanager、AI card、ACTION REQUIRED、CI/CD failure 共用 dispatcher;`suggestedMode`/kind 優先;5 分鐘先 group;同 fingerprint 只建立一個 active incident;保留 acceptance receipt |
|
||||
| 3 | AG99-P0-003 | 強化 ingress:relay token 必填、Telegram chat+sender allowlist、idempotency/single consumer | 待辦 | 未認證 LAN request 拒絕;producer `auto_repair`/policy 被保留;所有 request 可追 trace |
|
||||
| 4 | AG99-P0-004 | 新增 EvidenceCollect / RepairCandidate / MCP lane | 待辦 | 缺 evidence 的事件會建立可查 candidate,不再錯送 Perf/AwoooRepair;`INC-*` 可從收件追到 candidate |
|
||||
|
||||
@@ -150,6 +150,18 @@
|
||||
- Velero missed weekly restore dry-run 已由 controller 自動 backfill 成功;20:17 Prometheus guard 修復 bind-mount inode drift 後,source/canonical/active/runtime 四方 SHA 一致、145 rules healthy,Backup/Gitea active alerts 為 0。
|
||||
- 以上只關閉 `AG99-P0-012/014` 的 Gitea/backup 子切片;全域 backup domain、Harbor/Registry/CD DR、99 Agent runtime 與 10 分鐘 reboot SLO 仍未完成。
|
||||
|
||||
### 2026-07-11 09:35 Current P0 增量與 live truth
|
||||
|
||||
- 已補上 `ACTION REQUIRED` 與失敗 `AWOOOI CI/CD` 卡片的 Agent99 dispatcher;cold-start 明確走 `host_recovery/Recover`,不再產生不存在的 Kubernetes deployment restart。
|
||||
- Direct Alertmanager 與 Telegram outbound mirror 都保留 sanitized relay receipt;只有 `accepted=true` 才算 dispatched,未接受會釋放 single-flight 並允許重試。
|
||||
- Source event closure 分成 `mode_verifier_can_resolve` 與 `external_source_receipt_required`;CI/CD failure 不會因 Agent99 本機命令成功而假結案。
|
||||
- Rebase 到 Telegram single-owner mainline 後 focused 回歸 `131 passed`,擴大到全部 Telegram 與 Agent99/RepairCandidate 為 `276 passed`;repo static governance checks 綠。99 relay 的 no-page smoke 回覆 HTTP 202、`ok=true`、`inboxTriggered=true`,只證明 transport/inbox 可達,不證明新 runtime 已部署。
|
||||
- Windows 99 live manifest 為 `sourceRevision=e478ee128dfa046a9ddad872d3f60446e4477486e`、`runtimeMatched=true`、`14/14` files matched、mismatch `0`。這證明現行舊 package 內部一致,也明確證明本次 P0 source 尚未部署到 99。
|
||||
- 前一張 CD `#4899` 已 Success,Gitea deploy marker/main 是 `f102616f0`,production build readback 是 `4a4b95e513`;production Agent99 work-item endpoint 仍投影 2026-07-10 舊總帳,因此 current P0 runtime closure 尚未成立。
|
||||
- `#4898` 與 `#4899` rollout 初始階段都曾短暫回裸 502,下一次 health probe 才恢復 HTTP 200,deployment 後續成功。這是 maintenance fallback / zero-raw-502 仍需關閉的 runtime evidence,不得因最終成功而刪除。
|
||||
- Production health 仍為 `degraded`:API/PostgreSQL/Redis/OpenClaw 可用,SignOz down,GCP-A Ollama timeout,GCP-B/local fallback 可用。
|
||||
- Reboot SLO live scorecard 為 `blocked_reboot_auto_recovery_slo_not_ready`、readiness 33%、24 blockers;不得宣稱全主機 10 分鐘恢復完成。
|
||||
|
||||
## 7. 工作項強制欄位與治理
|
||||
|
||||
每個工作項必須先有:`id`、priority/order/status、scope、owner lane、dependencies、acceptance 與 next action。任何工作項升到 `candidate_ready`、`in_progress` 或 `executing` 前,還必須補齊 problem、trigger、evidence sources、AI role、action mode、controlled actions、prohibited actions、rollback、mode-specific verifier、callback、evidence refs 與 last update;缺一不可執行。
|
||||
@@ -165,4 +177,4 @@
|
||||
|
||||
## 8. 立即下一步
|
||||
|
||||
目前唯一 P0 仍是 `AG99-P0-002`。本機 Agent99 package 已完成 routing/dedupe/card/reboot identity 契約與 synthetic replay;下一步是先完成本分支 source/test/Gitea receipt,再將測試過的 Agent99 package 原子部署到 99,重跑 backup、provider、reboot、PostgreSQL、host、K3s replay corpus 並讀回 runtime manifest。99 尚未取得可用的 no-secret authenticated management channel前,狀態只能是 local green/runtime pending;不能把 `exitCode=0` 或本機測試當 production resolved。其後依序做 secure ingress、RepairCandidate、mode verifier/callback,再回到 reboot state machine live rerun。
|
||||
目前唯一 P0 仍是 `AG99-P0-002`。source/test candidate 已完成 Alertmanager 與 Telegram actionable card 共用 dispatcher、acceptance receipt、cold-start/CI-CD taxonomy、resolution policy 與 synthetic relay smoke。下一步依序是:完成 Gitea source receipt;等待單一視窗 CD 協調;原子部署測試過的 Agent99 package 到 99;驗證 runtime manifest;以 do-not-page 重播 ACTION REQUIRED cold-start、CI/CD failure、backup、provider、PostgreSQL、host、K3s corpus;讀回 Agent99 evidence、source-event closure 與 outbound envelope receipt。完成前只能標示 source green/runtime pending。其後再依序做 secure ingress、RepairCandidate、mode verifier/callback,最後回到 reboot state machine live rerun。
|
||||
|
||||
Reference in New Issue
Block a user