fix(reboot): add edge maintenance fallback apply gate
All checks were successful
CD Pipeline / workflow-shape (push) Successful in 0s
CD Pipeline / cancel-stale-cd (push) Has been skipped
CD Pipeline / tests (push) Successful in 56s
CD Pipeline / build-and-deploy (push) Successful in 4m16s
CD Pipeline / post-deploy-checks (push) Successful in 1m39s

This commit is contained in:
Your Name
2026-07-03 04:38:49 +08:00
parent 3aca484927
commit 84f6c6d4e3
9 changed files with 293 additions and 7 deletions

View File

@@ -20,7 +20,7 @@
| 優先 | 狀態 | 工作項 | 最新證據 | 下一步 / 完成條件 |
|------|------|--------|----------|-------------------|
| P0-1 | BLOCKED_HOST_WINDOWS | 全主機 reboot auto-detection / auto-trigger / 10 分鐘恢復 SLO | 2026-07-03 03:55 production scorecard`active_blocker_count=8``readiness_percent=67``can_claim_all_services_recovered_within_target=false``observed_host_count=7``missing_host_count=0``unreachable_host_count=1`、111 unreachable、99 uptime unknown。 | 先收斂 99 / Windows99 / VMware 與 111恢復 no-secret management channel 或 console verify stdout讀回 VMX / VM power / host uptime再 rerun host probe + reboot-event detector不得 reboot、不得 VM power change、不得讀 Windows 密碼。 |
| P0-2 | IN_PROGRESS_PUBLIC_UX | Deploy / reboot 期間 public 502 維護頁與外部 fallback | Gitea CD `#4517` marker 切換時再次讀到 API / SLO route `502` 且沒有 maintenance body / fallback headersource 已改成 location-level `proxy_intercept_errors``error_page 502 503 504 =503 /__awoooi-maintenance.html`probe 也改成能把帶 fallback header/body 的 5xx 分類為 maintenance fallback。 | 推送 source hardening 後觀察 CD 與 marker-time public route watch完成條件是 marker-time probe 不再看到 raw 502或看到明確 maintenance fallback header/body。若 source 已部署但 live 仍裸 502下一步是受控 edge config readback / staging vhost / `nginx -t`,不得用 app restart 當測試。 |
| P0-2 | BLOCKED_EDGE_PRIVILEGED_APPLY | Deploy / reboot 期間 public 502 維護頁與外部 fallback | Gitea CD `#4519` 已推 deploy marker `3aca484 -> a94ddd5`,但 marker 後 public probe 仍讀到 `https://awoooi.wooo.work/api/v1/health` raw `502`、fallback header/body 空live 188 `/etc/nginx/sites-enabled/awoooi.wooo.work.conf` 缺 maintenance fallback`/var/www/maintenance/maintenance.html` 缺失,`ollama@188` 無 passwordless sudo。 | 先跑 `scripts/reboot-recovery/public-maintenance-edge-fallback-apply.sh --check` 留 drift receipt具備 privileged channel 後執行 `--apply`,要求 backup、`nginx -t`、reload、public route probe 全綠。不得讀密碼、不得用 app restart 掩蓋 edge fallback drift。 |
| P0-3 | PARTIAL_GREEN_SOURCE_RUNTIME | 所有產品 / 網站版本與資料最新性 | Gitea `main=89d4d6112`Production SLO readback 對齊 deploy marker `89d4d6112 chore(cd): deploy 17ba08c [skip ci]`Stock freshness `status=ok``latest_trading_date=2026-07-02`、blockers `[]`。AWOOOI health HTTP 200 但整體 `degraded`SignOz / local Ollama 仍需列為 runtime degraded evidence。 | 將 source SHA / deploy marker / runtime endpoint / public route watch 固定進 scorecard完成條件是每個 public product 都有 source、deploy、runtime、freshness 四層 readback且 degraded components 有 owner lane。 |
| P0-4 | BLOCKED_WINDOWS99_AUTOSTART | 192.168.0.99 VMware 自動啟動與 VM guest 111 / 188 / 120 / 121 / 112 | Scorecard`windows99_update_no_auto_reboot_ready=true``windows99_vmware_verify_ready=false`、VMX missing alias `111`、powered off aliases `111/112/120/121/188`、WinRM unavailable、SSH BatchMode permission denied、RDP / Hyper-V console channel reachable。 | 只用 no-secret console / management collector 取得 `windows99-vmware-autostart.ps1 -Mode Verify` stdout先以 `validate-windows99-console-verify-artifact.py` 轉成 normalized artifact再重跑 SLO scorecard完成條件是 VMX config ready、guest power ready、99 uptime known、all required host reachable。 |
| P0-5 | PARTIAL_GREEN_BACKUP_MONITORING | Gitea / 主機 / DB / 網站 / 服務 / 套件 / 工具 / log 備份監控告警 | Gitea repo bundle readback readyexpected `12`、rows `12`、missing `0`、failed `0`、sample restore dry-run okbackup core green。這只證明 repo bundle / core backup不等於 Gitea full dump、DB/settings/issues/packages/LFS、所有工具與 log 全量備份監控完成。 | 補齊 backup health textfile / Prometheus / Telegram receipt matrix每個 backup scope 必須有 fresh、failed、age、restore-drill、alert-receipt完成條件是沒有「不知道備份有沒有跑」的盲點。 |