fix(reboot): preserve edge maintenance blocker in api overlay
All checks were successful
CD Pipeline / workflow-shape (push) Successful in 0s
CD Pipeline / cancel-stale-cd (push) Has been skipped
CD Pipeline / tests (push) Successful in 1m7s
CD Pipeline / build-and-deploy (push) Successful in 4m34s
CD Pipeline / post-deploy-checks (push) Successful in 1m45s
All checks were successful
CD Pipeline / workflow-shape (push) Successful in 0s
CD Pipeline / cancel-stale-cd (push) Has been skipped
CD Pipeline / tests (push) Successful in 1m7s
CD Pipeline / build-and-deploy (push) Successful in 4m34s
CD Pipeline / post-deploy-checks (push) Successful in 1m45s
This commit is contained in:
@@ -20,7 +20,7 @@
|
||||
| 優先 | 狀態 | 工作項 | 最新證據 | 下一步 / 完成條件 |
|
||||
|------|------|--------|----------|-------------------|
|
||||
| P0-1 | BLOCKED_HOST_WINDOWS | 全主機 reboot auto-detection / auto-trigger / 10 分鐘恢復 SLO | 2026-07-03 10:44 production scorecard:`status=blocked_reboot_auto_recovery_slo_not_ready`、`active_blocker_count=10`、`readiness_percent=60`、active blockers 仍含 reboot event / host unreachable / 99+111+Windows99 VMX / guest power,且新增 `windows99_vmware_autostart_config_not_ready`、`windows99_update_no_auto_reboot_policy_not_ready`。no-write artifact `/tmp/awoooi-reboot-continue-20260703-085205`:111 socket probe 仍 timeout / unreachable;99 RDP / VMConnect reachable 但 uptime unknown;188 socket probe reachable,前次 host probe 仍為 `systemd_state=degraded` / `startup_active=failed`。 | 先收斂 99 / Windows99 / VMware 與 111:恢復 no-secret management channel 或取得 validator 可接受的完整 console Verify stdout,讀回 VMX / VM power / host uptime,再 rerun host probe + reboot-event detector;不得 reboot、不得 VM power change、不得讀 Windows 密碼,不得用 RDP clipboard 片段當完成證據。 |
|
||||
| P0-2 | RUNTIME_EDGE_DRIFT_ACTIVE_PROD_API_READBACK_PENDING | Deploy / reboot 期間 public 502 維護頁與外部 fallback | 2026-07-03 15:13 110 SLO controlled install 已套用新版 exporter,並修正 installer payload 缺 `ops/nginx` 導致的 `local_nginx_conf_missing` 誤判。最新 artifact `/home/wooo/reboot-recovery/reboot-auto-recovery-slo-20260703-151219` 正確讀到 188 live edge:`REMOTE_CONF_READABLE=1`、`LIVE_CONF_HAS_INTERCEPT=0`、`LIVE_CONF_HAS_ERROR_PAGE_503=0`、`LIVE_CONF_HAS_FALLBACK_HEADER=0`、`LIVE_CONF_HAS_INTERNAL_LOCATION=0`、`LIVE_CONF_HAS_API_VIP=1`、`LIVE_CONF_HAS_WEB_VIP=1`、`MAINTENANCE_FILE_PRESENT=0`、`SUDO_N_READY=0`、`EDGE_FALLBACK_READY=0`。110 textfile 已輸出 `awoooi_public_maintenance_edge_fallback_ready{scope="public_routes"} 0`、`awoooi_reboot_auto_recovery_slo_blocker_count=8` 與 active blocker `edge_public_maintenance_live_config_drift`。 | 等最新 CD deploy marker readback matched 後,production `/api/v1/agents/reboot-auto-recovery-slo-scorecard` 必須上卷 edge drift;具備 privileged channel 後才可受控 `public-maintenance-edge-fallback-apply.sh --apply`,要求 backup、`nginx -t`、reload、public header verifier 全綠;或完成 L1 external cloud/CDN fallback。不得讀密碼、不得用 app / Docker / host restart 掩蓋 edge fallback drift。 |
|
||||
| P0-2 | API_OVERLAY_FIX_PENDING_DEPLOY_READBACK | Deploy / reboot 期間 public 502 維護頁與外部 fallback | 2026-07-03 15:35 source fix 已把 production API overlay 的 public maintenance blockers 拆成 route-level 與 edge-level:route probe 200 / ready 只能清 route blockers,不能清 `edge_public_maintenance_live_config_drift`;只有 `edge_fallback_runtime_readback_present=true` 且 `edge_fallback_ready=true` 才能清 edge drift。110 SLO artifact `/home/wooo/reboot-recovery/reboot-auto-recovery-slo-20260703-151219` 與 Prometheus/textfile 已證明 edge 未 ready:`awoooi_public_maintenance_edge_fallback_ready{scope="public_routes"} 0`、`awoooi_reboot_auto_recovery_slo_blocker_count=8`、active blocker `edge_public_maintenance_live_config_drift`。本地 regression:route ready + edge metric blocker 時 API active list 必須保留 edge,22 個 scorecard API 測試通過。 | commit / push 後等待 Gitea CD deploy marker matched;production `/api/v1/agents/reboot-auto-recovery-slo-scorecard` 必須讀回 `active_blockers` 含 `edge_public_maintenance_live_config_drift` 且 `runtime_metric_active_blocker_count=8`。後續才進 privileged edge apply 或 L1 cloud/CDN fallback decision;不得讀密碼、不得用 app / Docker / host restart 掩蓋 edge fallback drift。 |
|
||||
| P0-3 | PARTIAL_GREEN_DEPLOYED_SOURCE | 所有產品 / 網站版本與資料最新性 | Gitea `main=bc8d3af3f chore(cd): deploy dacdd90 [skip ci]`;CD `#4558` log 顯示 API/Web build/push wrapper 全部 `rc=0`、deploy marker `dacdd90`、production deploy readback matched。Production SLO route 已讀回最新 API 行為:`runtime_metric_runtime_readback_added_blockers=["windows99_vmware_autostart_config_not_ready","windows99_update_no_auto_reboot_policy_not_ready"]`、`windows99_update_no_auto_reboot_ready=false`。Stock freshness 仍為 `status=ok`、`latest_trading_date=2026-07-02`、blockers `[]`。 | 版本層已補到 AWOOOI production;完成整體 P0 仍需收斂 Windows99 VMX / service / policy / power 與 host reboot-event blockers。後續每個 public product 仍要維持 source SHA、deploy marker、runtime endpoint、public route watch、freshness 四層一致。 |
|
||||
| P0-4 | BLOCKED_WINDOWS99_AUTOSTART | 192.168.0.99 VMware 自動啟動與 VM guest 111 / 188 / 120 / 121 / 112 | 10:44 production readback 已把 nested Windows99 evidence 上卷到 active blockers:`windows99_vmware_vmx_missing`、`windows99_vmware_guest_power_not_ready`、`windows99_vmware_autostart_config_not_ready`、`windows99_update_no_auto_reboot_policy_not_ready`;`runtime_metric_runtime_readback_added_blockers` 含 config / policy 兩項,`windows99_update_no_auto_reboot_ready=false`。09:53 no-secret management probe:99 reachable、RDP / Hyper-V VMConnect reachable,WinRM timeout,Mac vantage collector 仍為 `ssh_batchmode_auth_ready=0` / `blocked_ssh_publickey_auth_missing`;completion 仍取決於 VMX / service / policy / power 全綠。 | 先修 111 VMX source 與 VMware service / policy readback,使 check-mode package 能解釋 powered-off aliases;不得從 scorecard 直接重啟 Windows service、改 registry、啟動 / 關閉 VM 或 host reboot。完成條件是 VMX config ready、guest power ready、99 uptime known、all required host reachable,且 active blocker matrix 顯示 service / policy / VMX / power 分開收斂。 |
|
||||
| P0-5 | RUNTIME_READY_BACKUP_RECEIPT_GAP | Gitea / 主機 / DB / 網站 / 服務 / 套件 / 工具 / log 備份監控告警 | Gitea repo bundle readback ready:expected `12`、rows `12`、missing `0`、failed `0`、sample restore dry-run ok;backup core green。2026-07-03 source / runtime 已部署 `awoooi_backup_alert_receipt_*` 指標與 Prometheus rules;110 exporter 讀回 88 個 stage requirement、188 讀回 12 個 stage requirement,`BackupAlertReceiptMetricMissing*` inactive,`BackupAlertReceiptStageMissing` 已修成每 `host / receipt_channel` 聚合 pending:110 一條、188 一條。 | 補 `/backup/alert-receipts/*.last_success` 脫敏 marker;下一層仍要補 Gitea full dump、DB/settings/issues/packages/LFS、所有工具與 log 全量備份監控。 |
|
||||
|
||||
Reference in New Issue
Block a user