fix(runner): flag stale gitea jobs readback
Some checks failed
CD Pipeline / workflow-shape (push) Successful in 0s
CD Pipeline / cancel-stale-cd (push) Has been skipped
CD Pipeline / tests (push) Successful in 28s
CD Pipeline / build-and-deploy (push) Failing after 2m44s
CD Pipeline / post-deploy-checks (push) Has been skipped

This commit is contained in:
Your Name
2026-06-30 19:31:46 +08:00
parent 06742b56e7
commit c4fe100620
4 changed files with 113 additions and 2 deletions

View File

@@ -21,6 +21,15 @@
**邊界**:未 SSH 寫 110未重啟主機未 restart Docker daemon / host Nginx / K3s / DB / Redis / firewall未 restore / prune / DB write未讀 secret / token / raw sessions / SQLite / `.env`,未使用 GitHub / `gh` / GitHub API。
## 2026-06-30 — current P0-4 Gitea Actions jobs readback stale guard
**照主線修正的問題**
- Public Gitea visible run 顯示 `cd.yaml #4041``78418babf``Failure`,但 `actions/runs/4041/jobs` 回來的 jobs `head_sha` 是舊 commit不能拿 jobs conclusion 當作這次 CD 成功證據。
- `ops/runner/read-public-gitea-actions-queue.py` 新增 `cd_run_jobs_head_shas``cd_run_jobs_head_sha_matches_visible``cd_run_jobs_run_id_matches_visible``cd_run_jobs_stale_or_mismatched`live #4041 現在正確分類為 `status=cd_jobs_stale_or_mismatched`
- `test_read_public_gitea_actions_queue.py` 增加 stale jobs payload regression testP0-4 後續只以 visible run / production deploy readback / deploy marker 三層一致作為成功,不再被 jobs API 舊資料誤導。
**邊界**:只讀 public Gitea HTML / APIjob logs endpoint 回 `401 token is required`,未讀 token / secret / raw sessions / SQLite / `.env`,未 workflow_dispatch未使用 GitHub / `gh` / GitHub API。
## 2026-06-30 — current P0 readback after verified Gitea bundle
**目前不能宣稱完成的事實**

View File

@@ -18,7 +18,7 @@
| P0-1 | BLOCKED | 全主機 cold-start / 10 分鐘自動恢復 SLO | 最新 `full-stack-cold-start-check.sh --monitor-read-only --no-color``PASS=67 WARN=4 BLOCKED=5`110 registry external `/v2`、110 SSH read-only check、K3s registry pull refused、AWOOOI internal API probe、SigNoz TLS/public route 仍 blocked。 | 先修第一個 cold-start blocker重跑同一 scorecard 到 `BLOCKED=0`;不可只用 route 200 宣稱恢復。 |
| P0-2 | DONE_THIS_INCIDENT | 使用者可見 502Tsenyang | `www.tsenyang.com` / `tsenyang.com` 由 502 恢復為 200188 `tsenyang-website` container runninglocal `127.0.0.1:3000` 回 200。 | 下次同類 502 先查 release symlink / image / container不先動 Nginx、DNS、DB、主機重啟。 |
| P0-3 | BLOCKED | StockPlatform data freshness | public `/healthz``/api/healthz` 回 200freshness / ingestion 回 `not_configured``postgres_not_ready`。 | 恢復 110 control path 後read-only 查 `/home/wooo/stockplatform-v2` compose / DB schema / migration status禁止 fake freshness、manual DB rows、restore/prune。 |
| P0-4 | BLOCKED | AWOOOI production 版本最新性 | source/Gitea `main=3b65f8762`production runtime readback 仍為 `7890778b83``runtime_build_readback_status=runtime_build_diverges_from_committed_deploy_readback`。 | 補 deploy marker / runtime SHA / endpoint readback 一致;未一致前不可宣稱 AWOOOI 最新。 |
| P0-4 | BLOCKED | AWOOOI production 版本最新性 | source/Gitea `main=78418babf`production runtime readback 仍為 `7890778b83``runtime_build_readback_status=runtime_build_diverges_from_committed_deploy_readback`Public Gitea visible run `cd.yaml #4041` 是 Failurejobs API 與 visible run `head_sha` 不一致,已標 `cd_jobs_stale_or_mismatched` | 補 deploy marker / runtime SHA / endpoint readback 一致;未一致前不可宣稱 AWOOOI 最新。 |
| P0-5 | BLOCKED | 110 control path | `diagnose-110-ssh-publickey-auth.sh`node-exporter / SSH banner 正常;`NODE_LOAD_CLASSIFIER=high_load``NODE_PROCS_BLOCKED=0``wooo` publickey `publickey_offer_timeout``root` publickey `permission_denied``git` / `ollama` `preauth_timeout`。 | 集中查 110 sshd publickey auth / authorized_keys / PAM / account lookup path並把 load / runner pressure 視為同一 blocker 的共因;可在 110 console/local 跑 `repair-110-ssh-publickey-auth-local.sh --check` / `--apply` 修 metadata 權限。恢復 SSH read-only command path 後才能驗證 Stock DB、Gitea dump、110 backup completeness。 |
| P0-6 | BLOCKED_BACKUP_COMPLETENESS | Gitea repo visibility 與完整備份 | Gitea version API 200public repo search 只列 4 個 public repo`stockplatform-v2` public page/API 404但 internal `git ls-remote` 成功188 `/home/ollama/backup/110/gitea` 起初為空。已建立 verified emergency bundle `/home/ollama/backup/110/gitea/git-bundles/20260630-190931`4 個 public/internal repo bundle verify + checksum 成功,`AwoooGo``stockplatform-v2``vibework` 因 private auth fail-closed。 | 188 `gitea_repo_mirror_from_110` subtree metric / alert 已補;下一步仍是恢復 110 SSH command path 後跑正式 `gitea dump`、private repo 非互動備份、repo count 與 restore drill readback。 |
| P0-7 | SOURCE_READY_RUNTIME_BLOCKED | 99 VMware / VM autostart | repo 已有 `windows99-vmware-autostart.ps1`,但 99 SSH/WinRM 尚未可用VM host 111 仍不可達。 | 恢復 99 可控通道或由 console 套用腳本;完成後讀回 111/188/120/121/112 boot evidence。 |