feat(ai): 接入 PixelRAG RAG replay smoke
Some checks failed
CD Pipeline / deploy (push) Has been cancelled

This commit is contained in:
ogt
2026-07-09 22:40:51 +08:00
parent 26b5d229d4
commit 48c978b455
13 changed files with 886 additions and 18 deletions

View File

@@ -1,8 +1,8 @@
# PChome 業績成長自動化作戰系統 — AI 競價情報模組 Single Source of Truth
> **最後更新**: 2026-07-09 (台北時間)
> **狀態**: 🟢 四 AI Agent 自動化閉環已落地LLM 路由紅線升級為 Ollama-first 三主機級聯PChome 後台業績匯入韌性已補強產品定位正名為「PChome 業績成長自動化作戰系統」外部市場來源正規化層、自動同步、作戰清單與價格參考表優先讀取、CSV 備援預檢、前台操作入口、高可見頁面繁中化守門、比價/作戰 UI 工作台化、AI 密集工作台文字密度守門、跨平台來源治理、商品身份 UI 契約、sitewide visual QA runtime readback、PChome auto-policy authorization guard monitoring、decision preflight machine evidence monitoring、decision closeout monitoring、authorization issuer gate monitoring、signing decision preflight monitoring、signing decision closeout monitoring、signing issuer guard monitoring、signing issuer closeout monitoring、signing execution preflight monitoring、signing execution closeout monitoring、signed receipt preflight monitoring、signed receipt closeout monitoring、signed receipt evidence intake monitoring、detached verification evidence validation monitoring、verifier receipt closeout monitoring、authorization evidence execution preflight monitoring、authorization evidence execution closeout monitoring、controlled apply final preflight monitoring、controlled dry-run package monitoring 與 controlled dry-run receipt closeout monitoring 已建立GCP embedding 熔斷延後處理、110 proxy rescue 與 direct host health skip 已建立
> **適用版本**: V10.745
> **狀態**: 🟢 四 AI Agent 自動化閉環已落地LLM 路由紅線升級為 Ollama-first 三主機級聯PChome 後台業績匯入韌性已補強產品定位正名為「PChome 業績成長自動化作戰系統」外部市場來源正規化層、自動同步、作戰清單與價格參考表優先讀取、CSV 備援預檢、前台操作入口、高可見頁面繁中化守門、比價/作戰 UI 工作台化、AI 密集工作台文字密度守門、跨平台來源治理、商品身份 UI 契約、sitewide visual QA runtime readback、external MCP/RAG integration monitoring、PixelRAG RAG candidate replay monitoring、PChome auto-policy authorization guard monitoring、decision preflight machine evidence monitoring、decision closeout monitoring、authorization issuer gate monitoring、signing decision preflight monitoring、signing decision closeout monitoring、signing issuer guard monitoring、signing issuer closeout monitoring、signing execution preflight monitoring、signing execution closeout monitoring、signed receipt preflight monitoring、signed receipt closeout monitoring、signed receipt evidence intake monitoring、detached verification evidence validation monitoring、verifier receipt closeout monitoring、authorization evidence execution preflight monitoring、authorization evidence execution closeout monitoring、controlled apply final preflight monitoring、controlled dry-run package monitoring 與 controlled dry-run receipt closeout monitoring 已建立GCP embedding 熔斷延後處理、110 proxy rescue 與 direct host health skip 已建立
> **適用版本**: V10.746
---
@@ -112,6 +112,8 @@
- 2026-07-03 起整站 UI/UX 專業化必須由 Sitewide UI/UX Agent 管理:`/api/ai-automation/sitewide-ui-ux-agent` 會掃描真正頁面模板、排除 partial/components/base layouts輸出高優先頁、compact guardrail、raw engineering issue 與 next machine action`/api/ai-automation/sitewide-ui-ux-repair-package` 會產生 no-write controlled repair package。當前 Agent 判定為 55 個頁面模板、31 個高優先模板、29 個已具 compact guardrail、0 個待專業化、0 個 raw engineering leakSitewide Agent 狀態為 `ok`repair package 為 `no_op`。四批 controlled UI/UX repair 已完成 AI observability、AI 建議、PPT/CI/CD/業績/比價/成長/供貨風險高優先 surface下一個機器動作為 `keep_sitewide_ui_ux_agent_monitoring`,後續只接受 Agent 新增 issue 或更高階 visual/mobile QA 的證據驅動修復。
- 2026-07-03 起 Sitewide UI/UX Agent 歸零狀態必須納入 AI automation runtime monitoring`/api/ai-automation/smoke` 需包含 `Sitewide UI/UX Agent readback` 檢查,要求 `issue_surface_count=0``repair_package_status=no_op``repair_action_count=0``/api/ai-automation/scheduled-health-summary` 需輸出 `sitewide_ui_ux_agent` family`/metrics` 需以 `momo_ai_automation_scheduled_health_family_status{family="sitewide_ui_ux_agent",...}` 暴露 scheduled family。此監控不得寫 DB、不得送通知、不得要求人工審核。
- 2026-07-03 起整站 desktop/tablet/mobile visual QA 必須進入 AI automation runtime monitoring`scripts/check_responsive_overflow.js --artifact-output data/ai_automation/sitewide_visual_qa_latest.json` 會輸出機器可讀 artifact`/api/ai-automation/sitewide-visual-qa-readback` 只讀該 artifact 並回報 route_count、viewport_count、failed_count、overflow_issue_count、stale 與 next_machine_action`/api/ai-automation/smoke` 需包含 `Sitewide visual QA readback``/api/ai-automation/scheduled-health-summary` 需輸出 `sitewide_visual_qa` family`/metrics` 需暴露 `momo_ai_automation_scheduled_health_family_status{family="sitewide_visual_qa",...}`。readback 不啟動 browser、不寫 DB、不送通知artifact 生成才需要 Playwright/瀏覽器。
- 2026-07-09 起 external MCP/RAG integration 必須進入 AI automation runtime monitoring`/api/ai-automation/external-mcp-rag-integration``scripts/ops/report_external_mcp_rag_integration.py` 需輸出 9 個 capability 的 absorbed / unresolved 狀態、內部治理落點與 runtime flags`/api/ai-automation/smoke` 需包含 `External MCP/RAG integration readback``/api/ai-automation/scheduled-health-summary` 需輸出 `external_mcp_rag_integration` family。`MCP_ROUTER_ENABLED=false``RAG_ENABLED=false` 必須被視為 runtime gap不得因 registry 已完成就宣稱 MCP/RAG runtime 已全面啟用。
- 2026-07-09 起 PixelRAG visual receipts 必須可轉為內部 RAG candidate replay readback`/api/ai-automation/pixelrag-rag-candidate-replay``scripts/ops/report_pixelrag_rag_candidate_replay.py` 只讀 `capture_receipt.json`,拆分 eligible / blocked / invalid receipts輸出 tile/missing/barrier count、candidate text、promotion boundary 與 next machine action`/api/ai-automation/smoke` 需包含 `PixelRAG RAG candidate replay``/api/ai-automation/scheduled-health-summary` 需輸出 `pixelrag_rag_candidate_replay` family。blocked / 403 / captcha / access denied / verify traffic page 只能進 platform probe 或 structured API 策略,不得當作商品資料、不得寫正式價格表、不得直接寫 `ai_insights`
- 2026-07-02 起 `/ai_intelligence` 商品明細與單品作戰詳情的四格價格證據必須可測PChome 價格、MOMO 參考價、差距、可信度需以 `data-evidence` 固定,並以 `aria-label="價格證據"` 對應可掃描區塊;候選待確認或缺資料只能顯示「候選待確認 / 待補」,不得捏造價格或讓使用者打開 raw payload 才知道判斷依據。
- 2026-07-02 起 `/ai_intelligence` 必須是密集 AI 工作台不得退回大段文字說明頁首屏與明細可見內容只保留短狀態、數字、四格證據與下一步按鈕KPI note、benchmark detail、alert 副句、策略說明、decision copy、來源長句與單品 reason list 不得佔用第一層視覺。`tests/test_ai_intelligence_text_density_guardrails.py` 必須鎖住 `data-density-guardrail="compact-ai-workbench"`、短任務文案、detail meta 與 hidden explanatory copy。
- 2026-07-02 起 `/observability/overview` 也必須採密集 AI 觀測工作台:首屏以 `data-density-guardrail="compact-observability-workbench"``AI 觀測 / 風險優先 / 下一步` 與 golden signals 先呈現狀態、數字與操作入口hero lede、signal note、route desc、host meta 與資料來源長句不得佔用第一層視覺。`tests/test_observability_text_density_guardrails.py` 必須鎖住 compact marker 與 hidden explanatory copy。
@@ -871,6 +873,7 @@ POSTGRES_HOST=momo-db
| 2026-07-03 | PChome auto-policy controlled apply final preflight 必須有 runtime monitoring | V10.743 起 `/api/ai-automation/smoke``/api/ai-automation/scheduled-health-summary``/metrics` 必須輸出 `PChome auto-policy controlled apply final preflight` / `pchome_auto_policy_controlled_apply_final_preflight`;此 runtime check 自動驗證 authorization evidence execution closeout carry-forward、database apply authorization final verifier gate、controlled apply final preflight package、rollback binding、post-apply verifier binding、12 個 final preflight checks、12 個 preflight fields、10 個 acceptance gates、dry-run-only、check-mode-only、same-run production truth requirement 與 future controlled dry-run package readiness但明確標記 `reads_secret_count=0``executes_script_count=0``executes_sql_count=0``writes_database_count=0``signs_database_apply_authorization_count=0``primary_human_gate_count=0``rollback_execution_authorized=false``rollback_executes_sql=false``rollback_writes_database=false``post_apply_verifier_execution_authorized_in_preview=false``executes_database_apply=false``ready_for_database_apply_now=false``database_apply_authorized=false``issues_database_apply_authorization=false``signs_database_apply_authorization=false`,不讀 secret、不執行 authorization evidence、不執行 database apply、不執行 endpoint/SQL、不寫 DB、不簽發 apply authorization只允許進入下一段 controlled dry-run package lane。 |
| 2026-07-03 | PChome auto-policy controlled dry-run package 必須有 runtime monitoring | V10.744 起 `/api/ai-automation/smoke``/api/ai-automation/scheduled-health-summary``/metrics` 必須輸出 `PChome auto-policy controlled dry-run package` / `pchome_auto_policy_controlled_dry_run_package`;此 runtime check 自動驗證 controlled apply final preflight carry-forward、rollback binding、post-apply verifier binding、12 個 dry-run package checks、12 個 package fields、10 個 acceptance gates、receipt preview、result parser、non-executable command shape、same-run production truth 與 future dry-run receipt closeout readiness但明確標記 `receipt_execution_performed=false``receipt_stdout_included=false``receipt_stderr_included=false``command_shape_execution_allowed=false``database_apply_authorized=false``reads_secret_count=0``executes_script_count=0``executes_sql_count=0``writes_database_count=0``signs_database_apply_authorization_count=0``primary_human_gate_count=0``executes_authorization_evidence=false``executes_database_apply=false``ready_for_database_apply_now=false``issues_database_apply_authorization=false``signs_database_apply_authorization=false`,不讀 secret、不執行 dry-run、不捕捉 stdout/stderr、不執行 authorization evidence、不執行 database apply、不執行 endpoint/SQL、不寫 DB、不簽發 apply authorization只允許進入下一段 dry-run receipt closeout lane。 |
| 2026-07-09 | PChome auto-policy controlled dry-run receipt closeout 必須有 runtime monitoring | V10.745 起 `/api/ai-automation/smoke``/api/ai-automation/scheduled-health-summary``/metrics` 必須輸出 `PChome auto-policy controlled dry-run receipt closeout` / `pchome_auto_policy_controlled_dry_run_receipt_closeout`;此 runtime check 自動驗證 controlled dry-run package carry-forward、dry-run result parser、receipt validation report、command-shape hash、rollback binding、post-apply verifier binding、12 個 receipt closeout checks、12 個 closeout fields、10 個 acceptance gates、same-run production truth 與 future runner readiness但明確標記 `receipt_validation_status=preview_validated_not_executed``validation_execution_performed=false``validation_stdout_included=false``validation_stderr_included=false``validation_database_apply_authorized=false``reads_secret_count=0``executes_script_count=0``executes_sql_count=0``writes_database_count=0``signs_database_apply_authorization_count=0``primary_human_gate_count=0``executes_authorization_evidence=false``executes_database_apply=false``ready_for_database_apply_now=false``database_apply_authorized=false``issues_database_apply_authorization=false``signs_database_apply_authorization=false`,不讀 secret、不執行 dry-run、不捕捉 stdout/stderr、不執行 authorization evidence、不執行 database apply、不執行 endpoint/SQL、不寫 DB、不簽發 apply authorization只允許進入下一段 controlled dry-run runner readiness lane。 |
| 2026-07-09 | PixelRAG / external MCP/RAG 必須有 runtime monitoring 與 candidate replay | V10.746 起 `/api/ai-automation/smoke``/api/ai-automation/scheduled-health-summary` 必須輸出 `External MCP/RAG integration readback` / `external_mcp_rag_integration``PixelRAG RAG candidate replay` / `pixelrag_rag_candidate_replay`external MCP/RAG readback 回報 9 個 capability 的 absorbed / unresolved 與 `MCP_ROUTER_ENABLED``RAG_ENABLED` runtime flagsPixelRAG replay 只讀 visual receipt拆分 eligible / blocked / invalid並明確標記 blocked page 不是商品資料。此路徑不讀 secret、不呼叫外部網路、不寫 DB、不寫 `ai_insights`、不寫正式價格表eligible receipt 仍需 OCR/VLM replay、identity matcher replay、PromotionGate 與 embedding signature guard。 |
| 2026-06-29 | PChome DB apply 授權 lane 必須先通過 no-write guard / decision preflight / decision closeout / issuer gate / signing-decision preflight / signing-decision closeout / signing-issuer guard | V10.725 的 PChome mapping backlog auto-policy 已新增 `/api/ai/pchome-growth/mapping-backlog/auto-policy-db-apply-authorization-lane-guard``/api/ai/pchome-growth/mapping-backlog/auto-policy-db-apply-authorization-decision-preflight``/api/ai/pchome-growth/mapping-backlog/auto-policy-db-apply-authorization-decision-closeout``/api/ai/pchome-growth/mapping-backlog/auto-policy-db-apply-authorization-issuer-gate``/api/ai/pchome-growth/mapping-backlog/auto-policy-db-apply-authorization-signing-decision-preflight``/api/ai/pchome-growth/mapping-backlog/auto-policy-db-apply-authorization-signing-decision-closeout``/api/ai/pchome-growth/mapping-backlog/auto-policy-db-apply-authorization-signing-issuer-guard`;這些 endpoint 只驗證 final exact request package、same-run production truth requirement、secret rejection、rollback boundary、lane entry requirements、decision input requirements、rejection policy、post-apply verifier、future authorization decision package、final nonsecret authorization envelope、signing decision preflight inputs、unsigned signing decision package 與 signable request boundary不讀 secret、不執行 shell/SQL、不寫 DB也不簽發 database apply authorization。 |
| 2026-06-29 | PChome DB apply 授權簽署發行者 lane 必須先產出 final signable request package | V10.725 的 PChome mapping backlog auto-policy 新增 `/api/ai/pchome-growth/mapping-backlog/auto-policy-db-apply-authorization-signing-issuer-closeout`;此 endpoint 只把 signing-issuer guard 的 signable request boundary 收斂成 final signable request package 與 closeout contract確認 fresh production truth、post-apply verifier、migration hash、secret boundary 與 no-side-effect checks不讀 secret、不簽發 authorization、不執行 shell/SQL、不寫 DB也不代表正式 DB apply 已授權。 |
| 2026-06-29 | PChome DB apply 授權簽署執行 lane 必須先通過 operator-held secret boundary preflight | V10.725 的 PChome mapping backlog auto-policy 新增 `/api/ai/pchome-growth/mapping-backlog/auto-policy-db-apply-authorization-signing-execution-preflight`;此 endpoint 只把 final signable request package 轉成 future signing execution preflight package、operator-held secret boundary contract、nonsecret signing inputs、command-shape preview、rollback boundary 與 abort conditions不讀 secret、不接受 plaintext secret、不簽發 authorization、不執行 shell/SQL、不寫 DB也不代表正式 DB apply 已授權。 |

View File

@@ -0,0 +1,38 @@
# AI Automation Mainline Work Items
> Updated: 2026-07-09
> Scope: EwoooC / MOMO Pro System AI automation, PixelRAG, MCP/RAG, production UX.
## Source Of Truth
- Production readback stays authoritative after deploy: `/health`, container health, AI automation API readback, and smoke.
- Gitea is the only source-control truth. GitHub remains frozen and must not be used.
- Runtime truth must stay separate from source truth: a green test or registry does not mean MCP/RAG runtime is enabled.
- PixelRAG visual receipts are evidence, not formal price truth. They cannot write `competitor_prices`, `competitor_price_history`, or `ai_insights` without replay and promotion gates.
## P0
| Status | Work item | Evidence / next machine action |
|---|---|---|
| Completed | Multi-commerce PixelRAG visual evidence lane for momo, pchome, shopee_tw, coupang_tw, yahoo_shopping_tw, etmall_tw, friday_tw, rakuten_tw | API and CLI can emit expansion plans and marketplace manifests; PChome parser failures can queue manifests. |
| Completed | External MCP/RAG capability inventory absorbed into internal governance readback | `/api/ai-automation/external-mcp-rag-integration` and `scripts/ops/report_external_mcp_rag_integration.py` expose 9 capabilities, absorbed/unresolved counts, and runtime flags. |
| In progress | PixelRAG receipts to internal RAG candidate replay | `/api/ai-automation/pixelrag-rag-candidate-replay` and `scripts/ops/report_pixelrag_rag_candidate_replay.py` read receipts, split eligible vs blocked, and require OCR/VLM replay plus PromotionGate before knowledge writes. |
| In progress | MCP/RAG runtime health in AI automation smoke | `/api/ai-automation/smoke` and `/api/ai-automation/scheduled-health-summary` include external MCP/RAG integration and PixelRAG RAG candidate replay families. |
| In progress | Formal production deploy/readback discipline | Every mainline change must update version, push Gitea main/dev, deploy to 188 without touching `momo-db`, and read back `/health` plus new endpoints. |
## P1
| Status | Work item | Evidence / next machine action |
|---|---|---|
| Not started | OCR/VLM replay contract for visual fields | Build a no-write replay that extracts title, price, promo badge, seller, rating, delivery badge from saved tiles and emits confidence/evidence only. |
| Not started | Ollama-first multimodal embedding benchmark | Verify local Qwen3-VL or equivalent visual embedding on GCP-A -> GCP-B -> 111 before any visual vector retrieval. |
| Not started | pgvector-compatible visual evidence metadata | Design metadata-first retrieval without FAISS in production unless ADR approves a different store. |
| Not started | Coupang platform probe / structured API strategy | Treat 403 as platform barrier; prefer public structured data or approved probe, never count blocked pages as product data. |
| Not started | Marketplace source contracts beyond MOMO/PChome | Promote stable Shopee/Coupang/Yahoo/ETMall/friday/Rakuten receipts into source contracts only after provenance, rate-limit, public boundary, and replay evidence. |
| In progress | Professional product website UI/UX text reduction | Continue compact AI surfaces, visual QA, and user-facing copy guardrails; avoid engineering logs or work-session prose in the UI. |
## Always Enforced
- Manual review is exception-only for low-risk, machine-verifiable controlled apply.
- Break-glass still applies to secrets, destructive DB actions, force pushes, production provider switchovers, raw runtime volumes, and unverified core model replacement.
- AI automation smoke must expose health honestly: `MCP_ROUTER_ENABLED=false` or `RAG_ENABLED=false` is a runtime gap, not a source-code failure.

View File

@@ -28,6 +28,9 @@
- AI 自動化閉環變更必須確認 `/api/ai-automation/smoke``/ai_automation_smoke` 仍能反映新狀態。
- 外部 MCP / RAG 能力導入內部治理時,必須確認 `/api/ai-automation/external-mcp-rag-integration`
`python scripts/ops/report_external_mcp_rag_integration.py` 可讀回每個能力的內部落點、狀態、資料邊界與下一個機器動作。
- PixelRAG 視覺 receipt 導入內部 RAG 時,必須確認 `/api/ai-automation/pixelrag-rag-candidate-replay`
`python scripts/ops/report_pixelrag_rag_candidate_replay.py` 可讀回 eligible / blocked / invalid receipts且 blocked page 不得被視為商品資料。
- AI automation smoke 必須包含 external MCP/RAG integration 與 PixelRAG RAG candidate replay family避免 registry 已完成但 runtime flag / receipt replay 未完成時被誤報為全自動閉環。
- AI 自動化 Prometheus 指標變更必須同步檢查 `docker/grafana/provisioning/dashboards/json/ai-automation-overview.json` 是否需要新增 panel 或查詢。
- 188 線上 active monitoring stack 以 `monitoring/prometheus.yml` 為準110 gateway 另有 `/home/wooo/monitoring/prometheus.yml`。若 dashboard 無資料,先確認 Prometheus `momo-app` target 與 `momo-network` 連線;所有 Blackbox HTTP target 必須打 `/health`,不可打 Dashboard 首頁 `/`
- Smoke dashboard 會保存 JSONL 趨勢;若新增檢查項目,要確保 history compact record 仍保持小而可讀。

View File

@@ -122,6 +122,7 @@ python scripts/ops/report_pixelrag_crawler_integration.py \
/api/ai-automation/pixelrag-crawler-integration?platform=coupang_tw&marketplace_keyword=iphone
/api/ai-automation/pixelrag-crawler-integration?platform=pchome&manifest_url=https://24h.pchome.com.tw/prod/TEST-000000001&crawler=PChomeCrawler.search_products&trigger_reason=parser_empty
/api/ai-automation/pixelrag-visual-evidence-readback?platform=pchome&manifest_id=4a93e95e5afb414bc8c3
/api/ai-automation/pixelrag-rag-candidate-replay?platform=shopee_tw&platform=coupang_tw
```
視覺證據 capture worker
@@ -161,6 +162,15 @@ runtime_artifacts/pixelrag_visual_evidence/<platform>/<manifest_id>/capture_rece
API readback 只讀 receipt 和檔案存在狀態,不重新抓外站、不寫資料庫。
正式 app 容器內的 readback 預設讀 `/app/data/ai_automation/pixelrag_visual_evidence`host worker 若直接在 188 執行,預設仍可用 `runtime_artifacts/pixelrag_visual_evidence`,需要給 API 回讀時再同步到容器掛載的 `data/ai_automation`
RAG candidate replay
```bash
python scripts/ops/report_pixelrag_rag_candidate_replay.py
python scripts/ops/report_pixelrag_rag_candidate_replay.py --platform shopee_tw --platform coupang_tw
```
此 replay 只把 receipt 轉成內部 RAG candidate previeweligible receipt 仍需 OCR/VLM replay、identity matcher replay、PromotionGate 與 embedding signature guardblocked / 403 / captcha / access denied receipt 只能進 platform probe 或 structured API 策略,不得當成商品資料。
安全邊界:
- read-only不登入、不下單、不加入購物車、不寫第三方狀態。