feat(ai): 建立 PixelRAG VLM replay worker

This commit is contained in:
ogt
2026-07-09 23:30:43 +08:00
parent e34c9d777b
commit 55ce12e6b0
12 changed files with 1012 additions and 10 deletions

View File

@@ -116,6 +116,7 @@
- 2026-07-09 起 PixelRAG visual receipts 必須可轉為內部 RAG candidate replay readback`/api/ai-automation/pixelrag-rag-candidate-replay``scripts/ops/report_pixelrag_rag_candidate_replay.py` 只讀 `capture_receipt.json`,拆分 eligible / blocked / invalid receipts輸出 tile/missing/barrier count、candidate text、promotion boundary 與 next machine action`/api/ai-automation/smoke` 需包含 `PixelRAG RAG candidate replay``/api/ai-automation/scheduled-health-summary` 需輸出 `pixelrag_rag_candidate_replay` family。blocked / 403 / captcha / access denied / verify traffic page 只能進 platform probe 或 structured API 策略,不得當作商品資料、不得寫正式價格表、不得直接寫 `ai_insights`
- 2026-07-09 起 PixelRAG visual receipts 進入 OCR/VLM 前必須先輸出 no-write replay contract`/api/ai-automation/pixelrag-ocr-vlm-replay``scripts/ops/report_pixelrag_ocr_vlm_replay.py` 只讀 saved tiles輸出欄位 schema、輸出 schema、confidence/evidence validation rules、Ollama-first route contract 與 next machine action`/api/ai-automation/smoke` 需包含 `PixelRAG OCR/VLM replay contract``/api/ai-automation/scheduled-health-summary` 需輸出 `pixelrag_ocr_vlm_replay` family。此 contract 不執行 OCR/VLM、不呼叫模型、不讀 secret、不連外、不寫 DB、不寫 `ai_insights`、不寫正式價格表blocked / 403 / captcha / access denied / verify traffic page 只能進 platform probe 或 structured API 策略。
- 2026-07-09 起 PixelRAG 可整合/可運用範圍必須有機器可讀 application portfolio`/api/ai-automation/pixelrag-application-portfolio``scripts/ops/report_pixelrag_application_portfolio.py` 需輸出 commerce、RAG、UX、ops、marketing、governance lanes每條 lane 必須包含 priority、status、integrates_with、use_cases、current_capability、next_machine_action、no-write 邊界與 forbidden guardrails。此 portfolio 不抓外站、不呼叫模型、不讀 secret、不寫 DB其用途是把「還可以整合哪些」變成可排程、可驗證、可拒絕違規場景的主線工作項目。
- 2026-07-09 起 PixelRAG ready receipts 必須有 Ollama-first VLM replay worker`/api/ai-automation/pixelrag-vlm-replay-worker``scripts/ops/run_pixelrag_vlm_replay_worker.py` 預設 dry-run`execute=true&write_receipt=true` 才呼叫 approved Ollama VLM route 並寫 artifact receipt`/api/ai-automation/smoke` 需包含 `PixelRAG VLM replay worker``/api/ai-automation/scheduled-health-summary` 需輸出 `pixelrag_vlm_replay_worker` family。此 worker 不讀 secret、不寫 DB、不寫 `ai_insights`、不寫正式價格表blocked / 403 / captcha / access denied receipt 自動跳 platform probe 或 structured APIready receipt 的 VLM 結果仍須 identity matcher replay 與 PromotionGate 才能進候選知識層。
- 2026-07-02 起 `/ai_intelligence` 商品明細與單品作戰詳情的四格價格證據必須可測PChome 價格、MOMO 參考價、差距、可信度需以 `data-evidence` 固定,並以 `aria-label="價格證據"` 對應可掃描區塊;候選待確認或缺資料只能顯示「候選待確認 / 待補」,不得捏造價格或讓使用者打開 raw payload 才知道判斷依據。
- 2026-07-02 起 `/ai_intelligence` 必須是密集 AI 工作台不得退回大段文字說明頁首屏與明細可見內容只保留短狀態、數字、四格證據與下一步按鈕KPI note、benchmark detail、alert 副句、策略說明、decision copy、來源長句與單品 reason list 不得佔用第一層視覺。`tests/test_ai_intelligence_text_density_guardrails.py` 必須鎖住 `data-density-guardrail="compact-ai-workbench"`、短任務文案、detail meta 與 hidden explanatory copy。
- 2026-07-02 起 `/observability/overview` 也必須採密集 AI 觀測工作台:首屏以 `data-density-guardrail="compact-observability-workbench"``AI 觀測 / 風險優先 / 下一步` 與 golden signals 先呈現狀態、數字與操作入口hero lede、signal note、route desc、host meta 與資料來源長句不得佔用第一層視覺。`tests/test_observability_text_density_guardrails.py` 必須鎖住 compact marker 與 hidden explanatory copy。
@@ -878,6 +879,7 @@ POSTGRES_HOST=momo-db
| 2026-07-09 | PixelRAG / external MCP/RAG 必須有 runtime monitoring 與 candidate replay | V10.746 起 `/api/ai-automation/smoke``/api/ai-automation/scheduled-health-summary` 必須輸出 `External MCP/RAG integration readback` / `external_mcp_rag_integration``PixelRAG RAG candidate replay` / `pixelrag_rag_candidate_replay`external MCP/RAG readback 回報 9 個 capability 的 absorbed / unresolved 與 `MCP_ROUTER_ENABLED``RAG_ENABLED` runtime flagsPixelRAG replay 只讀 visual receipt拆分 eligible / blocked / invalid並明確標記 blocked page 不是商品資料。此路徑不讀 secret、不呼叫外部網路、不寫 DB、不寫 `ai_insights`、不寫正式價格表eligible receipt 仍需 OCR/VLM replay、identity matcher replay、PromotionGate 與 embedding signature guard。 |
| 2026-07-09 | PixelRAG OCR/VLM replay contract 必須有 runtime monitoring | V10.747 起 `/api/ai-automation/pixelrag-ocr-vlm-replay``scripts/ops/report_pixelrag_ocr_vlm_replay.py``/api/ai-automation/smoke``/api/ai-automation/scheduled-health-summary` 必須輸出 no-write OCR/VLM replay contract / `pixelrag_ocr_vlm_replay` familyreadback 只讀 saved tiles 與 RAG candidate replay輸出 ready / blocked / invalid contracts、field schema、output schema、validation rules、Ollama-first route contract、blocked page guard 與 next machine action。此階段明確標記 `extraction_execution_performed=false``ocr_execution_performed=false``vlm_execution_performed=false``writes_database=false``writes_ai_insights=false``writes_price_tables=false``network_call=false``secret_read=false``primary_human_gate_count=0`ready receipt 才能進下一段 `run_ollama_first_vlm_replay_worker`blocked receipt 只能進 platform probe 或 structured API 策略。 |
| 2026-07-09 | PixelRAG application portfolio 必須把可整合場景轉成主線工作項目 | V10.748 起 `/api/ai-automation/pixelrag-application-portfolio``scripts/ops/report_pixelrag_application_portfolio.py` 必須輸出 PixelRAG 在 commerce、RAG、UX、ops、marketing、governance 的可整合/可運用 lanes每條 lane 需有 priority、status、integrates_with、use_cases、current_capability、next_machine_action、no-write 邊界與 forbidden guardrails。此 readback 依據 PixelRAG visual-RAG pattern、Google Merchant product data、Google Product structured data 與 Baymard product list UX 轉成內部工作項目;它不抓外站、不呼叫模型、不讀 secret、不寫 DB、不把像素結果當正式價格。 |
| 2026-07-09 | PixelRAG Ollama-first VLM replay worker 必須有 runtime monitoring | V10.749 起 `/api/ai-automation/pixelrag-vlm-replay-worker``scripts/ops/run_pixelrag_vlm_replay_worker.py``/api/ai-automation/smoke``/api/ai-automation/scheduled-health-summary` 必須輸出 controlled VLM replay worker / `pixelrag_vlm_replay_worker` familyreadback 預設 dry-run不呼叫模型、不寫 artifactexecute 模式只讀 saved tiles、呼叫 approved Ollama VLM route、驗證 JSON field confidence/evidence refs並只寫 artifact receipt。此 worker 明確標記 `writes_database=false``writes_ai_insights=false``writes_price_tables=false``secret_read=false``primary_human_gate_count=0`blocked page 不得輸出商品欄位ready VLM 結果仍需 identity matcher replay 與 PromotionGate。 |
| 2026-06-29 | PChome DB apply 授權 lane 必須先通過 no-write guard / decision preflight / decision closeout / issuer gate / signing-decision preflight / signing-decision closeout / signing-issuer guard | V10.725 的 PChome mapping backlog auto-policy 已新增 `/api/ai/pchome-growth/mapping-backlog/auto-policy-db-apply-authorization-lane-guard``/api/ai/pchome-growth/mapping-backlog/auto-policy-db-apply-authorization-decision-preflight``/api/ai/pchome-growth/mapping-backlog/auto-policy-db-apply-authorization-decision-closeout``/api/ai/pchome-growth/mapping-backlog/auto-policy-db-apply-authorization-issuer-gate``/api/ai/pchome-growth/mapping-backlog/auto-policy-db-apply-authorization-signing-decision-preflight``/api/ai/pchome-growth/mapping-backlog/auto-policy-db-apply-authorization-signing-decision-closeout``/api/ai/pchome-growth/mapping-backlog/auto-policy-db-apply-authorization-signing-issuer-guard`;這些 endpoint 只驗證 final exact request package、same-run production truth requirement、secret rejection、rollback boundary、lane entry requirements、decision input requirements、rejection policy、post-apply verifier、future authorization decision package、final nonsecret authorization envelope、signing decision preflight inputs、unsigned signing decision package 與 signable request boundary不讀 secret、不執行 shell/SQL、不寫 DB也不簽發 database apply authorization。 |
| 2026-06-29 | PChome DB apply 授權簽署發行者 lane 必須先產出 final signable request package | V10.725 的 PChome mapping backlog auto-policy 新增 `/api/ai/pchome-growth/mapping-backlog/auto-policy-db-apply-authorization-signing-issuer-closeout`;此 endpoint 只把 signing-issuer guard 的 signable request boundary 收斂成 final signable request package 與 closeout contract確認 fresh production truth、post-apply verifier、migration hash、secret boundary 與 no-side-effect checks不讀 secret、不簽發 authorization、不執行 shell/SQL、不寫 DB也不代表正式 DB apply 已授權。 |
| 2026-06-29 | PChome DB apply 授權簽署執行 lane 必須先通過 operator-held secret boundary preflight | V10.725 的 PChome mapping backlog auto-policy 新增 `/api/ai/pchome-growth/mapping-backlog/auto-policy-db-apply-authorization-signing-execution-preflight`;此 endpoint 只把 final signable request package 轉成 future signing execution preflight package、operator-held secret boundary contract、nonsecret signing inputs、command-shape preview、rollback boundary 與 abort conditions不讀 secret、不接受 plaintext secret、不簽發 authorization、不執行 shell/SQL、不寫 DB也不代表正式 DB apply 已授權。 |

View File

@@ -18,7 +18,8 @@
| Completed | External MCP/RAG capability inventory absorbed into internal governance readback | `/api/ai-automation/external-mcp-rag-integration` and `scripts/ops/report_external_mcp_rag_integration.py` expose 9 capabilities, absorbed/unresolved counts, and runtime flags. |
| Completed | PixelRAG receipts to internal RAG candidate replay | `/api/ai-automation/pixelrag-rag-candidate-replay` and `scripts/ops/report_pixelrag_rag_candidate_replay.py` read receipts, split eligible vs blocked, and require OCR/VLM replay plus PromotionGate before knowledge writes. |
| Completed | PixelRAG application portfolio and integration lanes | `/api/ai-automation/pixelrag-application-portfolio` and `scripts/ops/report_pixelrag_application_portfolio.py` expose commerce, RAG, UX, ops, marketing, and governance uses with priority, status, next machine action, and forbidden guardrails. |
| In progress | MCP/RAG runtime health in AI automation smoke | `/api/ai-automation/smoke` and `/api/ai-automation/scheduled-health-summary` include external MCP/RAG integration, PixelRAG RAG candidate replay, and PixelRAG OCR/VLM replay contract families. |
| Completed | PixelRAG Ollama-first VLM replay worker | `/api/ai-automation/pixelrag-vlm-replay-worker` and `scripts/ops/run_pixelrag_vlm_replay_worker.py` dry-run or execute ready visual receipts against approved Ollama VLM routes, emit evidence-bound artifact receipts, and keep blocked pages out of product data. |
| In progress | MCP/RAG runtime health in AI automation smoke | `/api/ai-automation/smoke` and `/api/ai-automation/scheduled-health-summary` include external MCP/RAG integration, PixelRAG RAG candidate replay, PixelRAG OCR/VLM replay contract, and PixelRAG VLM replay worker families. |
| In progress | Formal production deploy/readback discipline | Every mainline change must update version, push Gitea main/dev, deploy to 188 without touching `momo-db`, and read back `/health` plus new endpoints. |
## P1
@@ -26,7 +27,7 @@
| Status | Work item | Evidence / next machine action |
|---|---|---|
| Completed | OCR/VLM replay contract for visual fields | `/api/ai-automation/pixelrag-ocr-vlm-replay` and `scripts/ops/report_pixelrag_ocr_vlm_replay.py` turn saved tiles into no-write field contracts, output schemas, validation rules, and Ollama-first worker actions. |
| Not started | Ollama-first OCR/VLM extraction worker | Execute the replay contract against approved local multimodal/OCR models and emit confidence/evidence receipts without writing formal price truth. |
| Completed | Ollama-first OCR/VLM extraction worker | `run_pixelrag_vlm_replay_worker.py --execute --write-receipt` executes the replay contract against approved Ollama multimodal routes and emits confidence/evidence artifact receipts without writing formal price truth. |
| Not started | Ollama-first multimodal embedding benchmark | Verify local Qwen3-VL or equivalent visual embedding on GCP-A -> GCP-B -> 111 before any visual vector retrieval. |
| Not started | pgvector-compatible visual evidence metadata | Design metadata-first retrieval without FAISS in production unless ADR approves a different store. |
| Not started | Coupang platform probe / structured API strategy | Treat 403 as platform barrier; prefer public structured data or approved probe, never count blocked pages as product data. |

View File

@@ -34,7 +34,9 @@
`python scripts/ops/report_pixelrag_ocr_vlm_replay.py` 可讀回 ready / blocked / invalid replay contracts、field schema、validation rules 與 Ollama-first 下一步,且目前不執行 OCR/VLM、不寫正式價格。
- PixelRAG 可整合/可運用場景盤點必須確認 `/api/ai-automation/pixelrag-application-portfolio`
`python scripts/ops/report_pixelrag_application_portfolio.py` 可讀回 area、priority、status、use cases、next machine action 與 forbidden guardrails不得只存在聊天結論。
- AI automation smoke 必須包含 external MCP/RAG integration、PixelRAG RAG candidate replay 與 PixelRAG OCR/VLM replay contract family避免 registry 已完成但 runtime flag / receipt replay / VLM contract 未完成時被誤報為全自動閉環。
- PixelRAG VLM replay worker 必須確認 `/api/ai-automation/pixelrag-vlm-replay-worker`
`python scripts/ops/run_pixelrag_vlm_replay_worker.py` 可讀回 dry-run/execute、model_call、artifact receipt、blocked/ready 分流、DB write=0 與 primary human gate=0execute 結果仍只是 candidate evidence。
- AI automation smoke 必須包含 external MCP/RAG integration、PixelRAG RAG candidate replay、PixelRAG OCR/VLM replay contract 與 PixelRAG VLM replay worker family避免 registry 已完成但 runtime flag / receipt replay / VLM worker 未完成時被誤報為全自動閉環。
- AI 自動化 Prometheus 指標變更必須同步檢查 `docker/grafana/provisioning/dashboards/json/ai-automation-overview.json` 是否需要新增 panel 或查詢。
- 188 線上 active monitoring stack 以 `monitoring/prometheus.yml` 為準110 gateway 另有 `/home/wooo/monitoring/prometheus.yml`。若 dashboard 無資料,先確認 Prometheus `momo-app` target 與 `momo-network` 連線;所有 Blackbox HTTP target 必須打 `/health`,不可打 Dashboard 首頁 `/`
- Smoke dashboard 會保存 JSONL 趨勢;若新增檢查項目,要確保 history compact record 仍保持小而可讀。

View File

@@ -184,6 +184,16 @@ python scripts/ops/report_pixelrag_ocr_vlm_replay.py --platform shopee_tw --plat
此 contract 只把 saved tiles、欄位 schema、輸出 schema、confidence/evidence 規則與 Ollama-first worker 下一步打包;目前不執行 OCR/VLM、不呼叫模型、不寫 RAG、不寫 `ai_insights`、不寫正式價格表。ready receipt 進 `run_ollama_first_vlm_replay_worker`blocked / 403 / captcha / access denied receipt 進 platform probe 或 structured API 策略。
Ollama-first VLM replay worker
```bash
python scripts/ops/run_pixelrag_vlm_replay_worker.py
python scripts/ops/run_pixelrag_vlm_replay_worker.py --platform shopee_tw --platform coupang_tw
python scripts/ops/run_pixelrag_vlm_replay_worker.py --platform shopee_tw --execute --write-receipt --limit 1
```
API readback: `/api/ai-automation/pixelrag-vlm-replay-worker?platform=shopee_tw`。預設為 dry-run不呼叫模型、不寫 artifact`execute=true&write_receipt=true` 才呼叫 Ollama VLM 並寫 artifact receipt。即使 execute結果仍只是 candidate evidence不得直接寫 `ai_insights`、正式價格表或競品價格歷史,且 missing confidence/evidence 會留在 replay / probe lane。
Application portfolio
```bash