feat(ai): 建立 PixelRAG VLM replay worker
This commit is contained in:
@@ -116,6 +116,7 @@
|
||||
- 2026-07-09 起 PixelRAG visual receipts 必須可轉為內部 RAG candidate replay readback:`/api/ai-automation/pixelrag-rag-candidate-replay` 與 `scripts/ops/report_pixelrag_rag_candidate_replay.py` 只讀 `capture_receipt.json`,拆分 eligible / blocked / invalid receipts,輸出 tile/missing/barrier count、candidate text、promotion boundary 與 next machine action;`/api/ai-automation/smoke` 需包含 `PixelRAG RAG candidate replay`,`/api/ai-automation/scheduled-health-summary` 需輸出 `pixelrag_rag_candidate_replay` family。blocked / 403 / captcha / access denied / verify traffic page 只能進 platform probe 或 structured API 策略,不得當作商品資料、不得寫正式價格表、不得直接寫 `ai_insights`。
|
||||
- 2026-07-09 起 PixelRAG visual receipts 進入 OCR/VLM 前必須先輸出 no-write replay contract:`/api/ai-automation/pixelrag-ocr-vlm-replay` 與 `scripts/ops/report_pixelrag_ocr_vlm_replay.py` 只讀 saved tiles,輸出欄位 schema、輸出 schema、confidence/evidence validation rules、Ollama-first route contract 與 next machine action;`/api/ai-automation/smoke` 需包含 `PixelRAG OCR/VLM replay contract`,`/api/ai-automation/scheduled-health-summary` 需輸出 `pixelrag_ocr_vlm_replay` family。此 contract 不執行 OCR/VLM、不呼叫模型、不讀 secret、不連外、不寫 DB、不寫 `ai_insights`、不寫正式價格表;blocked / 403 / captcha / access denied / verify traffic page 只能進 platform probe 或 structured API 策略。
|
||||
- 2026-07-09 起 PixelRAG 可整合/可運用範圍必須有機器可讀 application portfolio:`/api/ai-automation/pixelrag-application-portfolio` 與 `scripts/ops/report_pixelrag_application_portfolio.py` 需輸出 commerce、RAG、UX、ops、marketing、governance lanes,每條 lane 必須包含 priority、status、integrates_with、use_cases、current_capability、next_machine_action、no-write 邊界與 forbidden guardrails。此 portfolio 不抓外站、不呼叫模型、不讀 secret、不寫 DB;其用途是把「還可以整合哪些」變成可排程、可驗證、可拒絕違規場景的主線工作項目。
|
||||
- 2026-07-09 起 PixelRAG ready receipts 必須有 Ollama-first VLM replay worker:`/api/ai-automation/pixelrag-vlm-replay-worker` 與 `scripts/ops/run_pixelrag_vlm_replay_worker.py` 預設 dry-run,`execute=true&write_receipt=true` 才呼叫 approved Ollama VLM route 並寫 artifact receipt;`/api/ai-automation/smoke` 需包含 `PixelRAG VLM replay worker`,`/api/ai-automation/scheduled-health-summary` 需輸出 `pixelrag_vlm_replay_worker` family。此 worker 不讀 secret、不寫 DB、不寫 `ai_insights`、不寫正式價格表;blocked / 403 / captcha / access denied receipt 自動跳 platform probe 或 structured API,ready receipt 的 VLM 結果仍須 identity matcher replay 與 PromotionGate 才能進候選知識層。
|
||||
- 2026-07-02 起 `/ai_intelligence` 商品明細與單品作戰詳情的四格價格證據必須可測:PChome 價格、MOMO 參考價、差距、可信度需以 `data-evidence` 固定,並以 `aria-label="價格證據"` 對應可掃描區塊;候選待確認或缺資料只能顯示「候選待確認 / 待補」,不得捏造價格或讓使用者打開 raw payload 才知道判斷依據。
|
||||
- 2026-07-02 起 `/ai_intelligence` 必須是密集 AI 工作台,不得退回大段文字說明頁:首屏與明細可見內容只保留短狀態、數字、四格證據與下一步按鈕;KPI note、benchmark detail、alert 副句、策略說明、decision copy、來源長句與單品 reason list 不得佔用第一層視覺。`tests/test_ai_intelligence_text_density_guardrails.py` 必須鎖住 `data-density-guardrail="compact-ai-workbench"`、短任務文案、detail meta 與 hidden explanatory copy。
|
||||
- 2026-07-02 起 `/observability/overview` 也必須採密集 AI 觀測工作台:首屏以 `data-density-guardrail="compact-observability-workbench"`、`AI 觀測 / 風險優先 / 下一步` 與 golden signals 先呈現狀態、數字與操作入口;hero lede、signal note、route desc、host meta 與資料來源長句不得佔用第一層視覺。`tests/test_observability_text_density_guardrails.py` 必須鎖住 compact marker 與 hidden explanatory copy。
|
||||
@@ -878,6 +879,7 @@ POSTGRES_HOST=momo-db
|
||||
| 2026-07-09 | PixelRAG / external MCP/RAG 必須有 runtime monitoring 與 candidate replay | V10.746 起 `/api/ai-automation/smoke`、`/api/ai-automation/scheduled-health-summary` 必須輸出 `External MCP/RAG integration readback` / `external_mcp_rag_integration` 與 `PixelRAG RAG candidate replay` / `pixelrag_rag_candidate_replay`;external MCP/RAG readback 回報 9 個 capability 的 absorbed / unresolved 與 `MCP_ROUTER_ENABLED`、`RAG_ENABLED` runtime flags,PixelRAG replay 只讀 visual receipt,拆分 eligible / blocked / invalid,並明確標記 blocked page 不是商品資料。此路徑不讀 secret、不呼叫外部網路、不寫 DB、不寫 `ai_insights`、不寫正式價格表;eligible receipt 仍需 OCR/VLM replay、identity matcher replay、PromotionGate 與 embedding signature guard。 |
|
||||
| 2026-07-09 | PixelRAG OCR/VLM replay contract 必須有 runtime monitoring | V10.747 起 `/api/ai-automation/pixelrag-ocr-vlm-replay`、`scripts/ops/report_pixelrag_ocr_vlm_replay.py`、`/api/ai-automation/smoke` 與 `/api/ai-automation/scheduled-health-summary` 必須輸出 no-write OCR/VLM replay contract / `pixelrag_ocr_vlm_replay` family;readback 只讀 saved tiles 與 RAG candidate replay,輸出 ready / blocked / invalid contracts、field schema、output schema、validation rules、Ollama-first route contract、blocked page guard 與 next machine action。此階段明確標記 `extraction_execution_performed=false`、`ocr_execution_performed=false`、`vlm_execution_performed=false`、`writes_database=false`、`writes_ai_insights=false`、`writes_price_tables=false`、`network_call=false`、`secret_read=false`、`primary_human_gate_count=0`;ready receipt 才能進下一段 `run_ollama_first_vlm_replay_worker`,blocked receipt 只能進 platform probe 或 structured API 策略。 |
|
||||
| 2026-07-09 | PixelRAG application portfolio 必須把可整合場景轉成主線工作項目 | V10.748 起 `/api/ai-automation/pixelrag-application-portfolio` 與 `scripts/ops/report_pixelrag_application_portfolio.py` 必須輸出 PixelRAG 在 commerce、RAG、UX、ops、marketing、governance 的可整合/可運用 lanes;每條 lane 需有 priority、status、integrates_with、use_cases、current_capability、next_machine_action、no-write 邊界與 forbidden guardrails。此 readback 依據 PixelRAG visual-RAG pattern、Google Merchant product data、Google Product structured data 與 Baymard product list UX 轉成內部工作項目;它不抓外站、不呼叫模型、不讀 secret、不寫 DB、不把像素結果當正式價格。 |
|
||||
| 2026-07-09 | PixelRAG Ollama-first VLM replay worker 必須有 runtime monitoring | V10.749 起 `/api/ai-automation/pixelrag-vlm-replay-worker`、`scripts/ops/run_pixelrag_vlm_replay_worker.py`、`/api/ai-automation/smoke` 與 `/api/ai-automation/scheduled-health-summary` 必須輸出 controlled VLM replay worker / `pixelrag_vlm_replay_worker` family;readback 預設 dry-run,不呼叫模型、不寫 artifact,execute 模式只讀 saved tiles、呼叫 approved Ollama VLM route、驗證 JSON field confidence/evidence refs,並只寫 artifact receipt。此 worker 明確標記 `writes_database=false`、`writes_ai_insights=false`、`writes_price_tables=false`、`secret_read=false`、`primary_human_gate_count=0`;blocked page 不得輸出商品欄位,ready VLM 結果仍需 identity matcher replay 與 PromotionGate。 |
|
||||
| 2026-06-29 | PChome DB apply 授權 lane 必須先通過 no-write guard / decision preflight / decision closeout / issuer gate / signing-decision preflight / signing-decision closeout / signing-issuer guard | V10.725 的 PChome mapping backlog auto-policy 已新增 `/api/ai/pchome-growth/mapping-backlog/auto-policy-db-apply-authorization-lane-guard`、`/api/ai/pchome-growth/mapping-backlog/auto-policy-db-apply-authorization-decision-preflight`、`/api/ai/pchome-growth/mapping-backlog/auto-policy-db-apply-authorization-decision-closeout`、`/api/ai/pchome-growth/mapping-backlog/auto-policy-db-apply-authorization-issuer-gate`、`/api/ai/pchome-growth/mapping-backlog/auto-policy-db-apply-authorization-signing-decision-preflight`、`/api/ai/pchome-growth/mapping-backlog/auto-policy-db-apply-authorization-signing-decision-closeout` 與 `/api/ai/pchome-growth/mapping-backlog/auto-policy-db-apply-authorization-signing-issuer-guard`;這些 endpoint 只驗證 final exact request package、same-run production truth requirement、secret rejection、rollback boundary、lane entry requirements、decision input requirements、rejection policy、post-apply verifier、future authorization decision package、final nonsecret authorization envelope、signing decision preflight inputs、unsigned signing decision package 與 signable request boundary,不讀 secret、不執行 shell/SQL、不寫 DB,也不簽發 database apply authorization。 |
|
||||
| 2026-06-29 | PChome DB apply 授權簽署發行者 lane 必須先產出 final signable request package | V10.725 的 PChome mapping backlog auto-policy 新增 `/api/ai/pchome-growth/mapping-backlog/auto-policy-db-apply-authorization-signing-issuer-closeout`;此 endpoint 只把 signing-issuer guard 的 signable request boundary 收斂成 final signable request package 與 closeout contract,確認 fresh production truth、post-apply verifier、migration hash、secret boundary 與 no-side-effect checks,不讀 secret、不簽發 authorization、不執行 shell/SQL、不寫 DB,也不代表正式 DB apply 已授權。 |
|
||||
| 2026-06-29 | PChome DB apply 授權簽署執行 lane 必須先通過 operator-held secret boundary preflight | V10.725 的 PChome mapping backlog auto-policy 新增 `/api/ai/pchome-growth/mapping-backlog/auto-policy-db-apply-authorization-signing-execution-preflight`;此 endpoint 只把 final signable request package 轉成 future signing execution preflight package、operator-held secret boundary contract、nonsecret signing inputs、command-shape preview、rollback boundary 與 abort conditions,不讀 secret、不接受 plaintext secret、不簽發 authorization、不執行 shell/SQL、不寫 DB,也不代表正式 DB apply 已授權。 |
|
||||
|
||||
@@ -18,7 +18,8 @@
|
||||
| Completed | External MCP/RAG capability inventory absorbed into internal governance readback | `/api/ai-automation/external-mcp-rag-integration` and `scripts/ops/report_external_mcp_rag_integration.py` expose 9 capabilities, absorbed/unresolved counts, and runtime flags. |
|
||||
| Completed | PixelRAG receipts to internal RAG candidate replay | `/api/ai-automation/pixelrag-rag-candidate-replay` and `scripts/ops/report_pixelrag_rag_candidate_replay.py` read receipts, split eligible vs blocked, and require OCR/VLM replay plus PromotionGate before knowledge writes. |
|
||||
| Completed | PixelRAG application portfolio and integration lanes | `/api/ai-automation/pixelrag-application-portfolio` and `scripts/ops/report_pixelrag_application_portfolio.py` expose commerce, RAG, UX, ops, marketing, and governance uses with priority, status, next machine action, and forbidden guardrails. |
|
||||
| In progress | MCP/RAG runtime health in AI automation smoke | `/api/ai-automation/smoke` and `/api/ai-automation/scheduled-health-summary` include external MCP/RAG integration, PixelRAG RAG candidate replay, and PixelRAG OCR/VLM replay contract families. |
|
||||
| Completed | PixelRAG Ollama-first VLM replay worker | `/api/ai-automation/pixelrag-vlm-replay-worker` and `scripts/ops/run_pixelrag_vlm_replay_worker.py` dry-run or execute ready visual receipts against approved Ollama VLM routes, emit evidence-bound artifact receipts, and keep blocked pages out of product data. |
|
||||
| In progress | MCP/RAG runtime health in AI automation smoke | `/api/ai-automation/smoke` and `/api/ai-automation/scheduled-health-summary` include external MCP/RAG integration, PixelRAG RAG candidate replay, PixelRAG OCR/VLM replay contract, and PixelRAG VLM replay worker families. |
|
||||
| In progress | Formal production deploy/readback discipline | Every mainline change must update version, push Gitea main/dev, deploy to 188 without touching `momo-db`, and read back `/health` plus new endpoints. |
|
||||
|
||||
## P1
|
||||
@@ -26,7 +27,7 @@
|
||||
| Status | Work item | Evidence / next machine action |
|
||||
|---|---|---|
|
||||
| Completed | OCR/VLM replay contract for visual fields | `/api/ai-automation/pixelrag-ocr-vlm-replay` and `scripts/ops/report_pixelrag_ocr_vlm_replay.py` turn saved tiles into no-write field contracts, output schemas, validation rules, and Ollama-first worker actions. |
|
||||
| Not started | Ollama-first OCR/VLM extraction worker | Execute the replay contract against approved local multimodal/OCR models and emit confidence/evidence receipts without writing formal price truth. |
|
||||
| Completed | Ollama-first OCR/VLM extraction worker | `run_pixelrag_vlm_replay_worker.py --execute --write-receipt` executes the replay contract against approved Ollama multimodal routes and emits confidence/evidence artifact receipts without writing formal price truth. |
|
||||
| Not started | Ollama-first multimodal embedding benchmark | Verify local Qwen3-VL or equivalent visual embedding on GCP-A -> GCP-B -> 111 before any visual vector retrieval. |
|
||||
| Not started | pgvector-compatible visual evidence metadata | Design metadata-first retrieval without FAISS in production unless ADR approves a different store. |
|
||||
| Not started | Coupang platform probe / structured API strategy | Treat 403 as platform barrier; prefer public structured data or approved probe, never count blocked pages as product data. |
|
||||
|
||||
@@ -34,7 +34,9 @@
|
||||
或 `python scripts/ops/report_pixelrag_ocr_vlm_replay.py` 可讀回 ready / blocked / invalid replay contracts、field schema、validation rules 與 Ollama-first 下一步,且目前不執行 OCR/VLM、不寫正式價格。
|
||||
- PixelRAG 可整合/可運用場景盤點必須確認 `/api/ai-automation/pixelrag-application-portfolio`
|
||||
或 `python scripts/ops/report_pixelrag_application_portfolio.py` 可讀回 area、priority、status、use cases、next machine action 與 forbidden guardrails;不得只存在聊天結論。
|
||||
- AI automation smoke 必須包含 external MCP/RAG integration、PixelRAG RAG candidate replay 與 PixelRAG OCR/VLM replay contract family,避免 registry 已完成但 runtime flag / receipt replay / VLM contract 未完成時被誤報為全自動閉環。
|
||||
- PixelRAG VLM replay worker 必須確認 `/api/ai-automation/pixelrag-vlm-replay-worker`
|
||||
或 `python scripts/ops/run_pixelrag_vlm_replay_worker.py` 可讀回 dry-run/execute、model_call、artifact receipt、blocked/ready 分流、DB write=0 與 primary human gate=0;execute 結果仍只是 candidate evidence。
|
||||
- AI automation smoke 必須包含 external MCP/RAG integration、PixelRAG RAG candidate replay、PixelRAG OCR/VLM replay contract 與 PixelRAG VLM replay worker family,避免 registry 已完成但 runtime flag / receipt replay / VLM worker 未完成時被誤報為全自動閉環。
|
||||
- AI 自動化 Prometheus 指標變更必須同步檢查 `docker/grafana/provisioning/dashboards/json/ai-automation-overview.json` 是否需要新增 panel 或查詢。
|
||||
- 188 線上 active monitoring stack 以 `monitoring/prometheus.yml` 為準;110 gateway 另有 `/home/wooo/monitoring/prometheus.yml`。若 dashboard 無資料,先確認 Prometheus `momo-app` target 與 `momo-network` 連線;所有 Blackbox HTTP target 必須打 `/health`,不可打 Dashboard 首頁 `/`。
|
||||
- Smoke dashboard 會保存 JSONL 趨勢;若新增檢查項目,要確保 history compact record 仍保持小而可讀。
|
||||
|
||||
@@ -184,6 +184,16 @@ python scripts/ops/report_pixelrag_ocr_vlm_replay.py --platform shopee_tw --plat
|
||||
|
||||
此 contract 只把 saved tiles、欄位 schema、輸出 schema、confidence/evidence 規則與 Ollama-first worker 下一步打包;目前不執行 OCR/VLM、不呼叫模型、不寫 RAG、不寫 `ai_insights`、不寫正式價格表。ready receipt 進 `run_ollama_first_vlm_replay_worker`,blocked / 403 / captcha / access denied receipt 進 platform probe 或 structured API 策略。
|
||||
|
||||
Ollama-first VLM replay worker:
|
||||
|
||||
```bash
|
||||
python scripts/ops/run_pixelrag_vlm_replay_worker.py
|
||||
python scripts/ops/run_pixelrag_vlm_replay_worker.py --platform shopee_tw --platform coupang_tw
|
||||
python scripts/ops/run_pixelrag_vlm_replay_worker.py --platform shopee_tw --execute --write-receipt --limit 1
|
||||
```
|
||||
|
||||
API readback: `/api/ai-automation/pixelrag-vlm-replay-worker?platform=shopee_tw`。預設為 dry-run,不呼叫模型、不寫 artifact;`execute=true&write_receipt=true` 才呼叫 Ollama VLM 並寫 artifact receipt。即使 execute,結果仍只是 candidate evidence;不得直接寫 `ai_insights`、正式價格表或競品價格歷史,且 missing confidence/evidence 會留在 replay / probe lane。
|
||||
|
||||
Application portfolio:
|
||||
|
||||
```bash
|
||||
|
||||
Reference in New Issue
Block a user