|
|
|
|
@@ -1,50 +1,85 @@
|
|
|
|
|
# AI Automation Mainline Work Items
|
|
|
|
|
|
|
|
|
|
> Updated: 2026-07-10
|
|
|
|
|
> Scope: EwoooC / MOMO Pro System AI automation, PixelRAG, MCP/RAG, production UX.
|
|
|
|
|
> Governance: `global_product_governance_v2` + ADR-038
|
|
|
|
|
> Current P0: `SEC-P0-001 deny-by-default access control`
|
|
|
|
|
|
|
|
|
|
## Source Of Truth
|
|
|
|
|
|
|
|
|
|
- Production readback stays authoritative after deploy: `/health`, container health, AI automation API readback, and smoke.
|
|
|
|
|
- Gitea is the only source-control truth. GitHub remains frozen and must not be used.
|
|
|
|
|
- Runtime truth must stay separate from source truth: a green test or registry does not mean MCP/RAG runtime is enabled.
|
|
|
|
|
- PixelRAG visual receipts are evidence, not formal price truth. They cannot write `competitor_prices`, `competitor_price_history`, or `ai_insights` without replay and promotion gates.
|
|
|
|
|
- Production runtime receipt/post-verifier/durable DB receipt is authoritative.
|
|
|
|
|
- Gitea main, deploy marker, CD run and production readback are the only source/deploy truth; GitHub remains frozen.
|
|
|
|
|
- Source/test/UI/CD green does not mean runtime closure.
|
|
|
|
|
- Completion must report program, asset coverage and runtime closure separately.
|
|
|
|
|
- PixelRAG visual evidence cannot write formal prices or `ai_insights` before identity, PromotionGate and internal RAG canary proof.
|
|
|
|
|
|
|
|
|
|
## P0
|
|
|
|
|
## Ordered P0
|
|
|
|
|
|
|
|
|
|
| Status | Work item | Evidence / next machine action |
|
|
|
|
|
|---|---|---|
|
|
|
|
|
| Completed | Multi-commerce PixelRAG visual evidence lane for momo, pchome, shopee_tw, coupang_tw, yahoo_shopping_tw, etmall_tw, friday_tw, rakuten_tw | API and CLI can emit expansion plans and marketplace manifests; PChome parser failures can queue manifests. |
|
|
|
|
|
| Completed | External MCP/RAG capability inventory absorbed into internal governance readback | `/api/ai-automation/external-mcp-rag-integration` and `scripts/ops/report_external_mcp_rag_integration.py` expose 9 capabilities, absorbed/unresolved counts, and runtime flags. |
|
|
|
|
|
| Completed | PixelRAG receipts to internal RAG candidate replay | `/api/ai-automation/pixelrag-rag-candidate-replay` and `scripts/ops/report_pixelrag_rag_candidate_replay.py` read visual capture receipts and platform probe worker receipts, split eligible / blocked / source-contract-fallback candidates, and expose combined OCR/VLM + source-contract next actions before identity matching and PromotionGate knowledge writes. |
|
|
|
|
|
| Completed | PixelRAG source-contract replay worker | `/api/ai-automation/pixelrag-source-contract-replay-worker` and `scripts/ops/run_pixelrag_source_contract_replay_worker.py` turn platform probe worker `source_contract_fallback` receipts into replayable source-contract artifact receipts with provenance, public-boundary, no-DB, and blocked-page-not-product-data guards. |
|
|
|
|
|
| Completed | PixelRAG marketplace source-contract adapter preflight | `/api/ai-automation/pixelrag-marketplace-source-contract-adapter-preflight` and `scripts/ops/run_pixelrag_marketplace_source_contract_adapter_preflight.py` validate source-contract replay receipts for adapter readiness, canonical offer fields, provenance/rate-limit/public-boundary requirements, and no-write promotion gates. |
|
|
|
|
|
| Completed | PixelRAG marketplace adapter dry-run | `/api/ai-automation/pixelrag-marketplace-source-contract-adapter-dry-run` and `scripts/ops/run_pixelrag_marketplace_source_contract_adapter_dry_run.py` turn adapter preflight receipts into deterministic no-write field contracts and offer candidate contracts, with price writes blocked until identity matcher replay and PromotionGate. |
|
|
|
|
|
| Completed | PixelRAG marketplace identity matcher replay | `/api/ai-automation/pixelrag-marketplace-identity-matcher-replay` and `scripts/ops/run_pixelrag_marketplace_identity_matcher_replay.py` turn adapter dry-run receipts into deterministic identity candidate contracts and source provenance checks, with price writes blocked until PromotionGate replay. |
|
|
|
|
|
| Completed | PixelRAG marketplace PromotionGate replay | `/api/ai-automation/pixelrag-marketplace-promotion-gate-replay` and `scripts/ops/run_pixelrag_marketplace_promotion_gate_replay.py` turn identity matcher receipts into deterministic PromotionGate contracts and embedding-signature readiness, with price writes still blocked. |
|
|
|
|
|
| Completed | PixelRAG marketplace embedding signature guard replay | `/api/ai-automation/pixelrag-marketplace-embedding-signature-guard-replay` and `scripts/ops/run_pixelrag_marketplace_embedding_signature_guard_replay.py` turn PromotionGate receipts into deterministic embedding-signature guard contracts and candidate-knowledge readiness, with DB and price writes still blocked. |
|
|
|
|
|
| Completed | PixelRAG marketplace candidate knowledge replay | `/api/ai-automation/pixelrag-marketplace-candidate-knowledge-replay` and `scripts/ops/run_pixelrag_marketplace_candidate_knowledge_replay.py` turn embedding-signature guard receipts into deterministic internal RAG candidate preview contracts, with DB, `ai_insights`, and price writes still blocked until canary. |
|
|
|
|
|
| Completed | PixelRAG application portfolio and integration lanes | `/api/ai-automation/pixelrag-application-portfolio` and `scripts/ops/report_pixelrag_application_portfolio.py` expose commerce, RAG, UX, ops, marketing, and governance uses with priority, status, next machine action, and forbidden guardrails. |
|
|
|
|
|
| Completed | PixelRAG Ollama-first VLM replay worker | `/api/ai-automation/pixelrag-vlm-replay-worker` and `scripts/ops/run_pixelrag_vlm_replay_worker.py` dry-run or execute ready visual receipts against approved Ollama VLM routes, emit evidence-bound artifact receipts, and keep blocked pages out of product data. |
|
|
|
|
|
| Completed | PixelRAG VLM route readiness and auto model select | `/api/ai-automation/pixelrag-vlm-route-readiness` and `scripts/ops/report_pixelrag_vlm_route_readiness.py` read approved Ollama `/api/tags`, expose configured/candidate model readiness, and let execute mode avoid missing-model blind generate calls. |
|
|
|
|
|
| Completed | PixelRAG platform probe worker | `/api/ai-automation/pixelrag-platform-probe-worker` and `scripts/ops/run_pixelrag_platform_probe_worker.py` turn platform probe plans into dry-run/execute automation: Shopee interstitials go to public empty-context capture when runtime is available, missing Playwright auto-falls back to structured-source/backoff receipt, and Coupang 403/access denied goes to structured-source/backoff artifact receipts. |
|
|
|
|
|
| In progress | MCP/RAG runtime health in AI automation smoke | `/api/ai-automation/smoke` and `/api/ai-automation/scheduled-health-summary` include external MCP/RAG integration, PixelRAG RAG candidate replay, PixelRAG OCR/VLM replay contract, PixelRAG VLM route readiness, PixelRAG VLM replay worker, PixelRAG platform probe, PixelRAG platform probe worker, source-contract replay, adapter preflight, adapter dry-run, identity matcher replay, PromotionGate replay, embedding signature guard replay, and candidate knowledge replay families. |
|
|
|
|
|
| In progress | Formal production deploy/readback discipline | Every mainline change must update version, push Gitea main/dev, deploy to 188 without touching `momo-db`, and read back `/health` plus new endpoints. |
|
|
|
|
|
| Order | ID | Status | Work item | Exit evidence / next machine action |
|
|
|
|
|
|---:|---|---|---|---|
|
|
|
|
|
| 1 | `SEC-P0-001` | In progress | Deny-by-default route access control | Central policy now protects vendor/import/CI-CD/crawler/category/log/operator and sales-detail surfaces; every anonymous GET is explicitly allowlisted. Exit: anonymous production matrix passes and `/metrics` moves behind edge/internal auth. Next: `verify_production_anonymous_matrix_and_move_metrics_behind_edge_auth`. |
|
|
|
|
|
| 2 | `SEC-P0-002` | Not started | Database identity + least-privilege RBAC | Replace shared admin password, default-admin fallback, process-local lockout and spoofable proxy identity. Exit: shadow auth, role matrix, durable audit/lockout, session revocation and canary cutover. |
|
|
|
|
|
| 3 | `SEC-P0-003` | In progress | Webhook trust and replay protection | Telegram secret-token verification code exists; production secret activation remains unproven. Exit: secret provisioned outside source, required mode enabled, invalid-secret 401 and authorized callback canary pass. |
|
|
|
|
|
| 4 | `SUPPLY-P0-001` | In progress | Gitea-only secure software supply chain | Gitea-native checkout and governance gate added. Exit: exact dependency lock, internal SAST/SCA/secret scan, SBOM, image digest/provenance, vulnerability SLA and production digest readback. |
|
|
|
|
|
| 5 | `GOV-P0-001` | In progress | Canonical full asset graph + runtime reconciliation | `governance/ewoooc_asset_inventory.json` seeds hosts, services, data, AI, routes, supply chain, observability and recovery. Exit: same-run probe receipt for every asset; drift auto-creates work items. |
|
|
|
|
|
| 6 | `GOV-P0-002` | Not started | Unified controlled-apply envelope | Introduce one `trace_id/run_id/work_item_id` across sensor, identity, SOT diff, decision, risk, dry-run, execution, verifier, rollback/retry and learning acknowledgement. Start with EventRouter + AutoHeal. |
|
|
|
|
|
| 7 | `RAG-P0-001` | Not started | Internal RAG candidate canary | V10.770 stops at candidate preview. Exit: bounded pgvector candidate canary, signature/readback/feedback receipt, no price write and no premature `ai_insights` promotion. Next: `run_internal_rag_candidate_canary`. |
|
|
|
|
|
| 8 | `MCP-P0-001` | In progress | MCP/RAG production runtime closure | Registry and smoke wiring exist, but runtime flags/ports must be live. Exit: MCP servers healthy, router enabled, RAG enabled, approved caller/tool boundary and production query canary pass. |
|
|
|
|
|
| 9 | `SEC-P0-004` | Not started | Security operations lifecycle and metrics | Add durable security incident state and publish MTTA, MTTR, recurrence, false positive, human intervention, verifier pass, rollback and freshness. Exit: detect-to-learn production receipt. |
|
|
|
|
|
| 10 | `REL-P0-001` | In progress | Formal deploy and visible proof discipline | Every functional change bumps version, passes Gitea CD, deploys only three app containers, leaves `momo-db` untouched, and reads back `/health`, new APIs and visible UI/security behavior. |
|
|
|
|
|
|
|
|
|
|
## P1
|
|
|
|
|
|
|
|
|
|
| Status | Work item | Evidence / next machine action |
|
|
|
|
|
| Order | ID | Status | Work item | Exit evidence / next machine action |
|
|
|
|
|
|---:|---|---|---|---|
|
|
|
|
|
| 11 | `RES-P1-001` | Not started | Backup/offsite/restore automation | Fresh checksum and offsite coverage plus isolated non-destructive restore drill with measured RPO/RTO. |
|
|
|
|
|
| 12 | `PLAT-P1-001` | Not started | Non-root container and capability hardening | Shadow non-root image, writable-path inventory, capability drop/read-only filesystem canary, three-app rollout. |
|
|
|
|
|
| 13 | `APPSEC-P1-001` | In progress | CSP and DOM/XSS hardening | Security headers are present and CSP is report-only. Collect violations, remove high-risk `innerHTML`/inline sinks, then enforce CSP by canary. |
|
|
|
|
|
| 14 | `APPSEC-P1-002` | Not started | Unsafe shared-cache serialization removal | Replace writable pickle caches in dashboard/daily-sales/EDM/sales with constrained JSON or signed schema. |
|
|
|
|
|
| 15 | `ARCH-P1-001` | In progress | Split oversized policy/executor/verifier modules | Current top debts include 44k-line PChome mapping and 14k-line smoke service. Split by bounded family and independent tests. |
|
|
|
|
|
| 16 | `UX-P1-001` | In progress | Professional full-site UI/UX | Complete first-viewport cockpit, progressive disclosure, text-density reduction, Traditional Chinese product copy, accessibility, loading/error/empty/degraded states and desktop/mobile visual smoke. |
|
|
|
|
|
| 17 | `PIXELRAG-P1-001` | Not started | Ollama-first multimodal embedding benchmark | Verify approved visual embedding on GCP-A -> GCP-B -> 111 and design pgvector-compatible visual metadata; FAISS remains disallowed without ADR. |
|
|
|
|
|
| 18 | `MARKET-P1-001` | In progress | Marketplace source contracts | Finish stable, public-boundary, provenance and rate-limit contracts for Shopee, Coupang, Yahoo, ETMall, Friday and Rakuten; blocked pages remain non-product data. |
|
|
|
|
|
| 19 | `QA-P1-001` | In progress | Deterministic test and CI governance | Current baseline is 1,701 passed / 30 legacy failures / 9 skipped plus one missing report module at collection. Restore or retire the missing contract, remove wall-clock/copy drift, and split deterministic release tests from nightly integration and production canaries. |
|
|
|
|
|
|
|
|
|
|
## P2
|
|
|
|
|
|
|
|
|
|
| Order | ID | Status | Work item | Exit evidence / next machine action |
|
|
|
|
|
|---:|---|---|---|---|
|
|
|
|
|
| 20 | `GOV-P2-001` | Not started | Continuous NIST/ASVS control trend | Persist governance snapshots, compare control/asset/runtime drift and create ordered work items automatically. |
|
|
|
|
|
| 21 | `DATA-P2-001` | Not started | Data classification and retention enforcement | Classify business, personal, operational and model data; automate retention, deletion eligibility and audit evidence without destructive default actions. |
|
|
|
|
|
| 22 | `CHAOS-P2-001` | Not started | Controlled resilience exercises | Run non-destructive model-host, MCP, queue, Telegram and app-container failure drills with rollback and learning receipts. |
|
|
|
|
|
|
|
|
|
|
## Completed Foundations
|
|
|
|
|
|
|
|
|
|
These are reusable foundations, not proof that the full program is complete.
|
|
|
|
|
|
|
|
|
|
| Status | Capability | Current boundary |
|
|
|
|
|
|---|---|---|
|
|
|
|
|
| Completed | OCR/VLM replay contract for visual fields | `/api/ai-automation/pixelrag-ocr-vlm-replay` and `scripts/ops/report_pixelrag_ocr_vlm_replay.py` turn saved tiles into no-write field contracts, output schemas, validation rules, and Ollama-first worker actions. |
|
|
|
|
|
| Completed | Ollama-first OCR/VLM extraction worker | `run_pixelrag_vlm_replay_worker.py --execute --write-receipt` executes the replay contract against approved Ollama multimodal routes and emits confidence/evidence artifact receipts without writing formal price truth. |
|
|
|
|
|
| Not started | Ollama-first multimodal embedding benchmark | Verify local Qwen3-VL or equivalent visual embedding on GCP-A -> GCP-B -> 111 before any visual vector retrieval. |
|
|
|
|
|
| Not started | pgvector-compatible visual evidence metadata | Design metadata-first retrieval without FAISS in production unless ADR approves a different store. |
|
|
|
|
|
| In progress | Coupang platform probe / structured API strategy | Treat 403 as platform barrier; `run_pixelrag_platform_probe_worker.py --execute --write-receipt` now emits structured-source/backoff receipts without network, DB writes, or product-data promotion. |
|
|
|
|
|
| In progress | Marketplace source contracts beyond MOMO/PChome | Source-contract replay receipts now enter marketplace adapter preflight as artifact receipts; next machine action is adapter dry-run for stable Shopee/Coupang/Yahoo/ETMall/friday/Rakuten contracts after provenance, rate-limit, public boundary, and replay evidence. |
|
|
|
|
|
| In progress | Professional product website UI/UX text reduction | Continue compact AI surfaces, visual QA, and user-facing copy guardrails; avoid engineering logs or work-session prose in the UI. |
|
|
|
|
|
| Completed | Multi-commerce PixelRAG visual evidence for momo, pchome, shopee_tw, coupang_tw, yahoo_shopping_tw, etmall_tw, friday_tw, rakuten_tw | Evidence-only; blocked pages are not product data. |
|
|
|
|
|
| Completed | External MCP/RAG capability inventory | Registry/integration readback exists; runtime enablement remains P0. |
|
|
|
|
|
| Completed | PixelRAG receipt -> RAG candidate replay | Candidate-only; no formal knowledge or price write. |
|
|
|
|
|
| Completed | Source-contract replay worker | Public-boundary artifact receipts only. |
|
|
|
|
|
| Completed | Marketplace adapter preflight and dry-run | Deterministic no-write contracts. |
|
|
|
|
|
| Completed | Marketplace identity matcher replay | Candidate identity only. |
|
|
|
|
|
| Completed | PromotionGate replay | No production write. |
|
|
|
|
|
| Completed | Embedding-signature guard replay | Signature readiness only. |
|
|
|
|
|
| Completed | Candidate knowledge replay | Internal RAG preview only; canary remains P0. |
|
|
|
|
|
| Completed | PixelRAG application portfolio | Commerce/RAG/UX/ops/marketing/governance inventory. |
|
|
|
|
|
| Completed | Ollama-first VLM route readiness and replay worker | Evidence-bound artifact output; no direct price write. |
|
|
|
|
|
| Completed | Platform probe worker | Shopee/Coupang barriers become structured fallback/backoff receipts. |
|
|
|
|
|
|
|
|
|
|
## Always Enforced
|
|
|
|
|
## Definition Of Done
|
|
|
|
|
|
|
|
|
|
- Manual review is exception-only for low-risk, machine-verifiable controlled apply.
|
|
|
|
|
- Break-glass still applies to secrets, destructive DB actions, force pushes, production provider switchovers, raw runtime volumes, and unverified core model replacement.
|
|
|
|
|
- AI automation smoke must expose health honestly: `MCP_ROUTER_ENABLED=false` or `RAG_ENABLED=false` is a runtime gap, not a source-code failure.
|
|
|
|
|
A work item can be `Completed` only when the same production run contains:
|
|
|
|
|
|
|
|
|
|
1. sensor/source receipt;
|
|
|
|
|
2. normalized canonical asset identity;
|
|
|
|
|
3. source-of-truth diff;
|
|
|
|
|
4. AI decision and candidate action;
|
|
|
|
|
5. risk/policy decision;
|
|
|
|
|
6. check-mode/dry-run receipt;
|
|
|
|
|
7. idempotent bounded execution receipt;
|
|
|
|
|
8. independent post-verifier and rollback/no-write terminal;
|
|
|
|
|
9. incident/Telegram/KM/RAG/MCP/PlayBook durable acknowledgement.
|
|
|
|
|
|
|
|
|
|
Missing any stage means `partial`, `degraded` or `blocked_with_safe_next_action`.
|
|
|
|
|
|
ogt