feat(governance): automate identity and comparison coverage

docs/AI_INTELLIGENCE_MODULE_SOT.md

@@ -1,8 +1,8 @@
 # PChome 業績成長自動化作戰系統 — AI 競價情報模組 Single Source of Truth
 
-> **最後更新**: 2026-07-11 (台北時間)
-> **狀態**: 🟠 Partial。SEC-P0-001 deny-by-default access control 已完成 production closure；四 Agent、PixelRAG、MCP/RAG registry、PChome controlled-preview families 與多項 smoke/readback 已建立，但 database identity/RBAC、full asset reconciliation、software supply chain、same-run controlled apply、restore drill、internal RAG canary 與 MCP/RAG runtime closure 尚未全部完成。任何局部 receipt 不得代表整體閉環完成。
-> **適用版本**: V10.787
+> **最後更新**: 2026-07-14 (台北時間)
+> **狀態**: 🟠 Partial。SEC-P0-001 deny-by-default access control 已完成 production closure；V10.788 database identity/RBAC source 與自動 cutover 已實作、等待 production canary/receipt。四 Agent、PixelRAG、MCP/RAG registry、PChome controlled-preview families 與多項 smoke/readback 已建立，但 full asset reconciliation、software supply chain、same-run controlled apply、restore drill、internal RAG canary 與 MCP/RAG runtime closure 尚未全部完成。任何局部 receipt 不得代表整體閉環完成。
+> **適用版本**: V10.788
 
 ---
 
@@ -17,6 +17,7 @@
 - 目前 ordered P0 以 `docs/guides/ai_automation_mainline_work_items.md` 為準：SEC-P0-001 access exposure 已關閉，current P0 是 identity/RBAC，之後依序為 webhook trust、supply chain、asset reconciliation、controlled-apply envelope、internal RAG canary 與 MCP/RAG runtime closure。
 - V10.786 起 `/metrics` 不再是 public exception；應用中央政策與 Nginx exact-match edge policy 僅允許 loopback / Docker transport，Prometheus 必須以內部 canary 證明 target `up`，且 public/LAN readback 必須拒絕。監控 Compose 不得保存明文 Grafana 管理密碼，Prometheus 對外埠預設只綁 loopback。
 - V10.787 依 live ingress 修正 metrics transport：production ingress 實際位於 188，110 global Prometheus 只能經專用 `172.20.0.1:19191` Docker-only Nginx bridge 抓取；bridge 以系統 CA、`mo.wooo.work` SNI/Host 轉送至 188，應用僅額外允許 `192.168.0.110/32` 單一監控主機，不得放行整段 LAN，也不得把 110 的其他 vhost 誤當 EwoooC production ingress。controlled apply 支援新 include 建立、Prometheus bind-mount reconciliation、`momo_app_info`/health/database product marker canary、精確 scrape URL、worker reload convergence 與失敗自動移除/回滾；production 已讀回 public 404、internal 200、target up、Prometheus identity preserved、8/8 anonymous matrix 與 `momo-db` unchanged，證據在 `governance/evidence/SEC-P0-001-20260711T122758Z.json`。
+- V10.788 把既有 `users/login_history/user_permissions` 正式接入登入：`auto` 模式先以 hybrid 維持回滾能力，只計 active admin 的 database-auth success durable receipts，預設連續兩次後自動轉為 database-only；legacy shared Session 下一次請求自動撤銷。鎖定改由 `login_history` 跨 worker 持久化，Session 綁定 password/role/active identity version，中央 policy 區分 read/operate/admin，且只有 allowlisted direct proxy peer 才能提供 `X-Forwarded-For/X-Real-IP`。帳號建立、身份更新、密碼變更、停用與權限異動會在同一交易寫入不含密碼的 audit event，commit 失敗時兩者一併 rollback。`/api/auth/governance` 與 `report_auth_identity_governance.py` 僅輸出非機密 readback；source ready 不等於 production cutover completed。
 - V10.777 起 web process 只能註冊 OpenClaw Blueprint，不得在 import 或 `record_once` 啟動 APScheduler、timed jobs 或 embedding thread；10 個 OpenClaw timed jobs 與唯一 embedding worker runtime owner 必須由 `momo-scheduler` 明確啟動。每日 04:15 的 codebase inventory 只掃描程式與前端資產並寫 artifact receipt，不讀 secret、不寫業務 DB。
 
 ---
@@ -155,7 +156,8 @@
 - V10.652 起正式首頁 `/` 必須顯示「PChome 業績成長自動化作戰系統」，舊商品看板僅保留在 `/dashboard` 或 `/product-dashboard`；「今日策略動作」必須放在首屏任務摘要後方，不能只藏在商品明細區；每列必須直接顯示價格證據，至少包含 PChome、MOMO、差距與可信度四格。候選待確認或缺資料時需以待確認/待補呈現，不得要求使用者先打開詳情才知道判斷依據。
 - V10.654 起全站側邊欄第一個主入口必須命名為「業績成長指揮台」；舊商品看板只能以「舊商品看板」保留在 `/dashboard`，比價工作台必須直連 `/dashboard?filter=pchome_review...`，不得再使用 `/` query 轉址，避免正式首頁與舊頁混淆。
 - V10.655 起正式首頁 `/` 必須以 HTTP 200 原地渲染「業績成長指揮台」，不得 302 跳轉到 `/ai_intelligence`；側邊欄第一個主入口必須直連 `/`。`/ai_intelligence` 只作為相容入口保留，不得成為主導流路由。
-- V10.656 起正式首頁首屏必須是「PChome 業績成長系統」專業儀表板，而非大段說明型頁首；第一屏需直接呈現近 7 天業績、比價可用率、下滑商品、待補比價、最大分類、下滑商品 TOP 5、PChome/MOMO 價格狀態圓環與處理狀態，且全部使用 `/api/ai/pchome-growth/opportunities` 真資料渲染。
+- V10.656 起正式首頁首屏必須是「PChome 業績成長系統」專業儀表板，而非大段說明型頁首；第一屏需直接呈現近 7 天業績、同款比價證據、下滑商品、待補同款、正式資料平台、下滑商品 TOP 5、比價覆蓋圓環與處理狀態，且全部使用 `/api/ai/pchome-growth/opportunities` 真資料渲染。
+- V10.789 起首頁禁止用無分母的「比價可用率」代表全站資料品質。`comparison_metric_contract.version=pchome_comparison_coverage_v2` 必須分開公布 TOP 高優先商品、近七日活躍商品與正式 runtime 平台三個 scope 的分子、分母與比率；`comparison_ready`、`candidate_validation`、`unmatched` 必須互斥且總和等於候選數。業績過期時 `operational_decision.ready=false`，UI 顯示決策暫停但保留歷史證據，不得把 stale 數字包裝成今日建議。`momo-scheduler` 必須以獨立防重入 watchdog 依序執行授權業績取得與 bounded comparison backfill，避免同步 `schedule.run_pending()` 被長任務阻塞後讓比價覆蓋永久停滯。
 - V10.658 起全站頁面必須回到「提升 PChome 業績」同一產品主線；共用 shell 需提供短流程骨架：評估、分析、建議、解法、治理。頁首、hero、任務卡與提示區不得依賴大段文字讓使用者理解狀況；首屏應以短指標、狀態分流、下一步動作與可執行入口呈現專業評估、分析、建議與解決方案。
 - V10.659 起主要入口頁的首屏文案必須改為流程職責語言，而不是功能說明書。當日業績負責找出下滑與價差壓力，匯入負責資料新鮮度，AI 建議負責把證據轉成銷售建議，比價負責同款與價差決策，缺貨負責避免主推商品斷貨，月結分析負責判斷成長、毛利與品類結構。
 

docs/adr/ADR-038-security-governance-automation-control-plane.md

@@ -60,6 +60,16 @@ Any missing stage produces `partial` or `degraded`, never completed.
 - `scripts/ops/report_security_governance_review.py --strict` is the source release gate.
 - `/api/ai-automation/security-governance-review` is the protected production readback.
 
+### 4.1 Database identity controlled cutover
+
+- `services/auth_identity_service.py` is the single policy for auth mode, trusted proxy CIDRs, role tiers and session identity versioning.
+- `login_history` is the durable login/audit/lockout source; process-local counters are not authoritative.
+- `auto` starts with bounded hybrid authority so the current operator is not locked out. Only successful database authentication by an active admin counts toward promotion; two durable success receipts are required by default.
+- Once the threshold is reached, effective authority becomes database-only without a manual review gate. Legacy shared sessions are revoked on their next request and cannot inherit an admin role from missing fields.
+- Any password, role, active-state or identity timestamp change invalidates the signed session through its durable identity version.
+- Account creation, identity updates, password changes, account disablement and permission changes must commit their no-secret `login_history` audit event in the same database transaction; failures roll back both the mutation and its audit.
+- `scripts/ops/report_auth_identity_governance.py` returns counts and control state only; it must never return password hashes, secret values or raw sessions.
+
 ### 5. Legacy policy sunset
 
 | Legacy policy | Owner | Expiry | Replacement | Verifier |

docs/guides/ai_automation_mainline_work_items.md

@@ -1,8 +1,8 @@
 # AI Automation Mainline Work Items
 
-> Updated: 2026-07-11
+> Updated: 2026-07-14
 > Governance: `global_product_governance_v2` + ADR-038
-> Current P0: `SEC-P0-002 database identity + least-privilege RBAC`
+> Current P0: `GROWTH-P0-001 comparison coverage truth + autonomous refresh`
 
 ## Source Of Truth
 
@@ -17,37 +17,38 @@
 | Order | ID | Status | Work item | Exit evidence / next machine action |
 |---:|---|---|---|---|
 | 1 | `SEC-P0-001` | Completed | Deny-by-default route access control | `governance/evidence/SEC-P0-001-20260711T122758Z.json` plus the current runtime receipt prove anonymous matrix 8/8 denied, public `/metrics` 404, exact internal target up, EwoooC product markers present, Prometheus identity preserved and `momo-db` unchanged. |
-| 2 | `SEC-P0-002` | In progress | Database identity + least-privilege RBAC | Replace shared admin password, default-admin fallback, process-local lockout and spoofable proxy identity. Next: inventory active users and auth call sites, then shadow database-backed auth before bounded canary cutover. Exit: role matrix, durable audit/lockout, session revocation and production readback. |
-| 3 | `SEC-P0-003` | In progress | Webhook trust and replay protection | Telegram secret-token verification code exists; production secret activation remains unproven. Exit: secret provisioned outside source, required mode enabled, invalid-secret 401 and authorized callback canary pass. |
-| 4 | `SUPPLY-P0-001` | In progress | Gitea-only secure software supply chain | Gitea-native checkout, secret-safe `.dockerignore`, commit-bound source receipt and governance gate are active. Exit: exact dependency lock, internal SAST/SCA/secret scan, SBOM, image digest/provenance, vulnerability SLA and production digest readback. |
-| 5 | `GOV-P0-001` | In progress | Canonical full asset graph + runtime reconciliation | `governance/ewoooc_asset_inventory.json` seeds hosts, services, data, AI, routes, supply chain, observability and recovery. Exit: same-run probe receipt for every asset; drift auto-creates work items. |
-| 6 | `GOV-P0-002` | Not started | Unified controlled-apply envelope | Introduce one `trace_id/run_id/work_item_id` across sensor, identity, SOT diff, decision, risk, dry-run, execution, verifier, rollback/retry and learning acknowledgement. Start with EventRouter + AutoHeal. |
-| 7 | `RAG-P0-001` | Not started | Internal RAG candidate canary | V10.770 stops at candidate preview. Exit: bounded pgvector candidate canary, signature/readback/feedback receipt, no price write and no premature `ai_insights` promotion. Next: `run_internal_rag_candidate_canary`. |
-| 8 | `MCP-P0-001` | In progress | MCP/RAG production runtime closure | Registry and smoke wiring exist, but runtime flags/ports must be live. Exit: MCP servers healthy, router enabled, RAG enabled, approved caller/tool boundary and production query canary pass. |
-| 9 | `SEC-P0-004` | Not started | Security operations lifecycle and metrics | Add durable security incident state and publish MTTA, MTTR, recurrence, false positive, human intervention, verifier pass, rollback and freshness. Exit: detect-to-learn production receipt. |
-| 10 | `REL-P0-001` | In progress | Formal deploy and visible proof discipline | V10.787 is healthy and the SEC-P0-001 runtime loop is closed with `momo-db` unchanged; Gitea CD #1120 remains waiting because no `ewoooc-host` runner is online. Exit: restore a dedicated EwoooC-only runner, replay the current source and close CD/readback automatically. |
+| 2 | `GROWTH-P0-001` | In progress | Comparison coverage truth + autonomous refresh | V10.789 source replaces the ambiguous percentage with numerator/denominator scopes for TOP opportunities, active seven-day catalog and registered runtime platforms; ready/candidate/unmatched states are mutually exclusive, stale sales block operational decisions, and a non-overlapping watchdog runs authorized sales acquisition before bounded comparison backfill. Exit: production API contract readback, newer scheduler receipts, desktop/mobile visible proof and unchanged `momo-db` identity. |
+| 3 | `SEC-P0-002` | In progress | Database identity + least-privilege RBAC | V10.789 release source connects `users/login_history/user_permissions` to login and role enforcement, adds durable lockout, identity-version session revocation, trusted-proxy CIDRs, and same-transaction no-secret audits for account/password/permission mutations. `auto` promotes from hybrid to database-only after two durable successful database logins by an active admin; no manual review gate. Next: deploy, write the runtime receipt, capture database-admin login receipts and verify automatic shared-authority retirement. |
+| 4 | `SEC-P0-003` | In progress | Webhook trust and replay protection | Telegram secret-token verification code exists; production secret activation remains unproven. Exit: secret provisioned outside source, required mode enabled, invalid-secret 401 and authorized callback canary pass. |
+| 5 | `SUPPLY-P0-001` | In progress | Gitea-only secure software supply chain | Gitea-native checkout, secret-safe `.dockerignore`, commit-bound source receipt and governance gate are active. Exit: exact dependency lock, internal SAST/SCA/secret scan, SBOM, image digest/provenance, vulnerability SLA and production digest readback. |
+| 6 | `GOV-P0-001` | In progress | Canonical full asset graph + runtime reconciliation | `governance/ewoooc_asset_inventory.json` seeds hosts, services, data, AI, routes, supply chain, observability and recovery. Exit: same-run probe receipt for every asset; drift auto-creates work items. |
+| 7 | `GOV-P0-002` | Not started | Unified controlled-apply envelope | Introduce one `trace_id/run_id/work_item_id` across sensor, identity, SOT diff, decision, risk, dry-run, execution, verifier, rollback/retry and learning acknowledgement. Start with EventRouter + AutoHeal. |
+| 8 | `RAG-P0-001` | Not started | Internal RAG candidate canary | V10.770 stops at candidate preview. Exit: bounded pgvector candidate canary, signature/readback/feedback receipt, no price write and no premature `ai_insights` promotion. Next: `run_internal_rag_candidate_canary`. |
+| 9 | `MCP-P0-001` | In progress | MCP/RAG production runtime closure | Registry and smoke wiring exist, but runtime flags/ports must be live. Exit: MCP servers healthy, router enabled, RAG enabled, approved caller/tool boundary and production query canary pass. |
+| 10 | `SEC-P0-004` | Not started | Security operations lifecycle and metrics | Add durable security incident state and publish MTTA, MTTR, recurrence, false positive, human intervention, verifier pass, rollback and freshness. Exit: detect-to-learn production receipt. |
+| 11 | `REL-P0-001` | In progress | Formal deploy and visible proof discipline | Production remains V10.787 while V10.789 identity and comparison-coverage source is under verification; SEC-P0-001 stays closed with `momo-db` unchanged. Gitea CD #1120 remains waiting because no `ewoooc-host` runner is online. Exit: deploy V10.789 through the bounded fallback, preserve the DB identity, and retain CD/runtime/visible proof separately. |
 
 ## P1
 
 | Order | ID | Status | Work item | Exit evidence / next machine action |
 |---:|---|---|---|---|
-| 11 | `RES-P1-001` | Not started | Backup/offsite/restore automation | Fresh checksum and offsite coverage plus isolated non-destructive restore drill with measured RPO/RTO. |
-| 12 | `PLAT-P1-001` | Not started | Non-root container and capability hardening | Shadow non-root image, writable-path inventory, capability drop/read-only filesystem canary, three-app rollout. |
-| 13 | `APPSEC-P1-001` | In progress | CSP and DOM/XSS hardening | Security headers are present and CSP is report-only. Collect violations, remove high-risk `innerHTML`/inline sinks, then enforce CSP by canary. |
-| 14 | `APPSEC-P1-002` | Not started | Unsafe shared-cache serialization removal | Replace writable pickle caches in dashboard/daily-sales/EDM/sales with constrained JSON or signed schema. |
-| 15 | `ARCH-P1-001` | In progress | Split oversized policy/executor/verifier modules | Current top debts include 44k-line PChome mapping and 14k-line smoke service. Split by bounded family and independent tests. |
-| 16 | `UX-P1-001` | In progress | Professional full-site UI/UX | Complete first-viewport cockpit, progressive disclosure, text-density reduction, Traditional Chinese product copy, accessibility, loading/error/empty/degraded states and desktop/mobile visual smoke. |
-| 17 | `PIXELRAG-P1-001` | Not started | Ollama-first multimodal embedding benchmark | Verify approved visual embedding on GCP-A -> GCP-B -> 111 and design pgvector-compatible visual metadata; FAISS remains disallowed without ADR. |
-| 18 | `MARKET-P1-001` | In progress | Marketplace source contracts | Finish stable, public-boundary, provenance and rate-limit contracts for Shopee, Coupang, Yahoo, ETMall, Friday and Rakuten; blocked pages remain non-product data. |
-| 19 | `QA-P1-001` | In progress | Deterministic test and CI governance | Current baseline is 1,701 passed / 30 legacy failures / 9 skipped plus one missing report module at collection. Restore or retire the missing contract, remove wall-clock/copy drift, and split deterministic release tests from nightly integration and production canaries. |
+| 12 | `RES-P1-001` | Not started | Backup/offsite/restore automation | Fresh checksum and offsite coverage plus isolated non-destructive restore drill with measured RPO/RTO. |
+| 13 | `PLAT-P1-001` | Not started | Non-root container and capability hardening | Shadow non-root image, writable-path inventory, capability drop/read-only filesystem canary, three-app rollout. |
+| 14 | `APPSEC-P1-001` | In progress | CSP and DOM/XSS hardening | Security headers are present and CSP is report-only. Collect violations, remove high-risk `innerHTML`/inline sinks, then enforce CSP by canary. |
+| 15 | `APPSEC-P1-002` | Not started | Unsafe shared-cache serialization removal | Replace writable pickle caches in dashboard/daily-sales/EDM/sales with constrained JSON or signed schema. |
+| 16 | `ARCH-P1-001` | In progress | Split oversized policy/executor/verifier modules | Current top debts include 44k-line PChome mapping and 14k-line smoke service. Split by bounded family and independent tests. |
+| 17 | `UX-P1-001` | In progress | Professional full-site UI/UX | Complete first-viewport cockpit, progressive disclosure, text-density reduction, Traditional Chinese product copy, accessibility, loading/error/empty/degraded states and desktop/mobile visual smoke. |
+| 18 | `PIXELRAG-P1-001` | Not started | Ollama-first multimodal embedding benchmark | Verify approved visual embedding on GCP-A -> GCP-B -> 111 and design pgvector-compatible visual metadata; FAISS remains disallowed without ADR. |
+| 19 | `MARKET-P1-001` | In progress | Marketplace source contracts | Finish stable, public-boundary, provenance and rate-limit contracts for Shopee, Coupang, Yahoo, ETMall, Friday and Rakuten; blocked pages remain non-product data. |
+| 20 | `QA-P1-001` | In progress | Deterministic test and CI governance | V10.789 full local baseline is 2,061 passed / 9 skipped / 0 failed with cache writes disabled after safe cache cleanup. Next: preserve the same terminal in Gitea CI, split deterministic release tests from nightly integration and production canaries, and prevent workstation disk pressure from invalidating evidence. |
 
 ## P2
 
 | Order | ID | Status | Work item | Exit evidence / next machine action |
 |---:|---|---|---|---|
-| 20 | `GOV-P2-001` | Not started | Continuous NIST/ASVS control trend | Persist governance snapshots, compare control/asset/runtime drift and create ordered work items automatically. |
-| 21 | `DATA-P2-001` | Not started | Data classification and retention enforcement | Classify business, personal, operational and model data; automate retention, deletion eligibility and audit evidence without destructive default actions. |
-| 22 | `CHAOS-P2-001` | Not started | Controlled resilience exercises | Run non-destructive model-host, MCP, queue, Telegram and app-container failure drills with rollback and learning receipts. |
+| 21 | `GOV-P2-001` | Not started | Continuous NIST/ASVS control trend | Persist governance snapshots, compare control/asset/runtime drift and create ordered work items automatically. |
+| 22 | `DATA-P2-001` | Not started | Data classification and retention enforcement | Classify business, personal, operational and model data; automate retention, deletion eligibility and audit evidence without destructive default actions. |
+| 23 | `CHAOS-P2-001` | Not started | Controlled resilience exercises | Run non-destructive model-host, MCP, queue, Telegram and app-container failure drills with rollback and learning receipts. |
 
 ## Completed Foundations
 

docs/memory/code_modularization_inventory_20260430.md

@@ -71,6 +71,7 @@
 - 2026-05-24 追記：同步 PChome review queue 決策信封合併後的 `services/competitor_intel_repository.py` 行數；此處只更新 inventory，不變更拆分策略。
 - 2026-05-25 追記：同步背景 embedding 讀取 `host_health_probes` skip guard 後的 `services/ollama_service.py` 行數；此處只更新 inventory，不變更 Ollama 路由決策。
 - 2026-05-29 追記：同步 PChome near-threshold / focused identity 回收系列後的 `services/marketplace_product_matcher.py` 行數；此處只更新 inventory，不變更拆分策略。
+- 2026-07-14 追記：`services/pchome_revenue_growth_service.py` 因新增可稽核比價覆蓋契約與 active-catalog scope 增至 1,366 行；此輪先完成 P0 正確性，後續 `ARCH-P1-001` 應拆出 catalog coverage query 與 metric-contract builder，主 service 保留 orchestration。
 
 ## 達到或超過 800 行檔案清單
 
@@ -146,7 +147,7 @@
 | 1306 | `routes/system_public_routes.py` | P1 public routes | health、metrics、public projection 分離 |
 | 1269 | `app.py` | P1 bootstrap、拆分進行中 | web scheduler 相容層已移到 `services/web_scheduler_compat.py`；下一步拆 extension setup |
 | 1272 | `services/code_review_pipeline_service.py` | P2 code review | scan、normalize、persist、report 分離 |
-| 1202 | `services/pchome_revenue_growth_service.py` | P1 revenue growth | sales query、offer comparison、scoring、projection 分離 |
+| 1366 | `services/pchome_revenue_growth_service.py` | P1 revenue growth | sales query、catalog coverage、metric contract、offer comparison、scoring、projection 分離 |
 | 1188 | `services/ppt_auto_generation_service.py` | P2 PPT automation | resolver、queue、generator、catchup 分離 |
 | 1121 | `services/pixelrag_crawler_integration_service.py` | P1 PixelRAG | capture routing、manifest、queue、readback 分離 |
 | 1023 | `routes/cicd_routes.py` | P2 CI/CD Blueprint | query、deploy action、route glue 分離 |
@@ -160,7 +161,7 @@
 | 895 | `routes/market_intel_review_routes.py` | P2 review Blueprint | command、query、route glue 分離 |
 | 867 | `services/token_report_service.py` | P2 token report | query、aggregate、chart、notify 分離 |
 | 864 | `services/ppt_vision_service.py` | P2 vision QA | runtime、queue、probe、audit 分離 |
-| 862 | `services/security_governance_review_service.py` | P1 governance | inventory、controls、runtime receipt、scorecard 分離 |
+| 1005 | `services/security_governance_review_service.py` | P1 governance | inventory、controls、runtime receipt、scorecard 分離 |
 | 861 | `routes/market_intel_review_post_routes.py` | P2 review POST routes | command handlers 與 verifier 分離 |
 | 846 | `services/market_intel/candidate_queue_review_ai_summary_persistence_telegram_dispatch_report_catalog_record_run_receipt.py` | P2 receipt pipeline | AI summary、persistence、delivery、catalog 分離 |
 | 843 | `services/pixelrag_vlm_replay_worker_service.py` | P1 VLM worker | route selection、inference、validation、receipt 分離 |