feat(awooop): Phase 1-8 完整實作 — AwoooP Agent Platform 六平面架構
## Phase 1-3: Control Plane + Contract System - awooop_phase1_control_plane_2026-05-04.sql: 12 張核心表 + RLS - awooop_phase1_batch1_rls_2026-05-04.sql: 全部 FORCE RLS + GRANT - packages/awooop-contracts/: 六合約 JSON Schema + golden fixtures - src/models/awooop_contracts.py: Pydantic v2 contract models(extra=forbid) - src/repositories/contract_repository.py: contract lifecycle(draft→published→active) - src/services/contract_service.py: HMAC publish sig + Redis multi-sig activate - src/services/schema_validator.py: LLM output validator(retry×3, E-SCHEMA-001) ## Phase 2: Tenant Isolation - awooop_phase2_budget_ledger_2026-05-04.sql: budget_ledger + RLS - src/services/budget_service.py: Token Budget Hard Kill 三層防線 - src/core/context.py: PROJECT_ID ContextVar(31 background loop 自動繼承) - src/db/base.py + models.py: project_id 欄位 + RLS set_config 注入 - src/hermes/nl_gateway.py: project_id Redis key 前綴(Phase A 雙寫) - src/services/anomaly_counter.py: per-project 改造(Phase A fallback) ## Phase 4: Platform Shell in Shadow Mode - awooop_phase4_run_state_2026-05-04.sql: run_state + step_journal + idempotency - src/services/run_state_machine.py: 8-state FSM + SKIP LOCKED + stale reaper - src/services/platform_runtime.py: UUID v7 + W3C trace_id + shadow_execute - src/services/audit_sink.py: PII/secret redaction 9 patterns - src/api/v1/platform/runs.py: POST/GET /v1/platform/runs(Router→Service 架構) - src/workers/platform_worker.py: SKIP LOCKED worker + heartbeat + reaper loop - src/main.py: platform router + lifespan worker start/stop ## Phase 5: MCP Gateway 五閘門 - awooop_phase5_mcp_gateway_2026-05-04.sql: 4 表 + RLS - src/plugins/mcp/gateway.py: McpGateway(Gate 1~5, E-MCP-GATE-001~009) - src/plugins/mcp/redaction_middleware.py: 雙層 redaction + 16K 截斷 - src/plugins/mcp/registry.py: __provider name mangling(ADR-116) - src/plugins/mcp/credential_resolver.py: k8s secret ref 解析 - tests/test_mcp_credential_isolation.py: 10 個迴歸測試(secret leak 防再現) ## Phase 6-8: EwoooC + Channel Hub + Approval Token - awooop_phase6_ewoooc_onboarding_2026-05-04.sql: ewoooc tenant + 4 read-only MCP tools - awooop_phase7_channel_hub_2026-05-04.sql: conversation_event + outbound_message - src/services/provider_proxy.py: ProviderProxy + PlatformEnvelope(ADR-115) - src/services/channel_hub.py: Telegram inbound mirror + Progressive Feedback(30s) - src/services/awooop_approval_token.py: HS256 + jti NX replay 防護 + suggest mode Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
285
docs/LOGBOOK.md
285
docs/LOGBOOK.md
@@ -6,6 +6,291 @@
|
||||
|
||||
---
|
||||
|
||||
## 2026-05-04 | AwoooP Phase 6-8 完收(EwoooC Onboarding / Channel Hub / Approval Token)
|
||||
|
||||
### Phase 6: EwoooC Tenant Onboarding(ADR-115)
|
||||
|
||||
**migrations/awooop_phase6_ewoooc_onboarding_2026-05-04.sql**:
|
||||
- `INSERT INTO awooop_projects` — project_id='ewoooc', migration_mode='shadow', budget_limit=50 USD
|
||||
- 4 個 read-only MCP tools 預置白名單(k8s_get / signoz_query / incident_read / km_read)
|
||||
- 所有 scope=['read'],environment_tags={env:any}(shadow phase 無環境限制)
|
||||
|
||||
**services/provider_proxy.py**(ProviderProxy + PlatformEnvelope):
|
||||
- `build_envelope()` → PlatformEnvelope(project_id / agent_id / trace_id / platform_subject_id)
|
||||
- `_validate_project()`:拒絕 legacy_awoooi_default mode
|
||||
- `_upsert_platform_subject()`:auto-provisioning(ON CONFLICT DO UPDATE last_seen_at)
|
||||
- `build_platform_subject_id()` → `"ewoooc:telegram:123456789"` 統一格式
|
||||
- `_new_trace_id()` → W3C traceparent(00-{32hex}-{16hex}-01)
|
||||
- 自驗:platform_subject_id 格式、trace_id 4段格式、PlatformEnvelope.as_dict() 正確
|
||||
|
||||
### Phase 7: Channel Hub(ADR-106 channel_event family)
|
||||
|
||||
**migrations/awooop_phase7_channel_hub_2026-05-04.sql**:
|
||||
- `awooop_conversation_event` — 入站事件鏡像(UNIQUE provider_event_id,dedup + run_id + content hash)
|
||||
- `awooop_outbound_message` — 出站訊息記錄(interim/final/error/approval_request + shadow status)
|
||||
- Progressive Feedback Policy 查詢 index(waiting_tool + pending)
|
||||
- 全部 FORCE ROW LEVEL SECURITY
|
||||
|
||||
**services/channel_hub.py**:
|
||||
- `mirror_inbound_event()` — raw_content 只存 sha256 hash + redacted preview(明文不入庫)
|
||||
- `record_outbound_message()` — shadow=True 時 status='shadow'(不發送)
|
||||
- `schedule_interim_feedback()` → asyncio.create_task(30s 計時器)
|
||||
- `_interim_feedback_task()` — 30s 後查 run state,仍 waiting_tool → 發 interim
|
||||
- `handle_telegram_inbound()` — 主入口:create_run + mirror + schedule_interim
|
||||
- 自驗:import 正確,INTERIM_WAIT_SECONDS=30
|
||||
|
||||
### Phase 8: Approval Token HS256 + Suggest Mode(ADR-116 Gate 5)
|
||||
|
||||
**services/awooop_approval_token.py**(獨立模組,不修改 legacy multi_sig_redis.py):
|
||||
- `issue_approval_token()` — HS256 自製 JWT(3 段 base64url),jti=uuid4.hex
|
||||
- `verify_approval_token()` — HMAC.compare_digest + exp 驗證,回傳 payload
|
||||
- `record_approval()` — verify token → Redis NX jti(防 replay)→ SADD approver_id → 回傳簽核數
|
||||
- `check_approval_quorum(required_count=1)` — SCARD ≥ required | QuorumNotMetError
|
||||
- Redis key 前綴:`awooop_appr:jti:*` / `awooop_appr:sigs:*`(與 legacy 不衝突)
|
||||
- `is_suggest_mode_enabled()` — AWOOOP_SUGGEST_MODE env flag
|
||||
- `build_suggest_action(rollback/scale/restart)` → SuggestedAction(dry_run=True, approval_required=True)
|
||||
- 錯誤碼:E-APPR-001/002/003/004
|
||||
- 自驗:6 個測試全部通過(issue/verify/tamper/expire/suggest mode/suggest rollback/scale)
|
||||
|
||||
---
|
||||
|
||||
## 2026-05-04 | AwoooP Phase 5 完收(MCP Gateway 五閘門 + Credential Isolation)
|
||||
|
||||
### Phase 5.1 MCP Gateway DB Migration(awooop_phase5_mcp_gateway_2026-05-04.sql)
|
||||
|
||||
四表 + 全部 RLS + GRANT:
|
||||
- `awooop_mcp_tool_registry` — Tool 白名單(Gate 3,tool_type 3 值、allowed_scopes JSONB、environment_tags Gate 4 用)
|
||||
- `awooop_mcp_grants` — Agent × Tool 授權(Grant 2+3,expires_at + is_revoked + granted_scopes + 撤銷一致性 CHECK)
|
||||
- `awooop_mcp_credential_refs` — k8s Secret 參照(ADR-118,只存路徑 namespace/secret#key,明文絕不入庫)
|
||||
- `awooop_mcp_gateway_audit` — Gateway call 稽核(gate_result JSONB 五閘結果 + block_gate/block_reason)
|
||||
- 全部 `FORCE ROW LEVEL SECURITY`;4 個查詢優化 partial index
|
||||
|
||||
ORM:`awooop_models.py` 新增 `AwoooPMcpToolRegistry` / `AwoooPMcpGrant` / `AwoooPMcpCredentialRef` / `AwoooPMcpGatewayAudit`
|
||||
自驗:4 個 ORM model import 正確(all_ok: True)
|
||||
|
||||
### Phase 5.2 Five-gate Enforcement Service(plugins/mcp/gateway.py)
|
||||
|
||||
`McpGateway.call()` 實作五閘門依序強制檢查:
|
||||
- Gate 1 — Project:`awooop_projects` 存在且 `migration_mode != legacy_awoooi_default`
|
||||
- Gate 2 — Agent:`awooop_active_revisions` 有 `family=agent, contract_id=agent_id` 的 active contract
|
||||
- Gate 3 — Tool+Grant:tool 在白名單 + grant 未到期/未撤銷 + scope 包含 required_scope
|
||||
- Gate 4 — Environment:tool.environment_tags 全部匹配(shadow mode 直接放行)
|
||||
- Gate 5 — Approval:write/admin scope 時查 Redis multi_sig approval key(shadow + read 直接放行)
|
||||
- 任一失敗:寫 blocked audit log + raise McpGatewayError(error_code E-MCP-GATE-001~009)
|
||||
- 通過後呼叫底層 provider,結果寫 success audit
|
||||
|
||||
自驗:所有 import 正確,GateCheckResult.all_passed / as_dict() 功能正常
|
||||
|
||||
### Phase 5.3 MCP Redaction Middleware(plugins/mcp/redaction_middleware.py)
|
||||
|
||||
雙層 redaction:
|
||||
- Layer 1(audit_sink)— 寫 audit log 前(已於 Phase 4.4 完成)
|
||||
- Layer 2(本層)— MCP tool call input/output 在進入 LLM context 前:
|
||||
- `redact_mcp_input()`: 移除 _mcp_audit injection + credential isolation(k8s_value 等)+ 欄位黑名單 + pattern redaction
|
||||
- `redact_mcp_output()`: 完整 pattern redaction + 大小限制(16,000 char,防 prompt stuffing)
|
||||
- `compute_safe_hash()`: sha256(redacted_data),供 gateway audit 使用
|
||||
- 自驗:4 個測試案例全部通過(all_ok: True)
|
||||
|
||||
### Phase 5.4 Provider 封裝強化(plugins/mcp/registry.py)
|
||||
|
||||
`AuditedMCPToolProvider._provider` → `__provider`(Python name mangling):
|
||||
- 防止 caller 透過 `wrapper._provider` 直接存取 inner provider(ADR-116 封裝要求)
|
||||
- Python 自動重命名為 `_AuditedMCPToolProvider__provider`,外部不可直接 access
|
||||
- 4 個 `self._provider.xxx` 引用全部更新為 `self.__provider.xxx`
|
||||
|
||||
### Phase 5.5 Credential Isolation(plugins/mcp/credential_resolver.py + 迴歸測試)
|
||||
|
||||
`credential_resolver.py`:
|
||||
- `resolve_k8s_secret(ref)` → `(actual_value, masked_value, sha256)` 三元組
|
||||
- ref 格式:`"namespace/secret-name#key"`,正則強驗
|
||||
- prod:kubernetes_asyncio in-cluster API
|
||||
- dev fallback:`AWOOOP_DEV_SECRETS_JSON` 環境變數(JSON dict)
|
||||
- actual_value 只在記憶體短暫存在,不寫任何持久化;masked_value(前4+***+後4)供 log
|
||||
|
||||
`tests/test_mcp_credential_isolation.py`(10 個測試全部通過 ✅):
|
||||
- bad ref 格式拒絕(5 個 case)
|
||||
- dev fallback 正確解析 / 找不到 key 拋錯
|
||||
- PG DSN / Telegram token / 內網 IP 在 output 被 redact(secret leak 迴歸測試)
|
||||
- _mcp_audit 在 input 被移除 / k8s_value credential isolation
|
||||
- name mangling:_provider 不可存取,_AuditedMCPToolProvider__provider 正確存在
|
||||
|
||||
---
|
||||
|
||||
## 2026-05-04 | AwoooP Phase 2 完收(P1-16/P1-17/2.3/2.4/2.6)
|
||||
|
||||
### P1-16 nl_gateway.py hermes Redis key 加 project 前綴
|
||||
- `_check_rate_limit`: `hermes:rl:{chat_id}` → `{project_id}:hermes:rl:{chat_id}`
|
||||
- `_load_session_context`: 讀新 key,Phase A fallback 到舊 key
|
||||
- `_save_session_turn`: 寫新 key + Phase A dual-write 舊 key
|
||||
- `process_nl_message`: 加 `project_id: str = "awoooi"` 並透傳
|
||||
|
||||
### P1-17 anomaly_counter.py per-project 改造
|
||||
- `__init__` 加 `project_id="awoooi"`,新增 `_pkey()` + `_redis_get_with_fallback()` 輔助方法
|
||||
- 所有寫路徑改用 `_pkey()`(timeline / repair_count / history / disposition / permanent_fix / metadata)
|
||||
- 所有讀路徑 Phase A fallback:先讀 `{project_id}:anomaly:*`,不存在才讀 `anomaly:*`
|
||||
- `get_all_disposition_stats` SCAN 先掃新前綴,無資料才 fallback 舊前綴
|
||||
- `get_anomaly_counter()` singleton 傳入 `project_id="awoooi"`
|
||||
|
||||
### Phase 2.3 Repository project_id filter
|
||||
- `db/base.py`: `get_db_context(project_id="awoooi")` 預設帶入 `SELECT set_config('app.project_id', :pid, TRUE)` → 所有現有呼叫端自動設置正確 tenant,RLS 生效
|
||||
- `db/models.py`: 4 個 ORM model(AuditLog / IncidentRecord / KnowledgeEntryRecord / PlaybookRecord)加 `project_id: Mapped[str]`
|
||||
- `incident_repository.py`: `_incident_to_record_data()` 加 `"project_id": getattr(incident, "project_id", "awoooi")`
|
||||
- `playbook_repository.py`: `get_session_factory()` 全部換成 `get_db_context()`;`_pg_upsert` 寫入 `project_id`
|
||||
- `db/base.py init_db()`: 防禦性 ALTER TABLE 四表加 `project_id VARCHAR(64) DEFAULT 'awoooi'` + index
|
||||
|
||||
### Phase 2.4 31 background loop project_id 標記(INV-8)
|
||||
- `core/context.py` 新建:`PROJECT_ID: ContextVar[str]`(default="awoooi")+ `get_current_project_id()`
|
||||
- `main.py lifespan()`: 冒頭 `PROJECT_ID.set("awoooi")`;asyncio.create_task 自動繼承父任務 ContextVar → 31 個 loop 全部標記
|
||||
- `get_db_context()`: 讀 contextvar 作 fallback(明確參數 > contextvar > "awoooi")
|
||||
|
||||
### Phase 2.6 Token Budget Hard Kill(ADR-120)
|
||||
- `migrations/awooop_phase2_budget_ledger_2026-05-04.sql`: `budget_ledger` 表 + RLS + GRANT
|
||||
- `db/models.py`: `BudgetLedgerRecord` ORM(UUID / NUMERIC(10,4) / project_id / run_id)
|
||||
- `services/budget_service.py`: 三層防線完整實作
|
||||
- `check_budget_before_llm_call()`: Layer 3 Emergency Kill → Layer 2 Tenant → Layer 1 Platform
|
||||
- `record_token_usage()`: POST-call accounting(async INSERT budget_ledger + Redis INCRBYFLOAT)
|
||||
- `activate_emergency_kill()` / `deactivate_emergency_kill()`:Admin 管理工具
|
||||
- Ollama 本地模型(deepseek/qwen3)自動 bypass(零費用)
|
||||
- `db/base.py init_db()`: 防禦性 CREATE TABLE IF NOT EXISTS budget_ledger
|
||||
|
||||
### 下一步(Phase 3)
|
||||
- Phase 3: Contract packages & validators(JSON Schema、Pydantic v2 contract models、contract lifecycle service)
|
||||
|
||||
---
|
||||
|
||||
## 2026-05-04 | AwoooP Phase 4 完收(Platform Shell in Shadow Mode)
|
||||
|
||||
### Phase 4.1 DB Migration(awooop_phase4_run_state_2026-05-04.sql)
|
||||
|
||||
三表 + 全部 RLS + GRANT:
|
||||
- `awooop_run_state` — Run FSM 主表(state enum 8 值、lease_until/heartbeat_at/worker_id SKIP LOCKED 欄位、is_shadow bool)
|
||||
- `awooop_run_step_journal` — SAGA step journal(step_seq unique per run、compensation_json JSONB、was_blocked 攔截記錄)
|
||||
- `awooop_run_idempotency` — 去重冪等(`uix_run_idempotency_key` = project_id + channel_type + provider_event_id)
|
||||
- 全部 `FORCE ROW LEVEL SECURITY`(ADR-118)
|
||||
|
||||
ORM:`awooop_models.py` 新增 `AwoooPRunState` / `AwoooPRunStepJournal` / `AwoooPRunIdempotency`(含 CheckConstraint + 4 個 partial index)
|
||||
|
||||
### Phase 4.2 Run State Machine(run_state_machine.py)
|
||||
|
||||
- `validate_transition(from, to)` — 8 狀態 × 合法轉換表,非法拋 `InvalidStateTransitionError`
|
||||
- `acquire_pending_run()` — `SELECT ... FOR UPDATE SKIP LOCKED`(多 worker 並行安全)
|
||||
- `heartbeat(run_id)` — 延長 lease TTL(每 15 秒,防 stale reaper 誤殺)
|
||||
- `transition(run_id, to_state, ...)` — 先讀 current state 驗合法性,再 UPDATE;terminal state 清 lease + 寫 completed_at
|
||||
- `reap_stale_runs()` — 掃 lease < NOW() 的 RUNNING:attempt < max → PENDING retry;attempt >= max → FAILED(E-RUN-002)
|
||||
|
||||
### Phase 4.3 Platform Runtime(platform_runtime.py)
|
||||
|
||||
- `_uuid7()` — 時間有序 UUID v7(適合 DB PK)
|
||||
- `_new_trace_id()` — W3C traceparent-compatible trace_id(128-bit trace + 64-bit span)
|
||||
- `check_idempotency()` — Redis NX 先攔(快)+ PG constraint 最後防(準確)
|
||||
- `create_run()` — 冪等建立 run,is_shadow=True,計算 input_sha256
|
||||
- `shadow_execute()` — 解析 agent contract → 記錄每個 tool → 攔截 is_destructive → COMPLETED(無 user response)
|
||||
- `is_destructive_tool()` — contract flag + keyword 雙層判斷(delete/drop/kill/exec 等 16 個關鍵字)
|
||||
|
||||
### Phase 4.4 Audit Sink(audit_sink.py)
|
||||
|
||||
PII/secret redaction pipeline(9 個 pattern):
|
||||
- Telegram token(8-12 位數字:32-64 位英數)✅
|
||||
- PostgreSQL DSN / password field
|
||||
- Bearer token / JWT(eyJ… 三段)
|
||||
- GCP/內網 IP(10.x, 172.16-31.x, 192.168.x)
|
||||
- SSH private key / API key
|
||||
- Hex secret ≥ 64 位
|
||||
- field 名稱黑名單(password/token/secret 等直接替換)
|
||||
- LLM raw 欄位(prompt/completion 只保留 sha256 hash 前 16 位)
|
||||
- 自驗測試:9 個 case 全部通過(all_ok: True)
|
||||
|
||||
### Phase 4.5 Platform API(api/v1/platform/runs.py)
|
||||
|
||||
- `POST /v1/platform/runs` — 建立 shadow run,202 Accepted,返回 `{run_id, is_duplicate, is_shadow, message}`
|
||||
- `GET /v1/platform/runs/{run_id}` — 查詢 run FSM 狀態
|
||||
- Idempotency 內建:provider_event_id + channel_type → 冪等命中返回既有 run_id
|
||||
- 所有 Phase 4 run 強制 is_shadow=True
|
||||
|
||||
### Phase 4.6 Platform Worker(workers/platform_worker.py)
|
||||
|
||||
- `PlatformWorker.run_loop()` — SKIP LOCKED 取 PENDING run,控制並行度(max 2),每 run 獨立 task
|
||||
- `PlatformWorker._execute_with_heartbeat()` — shadow_execute + 並行 heartbeat task 防 stale 誤殺
|
||||
- `PlatformWorker.reaper_loop()` — 每 60 秒 reap_stale_runs()
|
||||
- `start_platform_worker()` / `stop_platform_worker()` — lifespan hook
|
||||
|
||||
### main.py 掛載
|
||||
|
||||
- Import:`from src.api.v1 import platform as platform_v1`
|
||||
- Router:`app.include_router(platform_v1.router, prefix="/api/v1/platform")`
|
||||
- Lifespan startup:`await start_platform_worker()`
|
||||
- Lifespan shutdown:`await stop_platform_worker()`
|
||||
|
||||
### 語法驗證
|
||||
|
||||
8 個新建/修改 Python 檔案全部通過 `ast.parse()` ✅
|
||||
|
||||
---
|
||||
|
||||
## 2026-05-04 | AwoooP Phase 3 完收(Contract Packages & Validators)
|
||||
|
||||
### Phase 3.1 packages/awooop-contracts/(六合約 JSON Schema + golden fixtures)
|
||||
|
||||
新建 `packages/awooop-contracts/` 目錄,完整包含:
|
||||
|
||||
**六合約 JSON Schema**(`schemas/`):
|
||||
- `project_tenant.json` — 租戶/專案能力邊界(migration_mode enum、budget_limit_usd ge:0、allowed_channels uniqueItems)
|
||||
- `agent.json` — Agent 模型/工具/治理(sha256 pattern ^[0-9a-f]{64}$、temperature [0,2]、approval_timeout_seconds)
|
||||
- `mcp_gateway.json` — MCP Gateway(transport enum + if/then endpoint required、schema_sha256 完整性)
|
||||
- `policy_routing.json` — LLM 路由規則(routing_rules minItems:1、priority [0,9999]、retry_policy)
|
||||
- `runtime_run_state.json` — Run FSM(UUID pattern、state enum、input/output sha256、cost_usd ge:0)
|
||||
- `channel_event.json` — Channel Event 冪等(event_id UUID、payload minProperties:1、attachment sha256)
|
||||
|
||||
**Golden fixtures**(`fixtures/valid/` + `fixtures/invalid/`):
|
||||
- valid × 6 — 所有 Pydantic 驗證全通過 ✅
|
||||
- invalid × 6 — 各自包含 required 缺失/enum 不合法/format 錯誤,全數被拒絕 ✅
|
||||
- 自驗測試:`python3 -c validate_contract_body(...)` 通過(valid all pass: True, invalid all rejected: True)
|
||||
|
||||
### Phase 3.2 Contract lifecycle service
|
||||
|
||||
- **`src/models/awooop_contracts.py`**(新建):六合約 Pydantic v2 model
|
||||
- `ProjectTenantContract` / `AgentContract` / `MCPGatewayContract` / `PolicyRoutingContract` / `RuntimeRunStateContract` / `ChannelEventContract`
|
||||
- `ArtifactRef`(sha256 hex64 validator)、`ToolRef`、`ToolExposed`、`RoutingRule`、`AttachmentRef` 等共用子 model
|
||||
- `validate_contract_body(family, body)` — dispatcher,依 family 名稱驗證
|
||||
- `CONTRACT_FAMILY_MODELS` dict — 六合約映射表
|
||||
|
||||
- **`src/repositories/contract_repository.py`**(新建):append-only contract CRUD
|
||||
- `get_revision()` / `get_active_revision()` / `list_revisions()` — 讀取(RLS 透過 get_db_context 自動套用)
|
||||
- `create_draft()` — 建立 lifecycle_status='draft' revision
|
||||
- `mark_published()` — draft → published(HMAC 簽章後才能呼叫)
|
||||
- `mark_active()` — published → active(UPSERT active_pointer + 寫 outbox + revoke 舊版本,同一 transaction)
|
||||
|
||||
- **`src/services/contract_service.py`**(新建):完整 lifecycle orchestration
|
||||
- `draft()` — schema 驗證 + body_hash 計算(sha256 canonical JSON)+ DB 寫入 + audit log
|
||||
- `publish()` — HMAC 簽章驗證(settings.CONTRACT_HMAC_KEY)→ mark_published
|
||||
- `activate()` — Redis multi_sig approval 確認 → mark_active(bypass_approval 開關)
|
||||
- `get_active()` — runtime 唯一讀取路徑(active only + body_hash 完整性驗證)
|
||||
- `get_active_body()` — 便利方法,直接返回 body_json
|
||||
- `record_activation_approval()` — 記錄 approver 簽核(Redis TTL 24h)
|
||||
- 5 個自訂 Exception:ContractSchemaError / ContractSignatureError / ContractStateError / ContractApprovalError / ContractNotFoundError
|
||||
|
||||
### Phase 3.3 Output schema validator middleware
|
||||
|
||||
- **`src/services/schema_validator.py`**(新建):LLM 輸出驗證鏈
|
||||
- `extract_json_from_llm_output()` — 三策略容錯萃取(直接 parse / ```json``` block / regex {…})
|
||||
- `validate_llm_output()` — 主驗證器:驗證失敗 → retry prompt → 再試(上限 3 次)→ SchemaValidationError(E-SCHEMA-001)
|
||||
- `validate_llm_output_by_family()` — 依 contract_family 自動選 model
|
||||
- `validate_once()` — 單次驗證(測試 / 內部資料用)
|
||||
- `build_retry_prompt()` — 附錯誤回饋的 retry prompt builder
|
||||
- `SchemaValidationError` — error_code="E-SCHEMA-001"
|
||||
|
||||
### Phase 3 DoD 驗收
|
||||
- [x] schema 不符的 LLM 輸出無法到達 channel adapter(SchemaValidationError 阻擋)
|
||||
- [x] valid × 6 全部通過 Pydantic 驗證
|
||||
- [x] invalid × 6 全數被拒絕(涵蓋 required/enum/format/pattern 四類錯誤)
|
||||
- [x] prompt/schema ref 必含 sha256(ArtifactRef + ToolExposed.schema_sha256 + AttachmentRef.sha256)
|
||||
- [x] body_hash = sha256(canonical JSON),runtime get_active() 讀取時重算驗證
|
||||
|
||||
### 語法驗證
|
||||
- 4 個新建 Python 檔案全部通過 `ast.parse()` ✅
|
||||
|
||||
---
|
||||
|
||||
## 2026-05-04 | AwoooP Phase 2 初批 P0 修正 + Phase 1.7 Tests(commit 14bf86a4)
|
||||
|
||||
### 修正
|
||||
|
||||
1271
docs/awooop/DETAILED-IMPLEMENTATION-PLAN.md
Normal file
1271
docs/awooop/DETAILED-IMPLEMENTATION-PLAN.md
Normal file
File diff suppressed because it is too large
Load Diff
323
docs/awooop/IMPLEMENTATION-ROADMAP.md
Normal file
323
docs/awooop/IMPLEMENTATION-ROADMAP.md
Normal file
@@ -0,0 +1,323 @@
|
||||
# AwoooP Implementation Roadmap
|
||||
|
||||
**Status**: Planning baseline
|
||||
**Date**: 2026-05-01
|
||||
**Owner**: AWOOOI / AwoooP platform workstream
|
||||
**Primary ADRs**: ADR-106, ADR-107
|
||||
|
||||
## Purpose
|
||||
|
||||
This document is the implementation handoff for AwoooP, the AWOOOI Agent
|
||||
Platform.
|
||||
|
||||
ADRs record architectural decisions. This roadmap translates those decisions
|
||||
into execution phases, work items, acceptance gates, and Codex workflow
|
||||
guidance.
|
||||
|
||||
## Executive Summary
|
||||
|
||||
AwoooP is the multi-tenant Agent Platform extracted from the AI capabilities
|
||||
that first emerged inside AWOOOI.
|
||||
|
||||
AWOOOI is the first tenant and first runtime host. It is not the platform
|
||||
boundary.
|
||||
|
||||
AwoooP must let current and future products share AI agents and communication
|
||||
capabilities without copying project-specific code:
|
||||
|
||||
- AWOOOI: AIOps, SRE automation, K8s/MCP/Telegram workflows
|
||||
- EwoooC / MOMO PRO: ecommerce and business analysis, market intelligence, LINE
|
||||
and Telegram workflows
|
||||
- Tsenyang: existing Telegram webhook surface
|
||||
- Bitan and future products: repeatable onboarding to shared AI capabilities
|
||||
|
||||
The approved direction is not a pure HTTP hub, not only an SDK, and not only a
|
||||
shared config table. The approved direction is:
|
||||
|
||||
```text
|
||||
AwoooP = Control Plane + Agent Runtime + MCP Gateway + Context Firewall
|
||||
+ Communication Hub + Observability / Audit
|
||||
```
|
||||
|
||||
## Core Decisions
|
||||
|
||||
### Product and Layout
|
||||
|
||||
- Product name: `AwoooP`
|
||||
- Repository/package slug: `awooop`
|
||||
- Existing AWOOOI tenant id remains `awoooi`
|
||||
- Do not create empty runtime or package directories before implementation owns
|
||||
real code.
|
||||
- `docs/awooop/` is allowed now because it contains concrete implementation
|
||||
planning artifacts.
|
||||
|
||||
Future code layout should be created only when each path owns real artifacts:
|
||||
|
||||
| Path | Create when | Owns |
|
||||
|---|---|---|
|
||||
| `packages/awooop-contracts/` | Contract schemas are ready to validate | JSON Schema / Pydantic / TS types |
|
||||
| `packages/awooop-client/` | First downstream client integration begins | Python/TS SDK |
|
||||
| `apps/awooop-runtime/` | Runtime shell separates from `apps/api` | Platform API and run orchestration |
|
||||
| `apps/awooop-worker/` | Async workers are needed | Run state execution workers |
|
||||
| `docs/awooop/` | Planning and implementation docs exist | Roadmaps, schema notes, runbooks |
|
||||
|
||||
### Storage
|
||||
|
||||
AwoooP v1 is PostgreSQL-first.
|
||||
|
||||
- PostgreSQL is source of truth for contracts, active revisions, tenant records,
|
||||
policy, MCP grants, budget, ACL, run state, approval, channel event, and audit.
|
||||
- Redis is cache/watch/counter/coordination only.
|
||||
- Prompt, JSON Schema, eval, and replay artifacts are stored by ref + SHA-256
|
||||
hash.
|
||||
- Kubernetes CRDs are future runtime projection only, not the v1 control-plane
|
||||
source of truth.
|
||||
|
||||
### Six Contracts
|
||||
|
||||
| Contract | Purpose |
|
||||
|---|---|
|
||||
| Project / Tenant | Tenant identity, data boundary, budget, ACL, channels, approval gates |
|
||||
| Agent | Versioned capability module, I/O schema, context domain, safety ceiling |
|
||||
| MCP Gateway | Tool authorization, credential resolution, approval, result sanitization |
|
||||
| Policy / Routing | Effective model/provider route, fallback, privacy and budget gates |
|
||||
| Runtime / Run State | Durable async run lifecycle, shadow/canary/active, checkpoint/resume |
|
||||
| Communication / Channel Event | Telegram/LINE/Slack/Email/API receive, verify, normalize, send |
|
||||
|
||||
## Migration Strategy
|
||||
|
||||
AwoooP must migrate by strangler fig, not big-bang replacement:
|
||||
|
||||
1. `shadow`: mirror events, write audit/trace only, no user response, no side
|
||||
effects.
|
||||
2. `canary`: selected low-risk user-visible responses, no side effects by
|
||||
default.
|
||||
3. `read_only`: read-only queries and business chat move first.
|
||||
4. `suggest`: analysis and recommendations move next; approval still external.
|
||||
5. `auto_remediate`: write/execute tools move only after Gateway, approval,
|
||||
replay, audit, and rollback evidence are green.
|
||||
|
||||
## Implementation Phases
|
||||
|
||||
### Phase 0 - Documentation Freeze
|
||||
|
||||
Goal: make future implementation unambiguous.
|
||||
|
||||
Status: in progress / mostly complete.
|
||||
|
||||
Work items:
|
||||
|
||||
- ADR-106: AwoooP architecture and migration strategy.
|
||||
- ADR-107: AwoooP control-plane storage strategy.
|
||||
- This roadmap.
|
||||
- LOGBOOK entries for ADR-106/107 and roadmap.
|
||||
- Task-routing lookup in `docs/12-agent-game-rules.md`.
|
||||
|
||||
Acceptance:
|
||||
|
||||
- `git diff --check` passes.
|
||||
- No runtime code changes.
|
||||
- No empty code directories.
|
||||
|
||||
### Phase 1 - Control Plane Schema Foundation
|
||||
|
||||
Goal: create the minimum PostgreSQL foundation for contract materialization.
|
||||
|
||||
Suggested work items:
|
||||
|
||||
- Add `awooop_contract_revisions`.
|
||||
- Add active revision pointer tables.
|
||||
- Add `awooop_projects` / tenant records.
|
||||
- Add `awooop_artifact_refs`.
|
||||
- Add `awooop_project_migration_state`.
|
||||
- Add `project_id` strategy for existing AWOOOI records.
|
||||
- Add tests for immutable published revisions and draft isolation.
|
||||
|
||||
Acceptance:
|
||||
|
||||
- Published contract revision is append-only.
|
||||
- Runtime-facing reads cannot see mutable drafts.
|
||||
- Every active revision has `revision_id` and `body_hash`.
|
||||
- AWOOOI can be represented as `project_id=awoooi` without behavior changes.
|
||||
|
||||
### Phase 2 - Isolation and Namespace Hardening
|
||||
|
||||
Goal: make AWOOOI tenant-safe before sharing anything downstream.
|
||||
|
||||
Suggested work items:
|
||||
|
||||
- Prefix Redis sessions, budget, rate, and Telegram/Hermes keys with
|
||||
`project_id`.
|
||||
- Add `project_id` columns where required for logs, audit, sessions, and
|
||||
approval records.
|
||||
- Define `platform_resource` exceptions separately from tenant resources.
|
||||
- Convert global resources such as Ollama failover state into explicit platform
|
||||
resources.
|
||||
- Add regression tests for cross-project read/write rejection.
|
||||
|
||||
Acceptance:
|
||||
|
||||
- Tenant-scoped data cannot be read without matching `project_id`.
|
||||
- Platform resources are explicitly allowed and audited.
|
||||
- No prompt text is trusted as the data isolation boundary.
|
||||
|
||||
### Phase 3 - Contract Packages and Validators
|
||||
|
||||
Goal: make the six contracts executable rather than prose-only.
|
||||
|
||||
Suggested work items:
|
||||
|
||||
- Create `packages/awooop-contracts/` only when validators are implemented.
|
||||
- Define JSON Schema/Pydantic models for the six contracts.
|
||||
- Define envelope schemas for platform invocation, MCP tool call, run state, and
|
||||
channel events.
|
||||
- Add output schema validator middleware design.
|
||||
- Add golden fixtures for valid and invalid contracts.
|
||||
|
||||
Acceptance:
|
||||
|
||||
- Contract validation fails closed.
|
||||
- LLM output that fails schema validation cannot reach channel adapters.
|
||||
- Prompt/schema refs require SHA-256 hashes.
|
||||
|
||||
### Phase 4 - Platform Shell in Shadow Mode
|
||||
|
||||
Goal: build the first runtime shell without replacing legacy behavior.
|
||||
|
||||
Suggested work items:
|
||||
|
||||
- Add `/v1/platform/runs` shell around existing AWOOOI logic.
|
||||
- Generate `run_id` and `trace_id`.
|
||||
- Resolve project and agent contract revisions.
|
||||
- Resolve `EffectivePolicy` without changing provider behavior.
|
||||
- Write run state transitions and audit.
|
||||
- Mirror selected AWOOOI events into AwoooP shadow mode.
|
||||
|
||||
Acceptance:
|
||||
|
||||
- Shadow runs produce audit and trace.
|
||||
- Shadow runs never send user-visible responses.
|
||||
- Shadow runs never call write/execute/destructive tools.
|
||||
- Legacy AWOOOI behavior remains unchanged.
|
||||
|
||||
### Phase 5 - MCP Gateway First Slice
|
||||
|
||||
Goal: move tool authorization behind a real Gateway.
|
||||
|
||||
Suggested work items:
|
||||
|
||||
- Define MCP Gateway tables for tool registry, grants, credentials refs, and
|
||||
audit.
|
||||
- Wrap current read-only MCP tools behind Gateway checks.
|
||||
- Enforce `Project AND Agent AND Tool AND Environment AND Approval`.
|
||||
- Sanitize tool results before model context.
|
||||
- Record tool call `trace_id`, `run_id`, credential ref, latency, result status.
|
||||
|
||||
Acceptance:
|
||||
|
||||
- Agents never see raw credentials.
|
||||
- Read-only tools are auditable by `trace_id`.
|
||||
- Write/execute tools stay blocked until later phases.
|
||||
|
||||
### Phase 6 - EwoooC Read-Only Tenant Onboarding
|
||||
|
||||
Goal: validate AwoooP with a real downstream tenant without high-risk actions.
|
||||
|
||||
Suggested work items:
|
||||
|
||||
- Create `project_id=ewoooc`.
|
||||
- Register `openclaw-biz` as a specialized agent contract.
|
||||
- Register market-intelligence tools as read-only MCP Gateway tools.
|
||||
- Add a read-only platform client path.
|
||||
- Mirror EwoooC bot/business-analysis events into shadow/canary.
|
||||
|
||||
Acceptance:
|
||||
|
||||
- EwoooC can run read-only AwoooP shadow/canary without touching AWOOOI data.
|
||||
- Business-agent context cannot access infra-only AWOOOI context.
|
||||
- Budget and audit are project-scoped.
|
||||
|
||||
### Phase 7 - Communication Hub Increment
|
||||
|
||||
Goal: standardize channels without breaking existing bots.
|
||||
|
||||
Suggested work items:
|
||||
|
||||
- Define `ConversationEvent` and `OutboundMessage` tables.
|
||||
- Mirror Telegram inbound events first.
|
||||
- Add progressive status feedback policy for async runs.
|
||||
- Keep existing bot handlers authoritative until canary passes.
|
||||
- Add adapter-level escaping, redaction, idempotency, and delivery audit.
|
||||
|
||||
Acceptance:
|
||||
|
||||
- Channel adapters do not call LLM or MCP.
|
||||
- Async runs can produce progress updates.
|
||||
- Duplicate channel retries do not create duplicate runs.
|
||||
|
||||
### Phase 8 - Suggest and Controlled Write Paths
|
||||
|
||||
Goal: graduate from read-only to proposal and then controlled execution.
|
||||
|
||||
Suggested work items:
|
||||
|
||||
- Enable `suggest` mode for selected AWOOI SRE flows.
|
||||
- Add approval resume for `WAITING_APPROVAL`.
|
||||
- Add dry-run and rollback evidence gates.
|
||||
- Move low-risk write paths only after Gateway and audit evidence are stable.
|
||||
|
||||
Acceptance:
|
||||
|
||||
- `WAITING_APPROVAL` resumes without replaying the whole task.
|
||||
- Rejected or expired approvals fail with structured failure codes.
|
||||
- Write/execute paths are blocked by default and feature-flagged.
|
||||
|
||||
## Codex Workflow Recommendation
|
||||
|
||||
Use a new Codex conversation for implementation, but keep the working directory
|
||||
at the monorepo root:
|
||||
|
||||
```text
|
||||
/Users/ogt/awoooi
|
||||
```
|
||||
|
||||
Do not start from a new project directory until `apps/awooop-runtime` or
|
||||
`packages/awooop-*` actually exists and owns code.
|
||||
|
||||
Recommended implementation kickoff prompt:
|
||||
|
||||
```text
|
||||
Read AGENTS.md, docs/12-agent-game-rules.md, docs/LOGBOOK.md newest entry,
|
||||
docs/adr/ADR-106-agent-platform-architecture.md,
|
||||
docs/adr/ADR-107-awooop-control-plane-storage.md, and
|
||||
docs/awooop/IMPLEMENTATION-ROADMAP.md.
|
||||
|
||||
Start Phase 1 only: design and implement the minimum PostgreSQL control-plane
|
||||
schema foundation for AwoooP contract revisions. Do not create runtime APIs,
|
||||
do not change provider behavior, do not move Telegram/LINE webhooks, and do not
|
||||
create empty AwoooP code directories.
|
||||
```
|
||||
|
||||
If the current worktree is dirty, prefer a clean branch or clean worktree before
|
||||
runtime implementation. Documentation-only planning can remain in this thread.
|
||||
|
||||
## First Implementation Slice
|
||||
|
||||
The first code slice should be deliberately boring:
|
||||
|
||||
1. DB migration for contract revisions and active pointers.
|
||||
2. Pydantic/SQLAlchemy models for contract revisions.
|
||||
3. Repository/service methods for draft, publish, activate, and read active.
|
||||
4. Tests for immutability and draft isolation.
|
||||
5. LOGBOOK update.
|
||||
|
||||
Do not start with MCP Gateway, Telegram Hub, or model routing. Those are more
|
||||
visible, but they depend on the contract source of truth.
|
||||
|
||||
## References
|
||||
|
||||
- `docs/adr/ADR-106-agent-platform-architecture.md`
|
||||
- `docs/adr/ADR-107-awooop-control-plane-storage.md`
|
||||
- `docs/adr/ADR-105-mcp-agent-loop-governance.md`
|
||||
- `docs/12-agent-game-rules.md`
|
||||
- `docs/LOGBOOK.md`
|
||||
411
docs/awooop/MASTER-WORKPLAN.md
Normal file
411
docs/awooop/MASTER-WORKPLAN.md
Normal file
@@ -0,0 +1,411 @@
|
||||
# AwoooP Master Workplan(P0 防爆版)
|
||||
|
||||
**狀態**:規劃凍結基準
|
||||
**日期**:2026-05-03
|
||||
**主要 ADR**:ADR-106(架構)、ADR-107(控制面儲存)
|
||||
**取代**:本檔取代 `IMPLEMENTATION-ROADMAP.md` 作為 AwoooP 主索引;舊 roadmap 仍保留為一階草稿,僅供歷史對照
|
||||
|
||||
---
|
||||
|
||||
## 0. 為什麼有這份文件
|
||||
|
||||
12 位 Agent(critic / vuln-verifier / debugger / db-expert / planner / fullstack-engineer / refactor-specialist / migration-engineer / onboarder / tool-expert / web-researcher / frontend-designer)對舊版 Plan 1 與 ADR-106 做完獨立審查後,發現至少 12 個 P0 問題;後續再補了 12 個會在實作後咬人的設計缺口。
|
||||
|
||||
結論:**直接進 Phase 1 SQL migration 會立刻爆。** 必須先補足 5 份 ADR、4 份 Inventory,把 Strangler Fig 的「資料載體、雙寫遷移、邊界硬攔截、可重放、可審計」全部寫死,再下 code。
|
||||
|
||||
---
|
||||
|
||||
## 1. 共識:實作前必須先完成的修補
|
||||
|
||||
| # | 問題 | 風險等級 | 必補在 |
|
||||
|---|------|---------|--------|
|
||||
| 1 | Redis key 直接改名無雙寫期 → 費用計數歸零、Telegram 409、silence 失效、Ollama failover 雙寫不到 | 🔴🔴🔴 | Phase 2 之前 |
|
||||
| 2 | Migration SQL 表名錯(`incident_records` / `mcp_audit_snapshots`)、無 rollback、ORM 1.x vs 2.x | 🔴🔴🔴 | Phase 1 重寫 |
|
||||
| 3 | `project_id` / `tenant_id` 在 codebase 0 命中,30+ 業務表無此欄 | 🔴🔴🔴 | Phase 1 + Phase 2 |
|
||||
| 4 | `project_migration_state` 表缺失,Strangler Fig 無資料載體 | 🔴🔴 | Phase 1 |
|
||||
| 5 | AWOOOI 雙重身份 Bootstrap Paradox(cron/job/healthcheck 全無 `project_id`) | 🔴🔴 | Phase 0 補 ADR-111 |
|
||||
| 6 | EwoooC 接入零技術路徑,需要 Provider Proxy Adapter 設計 | 🔴🔴 | Phase 0 補 ADR-115 |
|
||||
| 7 | Strangler shadow→canary→active 無量化 gate 條件 | 🔴🔴 | Phase 0 寫進 ADR-106 補章 |
|
||||
| 8 | Layer 3 redaction 零實作(helper 有但無 enforcement point) | 🔴🔴🔴 | Phase 5 |
|
||||
| 9 | `_provider` 屬性是 public,可繞過 audit | 🔴🔴 | Phase 5 |
|
||||
| 10 | `WAITING_APPROVAL` resume 不驗 caller identity,無 approval_token 簽章 | 🔴🔴 | Phase 4 + Phase 8 |
|
||||
| 11 | Redis approval state 單點,無 PG sync | 🔴 | Phase 2 + Phase 8 |
|
||||
| 12 | Task 9(K8s ConfigMap)順序倒置,agent prompt 全回 None | 🔴🔴🔴 | Phase 1 之前先順序修正 |
|
||||
| 13 | Audit log 本身會洩密,redaction 必須做在 audit sink 前 | 🔴🔴 | Phase 5 |
|
||||
| 14 | `sanitization_service.py` 已存在 helper,但 MCP Gateway / AgentToolExecutor 沒強制使用 | 🔴🔴 | Phase 5 |
|
||||
| 15 | Redis working memory(`SCAN incident:*`)需要 project 邊界 | 🔴🔴 | Phase 2 |
|
||||
| 16 | Contract publish 權限與簽章未定義(誰可 publish/activate) | 🔴 | Phase 0 補 ADR-112 |
|
||||
| 17 | Active revision 切換無 transactional outbox,worker 可能吃舊 policy | 🔴 | Phase 0 補 ADR-113 |
|
||||
| 18 | Run/Channel idempotency 缺 key derivation 規則與 unique index | 🔴 | Phase 0 補 ADR-114 |
|
||||
| 19 | Async worker 缺 lease / heartbeat / stale reaper | 🔴 | Phase 4 |
|
||||
| 20 | 高流量表(`run_state` / `channel_event` / `mcp_audit` / `agent_audit`)partition 與 retention 需 Phase 1 就決定 | 🟠 | Phase 1 |
|
||||
| 21 | Observability metrics label cardinality 規則:`run_id`/`trace_id`/`session_id` 禁止進 metrics | 🟠 | Phase 4 |
|
||||
| 22 | Telegram/LINE/Slack/API/Internal 缺 canonical principal mapping | 🟠 | Phase 0 補 ADR-115 |
|
||||
| 23 | EwoooC Provider Proxy 不能只改 `OLLAMA_API_BASE`,必須補 envelope + audit 入口 | 🔴 | Phase 0 補 ADR-115 |
|
||||
| 24 | 所有 entrypoint(cron / job / webhook / CLI / healthcheck)需 inventory 並標 `requires_project_id` | 🔴 | Phase 0 Inventory |
|
||||
|
||||
---
|
||||
|
||||
## 2. Pre-flight Audit(Phase 0 擴張)
|
||||
|
||||
舊版 Phase 0 只凍 ADR-106/107。新版 Phase 0 還需要 **5 份 ADR + 4 份 Inventory**,全部 docs-only。
|
||||
|
||||
### 2.1 5 份必補 ADR(⚠️ ADR-108/109/110 已被其他 ADR 占用,AwoooP 從 ADR-111 開始)
|
||||
|
||||
| ADR | 主題 | 解決 |
|
||||
|-----|------|------|
|
||||
| **ADR-111** | AwoooP Bootstrap Order & Identity Paradox | #5、#24 — 定義 hard reject 啟用順序、`platform_internal` / `requires_project_id` / `legacy_awoooi_default` 標記、AWOOOI cron/job 過渡期豁免規則 |
|
||||
| **ADR-112** | Contract Governance & Publishing Workflow | #16 — 誰可 publish、誰可 activate、CODEOWNERS、簽章/HMAC、approval workflow、activation audit |
|
||||
| **ADR-113** | Active Revision Invalidation & Outbox | #17 — `awooop_contract_outbox` 表設計、Redis pub/sub 通知、worker revision-aware cache、split-brain 防禦 |
|
||||
| **ADR-114** | Idempotency, Worker Lease & Run Recovery | #18、#19 — channel event dedupe、`(project_id, channel_type, provider_event_id)` unique、worker `lease_until` / `heartbeat_at` / `attempt_count`、stale run reaper、SKIP LOCKED |
|
||||
| **ADR-115** | Canonical Principal Mapping & Tenant Onboarding Patterns | #6、#22、#23 — Telegram/LINE/Slack/API/Internal user → `platform_subject` 統一映射、EwoooC Provider Proxy Adapter 設計、Tsenyang/Bitan 接入模式範本 |
|
||||
|
||||
ADR-106 也需要補一節:**Strangler Fig Quantified Gates**,把 shadow → canary → active 的量化條件寫死(≥14 天、決策差異率 <5%、p95 退化 <10%、無 P1 incident、cost diff < 預算上限 50%)。
|
||||
|
||||
### 2.2 4 份必做 Inventory
|
||||
|
||||
| Inventory | 範圍 | 解決 |
|
||||
|-----------|------|------|
|
||||
| **INV-1:Redis Key Inventory** | 全 codebase grep `redis_client.*\(["']` + `r\.set/get/scan` → 列出 43+ 個 key、命名空間、TTL、用途、寫入點、讀取點、是否硬碼跨檔 | #1、#15 |
|
||||
| **INV-2:Repository Project-id Retrofit Map** | 列出全部 30+ 張業務表、目前有無 `project_id` 欄位、所有 repository 方法、需加 filter 的查詢、需 backfill 的歷史資料 | #3 |
|
||||
| **INV-3:Entrypoint Inventory** | 列出所有 cron job / scheduler / webhook / CLI script / healthcheck / internal service call,標記 `requires_project_id` / `platform_internal` / `legacy_awoooi_default` | #5、#24 |
|
||||
| **INV-4:Hardcoded Namespace & IP Inventory** | 列出所有硬碼 K8s namespace(`awoooi-prod`)、SSH 主機 IP、白名單,標記 tenant-scope 改造方案 | 配合 #3 完成多租戶啟用 |
|
||||
|
||||
### Phase 0 驗收
|
||||
|
||||
- ADR-111~115 全部 Accepted 並進 LOGBOOK
|
||||
- ADR-106 補 Quantified Gates 章節
|
||||
- 4 份 Inventory 寫入 `docs/awooop/inventory/`
|
||||
- 沒有任何 runtime code 變動
|
||||
- `git diff --check` 通過
|
||||
|
||||
---
|
||||
|
||||
## 3. 修訂版 8 階段實施計畫
|
||||
|
||||
> 階段順序與舊 roadmap 相同,但每階段範圍依 §1 共識重寫。
|
||||
|
||||
### Phase 1 — Control Plane Schema Foundation(重寫)
|
||||
|
||||
**目標**:建立 PostgreSQL contract control plane 最小可用骨架,並修正舊 SQL migration 三大 blocker。
|
||||
|
||||
工作項:
|
||||
|
||||
1. **核對真實表名**:在寫 SQL 前 grep 確認 `incidents`(非 `incident_records`)、`mcp_audit_log`(非 `mcp_audit_snapshots`),全部錯名修正
|
||||
2. **ORM 同步現況**:使用 SQLAlchemy 2.x `mapped_column`、補齊 `Numeric`/`UniqueConstraint`/`func` import
|
||||
3. **每個 migration 都有 rollback SQL**(down migration 強制)
|
||||
4. **新增 contract control 表**:
|
||||
- `awooop_projects`(tenant 主表,`project_id` PK)
|
||||
- `awooop_contract_revisions`(六合約共用 revision 表,append-only)
|
||||
- `awooop_active_revisions`(active pointer 表)
|
||||
- `awooop_artifact_refs`(prompt/schema/eval 的 ref + sha256)
|
||||
- `awooop_project_migration_state`(Strangler 階段追蹤)
|
||||
- `awooop_contract_outbox`(ADR-113,active revision 切換事件)
|
||||
- `awooop_channel_event_dedupe`(ADR-114,idempotency 唯一鍵)
|
||||
- `awooop_platform_subjects`(ADR-115,canonical principal)
|
||||
5. **高流量表 partition 策略決定(不延後)**:
|
||||
- `awooop_run_state`、`awooop_channel_event`、`awooop_mcp_gateway_audit`、`awooop_agent_audit_log` 一律按月 partition
|
||||
- 每 tenant retention policy(預設 90 天 hot + 1 年 warm)
|
||||
6. **`project_id` 對既有表的策略**:暫不在現有 30+ 業務表加欄位(留給 Phase 2),先在 AwoooP 自己的表強制 `project_id NOT NULL`
|
||||
7. **immutability 測試**:published revision 嘗試 UPDATE 必失敗、draft 與 active 隔離
|
||||
8. **Task 9 順序修正前置**:Dockerfile / ConfigMap / agent_loader 預設路徑改動先於任何 agent prompt 載入點變更
|
||||
|
||||
驗收:
|
||||
- AWOOOI 可被表示為 `project_id=awoooi` 且 0 行為改動
|
||||
- 每個 active revision 都有 `revision_id` 與 `body_hash`
|
||||
- runtime 讀取路徑看不到 mutable draft
|
||||
- migration up/down 都通過 dry-run
|
||||
- partition + retention 寫入 runbook
|
||||
|
||||
### Phase 2 — Tenant Isolation & Namespace Hardening(重寫)
|
||||
|
||||
**目標**:在開放任何下游 tenant 之前,把 AWOOOI 自己變成乾淨的 tenant。
|
||||
|
||||
工作項:
|
||||
|
||||
1. **Redis 三階段雙寫遷移**(依 INV-1):
|
||||
- **階段 A**:dual-write 新舊 key(30 天觀察)
|
||||
- **階段 B**:dual-read,新 key 為主、舊 key 為 fallback(14 天)
|
||||
- **階段 C**:移除舊 key 寫入,留 audit log
|
||||
- **必含**:`ai_rate:total_cost:gemini`(費用上限)、`telegram:polling:leader`(Pod 鎖)、`telegram_silence:{target}`(含 `decision_manager.py:230` 硬碼)、`ollama:current_primary`(含 `ollama_auto_recovery.py:230` 第二定義)
|
||||
2. **Repository project_id 改造(依 INV-2)**:
|
||||
- 所有 30+ repository 方法加 `project_id` filter
|
||||
- K8s namespace 白名單 → tenant-aware 設定(依 INV-4)
|
||||
- SSH 主機白名單 → tenant-aware
|
||||
3. **Redis working memory project 邊界**(#15):
|
||||
- `incident_service.py:603` 的 `SCAN incident:*` → `SCAN {project_id}:incident:*`
|
||||
- 所有 `SCAN`/`KEYS` 必須帶 prefix
|
||||
4. **`platform_resource` 例外名單**:Ollama failover state、global rate limit、leader election lock 等明確標記
|
||||
5. **回歸測試**:cross-project read/write 必拒絕;platform_resource 必允許但寫 audit
|
||||
6. **AWOOOI Bootstrap Paradox 修補**(依 ADR-111、INV-3):
|
||||
- 標記為 `platform_internal` 的 entrypoint 帶 `project_id=__platform__`,hard reject 例外但寫 audit
|
||||
- 標記為 `legacy_awoooi_default` 的舊 cron 暫時 fallback 到 `project_id=awoooi`,列退場時程
|
||||
|
||||
驗收:
|
||||
- INV-1 列出的所有 P0 key 完成三階段遷移
|
||||
- 30+ repository 全部加 `project_id` filter,cross-project test 全紅
|
||||
- 無任何 hardcode tenant 字串殘留(grep `awoooi-prod` / `192.168` 必為 0)
|
||||
|
||||
### Phase 3 — Contract Packages & Validators
|
||||
|
||||
**目標**:六合約從散文升級為可驗證程式。
|
||||
|
||||
工作項:
|
||||
1. 建立 `packages/awooop-contracts/`(此時才建立)
|
||||
2. 六合約 JSON Schema + Pydantic models
|
||||
3. envelope schema:platform invocation、MCP tool call、run state、channel events
|
||||
4. **Output schema validator middleware**:LLM 回傳必先過 schema、失敗 retry 上限硬碼、失敗不外漏到 channel
|
||||
5. golden fixtures(valid + invalid)
|
||||
6. **Contract publish governance**(依 ADR-112):
|
||||
- CODEOWNERS 對 `packages/awooop-contracts/`
|
||||
- publish API 簽章驗證
|
||||
- activate 動作要 approval workflow
|
||||
|
||||
驗收:
|
||||
- 任何 schema 不符的 LLM 輸出無法到達 channel adapter
|
||||
- prompt/schema ref 必含 sha256 hash
|
||||
- 無權限不能 publish 或 activate
|
||||
|
||||
### Phase 4 — Platform Shell in Shadow Mode(補 lease/idempotency/audit redaction)
|
||||
|
||||
**目標**:建立第一個 runtime shell,但只跑 shadow,不改 legacy 行為。
|
||||
|
||||
工作項:
|
||||
|
||||
1. `/v1/platform/runs` API(async)
|
||||
2. `run_id` / `trace_id` 生成(W3C tracecontext-compatible)
|
||||
3. project + agent contract revision 解析
|
||||
4. EffectivePolicy 解析(不改 provider 行為)
|
||||
5. **Run state machine**(依 ADR-114):
|
||||
- `lease_until`、`heartbeat_at`、`attempt_count` 欄位
|
||||
- SKIP LOCKED 取單
|
||||
- stale run reaper(每分鐘掃 expired lease)
|
||||
6. **Idempotency**(依 ADR-114):
|
||||
- `(project_id, channel_type, provider_event_id)` unique
|
||||
- duplicate event return 既有 run
|
||||
7. **Audit log redaction**(#13):
|
||||
- audit sink 寫入前過 `sanitization_service`
|
||||
- PII / secret pattern 硬攔
|
||||
8. **Observability label rules**(#21):
|
||||
- metrics label 限定 `project_id` / `agent_id` / `status` / `provider`
|
||||
- `run_id` / `trace_id` / `session_id` 只進 logs/traces
|
||||
9. mirror 選定 AWOOOI 事件到 shadow
|
||||
|
||||
驗收:
|
||||
- shadow run 永遠 0 user-visible response、0 destructive tool call
|
||||
- legacy AWOOOI 行為 0 改變
|
||||
- worker crash 後 stale run 1 分鐘內被回收
|
||||
- duplicate retry 不產生重複 run
|
||||
- audit log 0 secret 命中(vuln-verifier 抽樣 100 筆)
|
||||
|
||||
### Phase 5 — MCP Gateway First Slice(補 sanitization enforcement、_provider 修正、audit redaction)
|
||||
|
||||
**目標**:把 tool 授權搬到 Gateway,read-only 工具先進。
|
||||
|
||||
工作項:
|
||||
|
||||
1. Gateway 表:tool registry、grants、credential refs、audit
|
||||
2. wrap 既有 read-only MCP tool
|
||||
3. 強制:`Project AND Agent AND Tool AND Environment AND Approval` 五重交集
|
||||
4. **Result sanitization enforcement point**(#8、#14):
|
||||
- 所有 MCP result 必經 `sanitization_service` pipeline
|
||||
- 不允許 raw result 直接進 LLM context
|
||||
5. **`_provider` → `__provider`**(#9):
|
||||
- 雙底線真 private
|
||||
- 加 unit test:外部 reflect 取用 must fail
|
||||
6. **Audit log 雙層 redaction**(#13):
|
||||
- 進 LLM 前一層
|
||||
- 進 audit sink 一層
|
||||
7. tool call 記錄 `trace_id` / `run_id` / credential ref / latency / status
|
||||
|
||||
驗收:
|
||||
- agent 程式碼路徑 0 raw credential 接觸(sast 掃過)
|
||||
- raw result 不可能繞過 sanitization(單元測試 + 整合測試覆蓋)
|
||||
- 2026-04-18 secret leak 重演測試:kubectl describe configmap 輸出不會出現在任何 LLM context 或 audit row
|
||||
|
||||
### Phase 6 — EwoooC Read-Only Tenant Onboarding(依 ADR-115)
|
||||
|
||||
**目標**:以實際下游 tenant 驗證 AwoooP,全 read-only。
|
||||
|
||||
工作項:
|
||||
|
||||
1. 建立 `project_id=ewoooc`
|
||||
2. 註冊 `openclaw-biz` agent contract
|
||||
3. **Provider Proxy Adapter**(#23):
|
||||
- 不只是改 `OLLAMA_API_BASE`
|
||||
- Proxy 入口補 envelope(`project_id` / `agent_id` / `trace_id` / `run_id`)
|
||||
- 經過 EffectivePolicy + budget guard + audit
|
||||
- read-only / model-call 入口優先
|
||||
4. EwoooC 市場情報 tools 註冊為 read-only MCP Gateway tool
|
||||
5. EwoooC bot/business-analysis 事件先 mirror 到 shadow,14 天後升 canary
|
||||
|
||||
驗收:
|
||||
- EwoooC 可跑 read-only AwoooP shadow/canary,0 接觸 AWOOOI 資料
|
||||
- business-agent context 不可讀 infra-only AWOOOI context
|
||||
- budget / audit 完全 project-scoped
|
||||
|
||||
### Phase 7 — Communication Hub Increment(補 progressive feedback)
|
||||
|
||||
**目標**:標準化 channel 但不切斷既有 bot。
|
||||
|
||||
工作項:
|
||||
|
||||
1. `awooop_conversation_event` + `awooop_outbound_message` 表(partition + retention 已在 Phase 1)
|
||||
2. Telegram inbound mirror 先進
|
||||
3. **Progressive Feedback Policy**(async UX gap,#補充):
|
||||
- WAITING_TOOL / RUNNING / WAITING_APPROVAL 必發暫態訊息
|
||||
- 用 Telegram message edit 更新(非新訊息)
|
||||
4. 既有 bot handler 維持權威,直到 canary 通過量化 gate
|
||||
5. adapter-level escaping、redaction、idempotency、delivery audit
|
||||
6. **Canonical principal mapping**(依 ADR-115):所有 channel sender 寫入 `awooop_platform_subjects`
|
||||
|
||||
驗收:
|
||||
- channel adapter 0 LLM 呼叫、0 MCP 呼叫
|
||||
- async run 有進度更新(≤30s 必有第一則)
|
||||
- duplicate retry 不產生 duplicate run(INV-1 + Phase 4 idempotency 已就位)
|
||||
|
||||
### Phase 8 — Suggest & Controlled Write Paths(補 approval token signing)
|
||||
|
||||
**目標**:從 read-only 升級到 propose、再到 controlled execute。
|
||||
|
||||
工作項:
|
||||
|
||||
1. AWOOOI SRE 部分流程啟用 `suggest`
|
||||
2. **Approval resume 安全強化**(#10、#11):
|
||||
- resume API 強制驗 `approval_token`(HMAC-signed)
|
||||
- approval state PG 為 source of truth、Redis 為 cache
|
||||
- approval 過期 / 已決 / 重放都拒絕
|
||||
3. dry-run 與 rollback evidence gate
|
||||
4. write/execute 預設 disabled,feature flag 控制
|
||||
5. **量化 gate(依 ADR-106 補章)**:
|
||||
- shadow → canary:≥14 天 + decision divergence < 5% + p95 退化 < 10%
|
||||
- canary → active:≥7 天 + 0 P1 incident + cost diff < 預算 50%
|
||||
|
||||
驗收:
|
||||
- WAITING_APPROVAL resume 不能在沒 token 的情況下成功(vuln-verifier 寫 PoC)
|
||||
- Redis 宕機時 approval 仍可從 PG 恢復
|
||||
- write/execute 預設 OFF,需顯式 feature flag
|
||||
|
||||
---
|
||||
|
||||
## 4. 跨階段橫向工作項(Cross-Cutting)
|
||||
|
||||
| 工作項 | 跨哪些階段 | 負責 |
|
||||
|--------|-----------|------|
|
||||
| **Bootstrap order discipline**(ADR-111) | Phase 0、2、4、Forever | platform-runtime |
|
||||
| **Audit log redaction(雙層)** | Phase 4、5 | mcp-gateway + runtime |
|
||||
| **High-traffic table partition + retention** | Phase 1、4、7 | db-expert + sre |
|
||||
| **Observability label cardinality 規則** | Phase 4 起永久 | observability |
|
||||
| **Contract outbox / active revision invalidation** | Phase 1 表、Phase 4 worker 用 | platform-runtime |
|
||||
| **Canonical principal mapping** | Phase 0 ADR、Phase 7 落地 | identity |
|
||||
| **Approval token signing** | Phase 4 token 簽發、Phase 8 verify | security + runtime |
|
||||
| **EwoooC Provider Proxy Adapter** | Phase 0 設計、Phase 6 落地 | tenant-onboarding |
|
||||
|
||||
---
|
||||
|
||||
## 5. 工作排序總表(建議施作順序)
|
||||
|
||||
| 順序 | 工作 | 是否 docs-only | 阻擋誰 |
|
||||
|------|------|--------------|-------|
|
||||
| 1 | ADR-111 Bootstrap Order | ✅ | Phase 2 |
|
||||
| 2 | ADR-112 Contract Governance | ✅ | Phase 3 |
|
||||
| 3 | ADR-113 Active Revision Outbox | ✅ | Phase 1 |
|
||||
| 4 | ADR-114 Idempotency & Worker Lease | ✅ | Phase 4 |
|
||||
| 5 | ADR-115 Principal Mapping & EwoooC Proxy | ✅ | Phase 6、7 |
|
||||
| 6 | ADR-106 補 Quantified Gates 章節 | ✅ | Phase 8 |
|
||||
| 7 | INV-1 Redis Key Inventory | ✅ | Phase 2 |
|
||||
| 8 | INV-2 Repository Retrofit Map | ✅ | Phase 2 |
|
||||
| 9 | INV-3 Entrypoint Inventory | ✅ | Phase 2 |
|
||||
| 10 | INV-4 Namespace/IP Inventory | ✅ | Phase 2 |
|
||||
| 11 | Phase 1 schema migration(重寫版) | ❌ runtime | Phase 2-8 |
|
||||
| 12 | Task 9 順序修正(Dockerfile/ConfigMap 先) | ❌ runtime | Phase 1 prompt 任何更動 |
|
||||
| 13 | Phase 2 三階段 Redis 雙寫 + repository project_id | ❌ runtime | Phase 4 起所有 tenant 行為 |
|
||||
| 14 | Phase 3 contract packages | ❌ runtime | Phase 4 起 |
|
||||
| 15 | Phase 4 platform shell + shadow + idempotency + audit redaction | ❌ runtime | Phase 5-8 |
|
||||
| 16 | Phase 5 MCP Gateway + sanitization enforcement + `__provider` | ❌ runtime | Phase 6 read-only tool |
|
||||
| 17 | Phase 6 EwoooC onboarding via Provider Proxy | ❌ runtime | Phase 7 |
|
||||
| 18 | Phase 7 Communication Hub + progressive feedback | ❌ runtime | Phase 8 |
|
||||
| 19 | Phase 8 suggest + approval signing + controlled write | ❌ runtime | 平台 v1 GA |
|
||||
|
||||
**1~10 是 docs-only,可以在當前對話視窗連續做完,全部完成才開新對話進 Phase 1 code。**
|
||||
|
||||
---
|
||||
|
||||
## 6. 量化驗收門檻(Strangler Fig Gates)
|
||||
|
||||
每個 tenant × 每個 capability 切換階段都要過:
|
||||
|
||||
| 切換 | 必要條件 |
|
||||
|------|---------|
|
||||
| pre → shadow | tenant 已建、agent contract published、audit/trace 寫入正常 |
|
||||
| shadow → canary | ≥14 天 shadow 觀察 + decision divergence < 5% + p95 latency 退化 < 10% + 0 P0/P1 incident + audit 0 secret 命中 |
|
||||
| canary → read_only | ≥7 天 canary + user-visible response 錯誤率 < 0.5% + cost diff < 預算 50% |
|
||||
| read_only → suggest | ≥14 天 read_only + agent suggestion accept rate ≥50% + 0 hallucination escalation |
|
||||
| suggest → auto_remediate | ≥30 天 suggest + rollback evidence ≥3 次成功 + approval token signing live + dry-run pass rate ≥99% |
|
||||
|
||||
每個 gate 由 12-Agent critic + db-expert + vuln-verifier 三方簽核,寫進 LOGBOOK。
|
||||
|
||||
---
|
||||
|
||||
## 7. 授權需求清單(已獲統帥完整授權)
|
||||
|
||||
| 類別 | 動作 | 風險 |
|
||||
|------|------|------|
|
||||
| docs-only | 寫 ADR-111~115、4 份 Inventory、ADR-106 補章 | 低 |
|
||||
| schema | 新增 8 張 AwoooP 控制面表(Phase 1) | 中(DB migration) |
|
||||
| schema | 30+ 業務表加 `project_id` 欄位 + backfill(Phase 2) | 高(DB migration、要 db-expert review) |
|
||||
| Redis | 全 codebase 43+ key 三階段雙寫遷移(Phase 2) | 高(影響費用、Telegram、silence、ollama failover) |
|
||||
| code | 30+ repository 加 `project_id` filter(Phase 2) | 中(regression risk) |
|
||||
| code | wrap MCP provider 為 Gateway(Phase 5) | 中 |
|
||||
| infra | partition + retention runbook(Phase 1) | 低 |
|
||||
| infra | K8s ConfigMap 預載 agent prompt(Task 9 提前) | 中 |
|
||||
| security | `__provider` 雙底線、approval token 簽章(Phase 5、8) | 中 |
|
||||
| feature flag | suggest mode、controlled write(Phase 8) | 高(要 vuln-verifier PoC + dry-run evidence) |
|
||||
|
||||
不在本授權範圍:
|
||||
- 提高 paid provider 配額或啟用新雲端 provider(仍須 HARD_RULES feedback_cost_change_approval 流程)
|
||||
- 任何 destructive MCP tool 上線(要 Phase 8 evidence 才開)
|
||||
- Telegram/LINE/Slack webhook 直接切走(必須先 shadow → canary → 量化 gate)
|
||||
|
||||
---
|
||||
|
||||
## 8. Codex 工作模式建議
|
||||
|
||||
| 階段 | 對話 | cwd |
|
||||
|------|------|-----|
|
||||
| 排序 1~10(docs-only) | 當前對話可直接完成 | `/Users/ogt/awoooi` |
|
||||
| 排序 11 起(runtime code) | 新開 Codex 對話 + 乾淨 worktree | `/Users/ogt/awoooi` |
|
||||
|
||||
實作對話 kickoff prompt 模板:
|
||||
|
||||
```
|
||||
讀完:
|
||||
- AGENTS.md
|
||||
- docs/12-agent-game-rules.md
|
||||
- docs/LOGBOOK.md 最新一篇
|
||||
- docs/adr/ADR-106 / 107 / 108 / 109 / 110 / 111 / 112
|
||||
- docs/awooop/MASTER-WORKPLAN.md
|
||||
- docs/awooop/inventory/ INV-1 ~ INV-4
|
||||
|
||||
只做 Phase {N}。不擴張範圍、不改 provider 行為、不切 channel webhook、不建空目錄。
|
||||
完成後輸出 [P7-COMPLETION]。
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 9. 參考
|
||||
|
||||
- `docs/adr/ADR-106-agent-platform-architecture.md`
|
||||
- `docs/adr/ADR-107-awooop-control-plane-storage.md`
|
||||
- `docs/adr/ADR-105-revert-a2-ollama-primary.md`
|
||||
- `docs/adr/ADR-111-awooop-bootstrap-order.md`(待寫)
|
||||
- `docs/adr/ADR-112-awooop-contract-governance.md`(待寫)
|
||||
- `docs/adr/ADR-113-awooop-active-revision-outbox.md`(待寫)
|
||||
- `docs/adr/ADR-114-awooop-idempotency-worker-lease.md`(待寫)
|
||||
- `docs/adr/ADR-115-awooop-principal-mapping-tenant-onboarding.md`(待寫)
|
||||
- `docs/awooop/inventory/INV-1-redis-keys.md`(待寫)
|
||||
- `docs/awooop/inventory/INV-2-repository-project-id-retrofit.md`(待寫)
|
||||
- `docs/awooop/inventory/INV-3-entrypoints.md`(待寫)
|
||||
- `docs/awooop/inventory/INV-4-hardcoded-namespace-ip.md`(待寫)
|
||||
- `docs/12-agent-game-rules.md`
|
||||
- `docs/LOGBOOK.md`
|
||||
104
docs/awooop/inventory/INV-1-redis-keys.md
Normal file
104
docs/awooop/inventory/INV-1-redis-keys.md
Normal file
@@ -0,0 +1,104 @@
|
||||
# INV-1: Redis Key Inventory
|
||||
|
||||
**版本**:v1.0 初稿
|
||||
**日期**:2026-05-03(台北)
|
||||
**範圍**:`apps/api/src/` 全 codebase grep
|
||||
**用途**:Phase 2 三階段雙寫遷移的完整 Redis key 清單
|
||||
|
||||
---
|
||||
|
||||
## 使用方式
|
||||
|
||||
本 INV 列出所有 Redis key pattern、TTL、用途、寫入點、讀取點、是否硬碼跨兩處。
|
||||
Phase 2 遷移按批次執行:P0(Critical)→ P1(高風險)→ P2(業務資料)→ P3(低影響)
|
||||
|
||||
---
|
||||
|
||||
## 1. P0 — Critical(必須優先遷移)
|
||||
|
||||
| Key Pattern | 用途 | TTL | 寫入點 | 讀取點 | 問題 | 新 Pattern(Phase 2 後)|
|
||||
|-------------|------|-----|--------|--------|------|------------------------|
|
||||
| `ollama:current_primary` | Ollama primary URL 選擇 | 無 | `ollama_auto_recovery.py` (兩處!) | `ollama_failover_manager.py` | 🔴 P0-11:第二定義 at ollama_auto_recovery.py:230;三層 GCP 拓撲無法塞入單一 URL | `platform:ollama:topology`(JSON:primary/secondary/fallback)|
|
||||
| `ollama:gemini_daily_count:{date}` | Gemini 每日使用計數 | 24h | `ollama_auto_recovery.py` | same | 🔴 無 project 前綴,多 tenant 共用 | `platform:ollama:gemini_daily_count:{date}` |
|
||||
| `telegram:polling:leader` | Telegram polling pod 鎖 | ~30s | `telegram_gateway.py:55` | same | 🔴 platform_resource,應明確標記 | `platform:telegram:polling:leader` |
|
||||
| `telegram_silence:{target}` | 告警靜默控制 | 設定值 | `telegram_gateway.py:4079`(用 SILENCE_KEY_PREFIX)| `decision_manager.py:240`(🔴 硬碼 `telegram_silence:{target}`,未 import 常數)| P1-24:兩處定義不一致 | `{project_id}:telegram:silence:{target}` |
|
||||
| `ai_rate:total_cost:{provider}` | 費用上限控制 | 無 | `ai_rate_limiter.py` | `ai_router.py` | 🔴 無 project 前綴,AWOOOI 費用計數會與 EwoooC 混算 | `{project_id}:ai_rate:total_cost:{provider}` |
|
||||
|
||||
## 2. P1 — 高風險(Phase 2 第二批)
|
||||
|
||||
| Key Pattern | 用途 | TTL | 寫入點 | 讀取點 | 問題 | 新 Pattern |
|
||||
|-------------|------|-----|--------|--------|------|------------|
|
||||
| `approval:{run_id}:*` | approval 審核狀態 | 15min | `multi_sig_redis.py` | same | 🔴 P1-15:無 trace_id;approval_token 未簽章 | `{project_id}:approval:{run_id}:*`(Phase 8 後加 jti)|
|
||||
| `incident:{incident_id}` | incident working memory | 24h | `incident_service.py` | `incident_service.py:603`(SCAN `incident:*`)| 🔴 P1-18:SCAN 無 prefix 篩選 | `{project_id}:incident:{incident_id}` |
|
||||
| `consensus:{*}` | consensus engine 狀態 | varies | `consensus_engine.py`(CONSENSUS_PREFIX="consensus:")| same | 🔴 P0-12:無 project 前綴,multi-tenant 共用 | `{project_id}:consensus:{*}` |
|
||||
| `playbook:{playbook_id}` | playbook cache | 1h | `playbook_repository.py` | same | 🟠 無 project,但目前 AWOOOI 只有一個 tenant | `{project_id}:playbook:{playbook_id}` |
|
||||
| `playbook:embedding:index` | embedding index | 無 | `playbook_repository.py` | same | 🟠 無 project | `{project_id}:playbook:embedding:index` |
|
||||
| `playbook:index:alert:{alert_name}` | alert → playbook 映射 | 1h | `playbook_repository.py` | same | 🟠 無 project | `{project_id}:playbook:index:alert:{alert_name}` |
|
||||
| `playbook:embedding:{playbook_id}` | embedding vector | 無 | `playbook_repository.py` | same | 🟠 無 project | `{project_id}:playbook:embedding:{playbook_id}` |
|
||||
| `km:backfill:dlq` | KM backfill dead letter queue | 無 | `km_service.py` | same | 🟠 無 project | `{project_id}:km:backfill:dlq` |
|
||||
|
||||
## 3. P2 — 業務資料(Phase 2 第三批)
|
||||
|
||||
| Key Pattern | 用途 | TTL | 寫入點 | 讀取點 | 問題 | 新 Pattern |
|
||||
|-------------|------|-----|--------|--------|------|------------|
|
||||
| `anomaly:disposition:{alert_id}` | 告警處置決定 | 7d | `anomaly_counter.py:790` | same | 🔴 P1-17:AnomalyCounter 全域單例,6 個 prefix 無 tenant | `{project_id}:anomaly:disposition:{alert_id}` |
|
||||
| `anomaly:metadata:{alert_id}` | 告警 metadata | 7d | `anomaly_counter.py` | same | 同上 | `{project_id}:anomaly:metadata:{alert_id}` |
|
||||
| `anomaly:permanent_fix:{alert_id}` | 永久修復紀錄 | 30d | `anomaly_counter.py` | same | 同上 | `{project_id}:anomaly:permanent_fix:{alert_id}` |
|
||||
| `anomaly:repair_count:{alert_id}` | 修復計數 | 7d | `anomaly_counter.py` | same | 同上 | `{project_id}:anomaly:repair_count:{alert_id}` |
|
||||
| `anomaly:repair_history:{alert_id}` | 修復歷史 | 30d | `anomaly_counter.py` | same | 同上 | `{project_id}:anomaly:repair_history:{alert_id}` |
|
||||
| `anomaly:timeline:{alert_id}` | 告警時間線 | 7d | `anomaly_counter.py` | same | 同上 | `{project_id}:anomaly:timeline:{alert_id}` |
|
||||
| `hermes:session:{chat_id}:{user_id}` | Hermes 對話狀態 | 30min | `hermes/nl_gateway.py:146` | same | 🔴 P1-16:無 project 前綴 | `{project_id}:hermes:session:{chat_id}:{user_id}` |
|
||||
| `hermes:rl:{chat_id}` | Hermes rate limit | 1min | `hermes/nl_gateway.py:163` | same | 🔴 P1-16 | `{project_id}:hermes:rl:{chat_id}` |
|
||||
| `hermes:approvers` | Hermes 審核者清單 | 無 | `hermes/nl_gateway.py:7` | same | 🔴 P1-16 | `{project_id}:hermes:approvers` |
|
||||
| `session:{session_id}` | agent session | varies | `agent_sessions service` | same | 🟠 無 project | `{project_id}:session:{session_id}` |
|
||||
| `tg_sent:{fingerprint}` | Telegram 去重 dedup | 10min | `telegram_gateway.py` | same | 🟢 ADR-109 已定(dedup 相容性);注意 EwoooC Telegram 需要不同 fingerprint scope | `{project_id}:tg_sent:{fingerprint}` |
|
||||
| `telegram:{user_id}:{*}` | Telegram session / callback | varies | `telegram_gateway.py` | same | 🟠 無 project | `{project_id}:telegram:{user_id}:{*}` |
|
||||
|
||||
## 4. P3 — 低影響(Phase 2 最後)
|
||||
|
||||
| Key Pattern | 用途 | TTL | 問題 | 新 Pattern |
|
||||
|-------------|------|-----|------|------------|
|
||||
| `playbook:*`(SCAN)| playbook 全量掃描 | — | 🟠 SCAN 無 prefix 篩選 | 禁止裸 SCAN,改 `SCAN {project_id}:playbook:*` |
|
||||
| `approval:123:lock`(測試用)| 測試 key | — | 🔵 hardcode 測試字串,清理 | 移除 |
|
||||
|
||||
---
|
||||
|
||||
## 5. Platform Resource(不按 project 隔離,明確標記)
|
||||
|
||||
| Key Pattern | 理由 | 備註 |
|
||||
|-------------|------|------|
|
||||
| `platform:telegram:polling:leader` | 全域 polling leader 鎖,不屬於任何 tenant | ADR-111 platform_resource |
|
||||
| `platform:ollama:topology` | GCP-A/GCP-B/Local 三層路由狀態,所有 tenant 共用 | ADR-111 + ADR-110 |
|
||||
| `platform:ollama:gemini_daily_count:{date}` | 全 platform Gemini 緊急路由計數 | ADR-111 |
|
||||
|
||||
---
|
||||
|
||||
## 6. 遷移執行順序(三階段)
|
||||
|
||||
```
|
||||
Phase A(雙寫,30 天觀察):
|
||||
新 key 寫入 + 舊 key 繼續寫入
|
||||
讀取仍以舊 key 為主
|
||||
監控:舊 key 讀取次數 / 新 key 命中次數
|
||||
|
||||
Phase B(雙讀,14 天):
|
||||
新 key 為主,舊 key 為 fallback
|
||||
讀不到新 key → 讀舊 key,並回填新 key
|
||||
監控:舊 key fallback 率必須 < 1%
|
||||
|
||||
Phase C(移除舊 key 寫入):
|
||||
停止寫入舊 key
|
||||
舊 key 自然過期
|
||||
監控:確認舊 key 0 讀取後才進行
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7. 驗收標準
|
||||
|
||||
- [ ] 所有 P0 key 完成 Phase A 雙寫
|
||||
- [ ] `decision_manager.py:240` 改為 import `telegram_gateway.SILENCE_KEY_PREFIX`
|
||||
- [ ] `ollama:current_primary` 改為 `platform:ollama:topology`(JSON 格式)
|
||||
- [ ] 無任何裸 `SCAN *` 或 `SCAN prefix:*`(改為 `SCAN {project_id}:prefix:*`)
|
||||
|
||||
*最後更新:2026-05-03(台北)*
|
||||
137
docs/awooop/inventory/INV-2-repository-project-id-retrofit.md
Normal file
137
docs/awooop/inventory/INV-2-repository-project-id-retrofit.md
Normal file
@@ -0,0 +1,137 @@
|
||||
# INV-2: Repository project_id Retrofit Map
|
||||
|
||||
**版本**:v1.0 初稿
|
||||
**日期**:2026-05-03(台北)
|
||||
**範圍**:`apps/api/src/db/models.py` + `apps/api/migrations/` 全部
|
||||
**用途**:Phase 2 在 30+ 業務表加 project_id 欄位、backfill 歷史資料、repository filter 改造
|
||||
|
||||
---
|
||||
|
||||
## 1. 現有資料庫表清單(從 db/models.py 實際讀取)
|
||||
|
||||
| 表名 | SQLAlchemy 類名 | 有無 project_id | 是否需要加 | Phase |
|
||||
|------|----------------|----------------|------------|-------|
|
||||
| `approval_records` | ApprovalRecord | ❌ | ✅ 是(approval 需 tenant 隔離)| Phase 2 |
|
||||
| `timeline_events` | TimelineEvent | ❌ | ✅ 是(關聯 incident)| Phase 2 |
|
||||
| `audit_logs` | AuditLog | ❌ | ✅ 是(審計必須 project-scoped)| Phase 2 |
|
||||
| `mcp_audit_log` | McpAuditLog | ❌ | ✅ 是(MCP audit 必 project-scoped)+ 加 trace_id / run_id | Phase 2 |
|
||||
| `mcp_daily_stats` | McpDailyStats | ❌ | ✅ 是(統計分 project)| Phase 2 |
|
||||
| `k8s_state_snapshots` | K8sStateSnapshot | ❌ | ✅ 是(K8s 狀態屬於 tenant)| Phase 2 |
|
||||
| `prometheus_snapshots` | PrometheusSnapshot | ❌ | ✅ 是(metrics 屬於 tenant)| Phase 2 |
|
||||
| `auto_repair_executions` | AutoRepairExecution | ❌ | ✅ 是(修復操作屬於 tenant)| Phase 2 |
|
||||
| `alert_operation_log` | AlertOperationLog | ❌ | ✅ 是(告警操作屬於 tenant)| Phase 2 |
|
||||
| `incidents` | Incident | ❌ | ✅ 是(P0-03 核心問題)+ 加 trace_id / awooop_run_id | Phase 2 |
|
||||
| `knowledge_entries` | KnowledgeEntry | ❌ | ✅ 是(KM 需要 per-project namespace)| Phase 2 |
|
||||
| `incident_evidence` | IncidentEvidence | ❌ | ✅ 是(關聯 incident)| Phase 2 |
|
||||
| `playbooks` | Playbook | ❌ | ✅ 是(playbook 屬於 tenant)| Phase 2 |
|
||||
| `dynamic_baselines` | DynamicBaseline | ❌ | ✅ 是(基線屬於 tenant)| Phase 2 |
|
||||
| `log_clusters` | LogCluster | ❌ | ✅ 是(日誌分析屬於 tenant)| Phase 2 |
|
||||
| `agent_sessions` | AgentSession | ❌ | ✅ 是(session 必 tenant-scoped)| Phase 2 |
|
||||
| `ai_governance_events` | AiGovernanceEvent | ❌ | ✅ 是(治理事件屬於 tenant)| Phase 2 |
|
||||
| `governance_remediation_dispatch` | GovernanceRemediationDispatch | ❌ | ✅ 是 | Phase 2 |
|
||||
| `trust_records` | TrustRecord | ❌ | ✅ 是(trust 屬於 tenant)| Phase 2 |
|
||||
| `ai_provider_version_history` | AiProviderVersionHistory | ❌ | `platform_internal`(追蹤全平台 provider 版本,project_id = __platform__)| Phase 2 |
|
||||
|
||||
> 注:migrations 目錄還有其他表(adr*.sql 建立的),需逐一確認。
|
||||
|
||||
---
|
||||
|
||||
## 2. Migration SQL 表清單(從 migrations/*.sql 掃描)
|
||||
|
||||
| Migration 檔案 | 建立的表 | 是否需要 project_id |
|
||||
|----------------|---------|-------------------|
|
||||
| `adr071_notification_lifecycle.sql` | notification 相關表 | ✅ 需確認 |
|
||||
| `adr088_trust_score_persistence.sql` | trust 相關表 | ✅ 需確認 |
|
||||
| `adr090_asset_inventory_foundation.sql` | asset inventory 表 | ✅ 是(asset 屬於 tenant)|
|
||||
| `adr091_aider_events_schema.sql` | aider_events | ✅ 是 |
|
||||
| `adr092_p1_learning_chain_fix.sql` | learning chain 相關 | ✅ 是 |
|
||||
| `adr093_notification_routing.sql` | notification routing | ✅ 需確認 |
|
||||
| `adr094_hermes_dispatch_log.sql` | hermes_dispatch_log | ✅ 是 |
|
||||
| `adr104_playbook_versioning.sql` | playbook 版本相關 | ✅ 是 |
|
||||
| `adr105_mcp_audit_snapshots.sql` | ⚠️ 注意:表名是 `mcp_audit_snapshots` 但 models.py 用的是 `mcp_audit_log` → 需確認是否同一張表或已廢棄 | 確認後決定 |
|
||||
|
||||
---
|
||||
|
||||
## 3. project_id 加入策略
|
||||
|
||||
### 3.1 Migration 範本
|
||||
|
||||
```sql
|
||||
-- Phase 2:在既有表加 project_id(以 incidents 為例)
|
||||
ALTER TABLE incidents
|
||||
ADD COLUMN IF NOT EXISTS project_id VARCHAR(64)
|
||||
NOT NULL DEFAULT 'awoooi'; -- backfill default
|
||||
|
||||
-- backfill 驗證後移除 DEFAULT(不能永遠有 DEFAULT)
|
||||
-- 此步驟在 backfill 確認完成後再執行:
|
||||
-- ALTER TABLE incidents ALTER COLUMN project_id DROP DEFAULT;
|
||||
|
||||
-- Index
|
||||
CREATE INDEX CONCURRENTLY IF NOT EXISTS
|
||||
idx_incidents_project_id ON incidents(project_id);
|
||||
|
||||
-- Foreign key(可選,建議使用)
|
||||
ALTER TABLE incidents
|
||||
ADD CONSTRAINT fk_incidents_project
|
||||
FOREIGN KEY (project_id) REFERENCES awooop_projects(project_id);
|
||||
```
|
||||
|
||||
### 3.2 Backfill 策略
|
||||
|
||||
所有歷史資料:
|
||||
- `project_id = 'awoooi'`(AWOOOI first tenant,Phase 1 已建立)
|
||||
- 不需要轉換或映射
|
||||
- 在 migration SQL 中以 `DEFAULT 'awoooi'` 完成 backfill
|
||||
|
||||
### 3.3 Repository 方法改造(批次計畫)
|
||||
|
||||
Phase 2 PR-08(批次 1):
|
||||
- `IncidentRepository.get_by_id(incident_id)` → `get_by_id(incident_id, project_id)`
|
||||
- `IncidentRepository.list_active()` → `list_active(project_id)`
|
||||
- `PlaybookRepository.find_by_alert_type(alert_type)` → `find_by_alert_type(alert_type, project_id)`
|
||||
- `KnowledgeRepository.search(query)` → `search(query, project_id)`
|
||||
|
||||
Phase 2 PR-09(批次 2):
|
||||
- `McpAuditRepository.*` 全部加 project_id + trace_id
|
||||
- `AiDecisionRepository.*` 全部加 project_id + run_id
|
||||
- `ApprovalRepository.*` 全部加 project_id + trace_id
|
||||
|
||||
---
|
||||
|
||||
## 4. RLS 設計(ADR-118,補充)
|
||||
|
||||
```sql
|
||||
-- 每張表的 RLS policy(以 incidents 為例)
|
||||
ALTER TABLE incidents ENABLE ROW LEVEL SECURITY;
|
||||
|
||||
CREATE POLICY incidents_tenant_isolation ON incidents
|
||||
USING (
|
||||
project_id = current_setting('app.project_id', TRUE)
|
||||
OR current_user = 'awooop_platform'
|
||||
);
|
||||
|
||||
-- 應用程式在每次 connection/transaction 開始設定:
|
||||
-- SET LOCAL app.project_id = '{project_id}';
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. 特別注意項目
|
||||
|
||||
| 問題 | 影響 |
|
||||
|------|------|
|
||||
| `adr105_mcp_audit_snapshots.sql` 建的表名 `mcp_audit_snapshots` vs models.py 的 `mcp_audit_log` | P0-02 的一部分:表名不一致,migration 可能廢棄或重建 |
|
||||
| `knowledge_entries` 需要 pgvector namespace 隔離 | KM 向量搜索的 namespace 需同步加 project_id scope |
|
||||
| `agent_sessions` 已有 session_id,改造時要確保 session 不跨 tenant | 改造前先確認 session lookup 路徑 |
|
||||
|
||||
---
|
||||
|
||||
## 6. 驗收標準
|
||||
|
||||
- [ ] 所有 20 張表(db/models.py)完成 project_id migration
|
||||
- [ ] backfill:所有舊資料 project_id = 'awoooi'
|
||||
- [ ] 所有 repository 方法加 project_id filter
|
||||
- [ ] cross-project SELECT 全部被 RLS 拒絕(pytest 覆蓋)
|
||||
- [ ] `mcp_audit_snapshots` vs `mcp_audit_log` 衝突已解決
|
||||
|
||||
*最後更新:2026-05-03(台北)*
|
||||
110
docs/awooop/inventory/INV-3-entrypoints.md
Normal file
110
docs/awooop/inventory/INV-3-entrypoints.md
Normal file
@@ -0,0 +1,110 @@
|
||||
# INV-3: Entrypoint Inventory
|
||||
|
||||
**版本**:v1.0 初稿
|
||||
**日期**:2026-05-03(台北)
|
||||
**範圍**:`apps/api/src/main.py` + 所有 webhook / CLI / cron 入口點
|
||||
**用途**:Phase 2 Bootstrap Paradox 修補的 31 個 background loop 分類
|
||||
|
||||
---
|
||||
|
||||
## 1. 標記定義(ADR-111)
|
||||
|
||||
| 標記 | 意義 |
|
||||
|------|------|
|
||||
| `platform_internal` | 平台本身的維護工作,不屬於任何 tenant,帶 `project_id=__platform__` |
|
||||
| `legacy_awoooi_default` | 過渡期:暫時 fallback 到 `project_id=awoooi`,有退場時程 |
|
||||
| `requires_project_id` | 必須帶 project_id 才能運行,Phase 2 改造重點 |
|
||||
| `multi_tenant_ready` | 已支援 project_id(Phase 2 後才能設定)|
|
||||
|
||||
---
|
||||
|
||||
## 2. API Webhook 入口(同步接收)
|
||||
|
||||
| 入口 | 檔案 / 路由 | 標記 | project_id 來源 |
|
||||
|------|------------|------|----------------|
|
||||
| `POST /v1/webhooks/telegram` | `api/v1/webhooks.py:679` | `legacy_awoooi_default` | 固定 awoooi,Phase 7 後改為 principal mapping |
|
||||
| `POST /v1/webhooks/alertmanager` | `api/v1/webhooks.py` | `legacy_awoooi_default` | 固定 awoooi |
|
||||
| `POST /v1/webhooks/gitea` | `api/v1/webhooks.py` | `legacy_awoooi_default` | 固定 awoooi |
|
||||
| `GET /v1/health` | `api/v1/health.py` | `platform_internal` | `__platform__` |
|
||||
| `GET /v1/metrics` | `api/v1/metrics.py` | `platform_internal` | `__platform__` |
|
||||
| `POST /v1/decisions/` | `api/v1/decisions.py` | `legacy_awoooi_default` | 固定 awoooi,Phase 4 後改為 contract lookup |
|
||||
| `POST /v1/incidents/` | `api/v1/incidents.py` | `legacy_awoooi_default` | 固定 awoooi |
|
||||
|
||||
---
|
||||
|
||||
## 3. Background Loops(main.py asyncio.create_task,共 31 個)
|
||||
|
||||
> 來源:`apps/api/src/main.py` grep `asyncio.create_task`,行號已驗證
|
||||
|
||||
| # | 函數名 | main.py 行號 | 類別 | 標記 | 備註 |
|
||||
|---|--------|-------------|------|------|------|
|
||||
| 1 | `seed_playbooks_from_rules()` | 331 | 啟動一次性 | `legacy_awoooi_default` | playbook seed,只跑一次 |
|
||||
| 2 | `get_playbook_repository().backfill_redis_to_pg()` | 341 | 啟動一次性 | `legacy_awoooi_default` | Redis → PG backfill,只跑一次 |
|
||||
| 3 | `ensure_playbook_embeddings_indexed()` | 348 | 啟動一次性 | `legacy_awoooi_default` | embedding index,只跑一次 |
|
||||
| 4 | `get_decision_manager().resend_stale_ready_tokens()` | 362 | 持續 loop | `legacy_awoooi_default` | stale approval token 重送 |
|
||||
| 5 | `run_incident_analysis_sweeper()` | 373 | 持續 loop | `legacy_awoooi_default` | incident 週期分析 |
|
||||
| 6 | `run_asset_scanner_loop()` | 383 | 持續 loop | `legacy_awoooi_default` | K8s asset 掃描(ADR-090)|
|
||||
| 7 | `run_rule_catalog_sync_loop()` | 393 | 持續 loop | `legacy_awoooi_default` | 告警規則 catalog 同步 |
|
||||
| 8 | `run_capacity_scanner_loop()` | 403 | 持續 loop | `legacy_awoooi_default` | 容量掃描 |
|
||||
| 9 | `run_compliance_scanner_loop()` | 413 | 持續 loop | `legacy_awoooi_default` | 合規掃描 |
|
||||
| 10 | `run_aider_event_processor_loop()` | 423 | 持續 loop | `legacy_awoooi_default` | aider 事件處理 |
|
||||
| 11 | `run_coverage_evaluator_loop()` | 432 | 持續 loop | `legacy_awoooi_default` | 覆蓋率評估 |
|
||||
| 12 | `run_rule_stats_updater_loop()` | 442 | 持續 loop | `legacy_awoooi_default` | 規則統計更新 |
|
||||
| 13 | `run_asset_change_tracker_loop()` | 452 | 持續 loop | `legacy_awoooi_default` | 資產變更追蹤 |
|
||||
| 14 | `run_hermes_rule_quality_loop()` | 462 | 持續 loop | `legacy_awoooi_default` | Hermes 規則品質 |
|
||||
| 15 | `run_capacity_forecaster_loop()` | 472 | 持續 loop | `legacy_awoooi_default` | 容量預測 |
|
||||
| 16 | `run_daily_report_loop()` | 481 | 持續 loop | `legacy_awoooi_default` | 每日報告 |
|
||||
| 17 | `run_approval_timeout_resolver()` | 490 | 持續 loop | `legacy_awoooi_default` | 🔴 P1-15:無 trace_id |
|
||||
| 18 | `run_evolver_loop()` | 499 | 持續 loop | `legacy_awoooi_default` | playbook 進化 |
|
||||
| 19 | `run_playbook_generation_governance_loop()` | 507 | 持續 loop | `legacy_awoooi_default` | playbook 生成治理 |
|
||||
| 20 | `run_knowledge_decay_loop()` | 519 | 持續 loop | `legacy_awoooi_default` | KM 知識衰退 |
|
||||
| 21 | `run_km_backfill_reconciler_loop()` | 529 | 持續 loop | `legacy_awoooi_default` | KM backfill 核對 |
|
||||
| 22 | `run_aol_writeback_loop()` | 540 | 持續 loop | `legacy_awoooi_default` | AOL writeback(飛輪)|
|
||||
| 23 | `_run_kb_rot_cleaner_loop()` | 585 | 持續 loop | `legacy_awoooi_default` | KB 腐敗清理 |
|
||||
| 24 | `run_finetune_export_loop()` | 594 | 持續 loop | `legacy_awoooi_default` | finetune 資料匯出 |
|
||||
| 25 | `run_proactive_inspector_loop()` | 605 | 持續 loop | `legacy_awoooi_default` | 主動巡檢 |
|
||||
| 26 | `run_offline_replay_loop()` | 614 | 持續 loop | `legacy_awoooi_default` | 離線重放 |
|
||||
| 27 | `run_ai_slo_watchdog_loop()` | 623 | 持續 loop | `platform_internal` | SLO watchdog — 監控本平台健康,project_id=__platform__ |
|
||||
| 28 | `run_governance_loop()` | 632 | 持續 loop | `legacy_awoooi_default` | AI 治理主循環 |
|
||||
| 29 | `run_governance_dispatcher_loop()` | 640 | 持續 loop | `legacy_awoooi_default` | 治理事件派送 |
|
||||
| 30 | `_run_model_version_tracker_loop()` | 701 | 持續 loop | `platform_internal` | AI model 版本追蹤 — platform_resource |
|
||||
| 31 | (需確認 main.py 701 後是否還有)| 701+ | TBD | TBD | grep 計數 = 31,確認後補 |
|
||||
|
||||
---
|
||||
|
||||
## 4. 遷移策略(依 ADR-123)
|
||||
|
||||
### platform_internal(2 個,#27、#30)
|
||||
- 帶 `project_id=__platform__`
|
||||
- 不受 project RLS 限制
|
||||
- 但必須寫 audit log(標記 `platform_resource=true`)
|
||||
|
||||
### legacy_awoooi_default(29 個,其餘所有)
|
||||
- 過渡期:帶 `project_id=awoooi`
|
||||
- 退場時程:Phase 4 完成後 90 天內逐一改造為 `requires_project_id`
|
||||
- 每個 loop 改造後從 `legacy_awoooi_default` → `multi_tenant_ready`
|
||||
|
||||
### 特別注意
|
||||
- `run_approval_timeout_resolver()`(#17):改造時必須同步補入 trace_id(P1-15)
|
||||
- `run_aol_writeback_loop()`(#22):改造時確認 KM 雙路徑寫入(feedback_km_dual_path_design.md)
|
||||
|
||||
---
|
||||
|
||||
## 5. CLI / Script 入口
|
||||
|
||||
| 入口 | 類別 | 標記 |
|
||||
|------|------|------|
|
||||
| `python -m apps.api` / `uvicorn apps.api.src.main:app` | 主程序啟動 | `platform_internal` |
|
||||
| `python -m alembic upgrade head` | DB migration | `platform_internal` |
|
||||
| `python scripts/seed_*.py`(若有)| 資料 seed | `platform_internal` |
|
||||
|
||||
---
|
||||
|
||||
## 6. 驗收標準
|
||||
|
||||
- [ ] 31 個 background loop 全部有標記
|
||||
- [ ] `platform_internal` loop 帶 `project_id=__platform__`(可在 logging context 確認)
|
||||
- [ ] `legacy_awoooi_default` loop 帶 `project_id=awoooi`(fallback,不是最終形態)
|
||||
- [ ] 退場時程寫入 ADR-123
|
||||
|
||||
*最後更新:2026-05-03(台北)*
|
||||
108
docs/awooop/inventory/INV-4-hardcoded-namespace-ip.md
Normal file
108
docs/awooop/inventory/INV-4-hardcoded-namespace-ip.md
Normal file
@@ -0,0 +1,108 @@
|
||||
# INV-4: Hardcoded Namespace & IP Inventory
|
||||
|
||||
**版本**:v1.0 初稿
|
||||
**日期**:2026-05-03(台北)
|
||||
**範圍**:`apps/api/src/` 全 codebase + `k8s/`
|
||||
**用途**:Phase 2 多租戶改造 + EwoooC onboarding 前必須清理的硬碼
|
||||
|
||||
---
|
||||
|
||||
## 1. Hardcoded K8s Namespace
|
||||
|
||||
| 位置 | 行號 | 內容 | 影響 | 修補方式 |
|
||||
|------|------|------|------|---------|
|
||||
| `apps/api/src/plugins/mcp/providers/k8s_provider.py` | 40-41 | `ALLOWED_NAMESPACES = {"awoooi-prod"}` / `DEFAULT_NAMESPACE = "awoooi-prod"` | 🔴 P0-13:EwoooC K8s tool 無法操作自己的 namespace | 改為 config-driven:從 EffectivePolicy 或 project contract 讀 allowed namespaces |
|
||||
| `apps/api/src/plugins/mcp/mcp_bridge.py` | 592, 602, 631, 647, 681 | `namespace = parameters.get("namespace", "awoooi-prod")` | 🔴 P0-13:5 處預設值都寫死 | 改為 `namespace = parameters.get("namespace", get_project_default_namespace(project_id))` |
|
||||
| `apps/api/src/plugins/mcp/providers/signoz_provider.py` | 169 | `namespace = parameters.get("namespace", "awoooi-prod")` | 🟠 SignOz query 預設 namespace 錯誤 | 同上 |
|
||||
| `apps/api/src/main.py` | 175 | `sentry_sdk.set_tag("host", "k8s-awoooi-prod")` | 🔵 Sentry tag 寫死,EwoooC 看到錯誤 host tag | 改為 `settings.SENTRY_HOST_TAG` |
|
||||
| `apps/api/src/core/prompts.py` | 120, 178, 192 | 系統 prompt 中出現 `awoooi-prod` | 🔵 LLM 可能在其他 tenant 錯誤建議 awoooi-prod namespace | 改為 `{tenant_namespace}` template variable |
|
||||
|
||||
---
|
||||
|
||||
## 2. Hardcoded IP Addresses
|
||||
|
||||
### 內網 IP(已知用途)
|
||||
|
||||
| IP | 用途 | 位置 | 改法 |
|
||||
|----|------|------|------|
|
||||
| `192.168.0.110` | Gitea / Prometheus / Loki / MinIO | `config.py:226,413,460,813` | 已在 config 以 default 存在,OK(K8s 覆蓋)|
|
||||
| `192.168.0.111` | Ollama Local Fallback | `config.py` + `feedback_ollama_111_only.md`(已更新) | ADR-110 已改為第三層 fallback,config 需更新 |
|
||||
| `192.168.0.112` | ArgoCD | `config.py:396` | OK(config default)|
|
||||
| `192.168.0.120` | K3s API Server | `config.py:531` | OK(config default)|
|
||||
| `192.168.0.121` | K3s ingress? | `config.py:837` | 確認用途 |
|
||||
| `192.168.0.188` | PostgreSQL / Redis / SigNoz / Grafana / ClickHouse | `config.py` 多處 | OK(config default,K8s 以 env 覆蓋)|
|
||||
|
||||
### 🔴 問題:telemetry.py IP Assertion
|
||||
|
||||
| 位置 | 行號 | 內容 | 問題 |
|
||||
|------|------|------|------|
|
||||
| `apps/api/src/core/telemetry.py` | 71 | `if "192.168.0.188" not in endpoint: raise` | 🔴 P0-08:EwoooC 啟動必失敗(EwoooC SigNoz 可能是不同 endpoint)|
|
||||
| 修補方式 | | | 移除硬碼 assert,改為 `if endpoint not in settings.ALLOWED_TELEMETRY_ENDPOINTS:` |
|
||||
|
||||
### GCP IP(新增,ADR-110,2026-05-03 生效)
|
||||
|
||||
| IP | 用途 | 位置 |
|
||||
|----|------|------|
|
||||
| `34.143.170.20` | Ollama GCP-A Primary(SSD)| `config.py`(ADR-110 已加入 `_ALLOWED_PUBLIC_IPS`)|
|
||||
| `34.21.145.224` | Ollama GCP-B Secondary(SSD)| `config.py`(ADR-110 已加入 `_ALLOWED_PUBLIC_IPS`)|
|
||||
|
||||
**注意**:
|
||||
- K8s NetworkPolicy egress:已新增 GCP-A/GCP-B /32 出口規則(ADR-110)
|
||||
- INV-4 確認:GCP IP 已在 `config.py._ALLOWED_PUBLIC_IPS` 白名單,非新增需求
|
||||
- telemetry.py:71 assert:GCP IP 不影響(assert 是針對 OTEL endpoint,非 Ollama endpoint)
|
||||
|
||||
---
|
||||
|
||||
## 3. SSH Host Hardcodes
|
||||
|
||||
| 位置 | 內容 | 問題 | 修補 |
|
||||
|------|------|------|------|
|
||||
| `reference_four_hosts.md` | 110/120/121/188 四主機清單 | 文件,不是程式碼,OK | 無 |
|
||||
| `apps/api/src/plugins/mcp/providers/ssh_provider.py`(若有)| SSH 目標主機 | 需 grep 確認是否硬碼 | 改為 config-driven 白名單 |
|
||||
|
||||
---
|
||||
|
||||
## 4. 其他硬碼字串
|
||||
|
||||
| 位置 | 內容 | 問題 | 修補 |
|
||||
|------|------|------|------|
|
||||
| `apps/api/src/core/config.py:625` | `default="192.168.0.188=ollama"` | Ollama-to-host mapping,ADR-110 後需更新 | 改為 `192.168.0.188=ollama_old`(僅 fallback 相關)|
|
||||
| `apps/api/src/core/config.py:828` | `default="192.168.0.188,192.168.0.110,192.168.0.111"` | 主機清單 | 確認用途,若是 monitoring target 需加 GCP IP |
|
||||
|
||||
---
|
||||
|
||||
## 5. 改造策略(Phase 2)
|
||||
|
||||
### K8s Namespace(優先)
|
||||
```python
|
||||
# 方案:project contract 中定義允許的 K8s namespaces
|
||||
# awooop_projects.k8s_namespaces: ["awoooi-prod"](AWOOOI)/ ["ewoooc-prod"](EwoooC)
|
||||
# k8s_provider.py 從 project contract 讀,而非硬碼
|
||||
|
||||
def get_allowed_namespaces(project_id: str) -> set[str]:
|
||||
contract = get_active_project_contract(project_id)
|
||||
return set(contract.allowed_k8s_namespaces)
|
||||
```
|
||||
|
||||
### Telemetry Endpoint Assert(P0-08,PR-01 優先)
|
||||
```python
|
||||
# 修改前(telemetry.py:71)
|
||||
if "192.168.0.188" not in endpoint:
|
||||
raise ValueError(f"Forbidden OTEL endpoint: {endpoint}")
|
||||
|
||||
# 修改後
|
||||
allowed_endpoints = settings.ALLOWED_TELEMETRY_ENDPOINTS.split(",")
|
||||
if not any(allowed in endpoint for allowed in allowed_endpoints):
|
||||
raise ValueError(f"Forbidden OTEL endpoint: {endpoint}")
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. 驗收標準
|
||||
|
||||
- [ ] `grep -r "192.168.0.188" apps/api/src/` 中 `telemetry.py` 的 assert 行消失
|
||||
- [ ] `grep -r '"awoooi-prod"' apps/api/src/` 中的程式碼路徑(非 prompt 文字、非 comment)結果為 0
|
||||
- [ ] k8s_provider.py `ALLOWED_NAMESPACES` 改為 config-driven
|
||||
- [ ] INV-4 中標記 GCP IP 已確認加入 NetworkPolicy(ADR-110 完成)
|
||||
|
||||
*最後更新:2026-05-03(台北)*
|
||||
116
docs/awooop/inventory/INV-5-migration-compatibility-matrix.md
Normal file
116
docs/awooop/inventory/INV-5-migration-compatibility-matrix.md
Normal file
@@ -0,0 +1,116 @@
|
||||
# INV-5: Migration Compatibility Matrix
|
||||
|
||||
**版本**:v1.0 初稿
|
||||
**日期**:2026-05-03(台北)
|
||||
**範圍**:`apps/api/pyproject.toml` 實際版本 + AwoooP Phase 1 migration 需求
|
||||
**用途**:Phase 1 schema migration 前確認版本相容性,避免 breaking change 踩雷
|
||||
|
||||
---
|
||||
|
||||
## 1. 當前依賴版本(從 pyproject.toml 實際讀取)
|
||||
|
||||
| 套件 | 當前約束 | 對 AwoooP Phase 1 的影響 |
|
||||
|------|---------|------------------------|
|
||||
| Python | >=3.11 | ✅ 足夠;`asyncio.TaskGroup`(3.11)可用 |
|
||||
| FastAPI | >=0.115.0 | ✅ 0.115 已支援 lifespan context manager |
|
||||
| SQLAlchemy | **>=2.0.0** ✅ 已是 2.x | ✅ `mapped_column`、`DeclarativeBase`、async session 均可用 |
|
||||
| Pydantic | **>=2.5.0** ✅ 已是 v2 | ✅ `model_validator`、`field_validator` v2 語法可直接用 |
|
||||
| pydantic-settings | >=2.1.0 | ✅ |
|
||||
| asyncpg | >=0.29.0 | ✅ 支援 PostgreSQL 15/16 |
|
||||
| redis | >=5.0.0 | ✅ redis-py 5.x 支援 cluster mode + NX |
|
||||
| Alembic | **未列出**(透過 SQLAlchemy 間接)| ⚠️ 需確認版本;建議鎖定 >=1.13.0 |
|
||||
| Langfuse | >=2.0.0,<3.0.0(鎖定)| ✅ 無 AwoooP 影響,保持不變 |
|
||||
| opentelemetry | >=1.20.0 | ✅ GenAI semantic conventions 需要 >=1.25.0(ADR-121)|
|
||||
| claude-agent-sdk | >=0.1.50 | ✅ 已使用 |
|
||||
|
||||
---
|
||||
|
||||
## 2. 重要發現
|
||||
|
||||
**✅ SQLAlchemy 已是 2.x**:MASTER-WORKPLAN 中「ORM 1.x vs 2.x」問題**已不存在**。
|
||||
- `mapped_column`、`relationship`、async session 均可正常用
|
||||
- 但仍需確認:AwoooP 新 model 使用正確的 2.x 寫法(`DeclarativeBase` 而非舊 `declarative_base()`)
|
||||
|
||||
**✅ Pydantic 已是 v2**:六合約 Pydantic models 可直接用 v2 語法。
|
||||
|
||||
**⚠️ Alembic 版本未鎖定**:
|
||||
- `apps/api/migrations/` 有大量 `.sql` 手寫 migration(非 Alembic auto-generate)
|
||||
- AwoooP Phase 1 要決定:繼續手寫 SQL migration,還是引入 Alembic autogenerate?
|
||||
- **建議**:保持手寫 SQL(現有慣例),每個 migration 強制有 down migration
|
||||
|
||||
**⚠️ OTel 版本需升級(ADR-121)**:
|
||||
- GenAI Semantic Conventions 需要 `opentelemetry-semantic-conventions>=0.46b0`(含 `gen_ai.*` namespace)
|
||||
- 當前 `>=1.20.0` 可能不夠,需要 `>=1.25.0`
|
||||
- 升級風險:低(向後相容);需要加入 `opentelemetry-semantic-conventions` 依賴
|
||||
|
||||
---
|
||||
|
||||
## 3. AwoooP Phase 1 新依賴需求
|
||||
|
||||
| 套件 | 版本 | 用途 | 是否需要新增 |
|
||||
|------|------|------|------------|
|
||||
| `pg_partman`(PostgreSQL extension)| >=4.7.0 | 高流量表 partition 自動管理 | ⚠️ 需在 DB 層安裝,非 Python 套件 |
|
||||
| `opentelemetry-semantic-conventions` | >=0.46b0 | OTel GenAI attribute(ADR-121)| ✅ 需新增到 pyproject.toml |
|
||||
| `PgBouncer` | >=1.21.0 | Connection pool(tool-expert 需求)| ⚠️ Infrastructure,非 Python 套件 |
|
||||
|
||||
---
|
||||
|
||||
## 4. Migration SQL 版本相容性
|
||||
|
||||
| Migration 問題 | 當前狀態 | 修補方式 |
|
||||
|---------------|---------|---------|
|
||||
| `adr105_mcp_audit_snapshots.sql` 建表 `mcp_audit_snapshots`,但 models.py 用 `mcp_audit_log` | ⚠️ 衝突 | 確認是否已 rollback;若 `mcp_audit_log` 是現行表,則 `mcp_audit_snapshots` 已廢棄 |
|
||||
| 部分 migration 無 rollback SQL | ⚠️ MASTER-WORKPLAN P0-02 | Phase 1 新 migration 強制加 down migration |
|
||||
| `ARRAY` 型別在部分 migration 用 `TEXT[]`,在其他用 `JSONB` | 🟠 不一致 | AwoooP 新表一律用 `JSONB`(`fix_playbooks_array_to_jsonb.sql` 已示範)|
|
||||
|
||||
---
|
||||
|
||||
## 5. SQLAlchemy 2.x 正確寫法確認清單(Phase 1 新 model 必須遵守)
|
||||
|
||||
```python
|
||||
# ✅ 正確(SQLAlchemy 2.x)
|
||||
from sqlalchemy.orm import DeclarativeBase, mapped_column, Mapped
|
||||
from sqlalchemy import String, DateTime, func
|
||||
from typing import Optional
|
||||
|
||||
class Base(DeclarativeBase):
|
||||
pass
|
||||
|
||||
class AwooopProject(Base):
|
||||
__tablename__ = "awooop_projects"
|
||||
project_id: Mapped[str] = mapped_column(String(64), primary_key=True)
|
||||
display_name: Mapped[str] = mapped_column(String(128), nullable=False)
|
||||
created_at: Mapped[datetime] = mapped_column(
|
||||
DateTime(timezone=True), server_default=func.now()
|
||||
)
|
||||
migration_mode: Mapped[str] = mapped_column(
|
||||
String(32), nullable=False, default="legacy_awoooi_default"
|
||||
)
|
||||
|
||||
# ❌ 舊寫法(SQLAlchemy 1.x,不要用)
|
||||
# from sqlalchemy.ext.declarative import declarative_base
|
||||
# Base = declarative_base()
|
||||
# project_id = Column(String(64), primary_key=True)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. 版本兼容風險清單
|
||||
|
||||
| 風險 | 嚴重度 | 緩解方式 |
|
||||
|------|--------|---------|
|
||||
| pg_partman 未安裝,partition 手動建立出錯 | 中 | Phase 1 先不建 partition,只規劃;Phase 4 前安裝 pg_partman |
|
||||
| OTel semantic conventions 版本不足 | 低 | Phase 4 前升級(非 Phase 1 阻擋項)|
|
||||
| PgBouncer 未安裝,AwoooP worker 壓力測試可能爆連接 | 中 | Phase 4(platform shell)前安裝 |
|
||||
| Alembic 未鎖定,CI 環境版本飄移 | 低 | 鎖定 `alembic>=1.13.0` 加入 pyproject.toml |
|
||||
|
||||
---
|
||||
|
||||
## 7. 驗收標準
|
||||
|
||||
- [ ] Phase 1 所有新 migration 使用 SQLAlchemy 2.x `mapped_column` 語法
|
||||
- [ ] 所有新 migration 有 down migration SQL
|
||||
- [ ] `mcp_audit_snapshots` vs `mcp_audit_log` 衝突已解決(確認哪個是現行表)
|
||||
- [ ] `pg_partman` 安裝計畫已寫入 runbook
|
||||
|
||||
*最後更新:2026-05-03(台北)*
|
||||
213
docs/awooop/inventory/INV-6-rollback-playbook-register.md
Normal file
213
docs/awooop/inventory/INV-6-rollback-playbook-register.md
Normal file
@@ -0,0 +1,213 @@
|
||||
# INV-6: Rollback Playbook Register
|
||||
|
||||
**版本**:v1.0 初稿
|
||||
**日期**:2026-05-03(台北)
|
||||
**範圍**:AwoooP 全 8 Phase 的 rollback 場景
|
||||
**用途**:Phase 4 起的 gate 驗收「rollback evidence ≥ 3 次」需要這份登記冊
|
||||
|
||||
---
|
||||
|
||||
## 說明
|
||||
|
||||
每個 rollback playbook 需要:
|
||||
1. 觸發條件(什麼時候需要 rollback)
|
||||
2. rollback 步驟(精確指令)
|
||||
3. 驗證方式(rollback 是否成功)
|
||||
4. 影響範圍(什麼會受影響)
|
||||
|
||||
---
|
||||
|
||||
## RBP-01: Phase 1 Schema Migration Rollback
|
||||
|
||||
**觸發**:Phase 1 migration up 後發現業務功能異常、CI 回歸測試失敗
|
||||
|
||||
**rollback 步驟**:
|
||||
```bash
|
||||
# 1. 確認當前 migration head
|
||||
alembic current # 或直接查 DB schema
|
||||
|
||||
# 2. 執行 down migration(每個 AwoooP migration 都有 down SQL)
|
||||
# 手動執行 down migration SQL(deps/alembic down 或直接 psql)
|
||||
psql $DATABASE_URL -f apps/api/migrations/awooop_phase1_down.sql
|
||||
|
||||
# 3. 驗證 AwoooP 表已移除
|
||||
psql $DATABASE_URL -c "\dt awooop_*" # 應該 0 結果
|
||||
|
||||
# 4. 確認 AWOOOI 原有功能正常
|
||||
curl -f http://localhost:8000/v1/health
|
||||
```
|
||||
|
||||
**影響**:AwoooP 控制面表全部刪除,AWOOOI 業務功能不受影響(P0:AWOOOI 0 行為改動)
|
||||
**pre-condition**:AWOOOI 業務表無任何 AwoooP 外鍵依賴(Phase 1 設計邊界)
|
||||
|
||||
---
|
||||
|
||||
## RBP-02: Phase 2 Redis Key Rollback(三階段雙寫任一 Phase)
|
||||
|
||||
**觸發**:Redis 雙寫後發現舊服務讀取異常(Phase A 或 Phase B 期間)
|
||||
|
||||
**rollback 步驟**(Phase A 回滾):
|
||||
```bash
|
||||
# 停止寫入新 key(code revert 或 feature flag off)
|
||||
# 舊 key 本來就還在,直接恢復正常讀取
|
||||
# 確認舊 key 仍有效(TTL 未過期)
|
||||
redis-cli TTL "telegram_silence:{target}" # 確認大於 0
|
||||
|
||||
# 清理已寫入的新 key(避免混淆)
|
||||
redis-cli DEL "{project_id}:telegram:silence:{target}"
|
||||
```
|
||||
|
||||
**rollback 步驟**(Phase B 回滾 → 回到 Phase A):
|
||||
```bash
|
||||
# 將讀取邏輯改回「舊 key 為主」
|
||||
# 保持雙寫(不移除新 key 寫入)
|
||||
# 這是 Phase A 狀態,不需要清理資料
|
||||
```
|
||||
|
||||
**影響**:Redis 讀取回到舊 key,功能降級但不中斷
|
||||
**monitoring**:舊 key 命中率應在 24h 內回升到 100%
|
||||
|
||||
---
|
||||
|
||||
## RBP-03: Strangler Fig Phase Rollback(shadow/canary/read_only 回退)
|
||||
|
||||
**觸發**:量化 gate 失敗(error rate > threshold、p95 退化 > 10%、出現 P0/P1 incident)
|
||||
|
||||
**rollback 步驟**:
|
||||
```sql
|
||||
-- 回退 project_migration_state
|
||||
UPDATE awooop_project_migration_state
|
||||
SET current_mode = 'shadow', -- 回到上一個 mode
|
||||
rolled_back_at = NOW(),
|
||||
rollback_reason = '量化 gate 失敗:error_rate > 0.5%'
|
||||
WHERE project_id = 'awoooi'
|
||||
AND capability = 'sre_flows';
|
||||
|
||||
-- 讓 worker 重新讀 migration_state(invalidate cache)
|
||||
-- Redis: DEL platform:awooop:migration_state:awoooi
|
||||
```
|
||||
|
||||
```bash
|
||||
# feature flag 切回
|
||||
# AWOOOP_CANARY_MODE=false(K8s ConfigMap 或 env)
|
||||
kubectl patch configmap awoooi-config -n awoooi-prod \
|
||||
--type=merge -p '{"data":{"AWOOOP_CANARY_MODE":"false"}}'
|
||||
kubectl rollout restart deployment/awoooi-api -n awoooi-prod
|
||||
```
|
||||
|
||||
**影響**:AwoooP 回到上一個 mode,legacy AWOOOI handler 重新接管
|
||||
**monitoring**:Strangler Fig dashboard 顯示回退記錄(ADR-UI-03)
|
||||
|
||||
---
|
||||
|
||||
## RBP-04: Phase 5 MCP Gateway Rollback
|
||||
|
||||
**觸發**:MCP Gateway 導致工具呼叫失敗率 > 5%
|
||||
|
||||
**rollback 步驟**:
|
||||
```bash
|
||||
# 1. 關閉 MCP Gateway 攔截(feature flag)
|
||||
kubectl patch configmap awoooi-config -n awoooi-prod \
|
||||
--type=merge -p '{"data":{"AWOOOP_MCP_GATEWAY_ENABLED":"false"}}'
|
||||
|
||||
# 2. 重啟 pod 使設定生效
|
||||
kubectl rollout restart deployment/awoooi-api -n awoooi-prod
|
||||
|
||||
# 3. 確認工具呼叫直接通過(繞過 Gateway)
|
||||
# 監控:mcp_tool_call_success_rate 應在 5min 內回升
|
||||
```
|
||||
|
||||
**影響**:audit 仍記錄,但 Five-gate 不攔截;credential isolation 暫時失效
|
||||
**post-rollback action**:找 MCP Gateway 失敗原因後再重新啟用
|
||||
|
||||
---
|
||||
|
||||
## RBP-05: Phase 6 EwoooC Tenant Rollback
|
||||
|
||||
**觸發**:EwoooC shadow 期間發現跨 tenant 資料洩漏(E-TENANT-001 告警)
|
||||
|
||||
**rollback 步驟**:
|
||||
```sql
|
||||
-- 1. 立即停止 EwoooC 所有 run
|
||||
UPDATE awooop_run_state
|
||||
SET status = 'CANCELLED',
|
||||
failure_reason = 'Emergency rollback: cross-tenant leak detected'
|
||||
WHERE project_id = 'ewoooc'
|
||||
AND status NOT IN ('COMPLETED', 'FAILED', 'CANCELLED');
|
||||
|
||||
-- 2. 停用 EwoooC project
|
||||
UPDATE awooop_projects
|
||||
SET is_active = FALSE,
|
||||
suspension_reason = 'Emergency rollback 2026-05-xx'
|
||||
WHERE project_id = 'ewoooc';
|
||||
```
|
||||
|
||||
```bash
|
||||
# 3. feature flag 關閉
|
||||
kubectl patch configmap awoooi-config -n awoooi-prod \
|
||||
--type=merge -p '{"data":{"AWOOOP_EWOOOC_LIVE":"false"}}'
|
||||
|
||||
# 4. 通知 EwoooC 團隊
|
||||
```
|
||||
|
||||
**影響**:EwoooC 服務中斷;AWOOOI 不受影響
|
||||
**RCA 必要**:觸發 Runbook RB-06(cross-tenant leak);需確認 RLS 是否失效
|
||||
|
||||
---
|
||||
|
||||
## RBP-06: Phase 8 Approval Flow Rollback
|
||||
|
||||
**觸發**:approval_token 簽章機制導致正常 approval 無法完成
|
||||
|
||||
**rollback 步驟**:
|
||||
```bash
|
||||
# 1. 臨時切回舊 approval 路徑(feature flag)
|
||||
kubectl patch configmap awoooi-config -n awoooi-prod \
|
||||
--type=merge -p '{"data":{"AWOOOP_APPROVAL_TOKEN_REQUIRED":"false"}}'
|
||||
|
||||
# 2. 重啟
|
||||
kubectl rollout restart deployment/awoooi-api -n awoooi-prod
|
||||
|
||||
# 3. 手動處理 stuck WAITING_APPROVAL run(Runbook RB-02)
|
||||
```
|
||||
|
||||
**影響**:approval token 安全性降級,回到舊 approval 驗證邏輯
|
||||
**post-rollback action**:診斷 token 簽章失敗原因(clock skew?Secret rotation?)
|
||||
|
||||
---
|
||||
|
||||
## RBP-07: GCP Ollama Primary 失聯緊急 Rollback
|
||||
|
||||
**觸發**:GCP-A 和 GCP-B 同時不可達,Local 111 也異常(見 Runbook RB-07)
|
||||
|
||||
**rollback 步驟**:
|
||||
```bash
|
||||
# 1. 強制 Redis 設定 fallback
|
||||
redis-cli SET "platform:ollama:topology" \
|
||||
'{"primary":"http://192.168.0.111:11434","secondary":null,"fallback":"gemini"}'
|
||||
|
||||
# 2. 確認 paid provider 路由已啟用(config.py GEMINI_API_KEY 有效)
|
||||
# 3. 監控:LLM call 應切換到 Gemini
|
||||
```
|
||||
|
||||
**影響**:推理效能降低(Local HDD)或費用增加(Gemini)
|
||||
**post-rollback action**:確認 GCP-A/B 健康狀態後,Redis 恢復三層拓撲
|
||||
|
||||
---
|
||||
|
||||
## 登記表
|
||||
|
||||
| Rollback ID | Phase | 已驗證次數 | 最後驗證日期 | 備註 |
|
||||
|------------|-------|-----------|------------|------|
|
||||
| RBP-01 | Phase 1 | 0 | — | 待 Phase 1 完成後驗證 |
|
||||
| RBP-02 | Phase 2 | 0 | — | 待 Redis 遷移開始後驗證 |
|
||||
| RBP-03 | Phase 4+ | 0 | — | Strangler Fig gate |
|
||||
| RBP-04 | Phase 5 | 0 | — | MCP Gateway |
|
||||
| RBP-05 | Phase 6 | 0 | — | EwoooC |
|
||||
| RBP-06 | Phase 8 | 0 | — | Approval token |
|
||||
| RBP-07 | 任何時間 | 0 | — | GCP Ollama 緊急 |
|
||||
|
||||
**Phase 8 gate 要求**:進入 `suggest → auto_remediate` 前,上表「已驗證次數」必須 ≥ 3 次(每個 RBP)。
|
||||
由 critic + db-expert + vuln-verifier 三方在 LOGBOOK 簽核。
|
||||
|
||||
*最後更新:2026-05-03(台北)*
|
||||
277
docs/awooop/inventory/INV-7-pr-cutting-plan.md
Normal file
277
docs/awooop/inventory/INV-7-pr-cutting-plan.md
Normal file
@@ -0,0 +1,277 @@
|
||||
# INV-7: PR Cutting Plan
|
||||
|
||||
**版本**:v1.0 初稿
|
||||
**日期**:2026-05-03(台北)
|
||||
**範圍**:Phase 2 全部重構 PR 的切割方案(refactor-specialist 設計)
|
||||
**用途**:避免大 PR 難以 review;每個 PR 獨立可合併、有 rollback 能力
|
||||
|
||||
---
|
||||
|
||||
## 原則
|
||||
|
||||
1. 每個 PR 必須可獨立合併(不依賴後 PR 才能運行)
|
||||
2. 每個 PR 有 rollback 方式(revert commit 或 feature flag)
|
||||
3. PR-01~05 無依賴,可並行;PR-06+ 有順序依賴
|
||||
|
||||
---
|
||||
|
||||
## PR 清單
|
||||
|
||||
### PR-01: telemetry.py hardcoded IP assert 移除
|
||||
|
||||
**範圍**:`apps/api/src/core/telemetry.py:71`(約 1-3 行)
|
||||
**前置**:無
|
||||
**解決**:P0-08 EwoooC 啟動失敗
|
||||
|
||||
```python
|
||||
# 修改前
|
||||
if "192.168.0.188" not in endpoint:
|
||||
raise ValueError(f"Forbidden OTEL endpoint: {endpoint}")
|
||||
|
||||
# 修改後
|
||||
allowed = [e.strip() for e in settings.ALLOWED_TELEMETRY_ENDPOINTS.split(",")]
|
||||
if not any(a in endpoint for a in allowed):
|
||||
raise ValueError(f"Forbidden OTEL endpoint: {endpoint}")
|
||||
# 同步在 config.py 加入 ALLOWED_TELEMETRY_ENDPOINTS = "192.168.0.188"(default)
|
||||
```
|
||||
|
||||
**review**:vuln-verifier
|
||||
**風險**:低(僅移除一個 IP assert)
|
||||
|
||||
---
|
||||
|
||||
### PR-02: decision_manager.py silence key 常數化
|
||||
|
||||
**範圍**:`apps/api/src/services/decision_manager.py:240`(約 2 行)
|
||||
**前置**:無
|
||||
**解決**:P1-24 兩處定義不一致
|
||||
|
||||
```python
|
||||
# 修改前(decision_manager.py:240)
|
||||
silence_key = f"telegram_silence:{target}"
|
||||
|
||||
# 修改後
|
||||
from apps.api.src.services.telegram_gateway import SILENCE_KEY_PREFIX
|
||||
silence_key = f"{SILENCE_KEY_PREFIX}{target}"
|
||||
```
|
||||
|
||||
**review**:debugger(確認 SILENCE_KEY_PREFIX = "telegram_silence:",功能等效)
|
||||
**風險**:低
|
||||
|
||||
---
|
||||
|
||||
### PR-03: ollama_auto_recovery.py 第二定義移除
|
||||
|
||||
**範圍**:`apps/api/src/services/ollama_auto_recovery.py:230`(約 5 行)
|
||||
**前置**:需確認 INV-1 中 `ollama:current_primary` 的所有寫入點
|
||||
**解決**:P0-11 GCP 三層拓撲遷移必裂
|
||||
|
||||
```python
|
||||
# 移除 ollama_auto_recovery.py 中第二個 ollama:current_primary 寫入點
|
||||
# 統一從 config.py 或 ollama_failover_manager.py 讀取,不在 auto_recovery 自行決定 primary
|
||||
```
|
||||
|
||||
**review**:tool-expert(確認 GCP 拓撲三層 failover 邏輯)
|
||||
**風險**:中(影響 Ollama failover 路徑)
|
||||
|
||||
---
|
||||
|
||||
### PR-04: registry.py `_provider` → `__provider`
|
||||
|
||||
**範圍**:`apps/api/src/plugins/mcp/registry.py:24-71`(約 20 行)
|
||||
**前置**:無
|
||||
**解決**:P1-05 `_provider` public 可繞過 audit
|
||||
|
||||
```python
|
||||
# 修改所有 self._provider → self.__provider
|
||||
# 加 @property provider 讀取(需要時)
|
||||
# 加 unit test:
|
||||
# with pytest.raises(AttributeError):
|
||||
# registry.__provider # double underscore = name mangling
|
||||
```
|
||||
|
||||
**review**:vuln-verifier
|
||||
**風險**:低(Python name mangling,只影響直接反射存取)
|
||||
|
||||
---
|
||||
|
||||
### PR-05: mcp_bridge.py namespace 動態化
|
||||
|
||||
**範圍**:`apps/api/src/plugins/mcp/mcp_bridge.py:592,602,631,647,681`(約 30 行)
|
||||
**前置**:無(Phase 1 schema 未完成前,先改為 from settings fallback)
|
||||
**解決**:P0-13 EwoooC K8s tool 無法使用
|
||||
|
||||
```python
|
||||
# Phase 1 前的過渡做法(PR-05a):
|
||||
DEFAULT_NAMESPACE = os.getenv("K8S_DEFAULT_NAMESPACE", "awoooi-prod")
|
||||
namespace = parameters.get("namespace", DEFAULT_NAMESPACE)
|
||||
|
||||
# Phase 2 後的完整做法(PR-05b,需 Phase 1 schema 先完成):
|
||||
namespace = parameters.get("namespace",
|
||||
get_project_contract(project_id).allowed_k8s_namespaces[0])
|
||||
```
|
||||
|
||||
**review**:tool-expert
|
||||
**風險**:低(有 env fallback,功能等效)
|
||||
|
||||
---
|
||||
|
||||
### PR-06: consensus_engine.py CONSENSUS_PREFIX 加 project 前綴
|
||||
|
||||
**範圍**:`apps/api/src/services/consensus_engine.py`(約 15 行)
|
||||
**前置**:**Redis 雙寫 Phase A 已完成**(舊 `consensus:` key 已雙寫新 key)
|
||||
**解決**:P0-12 consensus engine 多租戶跨 tenant 共用
|
||||
|
||||
```python
|
||||
# 修改前
|
||||
CONSENSUS_PREFIX = "consensus:"
|
||||
|
||||
# 修改後(consensus engine 接受 project_id 參數)
|
||||
def get_consensus_prefix(project_id: str) -> str:
|
||||
return f"{project_id}:consensus:"
|
||||
```
|
||||
|
||||
**review**:db-expert(確認 consensus 語義正確)、debugger(regression)
|
||||
**風險**:中(需 Redis 雙寫先完成)
|
||||
|
||||
---
|
||||
|
||||
### PR-07: security hardening(nonce + webhook replay + approval policy)
|
||||
|
||||
**範圍**:
|
||||
- `apps/api/src/services/security_interceptor.py:451-490`(nonce 重設計)
|
||||
- `apps/api/src/api/v1/webhooks.py:679-728`(webhook timestamp/nonce)
|
||||
- `apps/api/src/services/decision_manager.py`(requires_approval policy-derived)
|
||||
**前置**:ADR-116 已 Accepted(規格凍結)
|
||||
**解決**:P0-04、P0-05、P0-06(3 個 PoC 確認漏洞)
|
||||
|
||||
```python
|
||||
# security_interceptor.py: nonce 必須 HMAC(server_secret + nonce)
|
||||
# webhooks.py: 加 X-Timestamp header 驗證 ±5min window + Redis NX nonce
|
||||
# decision_manager.py: requires_approval 從 policy contract 讀,禁止 LLM output 決定
|
||||
```
|
||||
|
||||
**review**:vuln-verifier(PoC 重跑驗證修補有效)
|
||||
**風險**:高(安全修補,必須 PoC 驗證)
|
||||
|
||||
---
|
||||
|
||||
### PR-08: Repository project_id filter 批次 1
|
||||
|
||||
**範圍**:
|
||||
- `IncidentRepository`(`incidents` 表)
|
||||
- `PlaybookRepository`(`playbooks` 表)
|
||||
- `KnowledgeRepository`(`knowledge_entries` 表)
|
||||
**前置**:**Phase 1 schema migration 完成**(`project_id` 欄位已存在)
|
||||
**解決**:P0-03 30+ 業務表無 project_id filter
|
||||
|
||||
```python
|
||||
# 每個 repository 方法加 project_id 參數
|
||||
# 例:
|
||||
async def get_incident(self, incident_id: str, project_id: str) -> Incident:
|
||||
return await self.session.execute(
|
||||
select(Incident)
|
||||
.where(Incident.incident_id == incident_id)
|
||||
.where(Incident.project_id == project_id) # 新增
|
||||
)
|
||||
```
|
||||
|
||||
**review**:db-expert(query plan 確認)
|
||||
**風險**:中(約 200 行;需 regression test)
|
||||
|
||||
---
|
||||
|
||||
### PR-09: Repository project_id filter 批次 2
|
||||
|
||||
**範圍**:
|
||||
- `McpAuditRepository`(加 trace_id + project_id)
|
||||
- `AiDecisionRepository`(加 run_id + project_id)
|
||||
- `ApprovalRepository`(加 trace_id + project_id)
|
||||
**前置**:PR-08 合併後
|
||||
**解決**:P0-03 繼續
|
||||
|
||||
**review**:db-expert + debugger(trace_id 貫穿驗證)
|
||||
**風險**:中
|
||||
|
||||
---
|
||||
|
||||
### PR-10: Background loop 標記(ADR-123)
|
||||
|
||||
**範圍**:`apps/api/src/main.py`(31 個 loop,約 150 行)
|
||||
**前置**:ADR-123 已 Accepted
|
||||
**解決**:P0-07 31 個 loop 無 project_id
|
||||
|
||||
```python
|
||||
# 每個 loop 的 context 加上標記
|
||||
# platform_internal: asyncio.create_task 前設定 contextvars
|
||||
# legacy_awoooi_default: fallback project_id = 'awoooi'
|
||||
|
||||
async def _run_with_project_context(coro, project_id: str):
|
||||
token = project_id_var.set(project_id)
|
||||
try:
|
||||
await coro
|
||||
finally:
|
||||
project_id_var.reset(token)
|
||||
```
|
||||
|
||||
**review**:critic(確認標記正確)
|
||||
**風險**:中(影響所有 background loop 的 logging context)
|
||||
|
||||
---
|
||||
|
||||
### PR-11: AnomalyCounter per-project 改造
|
||||
|
||||
**範圍**:`apps/api/src/services/anomaly_counter.py:790`(約 80 行)
|
||||
**前置**:PR-10 完成(loop 有 project_id context 後才能傳入)
|
||||
**解決**:P1-17 AnomalyCounter 全域單例
|
||||
|
||||
```python
|
||||
# 修改前:全域單例,6 個 prefix 無 tenant
|
||||
_anomaly_counter = AnomalyCounter(get_redis())
|
||||
|
||||
# 修改後:per-project 工廠
|
||||
def get_anomaly_counter(project_id: str) -> AnomalyCounter:
|
||||
return AnomalyCounter(get_redis(), project_prefix=f"{project_id}:anomaly")
|
||||
```
|
||||
|
||||
**review**:debugger(確認 6 個 prefix 全數改造)
|
||||
**風險**:中
|
||||
|
||||
---
|
||||
|
||||
## 執行順序圖
|
||||
|
||||
```
|
||||
並行群組 G-D(無依賴,可同時開):
|
||||
PR-01, PR-02, PR-03, PR-04, PR-05
|
||||
|
||||
↓ Phase 1 Schema 完成後
|
||||
|
||||
並行群組 G-E(部分並行):
|
||||
PR-07(安全修補,優先)
|
||||
PR-06(需 Redis 雙寫 Phase A)
|
||||
PR-08(需 Phase 1 schema)
|
||||
|
||||
↓ PR-08 合併後
|
||||
|
||||
PR-09(依賴 PR-08)
|
||||
|
||||
↓ ADR-123 Accepted 後
|
||||
|
||||
PR-10(background loop 標記)
|
||||
|
||||
↓ PR-10 合併後
|
||||
|
||||
PR-11(AnomalyCounter)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 驗收標準
|
||||
|
||||
- [ ] PR-01~05 全部合併(無依賴,Phase 0 完成後即可)
|
||||
- [ ] PR-07 通過 vuln-verifier PoC 重跑驗證
|
||||
- [ ] PR-08~09 通過 db-expert review(query plan 確認)
|
||||
- [ ] PR-10~11 通過 regression test(31 個 loop logging context 正確)
|
||||
|
||||
*最後更新:2026-05-03(台北)*
|
||||
110
docs/awooop/inventory/INV-8-background-loop-catalog.md
Normal file
110
docs/awooop/inventory/INV-8-background-loop-catalog.md
Normal file
@@ -0,0 +1,110 @@
|
||||
# INV-8: Background Loop Catalog
|
||||
|
||||
**版本**:v1.0 初稿
|
||||
**日期**:2026-05-03(台北)
|
||||
**來源**:`apps/api/src/main.py` grep asyncio.create_task(實測 31 個)
|
||||
**用途**:ADR-123 Background Loop Migration Strategy 的執行基礎
|
||||
|
||||
---
|
||||
|
||||
## 概覽
|
||||
|
||||
| 統計 | 數量 |
|
||||
|------|------|
|
||||
| 總 background loop | 31 |
|
||||
| platform_internal | 2(#27、#30)|
|
||||
| legacy_awoooi_default(過渡)| 29 |
|
||||
| requires_project_id(Phase 2+ 改造後)| 0(初始)|
|
||||
|
||||
---
|
||||
|
||||
## 完整清單
|
||||
|
||||
> 行號來自 `apps/api/src/main.py`
|
||||
> 「頻率」為估計值,需 code review 確認
|
||||
|
||||
| # | 函數名 | 行號 | 頻率 | 標記 | 改造優先度 | 備註 |
|
||||
|---|--------|------|------|------|-----------|------|
|
||||
| 1 | `seed_playbooks_from_rules()` | 331 | 啟動一次 | `legacy_awoooi_default` | P3(低)| 只在啟動時跑一次,風險低 |
|
||||
| 2 | `backfill_redis_to_pg()` | 341 | 啟動一次 | `legacy_awoooi_default` | P3 | 啟動一次性 backfill |
|
||||
| 3 | `ensure_playbook_embeddings_indexed()` | 348 | 啟動一次 | `legacy_awoooi_default` | P3 | 啟動一次性 index 確認 |
|
||||
| 4 | `resend_stale_ready_tokens()` | 362 | 每 30s | `legacy_awoooi_default` | P1 | 涉及 approval token,Phase 8 前必改 |
|
||||
| 5 | `run_incident_analysis_sweeper()` | 373 | 每 5min | `legacy_awoooi_default` | P2 | 掃描 incident,多 tenant 後需分 project |
|
||||
| 6 | `run_asset_scanner_loop()` | 383 | 每 10min | `legacy_awoooi_default` | P2 | K8s asset 掃描 |
|
||||
| 7 | `run_rule_catalog_sync_loop()` | 393 | 每 5min | `legacy_awoooi_default` | P2 | 規則 catalog 同步 |
|
||||
| 8 | `run_capacity_scanner_loop()` | 403 | 每 15min | `legacy_awoooi_default` | P2 | 容量掃描 |
|
||||
| 9 | `run_compliance_scanner_loop()` | 413 | 每 30min | `legacy_awoooi_default` | P2 | 合規掃描 |
|
||||
| 10 | `run_aider_event_processor_loop()` | 423 | 每 30s | `legacy_awoooi_default` | P2 | aider 事件處理 |
|
||||
| 11 | `run_coverage_evaluator_loop()` | 432 | 每 1h | `legacy_awoooi_default` | P3 | 覆蓋率評估 |
|
||||
| 12 | `run_rule_stats_updater_loop()` | 442 | 每 15min | `legacy_awoooi_default` | P3 | 規則統計 |
|
||||
| 13 | `run_asset_change_tracker_loop()` | 452 | 每 5min | `legacy_awoooi_default` | P2 | 資產變更追蹤 |
|
||||
| 14 | `run_hermes_rule_quality_loop()` | 462 | 每 1h | `legacy_awoooi_default` | P2 | Hermes 規則品質 |
|
||||
| 15 | `run_capacity_forecaster_loop()` | 472 | 每 6h | `legacy_awoooi_default` | P3 | 容量預測 |
|
||||
| 16 | `run_daily_report_loop()` | 481 | 每日 | `legacy_awoooi_default` | P3 | 每日報告 |
|
||||
| 17 | `run_approval_timeout_resolver()` | 490 | 每 30s | `legacy_awoooi_default` | **P0** | 🔴 P1-15:無 trace_id;Phase 8 前必改 |
|
||||
| 18 | `run_evolver_loop()` | 499 | 每 1h | `legacy_awoooi_default` | P2 | playbook 進化(飛輪關鍵)|
|
||||
| 19 | `run_playbook_generation_governance_loop()` | 507 | 每 30min | `legacy_awoooi_default` | P2 | playbook 生成治理 |
|
||||
| 20 | `run_knowledge_decay_loop()` | 519 | 每 6h | `legacy_awoooi_default` | P2 | KM 知識衰退 |
|
||||
| 21 | `run_km_backfill_reconciler_loop()` | 529 | 每 1h | `legacy_awoooi_default` | P2 | KM backfill 核對 |
|
||||
| 22 | `run_aol_writeback_loop()` | 540 | 每 30s | `legacy_awoooi_default` | P1 | AI 飛輪 AOL writeback,KM 雙路徑 |
|
||||
| 23 | `_run_kb_rot_cleaner_loop()` | 585 | 每 6h | `legacy_awoooi_default` | P3 | KB 腐敗清理 |
|
||||
| 24 | `run_finetune_export_loop()` | 594 | 每日 | `legacy_awoooi_default` | P3 | finetune 資料匯出 |
|
||||
| 25 | `run_proactive_inspector_loop()` | 605 | 每 5min | `legacy_awoooi_default` | P2 | 主動巡檢 |
|
||||
| 26 | `run_offline_replay_loop()` | 614 | 每 15min | `legacy_awoooi_default` | P3 | 離線重放 |
|
||||
| 27 | `run_ai_slo_watchdog_loop()` | 623 | 每 5min | **`platform_internal`** | N/A(已完成)| SLO watchdog,project_id=__platform__ |
|
||||
| 28 | `run_governance_loop()` | 632 | 每 5min | `legacy_awoooi_default` | P2 | AI 治理主循環 |
|
||||
| 29 | `run_governance_dispatcher_loop()` | 640 | 每 30s | `legacy_awoooi_default` | P2 | 治理事件派送 |
|
||||
| 30 | `_run_model_version_tracker_loop()` | 701 | 每 1h | **`platform_internal`** | N/A(已完成)| model 版本追蹤,platform_resource |
|
||||
| 31 | (需確認)| 701+ | TBD | TBD | TBD | grep 計數 = 31,需確認第 31 個 |
|
||||
|
||||
---
|
||||
|
||||
## 改造策略(ADR-123)
|
||||
|
||||
### platform_internal(#27、#30)
|
||||
```python
|
||||
# main.py 改造示意
|
||||
async def _run_with_context(coro, project_id: str):
|
||||
ctx_token = project_id_ctx_var.set(project_id)
|
||||
try:
|
||||
await coro
|
||||
finally:
|
||||
project_id_ctx_var.reset(ctx_token)
|
||||
|
||||
# platform_internal loop:
|
||||
asyncio.create_task(_run_with_context(
|
||||
run_ai_slo_watchdog_loop(), "__platform__"
|
||||
))
|
||||
```
|
||||
|
||||
### legacy_awoooi_default(其餘 29 個)
|
||||
```python
|
||||
# Phase 2 過渡:fallback 到 awoooi
|
||||
asyncio.create_task(_run_with_context(
|
||||
run_incident_analysis_sweeper(), "awoooi" # 過渡期
|
||||
))
|
||||
# 退場時程:Phase 4 完成後 90 天內逐一改為真正 multi-tenant
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 改造優先度說明
|
||||
|
||||
| 優先度 | 說明 | 對應 loop |
|
||||
|--------|------|-----------|
|
||||
| P0(立即)| 直接影響安全性或正確性 | #17(approval_timeout_resolver)|
|
||||
| P1(Phase 4 前)| 影響 AwoooP run state 正確性 | #4、#22 |
|
||||
| P2(Phase 6 前)| 影響多 tenant 資料隔離 | #5~16、#18~20、#25、#28~29 |
|
||||
| P3(Phase 8 前)| 低頻率、低風險 | #1~3、#11~12、#15~16、#23~24、#26 |
|
||||
|
||||
---
|
||||
|
||||
## 驗收標準
|
||||
|
||||
- [ ] 第 31 個 loop 確認並列入
|
||||
- [ ] 全部 31 個 loop 在 main.py 有 logging context(project_id 可見)
|
||||
- [ ] `platform_internal` loop 帶 `project_id=__platform__`
|
||||
- [ ] `legacy_awoooi_default` loop 帶 `project_id=awoooi`(PR-10)
|
||||
- [ ] loop #17(approval_timeout_resolver)已補入 trace_id(PR-10 合併時)
|
||||
|
||||
*最後更新:2026-05-03(台北)*
|
||||
235
docs/awooop/inventory/INV-9-global-singleton-catalog.md
Normal file
235
docs/awooop/inventory/INV-9-global-singleton-catalog.md
Normal file
@@ -0,0 +1,235 @@
|
||||
# INV-9: Global Singleton Catalog
|
||||
|
||||
**版本**:v1.0 初稿
|
||||
**日期**:2026-05-03(台北)
|
||||
**來源**:`apps/api/src/` grep `_instance = None` / `get_*()` factory functions
|
||||
**用途**:ADR-124 Global Singleton Decomposition 的執行基礎
|
||||
|
||||
---
|
||||
|
||||
## 概覽
|
||||
|
||||
| 統計 | 數量 |
|
||||
|------|------|
|
||||
| 確認全域單例 | 13 |
|
||||
| 需要分解(per-project)| 9 |
|
||||
| 可保持 platform-level | 4 |
|
||||
|
||||
---
|
||||
|
||||
## 完整清單
|
||||
|
||||
### 1. TrustEngine
|
||||
|
||||
| 屬性 | 值 |
|
||||
|------|----|
|
||||
| 位置 | `apps/api/src/core/trust_engine.py:189` |
|
||||
| Factory | `get_trust_engine()` at line 393 |
|
||||
| 單例模式 | module-level `_trust_engine` variable |
|
||||
| 目前狀態 | 全域,所有 tenant 共用 |
|
||||
| 影響 | trust_records(無 project_id),trust score 跨 tenant 混算 |
|
||||
| 分解策略 | per-project trust engine instance,從 project contract 讀 trust_policy |
|
||||
| Phase | Phase 2(repository 加 project_id filter 後)|
|
||||
| 風險 | 高(TrustEngine 是飛輪核心,分解需仔細 regression test)|
|
||||
|
||||
---
|
||||
|
||||
### 2. ProviderRegistry(MCP)
|
||||
|
||||
| 屬性 | 值 |
|
||||
|------|----|
|
||||
| 位置 | `apps/api/src/plugins/mcp/registry.py:74` |
|
||||
| Factory | module-level `get_registry()` at line 80 / `_registry` at line 209 |
|
||||
| 目前狀態 | 全域,所有 tenant 共用 MCP provider 清單 |
|
||||
| 影響 | Phase 5 MCP Gateway:不同 tenant 應有不同可用 tool 清單 |
|
||||
| 分解策略 | platform-level registry(tool 清單)+ per-project grants(誰可用什麼);Registry 本身可保持 platform-level |
|
||||
| Phase | Phase 5(MCP Gateway 建立時)|
|
||||
| 風險 | 中(Registry 改為 read-only platform resource,grants 移到 DB)|
|
||||
|
||||
---
|
||||
|
||||
### 3. AnomalyCounter
|
||||
|
||||
| 屬性 | 值 |
|
||||
|------|----|
|
||||
| 位置 | `apps/api/src/services/anomaly_counter.py:85` |
|
||||
| Factory | `_anomaly_counter = AnomalyCounter(get_redis())` at line 800 |
|
||||
| 目前狀態 | 全域,6 個 Redis prefix 無 tenant 隔離 |
|
||||
| 影響 | P1-17:多 tenant 計數混算 |
|
||||
| 分解策略 | per-project 工廠:`get_anomaly_counter(project_id)` 注入 `project_prefix` |
|
||||
| Phase | Phase 2(PR-11)|
|
||||
| 風險 | 中(6 個 prefix 全部要改)|
|
||||
|
||||
---
|
||||
|
||||
### 4. AIRouter
|
||||
|
||||
| 屬性 | 值 |
|
||||
|------|----|
|
||||
| 位置 | `apps/api/src/services/ai_router.py:194` |
|
||||
| Factory | `_router = AIRouter()` at line 1324 |
|
||||
| 依賴 | AIProviderRegistry(923)、AIRouterExecutor(975)|
|
||||
| 目前狀態 | 全域,所有 tenant 共用 provider routing 決策 |
|
||||
| 影響 | EwoooC 可能使用 AWOOOI 的 model routing policy |
|
||||
| 分解策略 | AIRouter 接受 EffectivePolicy 參數(per-project policy 已在 Phase 4 設計);Router 邏輯保持 platform-level,policy 來源改為 per-project |
|
||||
| Phase | Phase 4(EffectivePolicy 設計完成後)|
|
||||
| 風險 | 中(ADR-052 AIRouter 已有 protocol 設計,改 policy 注入點)|
|
||||
|
||||
---
|
||||
|
||||
### 5. IntentClassifier
|
||||
|
||||
| 屬性 | 值 |
|
||||
|------|----|
|
||||
| 位置 | `apps/api/src/services/intent_classifier.py:357` |
|
||||
| Factory | `_classifier = IntentClassifier()` at line 744 |
|
||||
| 目前狀態 | 全域 |
|
||||
| 影響 | 不同 tenant 可能需要不同 intent classification domain |
|
||||
| 分解策略 | 短期保持 platform-level(intent schema 共用);長期 per-project intent domain config |
|
||||
| Phase | Phase 6(EwoooC 需要不同 intent domain 時才分解)|
|
||||
| 風險 | 低(intent classification 不帶 tenant 資料)|
|
||||
|
||||
---
|
||||
|
||||
### 6. TelegramGateway
|
||||
|
||||
| 屬性 | 值 |
|
||||
|------|----|
|
||||
| 位置 | `apps/api/src/services/telegram_gateway.py:1324` |
|
||||
| Factory | `_gateway = TelegramGateway()` at line 6375 |
|
||||
| 目前狀態 | 全域,持有 polling leader lock |
|
||||
| 影響 | `telegram:polling:leader` 是 platform_resource,但 gateway 本身可以 per-project |
|
||||
| 分解策略 | polling leader 保持 platform_resource;gateway 處理邏輯改為 per-project(不同 tenant 有不同 Telegram bot)|
|
||||
| Phase | Phase 7(Communication Hub 建立時)|
|
||||
| 風險 | 高(TelegramGateway 是最大的 service,約 6000+ 行)|
|
||||
|
||||
---
|
||||
|
||||
### 7. DecisionManager
|
||||
|
||||
| 屬性 | 值 |
|
||||
|------|----|
|
||||
| 位置 | `apps/api/src/services/decision_manager.py:1402` |
|
||||
| Factory | `_decision_manager = DecisionManager()` at line 3529 |
|
||||
| 目前狀態 | 全域,硬碼 silence key(P1-24 已在 PR-02 修補)|
|
||||
| 影響 | 決策邏輯跨 tenant 共用(不同 tenant 決策規則不同)|
|
||||
| 分解策略 | DecisionManager 接受 project_id 參數;規則從 project policy contract 讀 |
|
||||
| Phase | Phase 4(EffectivePolicy 設計完成後)|
|
||||
| 風險 | 極高(DecisionManager 是 Tier 3 核心,修改需 ADR 和架構審查)|
|
||||
|
||||
---
|
||||
|
||||
### 8. ConsensusEngine
|
||||
|
||||
| 屬性 | 值 |
|
||||
|------|----|
|
||||
| 位置 | `apps/api/src/services/consensus_engine.py:344` |
|
||||
| Factory | `_consensus_engine = ConsensusEngine()` at line 716 |
|
||||
| 目前狀態 | 全域,`CONSENSUS_PREFIX="consensus:"` 無 project |
|
||||
| 影響 | P0-12:多 tenant consensus 結果混合 |
|
||||
| 分解策略 | ConsensusEngine 接受 project_id;CONSENSUS_PREFIX 改為 `{project_id}:consensus:` |
|
||||
| Phase | Phase 2(PR-06)|
|
||||
| 風險 | 中(需 Redis 雙寫 Phase A 先完成)|
|
||||
|
||||
---
|
||||
|
||||
### 9. DecisionFusionAdapter
|
||||
|
||||
| 屬性 | 值 |
|
||||
|------|----|
|
||||
| 位置 | `apps/api/src/services/decision_fusion_adapter.py:538` |
|
||||
| 單例模式 | `_adapter_instance = None` |
|
||||
| 目前狀態 | 全域 |
|
||||
| 影響 | 決策融合結果跨 tenant |
|
||||
| 分解策略 | 同 DecisionManager(依賴它),Phase 4 一起改 |
|
||||
| Phase | Phase 4 |
|
||||
| 風險 | 中 |
|
||||
|
||||
---
|
||||
|
||||
### 10. FailoverAlerter
|
||||
|
||||
| 屬性 | 值 |
|
||||
|------|----|
|
||||
| 位置 | `apps/api/src/services/failover_alerter.py:395` |
|
||||
| 單例模式 | `_alerter_instance = None` |
|
||||
| 目前狀態 | 全域 |
|
||||
| 影響 | failover 告警沒有 tenant 隔離(GCP Ollama failover 是 platform_resource)|
|
||||
| 分解策略 | failover 告警本身是 `platform_resource`(保持全域),但告警發送目的地改為 per-project channel |
|
||||
| Phase | Phase 7(Communication Hub 建立後)|
|
||||
| 風險 | 低 |
|
||||
|
||||
---
|
||||
|
||||
### 11. HostRepairAgent
|
||||
|
||||
| 屬性 | 值 |
|
||||
|------|----|
|
||||
| 位置 | `apps/api/src/services/host_repair_agent.py:204` |
|
||||
| 單例模式 | `cls._instance = None` |
|
||||
| 目前狀態 | 全域 |
|
||||
| 影響 | 主機修復操作無 tenant scope(主機屬於 platform)|
|
||||
| 分解策略 | 保持 `platform_internal`(主機修復是 platform 操作,不是 tenant 操作)|
|
||||
| Phase | N/A(不需分解)|
|
||||
| 風險 | 低 |
|
||||
|
||||
---
|
||||
|
||||
### 12. AIProviderRegistry
|
||||
|
||||
| 屬性 | 值 |
|
||||
|------|----|
|
||||
| 位置 | `apps/api/src/services/ai_router.py:923` |
|
||||
| 目前狀態 | AIRouter 的內部 registry,全域 |
|
||||
| 影響 | provider 清單是 platform resource(GCP Ollama/Gemini/etc 所有 tenant 共用)|
|
||||
| 分解策略 | 保持 platform-level registry;per-project policy 控制各 tenant 的可用 provider |
|
||||
| Phase | Phase 4(EffectivePolicy 設計)|
|
||||
| 風險 | 低 |
|
||||
|
||||
---
|
||||
|
||||
### 13. AIRouterExecutor
|
||||
|
||||
| 屬性 | 值 |
|
||||
|------|----|
|
||||
| 位置 | `apps/api/src/services/ai_router.py:975` |
|
||||
| 目前狀態 | AIRouter 的執行器,全域 |
|
||||
| 影響 | 執行邏輯共用,但 policy 會 per-project |
|
||||
| 分解策略 | 同 AIRouter(Phase 4)|
|
||||
| Phase | Phase 4 |
|
||||
| 風險 | 中 |
|
||||
|
||||
---
|
||||
|
||||
## 分解優先序(ADR-124 執行計畫)
|
||||
|
||||
| Phase | 要分解的單例 | 策略 |
|
||||
|-------|-----------|------|
|
||||
| Phase 2 | AnomalyCounter(PR-11)、ConsensusEngine(PR-06)| 注入 project_id 參數 |
|
||||
| Phase 4 | AIRouter、AIProviderRegistry、AIRouterExecutor、DecisionFusionAdapter | EffectivePolicy 注入 |
|
||||
| Phase 5 | ProviderRegistry | per-project grants 移到 DB,registry 保持 platform-level |
|
||||
| Phase 6 | IntentClassifier | EwoooC 需要不同 intent domain 時 |
|
||||
| Phase 7 | TelegramGateway、FailoverAlerter | Communication Hub 建立 |
|
||||
| **特別謹慎** | **TrustEngine**、**DecisionManager** | Tier 3,需額外 ADR 和架構審查 |
|
||||
|
||||
> **TrustEngine 和 DecisionManager 分解需要統帥批准(Tier 3 核心,RED_ZONES.md)**
|
||||
|
||||
---
|
||||
|
||||
## 不需分解的單例(platform_resource)
|
||||
|
||||
| 單例 | 理由 |
|
||||
|------|------|
|
||||
| HostRepairAgent | 主機修復 = platform 操作,不屬於任何 tenant |
|
||||
| AIProviderRegistry | provider 清單 = platform resource |
|
||||
| TelegramGateway 的 polling leader | platform_resource(鎖,全域唯一)|
|
||||
|
||||
---
|
||||
|
||||
## 驗收標準
|
||||
|
||||
- [ ] 13 個單例全部分類完成(per-project / platform-level / platform_resource)
|
||||
- [ ] TrustEngine 和 DecisionManager 的分解計畫需獨立 ADR(Tier 3 審查)
|
||||
- [ ] Phase 2 前:AnomalyCounter + ConsensusEngine 分解完成
|
||||
|
||||
*最後更新:2026-05-03(台北)*
|
||||
Reference in New Issue
Block a user