wooo/awoooi
1
1
Fork 0
Code Issues 2 Pull Requests 1 Actions 2 Packages Projects Releases Wiki Activity

feat(awooop): Phase 1-8 完整實作 — AwoooP Agent Platform 六平面架構
Some checks failed
run-migration / migrate (push) Failing after 59s
Details
Code Review / ai-code-review (push) Successful in 1m8s
Details
Type Sync Check / check-type-sync (push) Successful in 2m27s
Details

Browse Source 
## Phase 1-3: Control Plane + Contract System
- awooop_phase1_control_plane_2026-05-04.sql: 12 張核心表 + RLS
- awooop_phase1_batch1_rls_2026-05-04.sql: 全部 FORCE RLS + GRANT
- packages/awooop-contracts/: 六合約 JSON Schema + golden fixtures
- src/models/awooop_contracts.py: Pydantic v2 contract models（extra=forbid）
- src/repositories/contract_repository.py: contract lifecycle（draft→published→active）
- src/services/contract_service.py: HMAC publish sig + Redis multi-sig activate
- src/services/schema_validator.py: LLM output validator（retry×3, E-SCHEMA-001）

## Phase 2: Tenant Isolation
- awooop_phase2_budget_ledger_2026-05-04.sql: budget_ledger + RLS
- src/services/budget_service.py: Token Budget Hard Kill 三層防線
- src/core/context.py: PROJECT_ID ContextVar（31 background loop 自動繼承）
- src/db/base.py + models.py: project_id 欄位 + RLS set_config 注入
- src/hermes/nl_gateway.py: project_id Redis key 前綴（Phase A 雙寫）
- src/services/anomaly_counter.py: per-project 改造（Phase A fallback）

## Phase 4: Platform Shell in Shadow Mode
- awooop_phase4_run_state_2026-05-04.sql: run_state + step_journal + idempotency
- src/services/run_state_machine.py: 8-state FSM + SKIP LOCKED + stale reaper
- src/services/platform_runtime.py: UUID v7 + W3C trace_id + shadow_execute
- src/services/audit_sink.py: PII/secret redaction 9 patterns
- src/api/v1/platform/runs.py: POST/GET /v1/platform/runs（Router→Service 架構）
- src/workers/platform_worker.py: SKIP LOCKED worker + heartbeat + reaper loop
- src/main.py: platform router + lifespan worker start/stop

## Phase 5: MCP Gateway 五閘門
- awooop_phase5_mcp_gateway_2026-05-04.sql: 4 表 + RLS
- src/plugins/mcp/gateway.py: McpGateway（Gate 1~5, E-MCP-GATE-001~009）
- src/plugins/mcp/redaction_middleware.py: 雙層 redaction + 16K 截斷
- src/plugins/mcp/registry.py: __provider name mangling（ADR-116）
- src/plugins/mcp/credential_resolver.py: k8s secret ref 解析
- tests/test_mcp_credential_isolation.py: 10 個迴歸測試（secret leak 防再現）

## Phase 6-8: EwoooC + Channel Hub + Approval Token
- awooop_phase6_ewoooc_onboarding_2026-05-04.sql: ewoooc tenant + 4 read-only MCP tools
- awooop_phase7_channel_hub_2026-05-04.sql: conversation_event + outbound_message
- src/services/provider_proxy.py: ProviderProxy + PlatformEnvelope（ADR-115）
- src/services/channel_hub.py: Telegram inbound mirror + Progressive Feedback（30s）
- src/services/awooop_approval_token.py: HS256 + jti NX replay 防護 + suggest mode

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Your Name
2026-05-04 19:31:53 +08:00
parent 0a90dab1e9
commit 8629ac709b
69 changed files with 10999 additions and 77 deletions
Download Patch File Download Diff File Expand all files Collapse all files

66
apps/api/migrations/awooop_phase2_budget_ledger_2026-05-04.sql Normal file
View File

@@ -0,0 +1,66 @@
-- AwoooP Phase 2.6: budget_ledger 建表 + 欄位定義
-- 2026-05-04 ogt + Claude Sonnet 4.6（ADR-120 D5 實作）
--
-- 防止 $47k 事故的三層 Hard Kill 架構中的 accounting 層：
-- - 每次 LLM call 完成後寫入一筆 ledger record
-- - 供 Tenant Budget Cache 計算 / 儀表板消費統計 / 告警閾值觸發
--
-- Phase 1 Control Plane migration 必須先執行（awooop_projects 表存在）
-- awooop_run_state 欄位在 Phase 3 SAGA 實作後補加
-- =========================================================
-- STEP 1: 建立 budget_ledger 表
-- =========================================================
CREATE TABLE IF NOT EXISTS budget_ledger (
id UUID DEFAULT gen_random_uuid() PRIMARY KEY,
project_id VARCHAR(64) NOT NULL DEFAULT 'awoooi',
agent_id VARCHAR(128),
run_id UUID,
model VARCHAR(64),
provider VARCHAR(32),
prompt_tokens INT,
completion_tokens INT,
cost_usd NUMERIC(10, 4) NOT NULL DEFAULT 0.0000,
recorded_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
COMMENT ON TABLE budget_ledger IS 'ADR-120: 每次 LLM call 的 token/cost accounting 記錄';
COMMENT ON COLUMN budget_ledger.cost_usd IS 'prompt + completion token 的估算費用（USD）';
-- =========================================================
-- STEP 2: Index（分析 + 查詢效率）
-- =========================================================
CREATE INDEX IF NOT EXISTS idx_budget_ledger_project_date
ON budget_ledger(project_id, recorded_at DESC);
CREATE INDEX IF NOT EXISTS idx_budget_ledger_run
ON budget_ledger(run_id)
WHERE run_id IS NOT NULL;
CREATE INDEX IF NOT EXISTS idx_budget_ledger_agent
ON budget_ledger(project_id, agent_id, recorded_at DESC)
WHERE agent_id IS NOT NULL;
-- =========================================================
-- STEP 3: RLS（ADR-118 多租戶隔離）
-- =========================================================
ALTER TABLE budget_ledger ENABLE ROW LEVEL SECURITY;
ALTER TABLE budget_ledger FORCE ROW LEVEL SECURITY;
DROP POLICY IF EXISTS budget_ledger_tenant_isolation ON budget_ledger;
CREATE POLICY budget_ledger_tenant_isolation ON budget_ledger
FOR ALL TO awooop_app
USING (project_id = current_setting('app.project_id', TRUE))
WITH CHECK (project_id = current_setting('app.project_id', TRUE));
-- =========================================================
-- STEP 4: GRANT
-- =========================================================
GRANT SELECT, INSERT ON budget_ledger TO awooop_app;
-- =========================================================
-- 驗收查詢
-- =========================================================
-- SELECT tablename, rowsecurity FROM pg_tables WHERE tablename = 'budget_ledger';
-- -- 結果：rowsecurity = true
-- SELECT count(*) FROM budget_ledger; -- = 0（剛建）

200
apps/api/migrations/awooop_phase4_run_state_2026-05-04.sql Normal file
View File

@@ -0,0 +1,200 @@
-- AwoooP Phase 4: Platform Shell in Shadow Mode
-- Run State Machine 持久化表
-- 2026-05-04 ogt + Claude Sonnet 4.6（ADR-114/ADR-119）
--
-- 前置：Phase 1 control plane（awooop_projects）必須已執行
--
-- 三表：
-- awooop_run_state — Run FSM 主表（lease + heartbeat + SKIP LOCKED）
-- awooop_run_step_journal — SAGA step journal（tool call + 補償指令，ADR-119）
-- awooop_run_idempotency — 去重冪等表（ADR-114）
-- =========================================================
-- STEP 1: awooop_run_state
-- =========================================================
CREATE TABLE IF NOT EXISTS awooop_run_state (
run_id UUID PRIMARY KEY,
project_id VARCHAR(64) NOT NULL REFERENCES awooop_projects(project_id),
agent_id VARCHAR(128) NOT NULL,
-- FSM 狀態
state VARCHAR(32) NOT NULL DEFAULT 'pending'
CHECK (state IN (
'pending','running','waiting_tool',
'waiting_approval','completed','failed',
'cancelled','timeout'
)),
-- Worker lease（SKIP LOCKED 防 double-pickup）
lease_until TIMESTAMPTZ,
heartbeat_at TIMESTAMPTZ,
worker_id VARCHAR(128),
-- Retry 計數
attempt_count SMALLINT NOT NULL DEFAULT 0,
max_attempts SMALLINT NOT NULL DEFAULT 3,
-- Observability
trace_id VARCHAR(128),
-- Trigger 來源
trigger_type VARCHAR(32),
trigger_ref VARCHAR(256), -- channel_event_id / schedule_id / etc.
-- Shadow mode flag
is_shadow BOOLEAN NOT NULL DEFAULT TRUE,
-- Artifact integrity（ADR-112）
input_sha256 CHAR(64),
output_sha256 CHAR(64),
-- Budget
cost_usd NUMERIC(10, 4) NOT NULL DEFAULT 0.0000,
step_count SMALLINT NOT NULL DEFAULT 0,
-- 結果
error_code VARCHAR(64),
error_detail TEXT,
-- 時間戳記
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
started_at TIMESTAMPTZ,
completed_at TIMESTAMPTZ,
timeout_at TIMESTAMPTZ
);
COMMENT ON TABLE awooop_run_state IS
'ADR-114: Run FSM 主表，SKIP LOCKED worker lease';
COMMENT ON COLUMN awooop_run_state.is_shadow IS
'Phase 4 shadow mode：TRUE = 不產生 user response，不執行 destructive tool';
-- Index: worker 掃 PENDING（SKIP LOCKED 用）
CREATE INDEX IF NOT EXISTS idx_run_state_pending
ON awooop_run_state (project_id, created_at)
WHERE state = 'pending' AND lease_until IS NULL;
-- Index: stale run reaper（找 lease 過期的 running run）
CREATE INDEX IF NOT EXISTS idx_run_state_stale
ON awooop_run_state (lease_until)
WHERE state = 'running' AND lease_until IS NOT NULL;
-- Index: project timeline（dashboard 查詢）
CREATE INDEX IF NOT EXISTS idx_run_state_project_timeline
ON awooop_run_state (project_id, created_at DESC);
-- Index: trace_id（跨系統追蹤）
CREATE INDEX IF NOT EXISTS idx_run_state_trace_id
ON awooop_run_state (trace_id)
WHERE trace_id IS NOT NULL;
-- =========================================================
-- STEP 2: awooop_run_step_journal（SAGA step journal，ADR-119）
-- =========================================================
CREATE TABLE IF NOT EXISTS awooop_run_step_journal (
step_id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
run_id UUID NOT NULL REFERENCES awooop_run_state(run_id) ON DELETE CASCADE,
project_id VARCHAR(64) NOT NULL,
-- Step 順序（每個 run 內遞增）
step_seq SMALLINT NOT NULL,
-- Tool call 資訊
tool_name VARCHAR(128) NOT NULL,
mcp_gateway_id VARCHAR(128),
-- Artifact integrity（ADR-112）
input_hash CHAR(64),
output_hash CHAR(64),
-- SAGA 補償指令（JSON）
compensation_json JSONB,
-- 執行結果
result_status VARCHAR(16) NOT NULL DEFAULT 'pending'
CHECK (result_status IN ('pending','success','failed','compensated')),
error_code VARCHAR(64),
-- Shadow 攔截記錄
was_blocked BOOLEAN NOT NULL DEFAULT FALSE,
block_reason VARCHAR(128),
-- 時間
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
completed_at TIMESTAMPTZ,
latency_ms INTEGER
);
COMMENT ON TABLE awooop_run_step_journal IS
'ADR-119 SAGA step journal：每個 tool call 獨立記錄 + 補償指令';
CREATE UNIQUE INDEX IF NOT EXISTS uix_run_step_seq
ON awooop_run_step_journal (run_id, step_seq);
CREATE INDEX IF NOT EXISTS idx_run_step_run_id
ON awooop_run_step_journal (run_id, step_seq);
-- =========================================================
-- STEP 3: awooop_run_idempotency（ADR-114 去重冪等）
-- =========================================================
CREATE TABLE IF NOT EXISTS awooop_run_idempotency (
idempotency_id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
project_id VARCHAR(64) NOT NULL,
channel_type VARCHAR(32) NOT NULL,
provider_event_id VARCHAR(256) NOT NULL,
-- 映射到的 run
run_id UUID NOT NULL REFERENCES awooop_run_state(run_id),
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
COMMENT ON TABLE awooop_run_idempotency IS
'ADR-114: (project_id, channel_type, provider_event_id) → run_id 去重';
CREATE UNIQUE INDEX IF NOT EXISTS uix_run_idempotency_key
ON awooop_run_idempotency (project_id, channel_type, provider_event_id);
CREATE INDEX IF NOT EXISTS idx_run_idempotency_run_id
ON awooop_run_idempotency (run_id);
-- =========================================================
-- STEP 4: RLS（ADR-118 多租戶隔離）
-- =========================================================
ALTER TABLE awooop_run_state ENABLE ROW LEVEL SECURITY;
ALTER TABLE awooop_run_state FORCE ROW LEVEL SECURITY;
ALTER TABLE awooop_run_step_journal ENABLE ROW LEVEL SECURITY;
ALTER TABLE awooop_run_step_journal FORCE ROW LEVEL SECURITY;
ALTER TABLE awooop_run_idempotency ENABLE ROW LEVEL SECURITY;
ALTER TABLE awooop_run_idempotency FORCE ROW LEVEL SECURITY;
DROP POLICY IF EXISTS run_state_tenant_isolation ON awooop_run_state;
CREATE POLICY run_state_tenant_isolation ON awooop_run_state
FOR ALL TO awooop_app
USING (project_id = current_setting('app.project_id', TRUE))
WITH CHECK (project_id = current_setting('app.project_id', TRUE));
DROP POLICY IF EXISTS run_step_journal_tenant_isolation ON awooop_run_step_journal;
CREATE POLICY run_step_journal_tenant_isolation ON awooop_run_step_journal
FOR ALL TO awooop_app
USING (project_id = current_setting('app.project_id', TRUE))
WITH CHECK (project_id = current_setting('app.project_id', TRUE));
DROP POLICY IF EXISTS run_idempotency_tenant_isolation ON awooop_run_idempotency;
CREATE POLICY run_idempotency_tenant_isolation ON awooop_run_idempotency
FOR ALL TO awooop_app
USING (project_id = current_setting('app.project_id', TRUE))
WITH CHECK (project_id = current_setting('app.project_id', TRUE));
-- =========================================================
-- STEP 5: GRANT
-- =========================================================
GRANT SELECT, INSERT, UPDATE ON awooop_run_state TO awooop_app;
GRANT SELECT, INSERT, UPDATE ON awooop_run_step_journal TO awooop_app;
GRANT SELECT, INSERT ON awooop_run_idempotency TO awooop_app;
-- =========================================================
-- 驗收查詢
-- =========================================================
-- SELECT tablename, rowsecurity FROM pg_tables
-- WHERE tablename IN ('awooop_run_state','awooop_run_step_journal','awooop_run_idempotency');
-- 預期：所有 rowsecurity = true

198
apps/api/migrations/awooop_phase5_mcp_gateway_2026-05-04.sql Normal file
View File

@@ -0,0 +1,198 @@
-- =============================================================================
-- AwoooP Phase 5: MCP Gateway 四表
-- ADR-116（五閘門 enforcement）+ ADR-118（credential isolation）
-- 2026-05-04 ogt + Claude Sonnet 4.6
-- =============================================================================
-- 執行順序：
-- 1. awooop_mcp_tool_registry — Tool 白名單
-- 2. awooop_mcp_grants — Agent × Tool 授權記錄
-- 3. awooop_mcp_credential_refs — k8s Secret 參照（不儲存明文）
-- 4. awooop_mcp_gateway_audit — 每次 gateway call 稽核
-- =============================================================================
BEGIN;
-- ---------------------------------------------------------------------------
-- 1. awooop_mcp_tool_registry — Tool 白名單（Gate 3: Tool）
-- ---------------------------------------------------------------------------
CREATE TABLE IF NOT EXISTS awooop_mcp_tool_registry (
tool_id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
project_id VARCHAR(64) NOT NULL
REFERENCES awooop_projects(project_id) ON DELETE CASCADE,
tool_name VARCHAR(128) NOT NULL,
tool_type VARCHAR(32) NOT NULL, -- 'builtin' | 'mcp_server' | 'custom'
description TEXT,
allowed_scopes JSONB NOT NULL DEFAULT '[]'::jsonb, -- ["read","write","admin"]
environment_tags JSONB NOT NULL DEFAULT '{}'::jsonb, -- {"env": "prod"} gate 4 用
is_active BOOLEAN NOT NULL DEFAULT TRUE,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
CONSTRAINT chk_tool_type
CHECK (tool_type IN ('builtin','mcp_server','custom')),
CONSTRAINT chk_allowed_scopes_array
CHECK (jsonb_typeof(allowed_scopes) = 'array'),
CONSTRAINT uix_tool_registry_project_name
UNIQUE (project_id, tool_name)
);
CREATE INDEX IF NOT EXISTS idx_mcp_tool_registry_project
ON awooop_mcp_tool_registry (project_id, is_active);
-- ---------------------------------------------------------------------------
-- 2. awooop_mcp_grants — Agent × Tool 授權（Gate 2: Agent + Gate 3: Tool）
-- ---------------------------------------------------------------------------
CREATE TABLE IF NOT EXISTS awooop_mcp_grants (
grant_id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
project_id VARCHAR(64) NOT NULL
REFERENCES awooop_projects(project_id) ON DELETE CASCADE,
agent_id VARCHAR(128) NOT NULL, -- awooop_agents.agent_id
tool_id UUID NOT NULL
REFERENCES awooop_mcp_tool_registry(tool_id) ON DELETE CASCADE,
granted_by VARCHAR(128) NOT NULL, -- principal（human user / system）
granted_scopes JSONB NOT NULL DEFAULT '[]'::jsonb, -- subset of tool.allowed_scopes
expires_at TIMESTAMPTZ, -- NULL = 永不過期
is_revoked BOOLEAN NOT NULL DEFAULT FALSE,
revoked_at TIMESTAMPTZ,
revoked_by VARCHAR(128),
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
CONSTRAINT chk_grant_scopes_array
CHECK (jsonb_typeof(granted_scopes) = 'array'),
CONSTRAINT chk_revoke_consistency
CHECK (
(is_revoked = FALSE AND revoked_at IS NULL AND revoked_by IS NULL)
OR
(is_revoked = TRUE AND revoked_at IS NOT NULL)
),
CONSTRAINT uix_mcp_grant_agent_tool
UNIQUE (project_id, agent_id, tool_id)
);
CREATE INDEX IF NOT EXISTS idx_mcp_grants_lookup
ON awooop_mcp_grants (project_id, agent_id, tool_id)
WHERE is_revoked = FALSE;
CREATE INDEX IF NOT EXISTS idx_mcp_grants_expiry
ON awooop_mcp_grants (expires_at)
WHERE is_revoked = FALSE AND expires_at IS NOT NULL;
-- ---------------------------------------------------------------------------
-- 3. awooop_mcp_credential_refs — k8s Secret 參照（ADR-118 credential isolation）
-- 只儲存 ref 路徑 + sha256 指紋；明文絕不入庫
-- ---------------------------------------------------------------------------
CREATE TABLE IF NOT EXISTS awooop_mcp_credential_refs (
ref_id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
tool_id UUID NOT NULL
REFERENCES awooop_mcp_tool_registry(tool_id) ON DELETE CASCADE,
project_id VARCHAR(64) NOT NULL
REFERENCES awooop_projects(project_id) ON DELETE CASCADE,
-- k8s secret ref：格式 "namespace/secret-name#key"
k8s_secret_ref VARCHAR(256) NOT NULL,
-- sha256(actual_secret_value) — 用於 audit；不可還原原值
value_sha256 VARCHAR(64),
description TEXT,
is_active BOOLEAN NOT NULL DEFAULT TRUE,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
rotated_at TIMESTAMPTZ,
CONSTRAINT chk_k8s_ref_format
CHECK (k8s_secret_ref ~ '^[a-z0-9-]+/[a-z0-9-]+#[a-zA-Z0-9_-]+$'),
CONSTRAINT chk_value_sha256_hex
CHECK (value_sha256 IS NULL OR value_sha256 ~ '^[0-9a-f]{64}$'),
CONSTRAINT uix_credential_ref_tool
UNIQUE (tool_id, k8s_secret_ref)
);
CREATE INDEX IF NOT EXISTS idx_mcp_cred_refs_tool
ON awooop_mcp_credential_refs (tool_id)
WHERE is_active = TRUE;
-- ---------------------------------------------------------------------------
-- 4. awooop_mcp_gateway_audit — Gateway call 稽核日誌（ADR-116 P1-09）
-- 不儲存 raw input/output；只儲存 hash + 結果狀態
-- ---------------------------------------------------------------------------
CREATE TABLE IF NOT EXISTS awooop_mcp_gateway_audit (
call_id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
project_id VARCHAR(64) NOT NULL,
run_id UUID, -- FK soft（run 可能不存在）
trace_id VARCHAR(128),
agent_id VARCHAR(128),
tool_id UUID NOT NULL
REFERENCES awooop_mcp_tool_registry(tool_id),
tool_name VARCHAR(128) NOT NULL,
credential_ref VARCHAR(256), -- k8s_secret_ref 路徑（不含 key value）
input_hash VARCHAR(64), -- sha256(canonical input JSON)
output_hash VARCHAR(64), -- sha256(canonical output JSON)
gate_result JSONB NOT NULL DEFAULT '{}'::jsonb,
-- {"gate1_project": true, "gate2_agent": true, "gate3_tool": true,
-- "gate4_env": true, "gate5_approval": true}
result_status VARCHAR(16) NOT NULL, -- 'success' | 'blocked' | 'failed' | 'timeout'
block_gate SMALLINT, -- 哪個 gate 攔截（1-5，NULL=未攔截）
block_reason VARCHAR(256),
latency_ms INTEGER,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
CONSTRAINT chk_gateway_result_status
CHECK (result_status IN ('success','blocked','failed','timeout')),
CONSTRAINT chk_block_gate_range
CHECK (block_gate IS NULL OR (block_gate >= 1 AND block_gate <= 5)),
CONSTRAINT chk_input_hash_hex
CHECK (input_hash IS NULL OR input_hash ~ '^[0-9a-f]{64}$'),
CONSTRAINT chk_output_hash_hex
CHECK (output_hash IS NULL OR output_hash ~ '^[0-9a-f]{64}$')
);
-- 查詢熱路徑：by project + run
CREATE INDEX IF NOT EXISTS idx_mcp_audit_run
ON awooop_mcp_gateway_audit (project_id, run_id, created_at DESC);
-- 查詢熱路徑：blocked calls 分析
CREATE INDEX IF NOT EXISTS idx_mcp_audit_blocked
ON awooop_mcp_gateway_audit (project_id, block_gate, created_at DESC)
WHERE result_status = 'blocked';
-- 時序熱路徑（recent calls）
CREATE INDEX IF NOT EXISTS idx_mcp_audit_recent
ON awooop_mcp_gateway_audit (project_id, created_at DESC);
-- =============================================================================
-- Row Level Security
-- =============================================================================
ALTER TABLE awooop_mcp_tool_registry ENABLE ROW LEVEL SECURITY;
ALTER TABLE awooop_mcp_grants ENABLE ROW LEVEL SECURITY;
ALTER TABLE awooop_mcp_credential_refs ENABLE ROW LEVEL SECURITY;
ALTER TABLE awooop_mcp_gateway_audit ENABLE ROW LEVEL SECURITY;
ALTER TABLE awooop_mcp_tool_registry FORCE ROW LEVEL SECURITY;
ALTER TABLE awooop_mcp_grants FORCE ROW LEVEL SECURITY;
ALTER TABLE awooop_mcp_credential_refs FORCE ROW LEVEL SECURITY;
ALTER TABLE awooop_mcp_gateway_audit FORCE ROW LEVEL SECURITY;
-- awooop_app role：只能看自己 project 的資料
CREATE POLICY mcp_tool_registry_tenant_isolation ON awooop_mcp_tool_registry
USING (
project_id = current_setting('app.project_id', TRUE)
OR current_setting('app.project_id', TRUE) IS NULL
);
CREATE POLICY mcp_grants_tenant_isolation ON awooop_mcp_grants
USING (
project_id = current_setting('app.project_id', TRUE)
OR current_setting('app.project_id', TRUE) IS NULL
);
CREATE POLICY mcp_credential_refs_tenant_isolation ON awooop_mcp_credential_refs
USING (
project_id = current_setting('app.project_id', TRUE)
OR current_setting('app.project_id', TRUE) IS NULL
);
CREATE POLICY mcp_gateway_audit_tenant_isolation ON awooop_mcp_gateway_audit
USING (
project_id = current_setting('app.project_id', TRUE)
OR current_setting('app.project_id', TRUE) IS NULL
);
COMMIT;

93
apps/api/migrations/awooop_phase6_ewoooc_onboarding_2026-05-04.sql Normal file
View File

@@ -0,0 +1,93 @@
-- =============================================================================
-- AwoooP Phase 6: EwoooC Tenant Onboarding
-- ADR-115（Tenant Onboarding 模板）
-- 2026-05-04 ogt + Claude Sonnet 4.6
-- =============================================================================
-- 執行前提：Phase 1 migration（awooop_phase1_control_plane_2026-05-04.sql）已執行
-- 說明：
-- EwoooC 是第二個接入 AwoooP 的租戶（awoooi 為第一個）
-- migration_mode = 'shadow' 啟動，進入 canary 前需通過 shadow run 驗證
-- budget_limit_usd = 50.0（初始限制，可調整）
-- 4 個 read-only MCP tools 預先在白名單中（不需 approval）
-- =============================================================================
BEGIN;
-- ---------------------------------------------------------------------------
-- Step 1: INSERT awooop_projects（EwoooC 租戶）
-- ---------------------------------------------------------------------------
INSERT INTO awooop_projects (
project_id,
display_name,
migration_mode,
budget_limit_usd,
allowed_channels,
metadata
) VALUES (
'ewoooc',
'EwoooC Business Platform',
'shadow', -- Phase 6 啟動模式；通過驗證後升級為 canary
50.00, -- 初始 USD 預算上限
'["telegram","api"]'::jsonb,
'{
"onboarded_at": "2026-05-04",
"tier": "business",
"ollama_topology": "gcp_three_tier",
"note": "ADR-115 EwoooC 接入，共用 GCP Ollama 三層拓撲"
}'::jsonb
) ON CONFLICT (project_id) DO NOTHING;
-- ---------------------------------------------------------------------------
-- Step 2: awooop_mcp_tool_registry — 4 個 read-only MCP tools
-- （ewoooc 初始只允許唯讀工具，write/admin 需另外建 grant）
-- ---------------------------------------------------------------------------
-- Tool 1: k8s_get — 查詢 k8s resource（唯讀）
INSERT INTO awooop_mcp_tool_registry (
project_id, tool_name, tool_type, description, allowed_scopes, environment_tags
) VALUES (
'ewoooc',
'k8s_get',
'builtin',
'kubectl get 唯讀查詢（pod/deployment/service 狀態）',
'["read"]'::jsonb,
'{"env": "any"}'::jsonb
) ON CONFLICT (project_id, tool_name) DO NOTHING;
-- Tool 2: signoz_query — 查詢 SigNoz metrics/traces（唯讀）
INSERT INTO awooop_mcp_tool_registry (
project_id, tool_name, tool_type, description, allowed_scopes, environment_tags
) VALUES (
'ewoooc',
'signoz_query',
'builtin',
'SigNoz metrics/traces 查詢（唯讀，無告警修改）',
'["read"]'::jsonb,
'{"env": "any"}'::jsonb
) ON CONFLICT (project_id, tool_name) DO NOTHING;
-- Tool 3: incident_read — 讀取 EwoooC incident 記錄（唯讀，RLS 隔離）
INSERT INTO awooop_mcp_tool_registry (
project_id, tool_name, tool_type, description, allowed_scopes, environment_tags
) VALUES (
'ewoooc',
'incident_read',
'builtin',
'Incident 查詢（僅限 ewoooc 租戶資料，RLS 強制隔離）',
'["read"]'::jsonb,
'{"env": "any"}'::jsonb
) ON CONFLICT (project_id, tool_name) DO NOTHING;
-- Tool 4: km_read — 讀取 Knowledge Management 條目（唯讀）
INSERT INTO awooop_mcp_tool_registry (
project_id, tool_name, tool_type, description, allowed_scopes, environment_tags
) VALUES (
'ewoooc',
'km_read',
'builtin',
'Knowledge Management 讀取（ewoooc 租戶 KM，RLS 隔離）',
'["read"]'::jsonb,
'{"env": "any"}'::jsonb
) ON CONFLICT (project_id, tool_name) DO NOTHING;
COMMIT;

131
apps/api/migrations/awooop_phase7_channel_hub_2026-05-04.sql Normal file
View File

@@ -0,0 +1,131 @@
-- =============================================================================
-- AwoooP Phase 7: Channel Hub 雙表
-- ADR-106（channel_event family）+ Progressive Feedback Policy
-- 2026-05-04 ogt + Claude Sonnet 4.6
-- =============================================================================
-- 兩張表：
-- awooop_conversation_event — 入站事件鏡像（Telegram/LINE inbound）
-- awooop_outbound_message — 出站訊息記錄（interim + final reply）
-- =============================================================================
BEGIN;
-- ---------------------------------------------------------------------------
-- 1. awooop_conversation_event — 入站 Channel Event 鏡像
-- 目的：AwoooP 平台保留所有入站事件的不可變記錄，與 legacy 系統解耦
-- ---------------------------------------------------------------------------
CREATE TABLE IF NOT EXISTS awooop_conversation_event (
event_id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
project_id VARCHAR(64) NOT NULL
REFERENCES awooop_projects(project_id) ON DELETE CASCADE,
-- Channel 原始身份
channel_type VARCHAR(32) NOT NULL, -- 'telegram' | 'line' | 'slack' | 'api'
provider_event_id VARCHAR(256) NOT NULL, -- Telegram: message_id, LINE: webhook event_id
-- 統一身份（由 ProviderProxy 注入）
platform_subject_id VARCHAR(128),
channel_user_id VARCHAR(256),
channel_chat_id VARCHAR(256),
-- 關聯 run（若已建立）
run_id UUID, -- FK soft（run 可能晚於 event 建立）
-- 事件內容（只存摘要/hash，不存明文）
content_type VARCHAR(32) NOT NULL DEFAULT 'text', -- 'text' | 'photo' | 'document' | 'command'
content_hash VARCHAR(64), -- sha256(raw_content)，明文不入庫
content_preview VARCHAR(256), -- 前 256 字元（無 PII/secret）
attachment_sha256 VARCHAR(64), -- 附件 sha256
-- 去重（與 awooop_run_idempotency 對應）
is_duplicate BOOLEAN NOT NULL DEFAULT FALSE,
-- 時間
provider_ts TIMESTAMPTZ, -- provider 原始時間戳
received_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
CONSTRAINT chk_conv_event_channel_type
CHECK (channel_type IN ('telegram','line','slack','api','internal')),
CONSTRAINT chk_conv_event_content_type
CHECK (content_type IN ('text','photo','document','command','callback_query')),
CONSTRAINT uix_conv_event_dedup
UNIQUE (project_id, channel_type, provider_event_id)
);
CREATE INDEX IF NOT EXISTS idx_conv_event_run
ON awooop_conversation_event (project_id, run_id, received_at DESC);
CREATE INDEX IF NOT EXISTS idx_conv_event_subject
ON awooop_conversation_event (project_id, platform_subject_id, received_at DESC);
CREATE INDEX IF NOT EXISTS idx_conv_event_recent
ON awooop_conversation_event (project_id, channel_type, received_at DESC);
-- ---------------------------------------------------------------------------
-- 2. awooop_outbound_message — 出站訊息記錄（interim + final reply）
-- 目的：追蹤 AwoooP 發出的每一條訊息（shadow 不發、canary/active 發）
-- Progressive Feedback Policy：WAITING_TOOL 超過 30s → 發 interim message
-- ---------------------------------------------------------------------------
CREATE TABLE IF NOT EXISTS awooop_outbound_message (
message_id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
project_id VARCHAR(64) NOT NULL
REFERENCES awooop_projects(project_id) ON DELETE CASCADE,
run_id UUID NOT NULL, -- FK soft
conversation_event_id UUID, -- 觸發訊息的入站 event
-- 出站目的地
channel_type VARCHAR(32) NOT NULL,
channel_chat_id VARCHAR(256) NOT NULL,
-- 訊息分類
message_type VARCHAR(32) NOT NULL, -- 'interim' | 'final' | 'error' | 'approval_request'
-- 內容（只存 hash，不存明文）
content_hash VARCHAR(64), -- sha256(rendered_content)
content_preview VARCHAR(256), -- 前 256 字元（無 PII/secret）
-- provider 回報的 message_id（Telegram: message.message_id）
provider_message_id VARCHAR(64),
-- 狀態
send_status VARCHAR(16) NOT NULL DEFAULT 'pending', -- 'pending'|'sent'|'failed'|'shadow'
send_error TEXT,
-- 時間
queued_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
sent_at TIMESTAMPTZ,
-- Progressive Feedback Policy（WAITING_TOOL 超 30s 觸發 interim）
triggered_by_state VARCHAR(32), -- 觸發本訊息的 run state（'waiting_tool'等）
waiting_since TIMESTAMPTZ, -- 開始等待的時間（計算 30s 超時用）
CONSTRAINT chk_outbound_channel_type
CHECK (channel_type IN ('telegram','line','slack','api','internal')),
CONSTRAINT chk_outbound_message_type
CHECK (message_type IN ('interim','final','error','approval_request')),
CONSTRAINT chk_outbound_send_status
CHECK (send_status IN ('pending','sent','failed','shadow'))
);
CREATE INDEX IF NOT EXISTS idx_outbound_msg_run
ON awooop_outbound_message (project_id, run_id, queued_at DESC);
CREATE INDEX IF NOT EXISTS idx_outbound_msg_pending
ON awooop_outbound_message (project_id, channel_type, queued_at)
WHERE send_status = 'pending';
-- Progressive Feedback Policy 查詢：找等待超過 30s 的 runs
CREATE INDEX IF NOT EXISTS idx_outbound_msg_waiting
ON awooop_outbound_message (project_id, triggered_by_state, waiting_since)
WHERE triggered_by_state = 'waiting_tool' AND send_status = 'pending';
-- =============================================================================
-- Row Level Security
-- =============================================================================
ALTER TABLE awooop_conversation_event ENABLE ROW LEVEL SECURITY;
ALTER TABLE awooop_outbound_message ENABLE ROW LEVEL SECURITY;
ALTER TABLE awooop_conversation_event FORCE ROW LEVEL SECURITY;
ALTER TABLE awooop_outbound_message FORCE ROW LEVEL SECURITY;
CREATE POLICY conv_event_tenant_isolation ON awooop_conversation_event
USING (
project_id = current_setting('app.project_id', TRUE)
OR current_setting('app.project_id', TRUE) IS NULL
);
CREATE POLICY outbound_msg_tenant_isolation ON awooop_outbound_message
USING (
project_id = current_setting('app.project_id', TRUE)
OR current_setting('app.project_id', TRUE) IS NULL
);
COMMIT;

4
apps/api/src/api/v1/platform/__init__.py Normal file
View File

@@ -0,0 +1,4 @@
"""AwoooP Platform API — Phase 4 Shadow Mode Shell"""
from src.api.v1.platform.runs import router
__all__ = ["router"]

149
apps/api/src/api/v1/platform/runs.py Normal file
View File

@@ -0,0 +1,149 @@
"""
Platform Runs API
==================
AwoooP Phase 4: POST /v1/platform/runs — Shadow mode run 建立
2026-05-04 ogt + Claude Sonnet 4.6（ADR-106/ADR-114）
禁止碰：
- /v1/incidents/ — legacy 路由
- /v1/webhooks/ — legacy 路由
- Telegram bot handler — legacy 維持
Shadow mode 保證（Phase 4）：
- 建立的 run 全部 is_shadow=True
- 不發送任何 user-visible response
- 不執行任何 destructive tool call
"""
from __future__ import annotations
import uuid
from typing import Any
from fastapi import APIRouter, HTTPException, status
from pydantic import BaseModel, Field
from src.services.audit_sink import write_audit
from src.services.platform_runtime import create_run
router = APIRouter()
# ─────────────────────────────────────────────────────────────────────────────
# Request / Response models
# ─────────────────────────────────────────────────────────────────────────────
class CreateRunRequest(BaseModel):
"""POST /v1/platform/runs request body"""
project_id: str = Field(..., description="租戶 ID")
agent_id: str = Field(..., description="執行此 run 的 agent ID")
trigger_type: str = Field(
...,
pattern="^(channel_event|schedule|api|sub_agent|retry)$",
description="觸發來源類型",
)
trigger_ref: str | None = Field(None, description="觸發來源 ref（channel_event_id 等）")
input_payload: dict[str, Any] | None = Field(None, description="Run 輸入 payload")
channel_type: str | None = Field(None, description="Channel 類型（idempotency 用）")
provider_event_id: str | None = Field(
None, max_length=256,
description="Channel provider 原始事件 ID（idempotency 去重用）",
)
timeout_seconds: int = Field(600, ge=30, le=3600, description="Run 超時秒數")
class CreateRunResponse(BaseModel):
"""POST /v1/platform/runs response"""
run_id: str
is_duplicate: bool = Field(description="True = 冪等命中，返回既有 run_id")
is_shadow: bool = Field(True, description="Phase 4 固定 True")
message: str
# ─────────────────────────────────────────────────────────────────────────────
# Routes
# ─────────────────────────────────────────────────────────────────────────────
@router.post(
"/runs",
response_model=CreateRunResponse,
status_code=status.HTTP_202_ACCEPTED,
summary="建立 Platform Run（Shadow Mode）",
description=(
"AwoooP Phase 4 Shadow Mode：建立新 run，非同步執行。\n\n"
"- `is_shadow=true`：不產生任何 user-visible response\n"
"- `is_duplicate=true`：冪等命中，返回既有 run_id（不建立新 run）\n"
"- provider_event_id + channel_type 構成冪等 key（24h 視窗）"
),
)
async def create_platform_run(
request: CreateRunRequest,
) -> CreateRunResponse:
"""建立 shadow run。"""
try:
run_id, is_duplicate = await create_run(
project_id=request.project_id,
agent_id=request.agent_id,
trigger_type=request.trigger_type,
trigger_ref=request.trigger_ref,
input_payload=request.input_payload,
channel_type=request.channel_type,
provider_event_id=request.provider_event_id,
timeout_seconds=request.timeout_seconds,
)
except Exception as exc:
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail=f"Run 建立失敗: {exc}",
) from exc
# Audit log（非阻擋）
await write_audit(
project_id=request.project_id,
action="run.created",
resource_type="run",
resource_id=str(run_id),
details={
"agent_id": request.agent_id,
"trigger_type": request.trigger_type,
"is_duplicate": is_duplicate,
"is_shadow": True,
},
)
return CreateRunResponse(
run_id=str(run_id),
is_duplicate=is_duplicate,
is_shadow=True,
message="Run 已接受（shadow mode）" if not is_duplicate else "冪等命中，返回既有 run_id",
)
@router.get(
"/runs/{run_id}",
summary="查詢 Run 狀態",
)
async def get_run_status(
run_id: str,
project_id: str,
) -> dict[str, Any]:
"""查詢單一 run 的 FSM 狀態。"""
from src.services.platform_runtime import get_run_status as _svc_get_run_status
try:
uid = uuid.UUID(run_id)
except ValueError as exc:
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
detail=f"run_id 格式錯誤: {exc}",
) from exc
result = await _svc_get_run_status(uid, project_id)
if result is None:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"run {run_id!r} 不存在",
)
return result

22
apps/api/src/core/context.py Normal file
View File

@@ -0,0 +1,22 @@
"""AwoooP Phase 2.4: Project ID Context Variable
================================================
2026-05-04 ogt + Claude Sonnet 4.6（ADR-123 background loop tagging）
設計原則：
- Python asyncio.create_task() 自動繼承父任務的 ContextVar 值
- startup handler 設一次 PROJECT_ID.set("awoooi")，所有 31 個 loop 自動繼承
- get_db_context() 讀此 contextvar 作為 fallback，確保 RLS SET LOCAL 正確
- 多租戶未來：呼叫端傳入不同 project_id 即可隔離，無需改 loop 本體
"""
from __future__ import annotations
from contextvars import ContextVar
# 追蹤當前非同步任務的 project_id
# default="awoooi" 確保未設時也能正常查詢（RLS fail-open 保護）
PROJECT_ID: ContextVar[str] = ContextVar("project_id", default="awoooi")
def get_current_project_id() -> str:
"""取得當前任務的 project_id（給 service 層使用）"""
return PROJECT_ID.get()

380
apps/api/src/db/awooop_models.py
View File

@@ -309,3 +309,383 @@ class AwoooPProjectMigrationState(Base):
updated_at: Mapped[datetime] = mapped_column(
nullable=False, server_default=text("NOW()")
)
# ─────────────────────────────────────────────────────────────────────────────
# Phase 4: Run State Machine（ADR-114/ADR-119）
# 2026-05-04 ogt + Claude Sonnet 4.6
# ─────────────────────────────────────────────────────────────────────────────
class AwoooPRunState(Base):
"""Run FSM 主表（SKIP LOCKED worker lease，ADR-114）"""
__tablename__ = "awooop_run_state"
__table_args__ = (
CheckConstraint(
"state IN ("
"'pending','running','waiting_tool',"
"'waiting_approval','completed','failed','cancelled','timeout')",
name="chk_run_state",
),
Index("idx_run_state_pending", "project_id", "created_at",
postgresql_where=text("state = 'pending' AND lease_until IS NULL")),
Index("idx_run_state_stale", "lease_until",
postgresql_where=text("state = 'running' AND lease_until IS NOT NULL")),
Index("idx_run_state_project_timeline", "project_id", "created_at"),
Index("idx_run_state_trace_id", "trace_id",
postgresql_where=text("trace_id IS NOT NULL")),
)
run_id: Mapped[UUID] = mapped_column(primary_key=True)
project_id: Mapped[str] = mapped_column(
String(64), ForeignKey("awooop_projects.project_id"), nullable=False
)
agent_id: Mapped[str] = mapped_column(String(128), nullable=False)
state: Mapped[str] = mapped_column(String(32), nullable=False, default="pending")
lease_until: Mapped[datetime | None] = mapped_column(nullable=True)
heartbeat_at: Mapped[datetime | None] = mapped_column(nullable=True)
worker_id: Mapped[str | None] = mapped_column(String(128), nullable=True)
attempt_count: Mapped[int] = mapped_column(SmallInteger, nullable=False, default=0)
max_attempts: Mapped[int] = mapped_column(SmallInteger, nullable=False, default=3)
trace_id: Mapped[str | None] = mapped_column(String(128), nullable=True)
trigger_type: Mapped[str | None] = mapped_column(String(32), nullable=True)
trigger_ref: Mapped[str | None] = mapped_column(String(256), nullable=True)
is_shadow: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True)
input_sha256: Mapped[str | None] = mapped_column(String(64), nullable=True)
output_sha256: Mapped[str | None] = mapped_column(String(64), nullable=True)
cost_usd: Mapped[Decimal] = mapped_column(
Numeric(10, 4), nullable=False, default=Decimal("0.0000")
)
step_count: Mapped[int] = mapped_column(SmallInteger, nullable=False, default=0)
error_code: Mapped[str | None] = mapped_column(String(64), nullable=True)
error_detail: Mapped[str | None] = mapped_column(Text, nullable=True)
created_at: Mapped[datetime] = mapped_column(
nullable=False, server_default=text("NOW()")
)
started_at: Mapped[datetime | None] = mapped_column(nullable=True)
completed_at: Mapped[datetime | None] = mapped_column(nullable=True)
timeout_at: Mapped[datetime | None] = mapped_column(nullable=True)
class AwoooPRunStepJournal(Base):
"""SAGA step journal（ADR-119）— 每個 tool call 獨立記錄"""
__tablename__ = "awooop_run_step_journal"
__table_args__ = (
UniqueConstraint("run_id", "step_seq", name="uix_run_step_seq"),
CheckConstraint(
"result_status IN ('pending','success','failed','compensated')",
name="chk_step_result_status",
),
Index("idx_run_step_run_id", "run_id", "step_seq"),
)
step_id: Mapped[UUID] = mapped_column(
primary_key=True, server_default=text("gen_random_uuid()")
)
run_id: Mapped[UUID] = mapped_column(
ForeignKey("awooop_run_state.run_id", ondelete="CASCADE"), nullable=False
)
project_id: Mapped[str] = mapped_column(String(64), nullable=False)
step_seq: Mapped[int] = mapped_column(SmallInteger, nullable=False)
tool_name: Mapped[str] = mapped_column(String(128), nullable=False)
mcp_gateway_id: Mapped[str | None] = mapped_column(String(128), nullable=True)
input_hash: Mapped[str | None] = mapped_column(String(64), nullable=True)
output_hash: Mapped[str | None] = mapped_column(String(64), nullable=True)
compensation_json: Mapped[dict[str, Any] | None] = mapped_column(JSONB, nullable=True)
result_status: Mapped[str] = mapped_column(String(16), nullable=False, default="pending")
error_code: Mapped[str | None] = mapped_column(String(64), nullable=True)
was_blocked: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False)
block_reason: Mapped[str | None] = mapped_column(String(128), nullable=True)
created_at: Mapped[datetime] = mapped_column(
nullable=False, server_default=text("NOW()")
)
completed_at: Mapped[datetime | None] = mapped_column(nullable=True)
latency_ms: Mapped[int | None] = mapped_column(Integer, nullable=True)
class AwoooPRunIdempotency(Base):
"""Run 去重冪等表（ADR-114）— (project_id, channel_type, provider_event_id) → run_id"""
__tablename__ = "awooop_run_idempotency"
__table_args__ = (
UniqueConstraint(
"project_id", "channel_type", "provider_event_id",
name="uix_run_idempotency_key",
),
Index("idx_run_idempotency_run_id", "run_id"),
)
idempotency_id: Mapped[UUID] = mapped_column(
primary_key=True, server_default=text("gen_random_uuid()")
)
project_id: Mapped[str] = mapped_column(String(64), nullable=False)
channel_type: Mapped[str] = mapped_column(String(32), nullable=False)
provider_event_id: Mapped[str] = mapped_column(String(256), nullable=False)
run_id: Mapped[UUID] = mapped_column(
ForeignKey("awooop_run_state.run_id"), nullable=False
)
created_at: Mapped[datetime] = mapped_column(
nullable=False, server_default=text("NOW()")
)
# =============================================================================
# Phase 5: MCP Gateway 四表（ADR-116/ADR-118，2026-05-04）
# =============================================================================
class AwoooPMcpToolRegistry(Base):
"""MCP Tool 白名單（Gate 3: Tool）"""
__tablename__ = "awooop_mcp_tool_registry"
__table_args__ = (
CheckConstraint(
"tool_type IN ('builtin','mcp_server','custom')",
name="chk_tool_type",
),
CheckConstraint(
"jsonb_typeof(allowed_scopes) = 'array'",
name="chk_allowed_scopes_array",
),
UniqueConstraint("project_id", "tool_name", name="uix_tool_registry_project_name"),
Index("idx_mcp_tool_registry_project", "project_id", "is_active"),
)
tool_id: Mapped[UUID] = mapped_column(
primary_key=True, server_default=text("gen_random_uuid()")
)
project_id: Mapped[str] = mapped_column(
String(64), ForeignKey("awooop_projects.project_id", ondelete="CASCADE"), nullable=False
)
tool_name: Mapped[str] = mapped_column(String(128), nullable=False)
tool_type: Mapped[str] = mapped_column(String(32), nullable=False)
description: Mapped[str | None] = mapped_column(Text, nullable=True)
allowed_scopes: Mapped[list[Any]] = mapped_column(JSONB, nullable=False, default=list)
environment_tags: Mapped[dict[str, Any]] = mapped_column(JSONB, nullable=False, default=dict)
is_active: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True)
created_at: Mapped[datetime] = mapped_column(
nullable=False, server_default=text("NOW()")
)
updated_at: Mapped[datetime] = mapped_column(
nullable=False, server_default=text("NOW()")
)
class AwoooPMcpGrant(Base):
"""Agent × Tool 授權記錄（Gate 2 + Gate 3）"""
__tablename__ = "awooop_mcp_grants"
__table_args__ = (
CheckConstraint(
"jsonb_typeof(granted_scopes) = 'array'",
name="chk_grant_scopes_array",
),
CheckConstraint(
"(is_revoked = FALSE AND revoked_at IS NULL AND revoked_by IS NULL)"
" OR (is_revoked = TRUE AND revoked_at IS NOT NULL)",
name="chk_revoke_consistency",
),
UniqueConstraint("project_id", "agent_id", "tool_id", name="uix_mcp_grant_agent_tool"),
Index(
"idx_mcp_grants_lookup", "project_id", "agent_id", "tool_id",
postgresql_where=text("is_revoked = FALSE"),
),
)
grant_id: Mapped[UUID] = mapped_column(
primary_key=True, server_default=text("gen_random_uuid()")
)
project_id: Mapped[str] = mapped_column(
String(64), ForeignKey("awooop_projects.project_id", ondelete="CASCADE"), nullable=False
)
agent_id: Mapped[str] = mapped_column(String(128), nullable=False)
tool_id: Mapped[UUID] = mapped_column(
ForeignKey("awooop_mcp_tool_registry.tool_id", ondelete="CASCADE"), nullable=False
)
granted_by: Mapped[str] = mapped_column(String(128), nullable=False)
granted_scopes: Mapped[list[Any]] = mapped_column(JSONB, nullable=False, default=list)
expires_at: Mapped[datetime | None] = mapped_column(nullable=True)
is_revoked: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False)
revoked_at: Mapped[datetime | None] = mapped_column(nullable=True)
revoked_by: Mapped[str | None] = mapped_column(String(128), nullable=True)
created_at: Mapped[datetime] = mapped_column(
nullable=False, server_default=text("NOW()")
)
class AwoooPMcpCredentialRef(Base):
"""k8s Secret 參照（ADR-118 credential isolation）— 只存路徑，不存明文"""
__tablename__ = "awooop_mcp_credential_refs"
__table_args__ = (
CheckConstraint(
r"k8s_secret_ref ~ '^[a-z0-9-]+/[a-z0-9-]+#[a-zA-Z0-9_-]+$'",
name="chk_k8s_ref_format",
),
CheckConstraint(
r"value_sha256 IS NULL OR value_sha256 ~ '^[0-9a-f]{64}$'",
name="chk_value_sha256_hex",
),
UniqueConstraint("tool_id", "k8s_secret_ref", name="uix_credential_ref_tool"),
Index("idx_mcp_cred_refs_tool", "tool_id", postgresql_where=text("is_active = TRUE")),
)
ref_id: Mapped[UUID] = mapped_column(
primary_key=True, server_default=text("gen_random_uuid()")
)
tool_id: Mapped[UUID] = mapped_column(
ForeignKey("awooop_mcp_tool_registry.tool_id", ondelete="CASCADE"), nullable=False
)
project_id: Mapped[str] = mapped_column(
String(64), ForeignKey("awooop_projects.project_id", ondelete="CASCADE"), nullable=False
)
k8s_secret_ref: Mapped[str] = mapped_column(String(256), nullable=False)
value_sha256: Mapped[str | None] = mapped_column(String(64), nullable=True)
description: Mapped[str | None] = mapped_column(Text, nullable=True)
is_active: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True)
created_at: Mapped[datetime] = mapped_column(
nullable=False, server_default=text("NOW()")
)
rotated_at: Mapped[datetime | None] = mapped_column(nullable=True)
class AwoooPMcpGatewayAudit(Base):
"""MCP Gateway call 稽核日誌（ADR-116 P1-09）"""
__tablename__ = "awooop_mcp_gateway_audit"
__table_args__ = (
CheckConstraint(
"result_status IN ('success','blocked','failed','timeout')",
name="chk_gateway_result_status",
),
CheckConstraint(
"block_gate IS NULL OR (block_gate >= 1 AND block_gate <= 5)",
name="chk_block_gate_range",
),
Index("idx_mcp_audit_run", "project_id", "run_id", "created_at"),
Index(
"idx_mcp_audit_blocked", "project_id", "block_gate", "created_at",
postgresql_where=text("result_status = 'blocked'"),
),
)
call_id: Mapped[UUID] = mapped_column(
primary_key=True, server_default=text("gen_random_uuid()")
)
project_id: Mapped[str] = mapped_column(String(64), nullable=False)
run_id: Mapped[UUID | None] = mapped_column(nullable=True)
trace_id: Mapped[str | None] = mapped_column(String(128), nullable=True)
agent_id: Mapped[str | None] = mapped_column(String(128), nullable=True)
tool_id: Mapped[UUID] = mapped_column(
ForeignKey("awooop_mcp_tool_registry.tool_id"), nullable=False
)
tool_name: Mapped[str] = mapped_column(String(128), nullable=False)
credential_ref: Mapped[str | None] = mapped_column(String(256), nullable=True)
input_hash: Mapped[str | None] = mapped_column(String(64), nullable=True)
output_hash: Mapped[str | None] = mapped_column(String(64), nullable=True)
gate_result: Mapped[dict[str, Any]] = mapped_column(JSONB, nullable=False, default=dict)
result_status: Mapped[str] = mapped_column(String(16), nullable=False)
block_gate: Mapped[int | None] = mapped_column(SmallInteger, nullable=True)
block_reason: Mapped[str | None] = mapped_column(String(256), nullable=True)
latency_ms: Mapped[int | None] = mapped_column(Integer, nullable=True)
created_at: Mapped[datetime] = mapped_column(
nullable=False, server_default=text("NOW()")
)
# =============================================================================
# Phase 7: Channel Hub 雙表（ADR-106 channel_event family，2026-05-04）
# =============================================================================
class AwoooPConversationEvent(Base):
"""入站 Channel Event 鏡像（Telegram/LINE inbound，不儲存明文）"""
__tablename__ = "awooop_conversation_event"
__table_args__ = (
CheckConstraint(
"channel_type IN ('telegram','line','slack','api','internal')",
name="chk_conv_event_channel_type",
),
CheckConstraint(
"content_type IN ('text','photo','document','command','callback_query')",
name="chk_conv_event_content_type",
),
UniqueConstraint(
"project_id", "channel_type", "provider_event_id",
name="uix_conv_event_dedup",
),
Index("idx_conv_event_run", "project_id", "run_id", "received_at"),
Index("idx_conv_event_subject", "project_id", "platform_subject_id", "received_at"),
)
event_id: Mapped[UUID] = mapped_column(
primary_key=True, server_default=text("gen_random_uuid()")
)
project_id: Mapped[str] = mapped_column(
String(64), ForeignKey("awooop_projects.project_id", ondelete="CASCADE"), nullable=False
)
channel_type: Mapped[str] = mapped_column(String(32), nullable=False)
provider_event_id: Mapped[str] = mapped_column(String(256), nullable=False)
platform_subject_id: Mapped[str | None] = mapped_column(String(128), nullable=True)
channel_user_id: Mapped[str | None] = mapped_column(String(256), nullable=True)
channel_chat_id: Mapped[str | None] = mapped_column(String(256), nullable=True)
run_id: Mapped[UUID | None] = mapped_column(nullable=True)
content_type: Mapped[str] = mapped_column(String(32), nullable=False, default="text")
content_hash: Mapped[str | None] = mapped_column(String(64), nullable=True)
content_preview: Mapped[str | None] = mapped_column(String(256), nullable=True)
attachment_sha256: Mapped[str | None] = mapped_column(String(64), nullable=True)
is_duplicate: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False)
provider_ts: Mapped[datetime | None] = mapped_column(nullable=True)
received_at: Mapped[datetime] = mapped_column(
nullable=False, server_default=text("NOW()")
)
class AwoooPOutboundMessage(Base):
"""出站訊息記錄（interim/final/approval_request + shadow status）"""
__tablename__ = "awooop_outbound_message"
__table_args__ = (
CheckConstraint(
"channel_type IN ('telegram','line','slack','api','internal')",
name="chk_outbound_channel_type",
),
CheckConstraint(
"message_type IN ('interim','final','error','approval_request')",
name="chk_outbound_message_type",
),
CheckConstraint(
"send_status IN ('pending','sent','failed','shadow')",
name="chk_outbound_send_status",
),
Index("idx_outbound_msg_run", "project_id", "run_id", "queued_at"),
Index(
"idx_outbound_msg_pending", "project_id", "channel_type", "queued_at",
postgresql_where=text("send_status = 'pending'"),
),
)
message_id: Mapped[UUID] = mapped_column(
primary_key=True, server_default=text("gen_random_uuid()")
)
project_id: Mapped[str] = mapped_column(
String(64), ForeignKey("awooop_projects.project_id", ondelete="CASCADE"), nullable=False
)
run_id: Mapped[UUID] = mapped_column(nullable=False)
conversation_event_id: Mapped[UUID | None] = mapped_column(nullable=True)
channel_type: Mapped[str] = mapped_column(String(32), nullable=False)
channel_chat_id: Mapped[str] = mapped_column(String(256), nullable=False)
message_type: Mapped[str] = mapped_column(String(32), nullable=False)
content_hash: Mapped[str | None] = mapped_column(String(64), nullable=True)
content_preview: Mapped[str | None] = mapped_column(String(256), nullable=True)
provider_message_id: Mapped[str | None] = mapped_column(String(64), nullable=True)
send_status: Mapped[str] = mapped_column(String(16), nullable=False, default="pending")
send_error: Mapped[str | None] = mapped_column(Text, nullable=True)
queued_at: Mapped[datetime] = mapped_column(
nullable=False, server_default=text("NOW()")
)
sent_at: Mapped[datetime | None] = mapped_column(nullable=True)
triggered_by_state: Mapped[str | None] = mapped_column(String(32), nullable=True)
waiting_since: Mapped[datetime | None] = mapped_column(nullable=True)

78
apps/api/src/db/base.py
View File

@@ -106,6 +106,11 @@ async def get_db() -> AsyncGenerator[AsyncSession, None]:
factory = get_session_factory()
async with factory() as session:
try:
# AwoooP Phase 2.3 (2026-05-04 ogt): SET LOCAL app.project_id 讓 RLS Policy 生效
# 預設 'awoooi'，多租戶路由將在 middleware 注入實際 project_id
await session.execute(
text("SELECT set_config('app.project_id', 'awoooi', TRUE)")
)
yield session
await session.commit()
except Exception:
@@ -114,17 +119,30 @@ async def get_db() -> AsyncGenerator[AsyncSession, None]:
@asynccontextmanager
async def get_db_context() -> AsyncGenerator[AsyncSession, None]:
async def get_db_context(project_id: str | None = None) -> AsyncGenerator[AsyncSession, None]:
"""
Context manager for database session (non-FastAPI usage)
AwoooP Phase 2.3/2.4: 優先序 — 明確參數 > contextvar > "awoooi"
- Phase 2.3: 啟用 RLS tenant isolation（SET LOCAL app.project_id）
- Phase 2.4: 從 asyncio contextvar 讀取 background loop 的 project_id
Usage:
async with get_db_context() as db:
async with get_db_context() as db: # 繼承 contextvar 或預設 awoooi
...
async with get_db_context("other-tenant") as db: # 明確指定 tenant
...
"""
from src.core.context import get_current_project_id
effective_pid = project_id if project_id is not None else get_current_project_id()
factory = get_session_factory()
async with factory() as session:
try:
await session.execute(
text("SELECT set_config('app.project_id', :pid, TRUE)"),
{"pid": effective_pid},
)
yield session
await session.commit()
except Exception:
@@ -299,6 +317,62 @@ async def init_db() -> None:
"ON timeline_events(incident_id);"
))
# AwoooP Phase 2.6 (2026-05-04 ogt): budget_ledger 建表（ADR-120 Token Budget Hard Kill）
await conn.execute(text("""
CREATE TABLE IF NOT EXISTS budget_ledger (
id UUID DEFAULT gen_random_uuid() PRIMARY KEY,
project_id VARCHAR(64) NOT NULL DEFAULT 'awoooi',
agent_id VARCHAR(128),
run_id UUID,
model VARCHAR(64),
provider VARCHAR(32),
prompt_tokens INT,
completion_tokens INT,
cost_usd NUMERIC(10, 4) NOT NULL DEFAULT 0.0000,
recorded_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
"""))
await conn.execute(text(
"CREATE INDEX IF NOT EXISTS idx_budget_ledger_project_date "
"ON budget_ledger(project_id, recorded_at DESC);"
))
# AwoooP Phase 2.3 (2026-05-04 ogt): 四表加 project_id（RLS 多租戶隔離）
# 防禦性 ALTER — 已存在欄位為 no-op，安全。
# Batch 1 RLS migration 執行後，app.project_id 由 get_db_context() 自動設置。
await conn.execute(text(
"ALTER TABLE incidents "
"ADD COLUMN IF NOT EXISTS project_id VARCHAR(64) NOT NULL DEFAULT 'awoooi';"
))
await conn.execute(text(
"CREATE INDEX IF NOT EXISTS idx_incidents_project_id "
"ON incidents (project_id);"
))
await conn.execute(text(
"ALTER TABLE knowledge_entries "
"ADD COLUMN IF NOT EXISTS project_id VARCHAR(64) NOT NULL DEFAULT 'awoooi';"
))
await conn.execute(text(
"CREATE INDEX IF NOT EXISTS idx_knowledge_entries_project_id "
"ON knowledge_entries (project_id);"
))
await conn.execute(text(
"ALTER TABLE playbooks "
"ADD COLUMN IF NOT EXISTS project_id VARCHAR(64) NOT NULL DEFAULT 'awoooi';"
))
await conn.execute(text(
"CREATE INDEX IF NOT EXISTS idx_playbooks_project_id "
"ON playbooks (project_id);"
))
await conn.execute(text(
"ALTER TABLE audit_logs "
"ADD COLUMN IF NOT EXISTS project_id VARCHAR(64) NOT NULL DEFAULT 'awoooi';"
))
await conn.execute(text(
"CREATE INDEX IF NOT EXISTS idx_audit_logs_project_id "
"ON audit_logs (project_id);"
))
# 2026-04-15 ogt + Claude Sonnet 4.6（亞太）: Phase 6 自我治理閉環
# ADR-087: ai_governance_events 不可變 Event Sourcing 表
# asyncpg 不允許 prepared statement 內多條指令，必須分開 execute

75
apps/api/src/db/models.py
View File

@@ -11,8 +11,9 @@ Schema 設計原則：
"""
from datetime import datetime
from decimal import Decimal
from typing import Any
from uuid import uuid4
from uuid import UUID, uuid4
from sqlalchemy import (
JSON,
@@ -25,6 +26,7 @@ from sqlalchemy import (
ForeignKey,
Index,
Integer,
Numeric,
String,
Text,
text,
@@ -34,6 +36,7 @@ from sqlalchemy import (
)
from sqlalchemy.dialects.postgresql import ENUM as PgEnum
from sqlalchemy.dialects.postgresql import JSONB
from sqlalchemy.dialects.postgresql import UUID as pg_UUID
from sqlalchemy.orm import Mapped, mapped_column
from src.db.base import Base
@@ -368,6 +371,13 @@ class AuditLog(Base):
default="default",
nullable=False,
)
# AwoooP Phase 2.3 (2026-05-04 ogt): 多租戶隔離欄位，配合 Batch 1 RLS migration
project_id: Mapped[str] = mapped_column(
String(64),
default="awoooi",
nullable=False,
index=True,
)
# Execution Result
success: Mapped[bool] = mapped_column(default=False, nullable=False)
@@ -671,6 +681,13 @@ class IncidentRecord(Base):
primary_key=True,
comment="事件唯一識別碼 (如 INC-20260322-A1B2C3)",
)
# AwoooP Phase 2.3 (2026-05-04 ogt): 多租戶隔離欄位，配合 Batch 1 RLS migration
project_id: Mapped[str] = mapped_column(
String(64),
default="awoooi",
nullable=False,
index=True,
)
# === 狀態與嚴重度 ===
status: Mapped[str] = mapped_column(
@@ -813,6 +830,13 @@ class KnowledgeEntryRecord(Base):
primary_key=True,
default=generate_uuid,
)
# AwoooP Phase 2.3 (2026-05-04 ogt): 多租戶隔離欄位，配合 Batch 1 RLS migration
project_id: Mapped[str] = mapped_column(
String(64),
default="awoooi",
nullable=False,
index=True,
)
# Core Fields
title: Mapped[str] = mapped_column(String(255), nullable=False)
@@ -1075,6 +1099,13 @@ class PlaybookRecord(Base):
String(36), primary_key=True,
comment="Playbook 唯一識別碼 (PB-YYYYMMDD-XXXXXX)",
)
# AwoooP Phase 2.3 (2026-05-04 ogt): 多租戶隔離欄位，配合 Batch 1 RLS migration
project_id: Mapped[str] = mapped_column(
String(64),
default="awoooi",
nullable=False,
index=True,
)
# Core Fields
name: Mapped[str] = mapped_column(String(256), nullable=False)
@@ -1612,3 +1643,45 @@ class AIProviderVersionHistory(Base):
__table_args__ = (
Index("ix_provider_version_captured", "provider", "captured_at"),
)
# =============================================================================
# BudgetLedgerRecord — ADR-120 Token Budget Hard Kill（Phase 2.6）
# 2026-05-04 ogt + Claude Sonnet 4.6
# =============================================================================
class BudgetLedgerRecord(Base):
"""
LLM call 費用記帳表（ADR-120 D5）
每次 LLM call 完成後插入一筆記錄，供：
- Tenant Budget 累計計算（Redis 快取，每分鐘從此表同步）
- 儀表板消費統計
- 告警閾值觸發（80% / 95% / 100%）
"""
__tablename__ = "budget_ledger"
id: Mapped[UUID] = mapped_column(
pg_UUID(as_uuid=True),
primary_key=True,
server_default=text("gen_random_uuid()"),
)
project_id: Mapped[str] = mapped_column(
String(64), nullable=False, default="awoooi", index=True
)
agent_id: Mapped[str | None] = mapped_column(String(128), nullable=True)
run_id: Mapped[UUID | None] = mapped_column(pg_UUID(as_uuid=True), nullable=True)
model: Mapped[str | None] = mapped_column(String(64), nullable=True)
provider: Mapped[str | None] = mapped_column(String(32), nullable=True)
prompt_tokens: Mapped[int | None] = mapped_column(Integer, nullable=True)
completion_tokens: Mapped[int | None] = mapped_column(Integer, nullable=True)
cost_usd: Mapped[Decimal] = mapped_column(
Numeric(10, 4), nullable=False, default=Decimal("0.0000")
)
recorded_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), nullable=False, server_default=text("NOW()")
)
__table_args__ = (
Index("idx_budget_ledger_project_date", "project_id", "recorded_at"),
)

25
apps/api/src/hermes/nl_gateway.py
View File

@@ -139,11 +139,11 @@ async def _write_dispatch_log(
# T2：per-chat_id 速率限制（ADR-094，fail-open）
# ─────────────────────────────────────────────────────────────────────────────
async def _check_rate_limit(chat_id: str) -> bool:
async def _check_rate_limit(chat_id: str, project_id: str = "awoooi") -> bool:
"""True = 允許；False = 超過限制（20 req/min per chat_id）。Redis 不可用時放行。"""
try:
redis = get_redis()
key = f"hermes:rl:{chat_id}"
key = f"{project_id}:hermes:rl:{chat_id}"
count = await redis.incr(key)
if count == 1:
await redis.expire(key, _RATE_LIMIT_WINDOW_SEC)
@@ -156,12 +156,15 @@ async def _check_rate_limit(chat_id: str) -> bool:
# T3：Multi-turn session（Redis Hash TTL=300s，ADR-094）
# ─────────────────────────────────────────────────────────────────────────────
async def _load_session_context(chat_id: str, user_id: int) -> str:
async def _load_session_context(chat_id: str, user_id: int, project_id: str = "awoooi") -> str:
"""載入最近 3 輪對話歷史（最多 600 字），組成 context prefix。Redis 不可用時回空字串。"""
try:
redis = get_redis()
key = f"hermes:session:{chat_id}:{user_id}"
key = f"{project_id}:hermes:session:{chat_id}:{user_id}"
data = await redis.hgetall(key)
if not data:
# Phase A: fallback 到舊 key（滾動部署相容）
data = await redis.hgetall(f"hermes:session:{chat_id}:{user_id}")
if not data:
return ""
turns = sorted(
@@ -175,16 +178,19 @@ async def _load_session_context(chat_id: str, user_id: int) -> str:
async def _save_session_turn(
chat_id: str, user_id: int, user_msg: str, assistant_reply: str
chat_id: str, user_id: int, user_msg: str, assistant_reply: str, project_id: str = "awoooi"
) -> None:
"""將本輪對話存入 Redis Hash，並重置 TTL=300s。Redis 不可用時靜默忽略。"""
try:
redis = get_redis()
key = f"hermes:session:{chat_id}:{user_id}"
key = f"{project_id}:hermes:session:{chat_id}:{user_id}"
legacy_key = f"hermes:session:{chat_id}:{user_id}" # Phase A dual-write
turn_key = f"turn_{int(time.time())}"
value = f"用戶：{user_msg[:100]}\nHermes：{assistant_reply[:200]}"
await redis.hset(key, turn_key, value)
await redis.expire(key, 300)
await redis.hset(legacy_key, turn_key, value)
await redis.expire(legacy_key, 300)
except Exception:
pass
@@ -199,6 +205,7 @@ async def process_nl_message(
chat_id: str,
user_id: int,
username: str = "",
project_id: str = "awoooi",
) -> str:
"""
處理 NL 訊息，回傳 Telegram 格式的回覆文字。
@@ -231,7 +238,7 @@ async def process_nl_message(
)
# T2：速率限制
if not await _check_rate_limit(chat_id):
if not await _check_rate_limit(chat_id, project_id):
return "⚠️ 請求太頻繁，請稍後再試（每分鐘上限 20 次）。"
# Layer 1 意圖路由
@@ -249,7 +256,7 @@ async def process_nl_message(
system_prompt = get_agent_system_prompt(agent_name) or ""
# T3：載入 session context（最近 3 輪）
session_ctx = await _load_session_context(chat_id, user_id)
session_ctx = await _load_session_context(chat_id, user_id, project_id)
prompt_with_ctx = f"{session_ctx}{user_message}" if session_ctx else user_message
t0 = time.monotonic()
@@ -306,7 +313,7 @@ async def process_nl_message(
# T3：儲存本輪對話（只在成功時存）
if success:
await _save_session_turn(chat_id, user_id, user_message, result_text)
await _save_session_turn(chat_id, user_id, user_message, result_text, project_id)
# T1：非阻擋寫入 hermes_dispatch_log（失敗不影響回覆）
asyncio.create_task(

26
apps/api/src/main.py
View File

@@ -65,6 +65,7 @@ from src.api.v1 import (
signoz_webhook as signoz_webhook_v1, # Phase 21: SignOz → Telegram (ADR-037)
)
from src.api.v1 import drift as drift_v1 # Phase 25 P2: Config Drift Detection
from src.api.v1 import platform as platform_v1 # AwoooP Phase 4: Platform Shell（Shadow Mode）
from src.api.v1 import rag as rag_v1 # Phase 33 ADR-067: RAG 知識庫
from src.api.v1 import monitoring as monitoring_v1 # 2026-04-03: 監控工具狀態
from src.api.v1 import notifications as notifications_v1 # 2026-04-10: 通知頻道狀態
@@ -185,6 +186,11 @@ else:
@asynccontextmanager
async def lifespan(_app: FastAPI) -> AsyncGenerator[None, None]:
"""Application lifespan events"""
# AwoooP Phase 2.4 (2026-05-04 ogt): 設定 startup handler 的 project_id context
# asyncio.create_task() 自動繼承父任務的 ContextVar → 31 個 background loop 全部標記為 awoooi
from src.core.context import PROJECT_ID
PROJECT_ID.set("awoooi")
# Startup
logger.info(
"api_startup",
@@ -703,6 +709,16 @@ async def lifespan(_app: FastAPI) -> AsyncGenerator[None, None]:
except Exception as e:
logger.warning("model_version_tracker_schedule_failed", error=str(e))
# AwoooP Phase 4 (2026-05-04 ogt + Claude Sonnet 4.6): Platform Worker（Shadow Mode Shell）
# ADR-106 Strangler Fig Phase 4：SKIP LOCKED run worker + stale run reaper
# Shadow mode：is_shadow=True，0 user-visible response，0 destructive tool call
try:
from src.workers.platform_worker import start_platform_worker
await start_platform_worker()
logger.info("platform_worker_started", mode="shadow")
except Exception as e:
logger.warning("platform_worker_start_failed", error=str(e))
yield
# Shutdown
@@ -727,6 +743,14 @@ async def lifespan(_app: FastAPI) -> AsyncGenerator[None, None]:
except Exception as e:
logger.warning("auto_repair_drain_failed", error=str(e))
# AwoooP Phase 4: Platform Worker 優雅停機（2026-05-04 ogt）
try:
from src.workers.platform_worker import stop_platform_worker
await stop_platform_worker()
logger.info("platform_worker_stopped")
except Exception as e:
logger.warning("platform_worker_stop_failed", error=str(e))
# Phase 6.1: 關閉 Signal Worker (先關閉 Consumer)
await close_signal_worker()
await publisher.stop()
@@ -968,6 +992,8 @@ app.include_router(agent.router, prefix="/api/v1/agent", tags=["Agent"])
app.include_router(
notifications.router, prefix="/api/v1/notifications", tags=["Notifications"]
)
# AwoooP Phase 4 (2026-05-04 ogt): Platform Shell — Shadow Mode Run API
app.include_router(platform_v1.router, prefix="/api/v1/platform", tags=["AwoooP Platform"])
# =============================================================================

437
apps/api/src/models/awooop_contracts.py Normal file
View File

@@ -0,0 +1,437 @@
"""
AwoooP Contract Pydantic Models
================================
Phase 3: 六合約家族 Pydantic v2 驗證模型（ADR-112）
2026-05-04 ogt + Claude Sonnet 4.6
六合約家族：
1. ProjectTenantContract — 租戶/專案能力邊界
2. AgentContract — Agent 模型、工具、治理
3. MCPGatewayContract — MCP 工具閘道
4. PolicyRoutingContract — LLM 路由規則
5. RuntimeRunStateContract — Run FSM 狀態
6. ChannelEventContract — Channel 事件（冪等）
所有含 artifact ref 的欄位都附 sha256（ADR-112 artifact integrity）。
"""
from __future__ import annotations
import re
from datetime import datetime
from enum import Enum
from typing import Any
from uuid import UUID
from pydantic import BaseModel, Field, field_validator, model_validator
# ─────────────────────────────────────────────────────────────────────────────
# 共用型別
# ─────────────────────────────────────────────────────────────────────────────
_SHA256_RE = re.compile(r"^[0-9a-f]{64}$")
_PROJECT_ID_RE = re.compile(r"^[a-z0-9][a-z0-9_-]{1,63}$")
_AGENT_ID_RE = re.compile(r"^[a-z0-9][a-z0-9_-]{1,127}$")
_UUID_RE = re.compile(
r"^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$"
)
def _validate_sha256(v: str | None, field_name: str = "sha256") -> str | None:
if v is None:
return v
if not _SHA256_RE.match(v):
raise ValueError(f"{field_name} 必須為 64 位 hex 字串")
return v
class MigrationMode(str, Enum):
LEGACY = "legacy_awoooi_default"
SHADOW = "shadow"
CANARY = "canary"
ACTIVE = "active"
class ChannelType(str, Enum):
TELEGRAM = "telegram"
SLACK = "slack"
WEBHOOK = "webhook"
API = "api"
class Provider(str, Enum):
ANTHROPIC = "anthropic"
OPENAI = "openai"
OLLAMA = "ollama"
GEMINI = "gemini"
NVIDIA = "nvidia"
OPENROUTER = "openrouter"
class RunState(str, Enum):
PENDING = "pending"
RUNNING = "running"
WAITING_APPROVAL = "waiting_approval"
WAITING_TOOL = "waiting_tool"
COMPLETED = "completed"
FAILED = "failed"
CANCELLED = "cancelled"
TIMEOUT = "timeout"
class AuthScheme(str, Enum):
NONE = "none"
BEARER = "bearer"
HMAC = "hmac"
class Transport(str, Enum):
STDIO = "stdio"
HTTP = "http"
SSE = "sse"
class EventType(str, Enum):
MESSAGE_RECEIVED = "message_received"
CALLBACK_QUERY = "callback_query"
COMMAND_INVOKED = "command_invoked"
WEBHOOK_POST = "webhook_post"
API_REQUEST = "api_request"
APPROVAL_RESPONSE = "approval_response"
# ─────────────────────────────────────────────────────────────────────────────
# 1. Project Tenant Contract
# ─────────────────────────────────────────────────────────────────────────────
class ProjectTenantContract(BaseModel):
"""租戶/專案合約（ADR-111/115）"""
model_config = {"extra": "forbid"}
project_id: str = Field(..., description="全局唯一租戶識別符")
display_name: str = Field(..., min_length=1, max_length=256)
migration_mode: MigrationMode = MigrationMode.LEGACY
budget_limit_usd: float | None = Field(None, ge=0)
allowed_channels: list[ChannelType] = Field(default_factory=list)
is_active: bool = True
metadata: dict[str, Any] = Field(default_factory=dict)
@field_validator("project_id")
@classmethod
def validate_project_id(cls, v: str) -> str:
if not _PROJECT_ID_RE.match(v):
raise ValueError("project_id 只允許 a-z, 0-9, _, -，長度 2-64")
return v
@field_validator("allowed_channels")
@classmethod
def validate_unique_channels(cls, v: list[ChannelType]) -> list[ChannelType]:
if len(v) != len(set(v)):
raise ValueError("allowed_channels 不可包含重複項目")
return v
# ─────────────────────────────────────────────────────────────────────────────
# 2. Agent Contract
# ─────────────────────────────────────────────────────────────────────────────
class ArtifactRef(BaseModel):
"""含 SHA-256 的 artifact 參照（ADR-112 artifact integrity）"""
model_config = {"extra": "forbid"}
artifact_id: str
sha256: str = Field(..., description="SHA-256 hex digest（64 位）")
@field_validator("sha256")
@classmethod
def validate_sha256(cls, v: str) -> str:
return _validate_sha256(v, "sha256") # type: ignore[return-value]
class ToolRef(BaseModel):
"""Agent 工具參照"""
model_config = {"extra": "allow"}
tool_name: str
mcp_gateway_id: str | None = None
sha256: str | None = None
@field_validator("sha256")
@classmethod
def validate_sha256(cls, v: str | None) -> str | None:
return _validate_sha256(v, "tool sha256")
class AgentContract(BaseModel):
"""Agent 合約（ADR-112）"""
model_config = {"extra": "forbid"}
agent_id: str = Field(..., description="Agent 識別符")
agent_name: str = Field(..., min_length=1, max_length=256)
model: str = Field(..., min_length=1, max_length=128)
provider: Provider
max_tokens: int | None = Field(None, ge=1, le=200000)
temperature: float | None = Field(None, ge=0.0, le=2.0)
system_prompt_ref: ArtifactRef | None = None
tools: list[ToolRef] = Field(default_factory=list)
budget_limit_usd_per_run: float | None = Field(None, ge=0)
require_approval: bool = False
approval_timeout_seconds: int | None = Field(None, ge=60, le=86400)
max_parallel_runs: int = Field(1, ge=1, le=100)
tags: list[str] = Field(default_factory=list)
@field_validator("agent_id")
@classmethod
def validate_agent_id(cls, v: str) -> str:
if not _AGENT_ID_RE.match(v):
raise ValueError("agent_id 只允許 a-z, 0-9, _, -，長度 2-128")
return v
@model_validator(mode="after")
def validate_approval_config(self) -> AgentContract:
if self.require_approval and self.approval_timeout_seconds is None:
self.approval_timeout_seconds = 300
return self
# ─────────────────────────────────────────────────────────────────────────────
# 3. MCP Gateway Contract
# ─────────────────────────────────────────────────────────────────────────────
class ToolExposed(BaseModel):
"""Gateway 暴露的工具定義"""
model_config = {"extra": "forbid"}
tool_name: str
description: str | None = None
schema_sha256: str = Field(..., description="工具 input schema SHA-256")
is_destructive: bool = False
@field_validator("schema_sha256")
@classmethod
def validate_schema_sha256(cls, v: str) -> str:
return _validate_sha256(v, "schema_sha256") # type: ignore[return-value]
class MCPGatewayContract(BaseModel):
"""MCP Gateway 合約（ADR-113）"""
model_config = {"extra": "forbid"}
gateway_id: str
gateway_name: str = Field(..., min_length=1, max_length=256)
transport: Transport
endpoint: str | None = None
auth_scheme: AuthScheme = AuthScheme.NONE
hmac_secret_ref: str | None = None
tools_exposed: list[ToolExposed] = Field(default_factory=list)
rate_limit_rpm: int | None = Field(None, ge=1)
timeout_seconds: int = Field(30, ge=1, le=300)
is_enabled: bool = True
@model_validator(mode="after")
def validate_http_endpoint(self) -> MCPGatewayContract:
if self.transport in (Transport.HTTP, Transport.SSE) and not self.endpoint:
raise ValueError(f"transport={self.transport} 時 endpoint 為必填")
return self
# ─────────────────────────────────────────────────────────────────────────────
# 4. Policy Routing Contract
# ─────────────────────────────────────────────────────────────────────────────
class TimeRange(BaseModel):
model_config = {"extra": "forbid"}
start_utc: str = Field(..., pattern=r"^[0-2][0-9]:[0-5][0-9]$")
end_utc: str = Field(..., pattern=r"^[0-2][0-9]:[0-5][0-9]$")
class RoutingCondition(BaseModel):
model_config = {"extra": "forbid"}
task_types: list[str] = Field(default_factory=list)
max_prompt_tokens: int | None = Field(None, ge=1)
time_range: TimeRange | None = None
class RoutingRule(BaseModel):
model_config = {"extra": "forbid"}
rule_id: str
priority: int = Field(..., ge=0, le=9999)
provider: Provider
model: str
condition: RoutingCondition | None = None
weight: int = Field(100, ge=1, le=100)
class RetryPolicy(BaseModel):
model_config = {"extra": "forbid"}
max_retries: int = Field(3, ge=0, le=10)
backoff_base_seconds: float = Field(1.0, ge=0.1, le=60)
retry_on_provider_errors: bool = True
class PolicyRoutingContract(BaseModel):
"""路由/政策合約"""
model_config = {"extra": "forbid"}
policy_id: str
policy_name: str = Field(..., min_length=1, max_length=256)
routing_rules: list[RoutingRule] = Field(..., min_length=1)
fallback_provider: Provider | None = None
fallback_model: str | None = None
max_cost_per_run_usd: float | None = Field(None, ge=0)
retry_policy: RetryPolicy = Field(default_factory=RetryPolicy)
effective_from: datetime | None = None
effective_to: datetime | None = None
# ─────────────────────────────────────────────────────────────────────────────
# 5. Runtime Run State Contract
# ─────────────────────────────────────────────────────────────────────────────
class RunTrigger(BaseModel):
model_config = {"extra": "forbid"}
trigger_type: str = Field(
..., pattern="^(channel_event|schedule|api|sub_agent|retry)$"
)
channel_event_id: str | None = None
schedule_id: str | None = None
triggered_by: str | None = None
class RuntimeRunStateContract(BaseModel):
"""Run 狀態機合約（ADR-106 Phase 3）"""
model_config = {"extra": "forbid"}
run_id: str = Field(..., description="UUID v7")
project_id: str
agent_id: str
state: RunState
trace_id: str | None = None
parent_run_id: str | None = None
trigger: RunTrigger | None = None
input_sha256: str | None = None
output_sha256: str | None = None
started_at: datetime | None = None
completed_at: datetime | None = None
timeout_at: datetime | None = None
error_code: str | None = None
cost_usd: float | None = Field(None, ge=0)
step_count: int = Field(0, ge=0)
@field_validator("run_id", "parent_run_id")
@classmethod
def validate_uuid(cls, v: str | None) -> str | None:
if v is None:
return v
if not _UUID_RE.match(v):
raise ValueError("必須為標準 UUID 格式")
return v
@field_validator("input_sha256", "output_sha256")
@classmethod
def validate_sha256_fields(cls, v: str | None) -> str | None:
return _validate_sha256(v)
@field_validator("project_id")
@classmethod
def validate_project_id(cls, v: str) -> str:
if not _PROJECT_ID_RE.match(v):
raise ValueError("project_id 格式不合法")
return v
# ─────────────────────────────────────────────────────────────────────────────
# 6. Channel Event Contract
# ─────────────────────────────────────────────────────────────────────────────
class AttachmentRef(BaseModel):
model_config = {"extra": "forbid"}
attachment_type: str = Field(..., pattern="^(photo|document|audio|video)$")
file_id: str
sha256: str | None = None
@field_validator("sha256")
@classmethod
def validate_sha256(cls, v: str | None) -> str | None:
return _validate_sha256(v, "attachment sha256")
class ChannelEventContract(BaseModel):
"""Channel Event 合約（ADR-114 冪等去重）"""
model_config = {"extra": "forbid"}
event_id: str = Field(..., description="Platform 生成的 UUID")
project_id: str
channel_type: ChannelType
event_type: EventType
provider_event_id: str | None = Field(None, max_length=256)
user_id: str | None = None
chat_id: str | None = None
payload: dict[str, Any] = Field(..., min_length=1)
text: str | None = Field(None, max_length=4096)
attachments: list[AttachmentRef] = Field(default_factory=list)
run_id: str | None = None
is_duplicate: bool = False
received_at: datetime
@field_validator("event_id", "run_id")
@classmethod
def validate_uuid(cls, v: str | None) -> str | None:
if v is None:
return v
if not _UUID_RE.match(v):
raise ValueError("必須為標準 UUID 格式")
return v
@field_validator("project_id")
@classmethod
def validate_project_id(cls, v: str) -> str:
if not _PROJECT_ID_RE.match(v):
raise ValueError("project_id 格式不合法")
return v
# ─────────────────────────────────────────────────────────────────────────────
# Contract family dispatcher
# ─────────────────────────────────────────────────────────────────────────────
CONTRACT_FAMILY_MODELS: dict[str, type[BaseModel]] = {
"project_tenant": ProjectTenantContract,
"agent": AgentContract,
"mcp_gateway": MCPGatewayContract,
"policy_routing": PolicyRoutingContract,
"runtime_run_state": RuntimeRunStateContract,
"channel_event": ChannelEventContract,
}
VALID_CONTRACT_FAMILIES = frozenset(CONTRACT_FAMILY_MODELS.keys())
def validate_contract_body(family: str, body: dict[str, Any]) -> BaseModel:
"""
依 contract_family 驗證 body_json。
驗證失敗拋出 pydantic.ValidationError。
"""
model_cls = CONTRACT_FAMILY_MODELS.get(family)
if model_cls is None:
raise ValueError(
f"未知 contract_family: {family!r}。"
f"合法值：{sorted(VALID_CONTRACT_FAMILIES)}"
)
return model_cls.model_validate(body)

136
apps/api/src/plugins/mcp/credential_resolver.py Normal file
View File

@@ -0,0 +1,136 @@
"""
MCP Credential Resolver — k8s Secret 參照解析
=============================================
AwoooP Phase 5.5: ADR-118 Credential Isolation
2026-05-04 ogt + Claude Sonnet 4.6
設計原則（2026-04-18 Secret Leak 事故教訓）：
- 明文 credential 絕不進入 audit log / LLM context
- Gateway 只傳 k8s secret ref（格式："namespace/secret-name#key"）
- 真實 secret value 在記憶體中短暫存在，使用後立刻清除
- 回傳給 caller 時只提供「遮罩版」（前 4 字元 + *** + 後 4 字元）
- sha256(actual_value) 記入 awooop_mcp_credential_refs.value_sha256（指紋，不可還原）
k8s secret ref 格式：
"namespace/secret-name#key"
例："awoooi/telegram-bot#TELEGRAM_BOT_TOKEN"
解析方式（兩種，依環境）：
1. k8s in-cluster：使用 kubernetes asyncclient（prod）
2. 本機開發 fallback：讀 AWOOOP_DEV_SECRETS_JSON 環境變數（dev only）
"""
from __future__ import annotations
import hashlib
import os
import re
import structlog
logger = structlog.get_logger(__name__)
# k8s secret ref 格式正則（與 DB CHECK 一致）
_K8S_REF_RE = re.compile(r"^([a-z0-9-]+)/([a-z0-9-]+)#([a-zA-Z0-9_-]+)$")
# dev fallback：JSON 格式 {"namespace/secret-name#key": "actual_value"}
_DEV_SECRETS_ENV = "AWOOOP_DEV_SECRETS_JSON"
class CredentialResolutionError(Exception):
error_code = "E-MCP-GATE-009"
def _mask_secret(value: str) -> str:
"""回傳遮罩版：前 4 + *** + 後 4（若長度 < 8 則全遮罩）"""
if len(value) < 8:
return "***"
return f"{value[:4]}***{value[-4:]}"
def _sha256_secret(value: str) -> str:
return hashlib.sha256(value.encode()).hexdigest()
async def resolve_k8s_secret(ref: str) -> tuple[str, str, str]:
"""
解析 k8s secret ref，回傳 (actual_value, masked_value, sha256)。
actual_value：明文，caller 必須在使用後清除（不可存入任何持久化層）
masked_value：供 log / response 使用
sha256：供 awooop_mcp_credential_refs.value_sha256 記錄
Raises:
CredentialResolutionError: ref 格式錯誤或 secret 不存在
"""
m = _K8S_REF_RE.match(ref)
if not m:
raise CredentialResolutionError(
f"k8s secret ref 格式錯誤（期望 'namespace/secret-name#key'）：{ref!r}"
)
namespace, secret_name, key = m.group(1), m.group(2), m.group(3)
# Dev fallback：讀環境變數
dev_json = os.environ.get(_DEV_SECRETS_ENV)
if dev_json:
try:
import json
dev_secrets: dict[str, str] = json.loads(dev_json)
value = dev_secrets.get(ref)
if value is None:
raise CredentialResolutionError(
f"dev secrets 中找不到 ref={ref!r}"
)
logger.debug("credential_resolved_dev", ref=ref)
return value, _mask_secret(value), _sha256_secret(value)
except CredentialResolutionError:
raise
except Exception as exc:
raise CredentialResolutionError(
f"AWOOOP_DEV_SECRETS_JSON 解析失敗: {exc}"
) from exc
# Production：k8s in-cluster
try:
from kubernetes_asyncio import client, config # type: ignore[import]
from kubernetes_asyncio.client import CoreV1Api # type: ignore[import]
await config.load_incluster_config()
async with client.ApiClient() as api:
v1 = CoreV1Api(api)
secret = await v1.read_namespaced_secret(secret_name, namespace)
if secret.data is None or key not in secret.data:
raise CredentialResolutionError(
f"k8s secret '{namespace}/{secret_name}' 中找不到 key='{key}'"
)
import base64
encoded = secret.data[key]
value = base64.b64decode(encoded).decode()
logger.info(
"credential_resolved_k8s",
namespace=namespace,
secret_name=secret_name,
key=key,
masked=_mask_secret(value),
)
return value, _mask_secret(value), _sha256_secret(value)
except CredentialResolutionError:
raise
except ImportError:
raise CredentialResolutionError(
"kubernetes_asyncio 未安裝，且未設定 AWOOOP_DEV_SECRETS_JSON（dev fallback）"
)
except Exception as exc:
logger.exception(
"credential_resolution_k8s_failed",
ref=ref,
error=str(exc),
)
raise CredentialResolutionError(
f"k8s secret 解析失敗（{namespace}/{secret_name}#{key}）: {exc}"
) from exc

507
apps/api/src/plugins/mcp/gateway.py Normal file
View File

@@ -0,0 +1,507 @@
"""
MCP Gateway — 五閘門 Enforcement Service
=========================================
AwoooP Phase 5.2: ADR-116 五閘門強制執行
2026-05-04 ogt + Claude Sonnet 4.6
五閘門定義（依序，任一失敗即阻斷）：
Gate 1 — Project：project_id 在 awooop_projects 且 migration_mode != 'legacy_awoooi_default'
Gate 2 — Agent：agent_id 在 awooop_agents 且 status = 'active'
Gate 3 — Tool：tool_id 在 awooop_mcp_tool_registry 且 grant 存在且未到期
Gate 4 — Environment：tool.environment_tags 與 run context 匹配（shadow mode 強制放行）
Gate 5 — Approval：工具 scope 需要 approval 時，檢查 multi_sig 是否已核准
錯誤碼（E-MCP-GATE-XXX）：
E-MCP-GATE-001 Gate 1 project 不存在或 migration_mode 不符
E-MCP-GATE-002 Gate 2 agent 不存在或未啟用
E-MCP-GATE-003 Gate 3 tool 不在白名單或 grant 不存在/已到期/已撤銷
E-MCP-GATE-004 Gate 4 environment 標籤不匹配（非 shadow mode）
E-MCP-GATE-005 Gate 5 approval 尚未取得
E-MCP-GATE-009 credential 解析失敗（k8s secret 取不到）
使用方式：
from src.plugins.mcp.gateway import McpGateway, GatewayContext
ctx = GatewayContext(
project_id="awoooi",
agent_id="my-agent",
tool_name="kubectl_get",
run_id=run_id,
trace_id=trace_id,
is_shadow=True,
)
result = await McpGateway(db).call(ctx, parameters={"namespace": "default"})
"""
from __future__ import annotations
import hashlib
import json
import time
from dataclasses import dataclass, field
from datetime import datetime, timezone
from typing import Any
from uuid import UUID
import structlog
from sqlalchemy import select, text
from sqlalchemy.ext.asyncio import AsyncSession
from src.db.awooop_models import (
AwoooPActiveRevision,
AwoooPMcpGatewayAudit,
AwoooPMcpGrant,
AwoooPMcpToolRegistry,
AwoooPProject,
)
from src.plugins.mcp.interfaces import MCPToolResult
from src.plugins.mcp.registry import get_provider_registry
logger = structlog.get_logger(__name__)
# ─────────────────────────────────────────────────────────────────────────────
# 錯誤定義
# ─────────────────────────────────────────────────────────────────────────────
class McpGatewayError(Exception):
"""所有 Gateway 攔截錯誤的基礎類別"""
def __init__(self, error_code: str, message: str, gate: int) -> None:
super().__init__(message)
self.error_code = error_code
self.gate = gate
class GateProjectError(McpGatewayError):
def __init__(self, msg: str = "project 不存在或 migration_mode 不符") -> None:
super().__init__("E-MCP-GATE-001", msg, gate=1)
class GateAgentError(McpGatewayError):
def __init__(self, msg: str = "agent 不存在或未啟用") -> None:
super().__init__("E-MCP-GATE-002", msg, gate=2)
class GateToolError(McpGatewayError):
def __init__(self, msg: str = "tool 不在白名單或 grant 失效") -> None:
super().__init__("E-MCP-GATE-003", msg, gate=3)
class GateEnvironmentError(McpGatewayError):
def __init__(self, msg: str = "environment 標籤不匹配") -> None:
super().__init__("E-MCP-GATE-004", msg, gate=4)
class GateApprovalError(McpGatewayError):
def __init__(self, msg: str = "approval 尚未取得") -> None:
super().__init__("E-MCP-GATE-005", msg, gate=5)
class CredentialResolutionError(McpGatewayError):
def __init__(self, msg: str = "credential 解析失敗") -> None:
super().__init__("E-MCP-GATE-009", msg, gate=0)
# ─────────────────────────────────────────────────────────────────────────────
# Gateway Context（每次 call 一個）
# ─────────────────────────────────────────────────────────────────────────────
@dataclass
class GatewayContext:
project_id: str
agent_id: str
tool_name: str
run_id: UUID | None = None
trace_id: str | None = None
is_shadow: bool = True # shadow mode：Gate 4/5 放行，不執行 destructive
environment: dict[str, str] = field(default_factory=dict) # e.g. {"env": "prod"}
required_scope: str = "read" # "read" | "write" | "admin"
@dataclass
class GateCheckResult:
gate1_project: bool = False
gate2_agent: bool = False
gate3_tool: bool = False
gate4_env: bool = False
gate5_approval: bool = False
def as_dict(self) -> dict[str, bool]:
return {
"gate1_project": self.gate1_project,
"gate2_agent": self.gate2_agent,
"gate3_tool": self.gate3_tool,
"gate4_env": self.gate4_env,
"gate5_approval": self.gate5_approval,
}
@property
def all_passed(self) -> bool:
return all([
self.gate1_project,
self.gate2_agent,
self.gate3_tool,
self.gate4_env,
self.gate5_approval,
])
# ─────────────────────────────────────────────────────────────────────────────
# McpGateway
# ─────────────────────────────────────────────────────────────────────────────
class McpGateway:
"""
MCP Gateway：五閘門 enforcement + audit log + credential isolation。
每個 gateway call 都寫一筆 awooop_mcp_gateway_audit。
"""
def __init__(self, db: AsyncSession) -> None:
self._db = db
async def call(
self,
ctx: GatewayContext,
parameters: dict[str, Any],
) -> MCPToolResult:
"""
執行五閘門檢查後呼叫底層 MCP provider。
任一閘門失敗 → raise McpGatewayError + 寫 blocked audit。
"""
started = time.monotonic()
gate_result = GateCheckResult()
tool_row: AwoooPMcpToolRegistry | None = None
grant_row: AwoooPMcpGrant | None = None
try:
# Gate 1 — Project
tool_row, grant_row = await self._gate1_project(ctx, gate_result)
# Gate 2 — Agent
await self._gate2_agent(ctx, gate_result)
# Gate 3 — Tool + Grant
tool_row, grant_row = await self._gate3_tool(ctx, gate_result)
# Gate 4 — Environment（shadow mode 直接放行）
await self._gate4_environment(ctx, tool_row, gate_result)
# Gate 5 — Approval（shadow mode + scope=read 直接放行）
await self._gate5_approval(ctx, grant_row, gate_result)
except McpGatewayError as exc:
latency = int((time.monotonic() - started) * 1000)
await self._write_audit(
ctx=ctx,
tool_row=tool_row,
parameters=parameters,
result=None,
gate_result=gate_result,
result_status="blocked",
block_gate=exc.gate,
block_reason=f"{exc.error_code}: {exc}",
latency_ms=latency,
)
raise
# 五閘通過 → 執行 tool
result: MCPToolResult | None = None
result_status = "failed"
try:
result = await self._execute_tool(ctx, tool_row, parameters)
result_status = "success" if result.success else "failed"
return result
except Exception as exc:
logger.exception(
"mcp_gateway_execution_error",
project_id=ctx.project_id,
tool_name=ctx.tool_name,
error=str(exc),
)
raise
finally:
latency = int((time.monotonic() - started) * 1000)
await self._write_audit(
ctx=ctx,
tool_row=tool_row,
parameters=parameters,
result=result,
gate_result=gate_result,
result_status=result_status,
block_gate=None,
block_reason=None,
latency_ms=latency,
)
# ── 五閘門實作 ────────────────────────────────────────────────────────────
async def _gate1_project(
self, ctx: GatewayContext, gate_result: GateCheckResult
) -> tuple[AwoooPMcpToolRegistry | None, AwoooPMcpGrant | None]:
"""Gate 1：project 必須存在且 migration_mode != 'legacy_awoooi_default'"""
result = await self._db.execute(
select(AwoooPProject).where(
AwoooPProject.project_id == ctx.project_id,
AwoooPProject.migration_mode != "legacy_awoooi_default",
)
)
project = result.scalar_one_or_none()
if project is None:
raise GateProjectError(
f"project '{ctx.project_id}' 不存在或 migration_mode=legacy_awoooi_default"
)
gate_result.gate1_project = True
return None, None
async def _gate2_agent(
self, ctx: GatewayContext, gate_result: GateCheckResult
) -> None:
"""Gate 2：agent 必須在 awooop_active_revisions 中有 active contract（family='agent'）"""
result = await self._db.execute(
select(AwoooPActiveRevision).where(
AwoooPActiveRevision.project_id == ctx.project_id,
AwoooPActiveRevision.contract_family == "agent",
AwoooPActiveRevision.contract_id == ctx.agent_id,
)
)
active = result.scalar_one_or_none()
if active is None:
raise GateAgentError(
f"agent '{ctx.agent_id}' 在 '{ctx.project_id}' 無 active contract"
)
gate_result.gate2_agent = True
async def _gate3_tool(
self, ctx: GatewayContext, gate_result: GateCheckResult
) -> tuple[AwoooPMcpToolRegistry, AwoooPMcpGrant]:
"""Gate 3：tool 在白名單 + grant 有效（未到期、未撤銷）"""
now = datetime.now(timezone.utc)
# 查 tool registry
tool_result = await self._db.execute(
select(AwoooPMcpToolRegistry).where(
AwoooPMcpToolRegistry.project_id == ctx.project_id,
AwoooPMcpToolRegistry.tool_name == ctx.tool_name,
AwoooPMcpToolRegistry.is_active.is_(True),
)
)
tool_row = tool_result.scalar_one_or_none()
if tool_row is None:
raise GateToolError(f"tool '{ctx.tool_name}' 不在白名單")
# 查 grant（scope 必須包含 required_scope）
grant_result = await self._db.execute(
select(AwoooPMcpGrant).where(
AwoooPMcpGrant.project_id == ctx.project_id,
AwoooPMcpGrant.agent_id == ctx.agent_id,
AwoooPMcpGrant.tool_id == tool_row.tool_id,
AwoooPMcpGrant.is_revoked.is_(False),
)
)
grant_row = grant_result.scalar_one_or_none()
if grant_row is None:
raise GateToolError(
f"agent '{ctx.agent_id}' 對 tool '{ctx.tool_name}' 無有效 grant"
)
if grant_row.expires_at is not None and grant_row.expires_at < now:
raise GateToolError(
f"agent '{ctx.agent_id}' 對 tool '{ctx.tool_name}' 的 grant 已到期"
)
# scope 檢查：required_scope 必須在 granted_scopes 中
granted_scopes: list[str] = grant_row.granted_scopes or []
if ctx.required_scope not in granted_scopes:
raise GateToolError(
f"grant 未包含所需 scope '{ctx.required_scope}'（有：{granted_scopes}）"
)
gate_result.gate3_tool = True
return tool_row, grant_row
async def _gate4_environment(
self,
ctx: GatewayContext,
tool_row: AwoooPMcpToolRegistry | None,
gate_result: GateCheckResult,
) -> None:
"""Gate 4：environment 標籤匹配（shadow mode 強制放行）"""
if ctx.is_shadow:
gate_result.gate4_env = True
return
if tool_row is None:
gate_result.gate4_env = True
return
required_tags: dict[str, str] = tool_row.environment_tags or {}
for k, v in required_tags.items():
if ctx.environment.get(k) != v:
raise GateEnvironmentError(
f"environment tag '{k}' 期望 '{v}'，實際 '{ctx.environment.get(k)}'"
)
gate_result.gate4_env = True
async def _gate5_approval(
self,
ctx: GatewayContext,
grant_row: AwoooPMcpGrant | None,
gate_result: GateCheckResult,
) -> None:
"""Gate 5：需要 approval 時，檢查 Redis multi_sig（shadow + read scope 直接放行）"""
# shadow mode 或 read scope 不需 approval
if ctx.is_shadow or ctx.required_scope == "read":
gate_result.gate5_approval = True
return
# write/admin scope 需要檢查 approval
if ctx.run_id is None:
raise GateApprovalError("write/admin 操作需要 run_id（approval 追蹤用）")
try:
import aioredis
from src.core.config import settings
redis = aioredis.from_url(settings.REDIS_URL)
approval_key = f"mcp_approval:{ctx.project_id}:{ctx.agent_id}:{ctx.tool_name}:{ctx.run_id}"
approved = await redis.get(approval_key)
await redis.aclose()
except Exception as exc:
logger.warning(
"mcp_gate5_redis_error",
project_id=ctx.project_id,
tool_name=ctx.tool_name,
error=str(exc),
)
# Redis 失敗時 fail-closed（不放行）
raise GateApprovalError(f"approval Redis 查詢失敗: {exc}") from exc
if not approved:
raise GateApprovalError(
f"tool '{ctx.tool_name}' 需要 approval（key={approval_key}）"
)
gate_result.gate5_approval = True
# ── 執行層 ───────────────────────────────────────────────────────────────
async def _execute_tool(
self,
ctx: GatewayContext,
tool_row: AwoooPMcpToolRegistry | None,
parameters: dict[str, Any],
) -> MCPToolResult:
"""呼叫底層 MCP provider 執行工具"""
registry = get_provider_registry()
provider = registry.get(ctx.tool_name) or registry.get(
tool_row.tool_name if tool_row else ctx.tool_name
)
# 找不到 provider → 回傳 shadow no-op
if provider is None:
logger.warning(
"mcp_gateway_no_provider",
tool_name=ctx.tool_name,
is_shadow=ctx.is_shadow,
)
return MCPToolResult(
success=True,
execution_id=f"shadow-noop-{ctx.tool_name}",
output={"shadow": True, "message": "no provider registered, shadow no-op"},
)
audit_params = dict(parameters)
audit_params["_mcp_audit"] = {
"project_id": ctx.project_id,
"agent_id": ctx.agent_id,
"run_id": str(ctx.run_id) if ctx.run_id else None,
"trace_id": ctx.trace_id,
}
return await provider.execute(ctx.tool_name, audit_params)
# ── Audit log ─────────────────────────────────────────────────────────────
async def _write_audit(
self,
*,
ctx: GatewayContext,
tool_row: AwoooPMcpToolRegistry | None,
parameters: dict[str, Any],
result: MCPToolResult | None,
gate_result: GateCheckResult,
result_status: str,
block_gate: int | None,
block_reason: str | None,
latency_ms: int,
) -> None:
"""寫 awooop_mcp_gateway_audit — 只寫 hash，不寫明文 input/output"""
try:
input_hash = hashlib.sha256(
json.dumps(parameters, sort_keys=True, default=str).encode()
).hexdigest()
output_hash: str | None = None
if result is not None:
output_hash = hashlib.sha256(
json.dumps(result.output, sort_keys=True, default=str).encode()
).hexdigest()
audit = AwoooPMcpGatewayAudit(
project_id=ctx.project_id,
run_id=ctx.run_id,
trace_id=ctx.trace_id,
agent_id=ctx.agent_id,
tool_id=tool_row.tool_id if tool_row else None, # type: ignore[arg-type]
tool_name=ctx.tool_name,
input_hash=input_hash,
output_hash=output_hash,
gate_result=gate_result.as_dict(),
result_status=result_status,
block_gate=block_gate,
block_reason=block_reason,
latency_ms=latency_ms,
)
if tool_row is not None:
self._db.add(audit)
await self._db.flush()
except Exception as exc:
logger.warning(
"mcp_gateway_audit_write_failed",
project_id=ctx.project_id,
tool_name=ctx.tool_name,
error=str(exc),
)
# ─────────────────────────────────────────────────────────────────────────────
# 便捷函數
# ─────────────────────────────────────────────────────────────────────────────
async def gateway_call(
db: AsyncSession,
*,
project_id: str,
agent_id: str,
tool_name: str,
parameters: dict[str, Any],
run_id: UUID | None = None,
trace_id: str | None = None,
is_shadow: bool = True,
required_scope: str = "read",
environment: dict[str, str] | None = None,
) -> MCPToolResult:
"""
Stateless 便捷函數：建立 GatewayContext + 執行 McpGateway.call()。
"""
ctx = GatewayContext(
project_id=project_id,
agent_id=agent_id,
tool_name=tool_name,
run_id=run_id,
trace_id=trace_id,
is_shadow=is_shadow,
required_scope=required_scope,
environment=environment or {},
)
return await McpGateway(db).call(ctx, parameters)

159
apps/api/src/plugins/mcp/redaction_middleware.py Normal file
View File

@@ -0,0 +1,159 @@
"""
MCP Redaction Middleware — 雙層 PII/Secret Redaction
=====================================================
AwoooP Phase 5.3: ADR-116 P1-04 + P1-09
2026-05-04 ogt + Claude Sonnet 4.6
MCP tool call 的 input/output 必須經過雙層 redaction：
Layer 1（audit_sink）— 寫入 audit log 前的 sanitization（欄位黑名單 + pattern 攔截）
Layer 2（本層） — MCP tool call input/output 專用：
- 移除已知 secret 欄位（_mcp_audit 注入的 context）
- 對 output 套用 audit_sink 的完整 redaction patterns
- 限制 output 大小（防 prompt stuffing）
設計原則（ADR-118 credential isolation 延伸）：
- MCP tool 的 output 可能含 k8s secret 值 → 必須在 output 進入 LLM context 前 redact
- 只有「安全的」output 才能被 platform_runtime.shadow_execute 使用
- input credential 欄位（如 k8s_value）在送入 provider 前清除（credential isolation）
雙層保障的必要性：
- audit_sink 保護的是 audit log DB
- 本 middleware 保護的是 LLM context + gateway audit hash
- 兩者防護對象不同，不可互相替代
"""
from __future__ import annotations
import hashlib
import json
import re
from typing import Any
import structlog
from src.services.audit_sink import _BLOCKED_FIELD_NAMES, _REDACTION_PATTERNS, _redact_string
logger = structlog.get_logger(__name__)
# MCP output 進入 LLM context 的最大字元數（防 prompt stuffing）
_MCP_OUTPUT_MAX_CHARS = 16_000
# MCP gateway 注入的 audit context key（送 provider 前移除）
_MCP_AUDIT_KEY = "_mcp_audit"
# MCP credential 欄位名稱（Gate 5 credential isolation — 在 input 中清除）
_MCP_CREDENTIAL_FIELDS = frozenset({
"k8s_value", "secret_value", "credential", "credential_value",
"token_value", "api_key_value", "private_key_value",
})
def redact_mcp_input(parameters: dict[str, Any]) -> dict[str, Any]:
"""
Layer 2 Input Redaction：清理 MCP tool call 的 input parameters。
1. 移除 _mcp_audit（audit context，不應傳給 provider）
2. 移除 credential 欄位（credential isolation）
3. 對剩餘的 string values 套用 audit_sink patterns
"""
cleaned: dict[str, Any] = {}
for key, value in parameters.items():
# 移除 audit context injection
if key == _MCP_AUDIT_KEY:
continue
# credential isolation — 不讓 credential 明文流向 provider
if key.lower() in _MCP_CREDENTIAL_FIELDS:
cleaned[key] = "[REDACTED:CREDENTIAL_ISOLATION]"
continue
# 欄位名稱黑名單（與 audit_sink 對齊）
if key.lower() in _BLOCKED_FIELD_NAMES:
cleaned[key] = "[REDACTED:BLOCKED_FIELD]"
continue
# string value — 套用 pattern redaction
if isinstance(value, str):
cleaned[key] = _redact_string(value)
elif isinstance(value, dict):
cleaned[key] = redact_mcp_input(value)
elif isinstance(value, list):
cleaned[key] = [
redact_mcp_input(item) if isinstance(item, dict)
else (_redact_string(item) if isinstance(item, str) else item)
for item in value
]
else:
cleaned[key] = value
return cleaned
def redact_mcp_output(output: Any) -> Any:
"""
Layer 2 Output Redaction：清理 MCP tool call 的 output。
1. 對 output dict / string 套用 audit_sink patterns
2. 限制 output 大小（防 prompt stuffing）
3. 回傳清理後的 output（供 LLM context 使用）
"""
if output is None:
return None
if isinstance(output, str):
redacted = _redact_string(output)
if len(redacted) > _MCP_OUTPUT_MAX_CHARS:
redacted = redacted[:_MCP_OUTPUT_MAX_CHARS] + f"\n[TRUNCATED:{len(output)} chars]"
return redacted
if isinstance(output, dict):
return _redact_output_dict(output)
if isinstance(output, list):
result = []
total = 0
for item in output:
if total > _MCP_OUTPUT_MAX_CHARS:
result.append(f"[TRUNCATED:{len(output)} items total]")
break
cleaned = redact_mcp_output(item)
serialized = json.dumps(cleaned, ensure_ascii=False, default=str)
total += len(serialized)
result.append(cleaned)
return result
return output
def _redact_output_dict(d: dict[str, Any], depth: int = 0) -> dict[str, Any]:
"""遞迴 redact output dict"""
if depth > 8:
return {"[MAX_DEPTH]": True}
result: dict[str, Any] = {}
for key, value in d.items():
# 欄位名稱黑名單
if key.lower() in _BLOCKED_FIELD_NAMES:
result[key] = "[REDACTED:BLOCKED_FIELD]"
continue
if isinstance(value, str):
result[key] = _redact_string(value)
elif isinstance(value, dict):
result[key] = _redact_output_dict(value, depth + 1)
elif isinstance(value, list):
result[key] = [
_redact_output_dict(item, depth + 1) if isinstance(item, dict)
else (_redact_string(item) if isinstance(item, str) else item)
for item in value
]
else:
result[key] = value
return result
def compute_safe_hash(data: Any) -> str:
"""計算 redacted data 的 sha256（供 gateway audit 使用）"""
serialized = json.dumps(data, sort_keys=True, ensure_ascii=False, default=str)
return hashlib.sha256(serialized.encode()).hexdigest()

14
apps/api/src/plugins/mcp/registry.py
View File

@@ -21,18 +21,20 @@ class AuditedMCPToolProvider(MCPToolProvider):
"""Provider wrapper that writes every MCP tool call to the audit subsystem."""
def __init__(self, provider: MCPToolProvider) -> None:
self._provider = provider
# __provider 使用 Python name mangling（_AuditedMCPToolProvider__provider）
# 防止 caller 透過 wrapper._provider 直接存取 inner provider（ADR-116 封裝要求）
self.__provider = provider # noqa: SLF001 — intentional name mangling
@property
def name(self) -> str:
return self._provider.name
return self.__provider.name
@property
def enabled(self) -> bool:
return self._provider.enabled
return self.__provider.enabled
async def list_tools(self) -> list[MCPTool]:
return await self._provider.list_tools()
return await self.__provider.list_tools()
async def execute(
self,
@@ -49,7 +51,7 @@ class AuditedMCPToolProvider(MCPToolProvider):
started = monotonic_ms()
result: MCPToolResult | None = None
try:
result = await self._provider.execute(tool_name, provider_parameters)
result = await self.__provider.execute(tool_name, provider_parameters)
return result
finally:
duration_ms = monotonic_ms() - started
@@ -68,7 +70,7 @@ class AuditedMCPToolProvider(MCPToolProvider):
)
async def health_check(self) -> bool:
return await self._provider.health_check()
return await self.__provider.health_check()
class ProviderRegistry:

261
apps/api/src/repositories/contract_repository.py Normal file
View File

@@ -0,0 +1,261 @@
"""
Contract Repository
===================
AwoooP Phase 3: contract revision CRUD（append-only）
2026-05-04 ogt + Claude Sonnet 4.6（ADR-107/ADR-112）
設計原則：
- append-only：已 published 的 revision 不可修改
- active pointer 以 UPSERT 維護（awooop_active_revisions）
- outbox 事件在同一 transaction 寫入（ADR-113）
- RLS 透過 get_db_context() 自動套用
"""
from __future__ import annotations
from typing import Any
from uuid import UUID
import structlog
from sqlalchemy import select, update
from sqlalchemy.dialects.postgresql import insert as pg_insert
from src.db.awooop_models import (
AwoooPActiveRevision,
AwoooPContractOutbox,
AwoooPContractRevision,
)
from src.db.base import get_db_context
logger = structlog.get_logger(__name__)
# ─────────────────────────────────────────────────────────────────────────────
# Read
# ─────────────────────────────────────────────────────────────────────────────
async def get_revision(
revision_id: UUID,
project_id: str = "awoooi",
) -> AwoooPContractRevision | None:
"""依 revision_id 讀取單筆（含 RLS 驗證）"""
async with get_db_context(project_id) as db:
result = await db.execute(
select(AwoooPContractRevision).where(
AwoooPContractRevision.revision_id == revision_id,
AwoooPContractRevision.project_id == project_id,
)
)
return result.scalar_one_or_none()
async def get_active_revision(
project_id: str,
contract_family: str,
contract_id: str,
) -> AwoooPContractRevision | None:
"""
讀取 active revision（runtime 路徑）。
只返回 lifecycle_status='active' 的 revision。
"""
async with get_db_context(project_id) as db:
result = await db.execute(
select(AwoooPContractRevision)
.join(
AwoooPActiveRevision,
AwoooPActiveRevision.active_revision_id == AwoooPContractRevision.revision_id,
)
.where(
AwoooPActiveRevision.project_id == project_id,
AwoooPActiveRevision.contract_family == contract_family,
AwoooPActiveRevision.contract_id == contract_id,
AwoooPContractRevision.lifecycle_status == "active",
)
)
return result.scalar_one_or_none()
async def list_revisions(
project_id: str,
contract_family: str,
contract_id: str,
lifecycle_status: str | None = None,
) -> list[AwoooPContractRevision]:
"""列出所有 revision（按 version 降序）"""
async with get_db_context(project_id) as db:
q = select(AwoooPContractRevision).where(
AwoooPContractRevision.project_id == project_id,
AwoooPContractRevision.contract_family == contract_family,
AwoooPContractRevision.contract_id == contract_id,
)
if lifecycle_status:
q = q.where(AwoooPContractRevision.lifecycle_status == lifecycle_status)
q = q.order_by(
AwoooPContractRevision.version_major.desc(),
AwoooPContractRevision.version_minor.desc(),
)
result = await db.execute(q)
return list(result.scalars().all())
# ─────────────────────────────────────────────────────────────────────────────
# Write（append-only）
# ─────────────────────────────────────────────────────────────────────────────
async def create_draft(
*,
project_id: str,
contract_family: str,
contract_id: str,
version_major: int,
version_minor: int,
body_json: dict[str, Any],
body_hash: str,
body_schema_version: str = "v1.0",
) -> AwoooPContractRevision:
"""建立 draft revision（不可被 runtime 讀取）"""
async with get_db_context(project_id) as db:
revision = AwoooPContractRevision(
project_id=project_id,
contract_family=contract_family,
contract_id=contract_id,
version_major=version_major,
version_minor=version_minor,
lifecycle_status="draft",
body_json=body_json,
body_hash=body_hash,
body_schema_version=body_schema_version,
)
db.add(revision)
await db.flush()
await db.refresh(revision)
logger.info(
"contract_draft_created",
revision_id=str(revision.revision_id),
project_id=project_id,
contract_family=contract_family,
contract_id=contract_id,
)
return revision
async def mark_published(
*,
revision_id: UUID,
project_id: str,
publisher_id: str,
publish_signature: str,
published_at: Any, # datetime
) -> AwoooPContractRevision:
"""
draft → published 轉換（HMAC 簽章驗證後由 service 呼叫）。
published revision 可被 activate，但不可被 runtime 直接讀取。
"""
async with get_db_context(project_id) as db:
await db.execute(
update(AwoooPContractRevision)
.where(
AwoooPContractRevision.revision_id == revision_id,
AwoooPContractRevision.project_id == project_id,
AwoooPContractRevision.lifecycle_status == "draft",
)
.values(
lifecycle_status="published",
publisher_id=publisher_id,
publish_signature=publish_signature,
published_at=published_at,
)
)
result = await db.execute(
select(AwoooPContractRevision).where(
AwoooPContractRevision.revision_id == revision_id
)
)
revision = result.scalar_one()
logger.info(
"contract_published",
revision_id=str(revision_id),
project_id=project_id,
publisher_id=publisher_id,
)
return revision
async def mark_active(
*,
revision_id: UUID,
project_id: str,
contract_family: str,
contract_id: str,
old_revision_id: UUID | None,
) -> AwoooPContractRevision:
"""
published → active 轉換 + 更新 active pointer + 寫入 outbox。
三個操作在同一 transaction（ADR-113 transactional outbox）。
"""
async with get_db_context(project_id) as db:
# 1. 更新 revision lifecycle_status
await db.execute(
update(AwoooPContractRevision)
.where(
AwoooPContractRevision.revision_id == revision_id,
AwoooPContractRevision.project_id == project_id,
AwoooPContractRevision.lifecycle_status == "published",
)
.values(lifecycle_status="active")
)
# 2. UPSERT active pointer
stmt = pg_insert(AwoooPActiveRevision).values(
project_id=project_id,
contract_family=contract_family,
contract_id=contract_id,
active_revision_id=revision_id,
)
stmt = stmt.on_conflict_do_update(
constraint="uq_active_pointer",
set_={
"active_revision_id": revision_id,
},
)
await db.execute(stmt)
# 3. 寫入 outbox event（ADR-113）
outbox_event = AwoooPContractOutbox(
event_type="contract.activated",
project_id=project_id,
contract_family=contract_family,
contract_id=contract_id,
old_revision_id=old_revision_id,
new_revision_id=revision_id,
)
db.add(outbox_event)
# 4. 如有舊 active revision，標記為 revoked
if old_revision_id:
await db.execute(
update(AwoooPContractRevision)
.where(
AwoooPContractRevision.revision_id == old_revision_id,
AwoooPContractRevision.lifecycle_status == "active",
)
.values(lifecycle_status="revoked")
)
result = await db.execute(
select(AwoooPContractRevision).where(
AwoooPContractRevision.revision_id == revision_id
)
)
revision = result.scalar_one()
logger.info(
"contract_activated",
revision_id=str(revision_id),
old_revision_id=str(old_revision_id) if old_revision_id else None,
project_id=project_id,
contract_family=contract_family,
contract_id=contract_id,
)
return revision

1
apps/api/src/repositories/incident_repository.py
View File

@@ -63,6 +63,7 @@ def _incident_to_record_data(incident: Incident) -> dict[str, Any]:
return {
"incident_id": incident.incident_id,
"project_id": getattr(incident, "project_id", "awoooi"), # AwoooP Phase 2.3
"status": incident.status.value,
"severity": incident.severity.value,
"signals": [

20
apps/api/src/repositories/playbook_repository.py
View File

@@ -24,7 +24,7 @@ import structlog
from sqlalchemy import select
from src.core.redis_client import get_redis
from src.db.base import get_session_factory
from src.db.base import get_db_context
from src.db.models import PlaybookRecord
from src.models.playbook import (
Playbook,
@@ -255,8 +255,7 @@ class PlaybookRepository:
Phase 3.5：改用 PG 查詢，效率更高，資料更完整
"""
try:
factory = get_session_factory()
async with factory() as session:
async with get_db_context() as session:
stmt = select(PlaybookRecord)
if status is not None:
stmt = stmt.where(PlaybookRecord.status == status.value)
@@ -356,8 +355,7 @@ class PlaybookRepository:
"""
try:
# 使用 SELECT FOR UPDATE 確保並行 update_stats 不會 lost update
factory = get_session_factory()
async with factory() as session:
async with get_db_context() as session:
async with session.begin():
stmt = (
select(PlaybookRecord)
@@ -411,8 +409,7 @@ class PlaybookRepository:
async def find_by_source_incident(self, incident_id: str) -> list[Playbook]:
"""根據來源 Incident ID 找 Playbook（從 PG 查詢）"""
try:
factory = get_session_factory()
async with factory() as session:
async with get_db_context() as session:
# PG JSONB contains 查詢
stmt = select(PlaybookRecord).where(
PlaybookRecord.source_incident_ids.contains([incident_id])
@@ -529,10 +526,12 @@ class PlaybookRepository:
try:
from sqlalchemy.dialects.postgresql import insert as pg_insert
factory = get_session_factory()
async with factory() as session:
async with get_db_context(
getattr(playbook, "project_id", "awoooi")
) as session:
stmt = pg_insert(PlaybookRecord).values(
playbook_id=playbook.playbook_id,
project_id=getattr(playbook, "project_id", "awoooi"), # AwoooP Phase 2.3
name=playbook.name,
description=playbook.description,
status=playbook.status.value,
@@ -600,8 +599,7 @@ class PlaybookRepository:
async def _pg_get(self, playbook_id: str) -> Playbook | None:
"""從 PostgreSQL 載入 Playbook"""
try:
factory = get_session_factory()
async with factory() as session:
async with get_db_context() as session:
result = await session.get(PlaybookRecord, playbook_id)
if result is None:
return None

115
apps/api/src/services/anomaly_counter.py
View File

@@ -115,8 +115,20 @@ class AnomalyCounter:
# TTL 設定 (35 天，比清理週期長一點)
TTL_SECONDS = 35 * 24 * 3600
def __init__(self, redis_client: redis.Redis) -> None:
def __init__(self, redis_client: redis.Redis, project_id: str = "awoooi") -> None:
self.redis = redis_client
self.project_id = project_id
def _pkey(self, prefix: str, key: str) -> str:
"""新格式 key: {project_id}:{prefix}{key}（Phase A 多租戶）"""
return f"{self.project_id}:{prefix}{key}"
async def _redis_get_with_fallback(self, prefix: str, key: str) -> bytes | None:
"""Phase A: 讀新 key，fallback 到舊 key。"""
val = await self.redis.get(self._pkey(prefix, key))
if val is None:
val = await self.redis.get(f"{prefix}{key}")
return val
@staticmethod
def derive_key_from_incident(incident) -> str | None:
@@ -217,7 +229,7 @@ class AnomalyCounter:
) -> AnomalyFrequency:
"""實際的異常記錄邏輯（可能拋出 Redis 異常）"""
timestamp = now.timestamp()
timeline_key = f"{self.PREFIX_TIMELINE}{anomaly_key}"
timeline_key = self._pkey(self.PREFIX_TIMELINE, anomaly_key)
# 1. 添加到 Sorted Set (score = timestamp, member = timestamp string)
await self.redis.zadd(timeline_key, {str(timestamp): timestamp})
@@ -270,27 +282,22 @@ class AnomalyCounter:
else now
)
# 6. 讀取修復統計
repair_count_str = await self.redis.get(
f"{self.PREFIX_REPAIR_COUNT}{anomaly_key}"
)
# 6. 讀取修復統計（Phase A: 讀新 key，fallback 到舊 key）
repair_count_str = await self._redis_get_with_fallback(self.PREFIX_REPAIR_COUNT, anomaly_key)
auto_repair_count = int(repair_count_str) if repair_count_str else 0
permanent_fix_str = await self.redis.get(
f"{self.PREFIX_PERMANENT_FIX}{anomaly_key}"
)
permanent_fix = permanent_fix_str == "1"
permanent_fix_str = await self._redis_get_with_fallback(self.PREFIX_PERMANENT_FIX, anomaly_key)
permanent_fix = permanent_fix_str == b"1" or permanent_fix_str == "1"
# 7. 儲存 metadata (首次記錄時)
metadata_key = f"{self.PREFIX_METADATA}{anomaly_key}"
if not await self.redis.exists(metadata_key):
await self.redis.hset(
metadata_key,
mapping={
"signature": json.dumps(anomaly_signature),
"first_seen": now.isoformat(),
},
)
metadata_key = self._pkey(self.PREFIX_METADATA, anomaly_key)
legacy_metadata_key = f"{self.PREFIX_METADATA}{anomaly_key}"
if not await self.redis.exists(metadata_key) and not await self.redis.exists(legacy_metadata_key):
metadata_payload = {
"signature": json.dumps(anomaly_signature),
"first_seen": now.isoformat(),
}
await self.redis.hset(metadata_key, mapping=metadata_payload)
await self.redis.expire(metadata_key, self.TTL_SECONDS)
# 8. 判斷升級等級
@@ -353,14 +360,14 @@ class AnomalyCounter:
success: 是否成功
"""
try:
repair_key = f"{self.PREFIX_REPAIR_COUNT}{anomaly_key}"
repair_key = self._pkey(self.PREFIX_REPAIR_COUNT, anomaly_key)
# 遞增修復嘗試次數
await self.redis.incr(repair_key)
await self.redis.expire(repair_key, self.TTL_SECONDS)
# 記錄修復歷史 (用於學習)
history_key = f"{self.PREFIX_REPAIR_HISTORY}{anomaly_key}"
history_key = self._pkey(self.PREFIX_REPAIR_HISTORY, anomaly_key)
await self.redis.lpush(
history_key,
json.dumps(
@@ -411,7 +418,7 @@ class AnomalyCounter:
return
try:
key = f"{self.PREFIX_DISPOSITION}{anomaly_key}"
key = self._pkey(self.PREFIX_DISPOSITION, anomaly_key)
await self.redis.hincrby(key, disposition_type, 1)
await self.redis.hincrby(key, "total", 1)
await self.redis.expire(key, self.TTL_SECONDS)
@@ -434,8 +441,11 @@ class AnomalyCounter:
"cold_start_trust": N, "total": N}
"""
try:
key = f"{self.PREFIX_DISPOSITION}{anomaly_key}"
key = self._pkey(self.PREFIX_DISPOSITION, anomaly_key)
raw = await self.redis.hgetall(key)
if not raw:
# Phase A: fallback 到舊 key
raw = await self.redis.hgetall(f"{self.PREFIX_DISPOSITION}{anomaly_key}")
return {
"auto_repair": int(raw.get(b"auto_repair", raw.get("auto_repair", 0))),
"human_approved": int(raw.get(b"human_approved", raw.get("human_approved", 0))),
@@ -471,11 +481,25 @@ class AnomalyCounter:
try:
# S2 Fix: 使用 Pipeline 批次查詢，消除 N+1 問題
pattern = f"{self.PREFIX_DISPOSITION}*"
# Phase A: 先掃新前綴，若無資料 fallback 到舊前綴
new_pattern = f"{self.project_id}:{self.PREFIX_DISPOSITION}*"
new_strip = f"{self.project_id}:{self.PREFIX_DISPOSITION}"
legacy_pattern = f"{self.PREFIX_DISPOSITION}*"
legacy_strip = self.PREFIX_DISPOSITION
keys: list = []
async for key in self.redis.scan_iter(match=pattern, count=100):
async for key in self.redis.scan_iter(match=new_pattern, count=100):
keys.append(key)
if keys:
strip_prefix = new_strip
meta_prefix = f"{self.project_id}:{self.PREFIX_METADATA}"
else:
async for key in self.redis.scan_iter(match=legacy_pattern, count=100):
keys.append(key)
strip_prefix = legacy_strip
meta_prefix = self.PREFIX_METADATA
if not keys:
return total_summary, by_anomaly
@@ -489,11 +513,11 @@ class AnomalyCounter:
anomaly_keys_str = []
for key in keys:
key_str = key.decode() if isinstance(key, bytes) else key
anomaly_keys_str.append(key_str.replace(self.PREFIX_DISPOSITION, ""))
anomaly_keys_str.append(key_str.replace(strip_prefix, ""))
meta_pipe = self.redis.pipeline(transaction=False)
for ak in anomaly_keys_str:
meta_pipe.hget(f"{self.PREFIX_METADATA}{ak}", "signature")
meta_pipe.hget(f"{meta_prefix}{ak}", "signature")
meta_results = await meta_pipe.execute()
for i, raw in enumerate(results):
@@ -547,13 +571,13 @@ class AnomalyCounter:
"""
try:
await self.redis.set(
f"{self.PREFIX_PERMANENT_FIX}{anomaly_key}",
self._pkey(self.PREFIX_PERMANENT_FIX, anomaly_key),
"1",
ex=90 * 24 * 3600, # 90 天
)
# 記錄修復詳情
metadata_key = f"{self.PREFIX_METADATA}{anomaly_key}"
metadata_key = self._pkey(self.PREFIX_METADATA, anomaly_key)
await self.redis.hset(
metadata_key,
mapping={
@@ -588,8 +612,11 @@ class AnomalyCounter:
}
"""
try:
history_key = f"{self.PREFIX_REPAIR_HISTORY}{anomaly_key}"
history_key = self._pkey(self.PREFIX_REPAIR_HISTORY, anomaly_key)
history = await self.redis.lrange(history_key, 0, -1)
if not history:
# Phase A: fallback 到舊 key
history = await self.redis.lrange(f"{self.PREFIX_REPAIR_HISTORY}{anomaly_key}", 0, -1)
total = 0
success_count = 0
@@ -627,8 +654,11 @@ class AnomalyCounter:
}
"""
try:
history_key = f"{self.PREFIX_REPAIR_HISTORY}{anomaly_key}"
history_key = self._pkey(self.PREFIX_REPAIR_HISTORY, anomaly_key)
history = await self.redis.lrange(history_key, 0, -1)
if not history:
# Phase A: fallback 到舊 key
history = await self.redis.lrange(f"{self.PREFIX_REPAIR_HISTORY}{anomaly_key}", 0, -1)
stats: dict[str, dict] = {}
@@ -666,11 +696,14 @@ class AnomalyCounter:
AnomalyFrequency 或 None (若無記錄 或 Redis 重連失敗)
"""
try:
timeline_key = f"{self.PREFIX_TIMELINE}{anomaly_key}"
timeline_key = self._pkey(self.PREFIX_TIMELINE, anomaly_key)
legacy_timeline_key = f"{self.PREFIX_TIMELINE}{anomaly_key}"
# 檢查是否有記錄
# Phase A: 若新 key 無資料，改用舊 key
if not await self.redis.exists(timeline_key):
return None
if not await self.redis.exists(legacy_timeline_key):
return None
timeline_key = legacy_timeline_key
now = datetime.now()
cutoff_30d = (now - timedelta(days=30)).timestamp()
@@ -716,16 +749,12 @@ class AnomalyCounter:
else now
)
# 讀取修復統計
repair_count_str = await self.redis.get(
f"{self.PREFIX_REPAIR_COUNT}{anomaly_key}"
)
# 讀取修復統計（Phase A: 讀新 key，fallback 到舊 key）
repair_count_str = await self._redis_get_with_fallback(self.PREFIX_REPAIR_COUNT, anomaly_key)
auto_repair_count = int(repair_count_str) if repair_count_str else 0
permanent_fix_str = await self.redis.get(
f"{self.PREFIX_PERMANENT_FIX}{anomaly_key}"
)
permanent_fix = permanent_fix_str == "1"
permanent_fix_str = await self._redis_get_with_fallback(self.PREFIX_PERMANENT_FIX, anomaly_key)
permanent_fix = permanent_fix_str in (b"1", "1")
escalation_level = self._get_escalation_level(count_24h)
@@ -797,7 +826,7 @@ def get_anomaly_counter() -> AnomalyCounter:
if _anomaly_counter is None:
from src.core.redis_client import get_redis
_anomaly_counter = AnomalyCounter(get_redis())
_anomaly_counter = AnomalyCounter(get_redis(), project_id="awoooi")
return _anomaly_counter

227
apps/api/src/services/audit_sink.py Normal file
View File

@@ -0,0 +1,227 @@
"""
Audit Sink with PII/Secret Redaction
======================================
AwoooP Phase 4.4: Audit log 寫入前的 sanitization pipeline（ADR-116）
2026-05-04 ogt + Claude Sonnet 4.6
設計原則：
- audit log 不記錄 raw LLM input/output，只記 hash + schema validation result
- PII / secret pattern 硬攔（不可被 caller 繞過）
- 攔截清單：GCP IP、PostgreSQL password、Telegram token、SSH key、Bearer token 等
- redaction 後原值不可還原（替換為 [REDACTED:<type>]）
- 所有 audit 寫入透過此 sink（禁止其他 service 直接 INSERT audit_logs）
使用：
from src.services.audit_sink import write_audit
await write_audit(
project_id="awoooi",
action="run.completed",
resource_type="run",
resource_id=str(run_id),
details={"trace_id": trace_id, "cost_usd": 0.012},
)
"""
from __future__ import annotations
import hashlib
import json
import re
from typing import Any
import structlog
logger = structlog.get_logger(__name__)
# ─────────────────────────────────────────────────────────────────────────────
# Redaction patterns（ADR-116 P1-08）
# ─────────────────────────────────────────────────────────────────────────────
# 每個 pattern: (compiled_re, replacement_tag)
_REDACTION_PATTERNS: list[tuple[re.Pattern[str], str]] = [
# Telegram bot token（數字:英數字母混合 32~64 字元）
(re.compile(r"\d{8,12}:[A-Za-z0-9_-]{32,64}"), "TELEGRAM_TOKEN"),
# PostgreSQL connection string
(re.compile(r"postgresql(?:\+asyncpg)?://[^:]+:[^@]+@[^/\s]+"), "PG_DSN"),
# Generic password in URL / config
(re.compile(r"(?i)(?:password|passwd|pwd)\s*[:=]\s*\S+"), "PASSWORD"),
# Bearer / Authorization header value
(re.compile(r"(?i)(?:bearer|token)\s+[A-Za-z0-9\-._~+/]+=*"), "BEARER_TOKEN"),
# AWS / GCP / NVIDIA API key patterns
(re.compile(r"(?i)(?:api[_-]?key|apikey)\s*[:=]\s*[A-Za-z0-9\-._]{20,}"), "API_KEY"),
# Private GCP internal IPs（ADR-116 禁止 GCP 內網 IP 進 log）
(re.compile(r"\b10\.\d{1,3}\.\d{1,3}\.\d{1,3}\b"), "INTERNAL_IP"),
(re.compile(r"\b172\.(?:1[6-9]|2\d|3[0-1])\.\d{1,3}\.\d{1,3}\b"), "INTERNAL_IP"),
(re.compile(r"\b192\.168\.\d{1,3}\.\d{1,3}\b"), "INTERNAL_IP"),
# SSH private key
(re.compile(r"-----BEGIN (?:RSA|EC|OPENSSH) PRIVATE KEY-----[\s\S]*?-----END [A-Z ]+ PRIVATE KEY-----"), "SSH_PRIVATE_KEY"),
# JWT（三段 base64 以 . 分隔）
(re.compile(r"eyJ[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+"), "JWT_TOKEN"),
# Hex secret >= 32 位（可能是 HMAC key / session token）
(re.compile(r"\b[0-9a-f]{64}\b"), "HEX_SECRET_64"),
]
# 欄位名稱黑名單：這些 key 的 value 直接替換（不做 pattern 掃描）
_BLOCKED_FIELD_NAMES = frozenset({
"password", "passwd", "pwd", "secret", "token", "api_key", "apikey",
"private_key", "private_key_pem", "bot_token", "telegram_token",
"hmac_key", "jwt", "authorization", "cookie", "session",
})
# LLM raw input/output 欄位名稱（只記 hash）
_LLM_RAW_FIELDS = frozenset({
"raw_input", "raw_output", "llm_input", "llm_output",
"prompt", "completion", "system_prompt",
})
# ─────────────────────────────────────────────────────────────────────────────
# Sanitization pipeline
# ─────────────────────────────────────────────────────────────────────────────
def _redact_string(value: str) -> str:
"""對字串套用所有 redaction patterns"""
for pattern, tag in _REDACTION_PATTERNS:
value = pattern.sub(f"[REDACTED:{tag}]", value)
return value
def sanitize(details: dict[str, Any]) -> dict[str, Any]:
"""
遞迴處理 details dict，套用所有 redaction 規則。
規則優先序：
1. key 在 _BLOCKED_FIELD_NAMES → value 替換為 [REDACTED:BLOCKED_FIELD]
2. key 在 _LLM_RAW_FIELDS → value 替換為 sha256(str(value))（只記 hash）
3. string value → pattern redaction
4. nested dict/list → 遞迴處理
"""
return _sanitize_value(details, depth=0)
def _sanitize_value(value: Any, depth: int = 0) -> Any:
if depth > 10:
return "[REDACTED:MAX_DEPTH]"
if isinstance(value, dict):
return {k: _sanitize_dict_entry(k, v, depth) for k, v in value.items()}
if isinstance(value, list):
return [_sanitize_value(item, depth + 1) for item in value]
if isinstance(value, str):
return _redact_string(value)
return value
def _sanitize_dict_entry(key: str, value: Any, depth: int) -> Any:
key_lower = key.lower()
if key_lower in _BLOCKED_FIELD_NAMES:
return "[REDACTED:BLOCKED_FIELD]"
if key_lower in _LLM_RAW_FIELDS:
# 只記 sha256 hash，不記原始內容
raw_str = json.dumps(value, ensure_ascii=False) if not isinstance(value, str) else value
return f"[LLM_RAW_HASH:{hashlib.sha256(raw_str.encode()).hexdigest()[:16]}]"
return _sanitize_value(value, depth + 1)
# ─────────────────────────────────────────────────────────────────────────────
# Audit write
# ─────────────────────────────────────────────────────────────────────────────
async def write_audit(
*,
project_id: str,
action: str,
resource_type: str,
resource_id: str,
details: dict[str, Any] | None = None,
run_id: str | None = None,
trace_id: str | None = None,
) -> None:
"""
統一 audit log 寫入入口（Phase 4+ 所有 service 必須透過此方法）。
1. sanitize details（PII / secret redaction）
2. 附加 run_id / trace_id（可觀測性）
3. INSERT audit_logs（非阻擋 background task）
"""
import asyncio
asyncio.create_task(
_write_audit_impl(
project_id=project_id,
action=action,
resource_type=resource_type,
resource_id=resource_id,
details=details,
run_id=run_id,
trace_id=trace_id,
),
name="audit_sink_write",
)
async def _write_audit_impl(
*,
project_id: str,
action: str,
resource_type: str,
resource_id: str,
details: dict[str, Any] | None,
run_id: str | None,
trace_id: str | None,
) -> None:
try:
from sqlalchemy import text as sa_text
from src.db.base import get_db_context
clean_details: dict[str, Any] = sanitize(details or {})
if run_id:
clean_details["_run_id"] = run_id
if trace_id:
clean_details["_trace_id"] = trace_id
async with get_db_context(project_id) as db:
await db.execute(
sa_text("""
INSERT INTO audit_logs
(project_id, action, resource_type, resource_id, details)
VALUES
(:project_id, :action, :resource_type, :resource_id, :details::jsonb)
"""),
{
"project_id": project_id,
"action": action,
"resource_type": resource_type,
"resource_id": resource_id,
"details": json.dumps(clean_details),
},
)
except Exception as exc:
logger.warning(
"audit_sink_write_failed",
action=action,
resource_id=resource_id,
error=str(exc),
)
# ─────────────────────────────────────────────────────────────────────────────
# Convenience：可在測試中驗證 sanitization 結果
# ─────────────────────────────────────────────────────────────────────────────
def sanitize_for_test(details: dict[str, Any]) -> dict[str, Any]:
"""同步 sanitize，供測試使用"""
return sanitize(details)

357
apps/api/src/services/awooop_approval_token.py Normal file
View File

@@ -0,0 +1,357 @@
"""
AwoooP Approval Token — HS256 簽核令牌 + Multi-sig + Suggest Mode
==================================================================
AwoooP Phase 8: ADR-116 Gate 5 approval flow
2026-05-04 ogt + Claude Sonnet 4.6
功能：
1. HS256 Approval Token（自製，不依賴 PyJWT）：
- issue_approval_token() → signed token（3 段 base64url）
- verify_approval_token() → payload（含 jti/exp/sub/approver）
- jti 存 Redis NX（TTL = exp - now）防 token replay
- TTL = 15 分鐘（APPROVAL_TOKEN_TTL = 900s）
2. Multi-sig quorum：
- record_approval() → 驗 token + NX jti + SADD approver_id → 目前簽核數
- check_approval_quorum(required=1) → bool | raise QuorumNotMetError
- Redis Set TTL = 1h
3. Suggest Mode（AWOOOP_SUGGEST_MODE feature flag）：
- is_suggest_mode_enabled() → bool
- build_suggest_action(action_type, target) → SuggestedAction（dry-run）
- 支援 3 個 SRE flow：rollback / scale / restart
Redis key 前綴（與 legacy multi_sig_redis.py 不衝突）：
awooop_appr:jti:{jti} — NX token replay 防護
awooop_appr:sigs:{project_id}:{run_id}:{tool_name} — 簽核人 Set
錯誤碼：
E-APPR-001 token 無效或已過期
E-APPR-002 jti 已使用（replay attack）
E-APPR-003 quorum 未達
E-APPR-004 approver 重複簽核
"""
from __future__ import annotations
import base64
import hashlib
import hmac as _hmac_module
import json
import os
import time
import uuid
from dataclasses import dataclass, field
from typing import Any
import structlog
logger = structlog.get_logger(__name__)
# ─────────────────────────────────────────────────────────────────────────────
# 常數
# ─────────────────────────────────────────────────────────────────────────────
APPROVAL_TOKEN_TTL = 900 # 15 分鐘
_JTI_KEY_PREFIX = "awooop_appr:jti:"
_SIG_SET_PREFIX = "awooop_appr:sigs:"
_SIG_TTL_SECONDS = 3600 # 簽核 Set 1h TTL
_SUGGEST_MODE_ENV = "AWOOOP_SUGGEST_MODE"
# ─────────────────────────────────────────────────────────────────────────────
# 錯誤定義
# ─────────────────────────────────────────────────────────────────────────────
class InvalidApprovalTokenError(Exception):
error_code = "E-APPR-001"
class TokenReplayError(Exception):
error_code = "E-APPR-002"
class QuorumNotMetError(Exception):
error_code = "E-APPR-003"
class DuplicateApproverError(Exception):
error_code = "E-APPR-004"
# ─────────────────────────────────────────────────────────────────────────────
# HS256 Token 實作
# ─────────────────────────────────────────────────────────────────────────────
def _b64url_encode(data: bytes) -> str:
return base64.urlsafe_b64encode(data).rstrip(b"=").decode()
def _b64url_decode(s: str) -> bytes:
padding = 4 - len(s) % 4
if padding != 4:
s += "=" * padding
return base64.urlsafe_b64decode(s)
def _get_hmac_key() -> bytes:
try:
from src.core.config import settings
key = getattr(settings, "APPROVAL_HMAC_KEY", None) or ""
except Exception:
key = ""
key = key or os.environ.get("APPROVAL_HMAC_KEY", "")
if not key:
logger.warning("approval_hmac_key_not_set_using_dev_fallback")
key = "dev-awooop-approval-hmac-fallback"
return key.encode()
def issue_approval_token(
*,
project_id: str,
run_id: str,
tool_name: str,
approver_id: str,
ttl_seconds: int = APPROVAL_TOKEN_TTL,
) -> str:
"""
產生 HS256 Approval Token。
payload：
jti = uuid4().hex（唯一 token ID，用於 Redis NX 防 replay）
iss = "awooop-approval"
sub = "{project_id}:{run_id}:{tool_name}"
approver = approver_id
iat / exp
"""
now = int(time.time())
jti = uuid.uuid4().hex
header = {"alg": "HS256", "typ": "JWT"}
payload = {
"jti": jti,
"iss": "awooop-approval",
"sub": f"{project_id}:{run_id}:{tool_name}",
"approver": approver_id,
"iat": now,
"exp": now + ttl_seconds,
}
h_b64 = _b64url_encode(json.dumps(header, separators=(",", ":")).encode())
p_b64 = _b64url_encode(json.dumps(payload, separators=(",", ":")).encode())
signing_input = f"{h_b64}.{p_b64}"
sig = _hmac_module.new(
_get_hmac_key(),
signing_input.encode(),
hashlib.sha256,
).digest()
return f"{signing_input}.{_b64url_encode(sig)}"
def verify_approval_token(token: str) -> dict[str, Any]:
"""
驗證 HS256 token，回傳 payload。
Raises:
InvalidApprovalTokenError: 簽名無效/過期/格式錯誤
"""
try:
parts = token.split(".")
if len(parts) != 3:
raise InvalidApprovalTokenError("token 非 3 段格式")
h_b64, p_b64, sig_b64 = parts
signing_input = f"{h_b64}.{p_b64}"
expected_sig = _hmac_module.new(
_get_hmac_key(),
signing_input.encode(),
hashlib.sha256,
).digest()
if not _hmac_module.compare_digest(sig_b64, _b64url_encode(expected_sig)):
raise InvalidApprovalTokenError("token 簽名無效")
payload = json.loads(_b64url_decode(p_b64))
if int(time.time()) > payload.get("exp", 0):
raise InvalidApprovalTokenError("token 已過期")
return payload
except InvalidApprovalTokenError:
raise
except Exception as exc:
raise InvalidApprovalTokenError(f"token 解析失敗: {exc}") from exc
# ─────────────────────────────────────────────────────────────────────────────
# Multi-sig Redis approval
# ─────────────────────────────────────────────────────────────────────────────
async def record_approval(
*,
project_id: str,
run_id: str,
tool_name: str,
approver_id: str,
token: str,
) -> int:
"""
記錄一筆簽核。步驟：
1. verify_approval_token（HS256 + exp）
2. sub 匹配驗證
3. Redis NX jti（防 replay）
4. Redis SADD approver_id（防重複）
5. 回傳目前簽核數
Raises:
InvalidApprovalTokenError, TokenReplayError, DuplicateApproverError
"""
payload = verify_approval_token(token)
expected_sub = f"{project_id}:{run_id}:{tool_name}"
if payload.get("sub") != expected_sub:
raise InvalidApprovalTokenError(
f"token sub 不符（期望 '{expected_sub}'，實際 '{payload.get('sub')}'）"
)
jti = payload["jti"]
exp = payload["exp"]
try:
import aioredis
from src.core.config import settings
redis = aioredis.from_url(settings.REDIS_URL)
# jti NX
jti_key = f"{_JTI_KEY_PREFIX}{jti}"
ttl_remaining = max(exp - int(time.time()), 1)
ok = await redis.set(jti_key, "1", nx=True, ex=ttl_remaining)
if not ok:
await redis.aclose()
raise TokenReplayError(f"jti={jti!r} 已使用")
# SADD approver
sig_key = f"{_SIG_SET_PREFIX}{project_id}:{run_id}:{tool_name}"
added = await redis.sadd(sig_key, approver_id)
if added == 0:
await redis.aclose()
raise DuplicateApproverError(f"approver '{approver_id}' 已簽核")
await redis.expire(sig_key, _SIG_TTL_SECONDS)
count = int(await redis.scard(sig_key))
await redis.aclose()
logger.info(
"awooop_approval_recorded",
project_id=project_id,
run_id=run_id,
tool_name=tool_name,
approver_id=approver_id,
count=count,
)
return count
except (InvalidApprovalTokenError, TokenReplayError, DuplicateApproverError):
raise
except Exception as exc:
logger.exception("awooop_approval_redis_error", error=str(exc))
raise InvalidApprovalTokenError(f"Redis 錯誤: {exc}") from exc
async def check_approval_quorum(
*,
project_id: str,
run_id: str,
tool_name: str,
required_count: int = 1,
) -> bool:
"""
檢查 quorum。Raises QuorumNotMetError if 不足。
"""
try:
import aioredis
from src.core.config import settings
redis = aioredis.from_url(settings.REDIS_URL)
sig_key = f"{_SIG_SET_PREFIX}{project_id}:{run_id}:{tool_name}"
count = int(await redis.scard(sig_key))
await redis.aclose()
if count < required_count:
raise QuorumNotMetError(f"簽核數不足（{count}/{required_count}）")
return True
except QuorumNotMetError:
raise
except Exception as exc:
raise QuorumNotMetError(f"Redis 查詢失敗: {exc}") from exc
# ─────────────────────────────────────────────────────────────────────────────
# Suggest Mode
# ─────────────────────────────────────────────────────────────────────────────
@dataclass
class SuggestedAction:
"""Suggest mode dry-run 結果（不真正執行）"""
action_type: str # 'rollback' | 'scale' | 'restart'
target: str
suggested_command: str
rollback_evidence: dict[str, Any] = field(default_factory=dict)
dry_run: bool = True
approval_required: bool = True
def is_suggest_mode_enabled() -> bool:
return os.environ.get(_SUGGEST_MODE_ENV, "").lower() in ("true", "1", "yes")
async def build_suggest_action(
action_type: str,
*,
target: str,
run_id: str,
project_id: str,
) -> SuggestedAction:
"""
Suggest mode：返回 dry-run 建議，不執行真實操作。
支援 rollback / scale / restart 三個 SRE flow。
"""
if action_type not in ("rollback", "scale", "restart"):
raise ValueError(f"不支援的 action_type: {action_type!r}")
if action_type == "rollback":
command = f"kubectl rollout undo deployment/{target}"
evidence: dict[str, Any] = {
"note": f"需確認 deployment/{target} 當前 image 與 rollout history",
"suggested_verification": f"kubectl rollout history deployment/{target}",
}
elif action_type == "scale":
command = f"kubectl scale deployment/{target} --replicas=<N>"
evidence = {
"note": f"需確認 deployment/{target} 當前 replicas 數量",
"suggested_verification": f"kubectl get deployment/{target} -o json | jq .spec.replicas",
}
else: # restart
command = f"kubectl rollout restart deployment/{target}"
evidence = {
"note": f"需確認 deployment/{target} 當前 pod 狀態",
"suggested_verification": f"kubectl get pods -l app={target}",
}
logger.info(
"suggest_action_built",
project_id=project_id,
run_id=run_id,
action_type=action_type,
target=target,
)
return SuggestedAction(
action_type=action_type,
target=target,
suggested_command=command,
rollback_evidence=evidence,
)

378
apps/api/src/services/budget_service.py Normal file
View File

@@ -0,0 +1,378 @@
"""AwoooP Token Budget Hard Kill Service
=======================================
ADR-120: 三層 Hard Kill 防護架構
2026-05-04 ogt + Claude Sonnet 4.6（Phase 2.6）
防線：
1. Pre-call check（呼叫前）— Layer 1 Tenant + Layer 2 Platform + Layer 3 Emergency Kill
2. Post-call accounting（呼叫後）— 寫 budget_ledger + 更新 Redis cache
3. 告警閾值通知（80% / 95% Telegram 告警）
注意：Layer 0 Run budget 需要 awooop_run_state（Phase 3 SAGA 實作後補加）
"""
from __future__ import annotations
import time
from decimal import Decimal
import structlog
from src.core.config import settings
logger = structlog.get_logger(__name__)
# ─────────────────────────────────────────────────────────────────────────────
# 告警閾值（ADR-120 D4）
# ─────────────────────────────────────────────────────────────────────────────
BUDGET_ALERT_THRESHOLDS = {
"warn": Decimal("0.80"),
"critical": Decimal("0.95"),
"hard_kill": Decimal("1.00"),
}
# Redis key 前綴
_EMERGENCY_KILL_KEY = "platform:budget:emergency_kill"
_TENANT_BUDGET_KEY_PREFIX = "budget:tenant:" # {project_id}:daily_used_usd
_PLATFORM_BUDGET_KEY = "budget:platform:daily_used_usd"
_BUDGET_CACHE_TTL = 300 # 5 分鐘，每次寫入後 refresh
class BudgetExhaustedError(Exception):
"""LLM call 被 hard kill 攔截"""
def __init__(self, error_code: str, message: str) -> None:
self.error_code = error_code
super().__init__(f"[{error_code}] {message}")
# ─────────────────────────────────────────────────────────────────────────────
# 費用計算（按模型定價估算）
# ─────────────────────────────────────────────────────────────────────────────
# USD per 1M tokens（in + out）
_COST_PER_MILLION_TOKENS: dict[str, tuple[float, float]] = {
# (prompt_per_M, completion_per_M)
"claude-opus-4-7": (15.0, 75.0),
"claude-sonnet-4-6": (3.0, 15.0),
"claude-haiku-4-5": (0.8, 4.0),
"gpt-4o": (5.0, 15.0),
"gpt-4o-mini": (0.15, 0.6),
"gemini-2.0-flash": (0.075, 0.3),
"deepseek-r1:14b": (0.0, 0.0), # local Ollama — 無費用
"qwen3:8b": (0.0, 0.0), # local Ollama — 無費用
}
_DEFAULT_COST_PER_M = (3.0, 15.0) # fallback → claude-sonnet
def estimate_cost(
prompt_tokens: int,
completion_tokens: int,
model: str,
) -> Decimal:
"""估算一次 LLM call 的費用（USD）"""
prompt_rate, completion_rate = _COST_PER_MILLION_TOKENS.get(
model, _DEFAULT_COST_PER_M
)
cost = (prompt_tokens / 1_000_000 * prompt_rate +
completion_tokens / 1_000_000 * completion_rate)
return Decimal(str(round(cost, 6)))
# ─────────────────────────────────────────────────────────────────────────────
# Pre-call Budget Check（ADR-120 D2 防線 1）
# ─────────────────────────────────────────────────────────────────────────────
async def check_budget_before_llm_call(
project_id: str,
model: str,
estimated_prompt_tokens: int = 4000,
*,
agent_id: str | None = None,
) -> None:
"""
LLM call 前的三層 budget check。
超出任一層預算 → 拋出 BudgetExhaustedError，阻止 API call。
Redis 不可用時 fail-open（不阻擋呼叫，但記 warning）。
Args:
project_id: 租戶 ID
model: 模型名稱（用於費用估算）
estimated_prompt_tokens: 預估 prompt token 數（保守估計 × 1.5 已含在外）
"""
# Layer 3：Emergency Kill Switch（最優先）
await check_emergency_kill()
# Local Ollama 模型無費用，跳過 Layer 1/2
if model in {"deepseek-r1:14b", "qwen3:8b"} or model.startswith("ollama/"):
return
estimated_cost = estimate_cost(estimated_prompt_tokens, 0, model)
# Layer 2：Tenant Budget
await _check_tenant_budget(project_id, estimated_cost)
# Layer 1：Platform Budget
await _check_platform_budget(estimated_cost)
async def check_emergency_kill() -> None:
"""Layer 3: Emergency Kill Switch — Redis key platform:budget:emergency_kill"""
try:
from src.core.redis_client import get_redis
redis = get_redis()
if await redis.exists(_EMERGENCY_KILL_KEY):
raise BudgetExhaustedError(
"E-BUDGET-004",
"Emergency kill switch activated — contact platform admin",
)
except BudgetExhaustedError:
raise
except Exception as exc:
logger.warning("budget_emergency_kill_check_failed", error=str(exc))
async def _check_tenant_budget(project_id: str, estimated_cost: Decimal) -> None:
"""Layer 2: Tenant Budget（Redis 快取 + awooop_projects.budget_limit_usd）"""
try:
from src.core.redis_client import get_redis
redis = get_redis()
# 讀取 Tenant 每日已用金額
cache_key = f"{_TENANT_BUDGET_KEY_PREFIX}{project_id}"
used_raw = await redis.get(cache_key)
used_usd = Decimal(used_raw.decode() if isinstance(used_raw, bytes) else used_raw or "0")
# 讀取 Tenant 預算上限（從 awooop_projects 表）
limit_usd = await _get_tenant_budget_limit(project_id)
if limit_usd is None:
return # 無上限 → 放行
if used_usd + estimated_cost > limit_usd:
raise BudgetExhaustedError(
"E-BUDGET-002",
f"Tenant {project_id} budget exhausted: "
f"used ${used_usd:.4f} / ${limit_usd:.4f}",
)
# 告警閾值
usage_pct = (used_usd + estimated_cost) / limit_usd
if usage_pct >= BUDGET_ALERT_THRESHOLDS["critical"]:
logger.warning(
"budget_tenant_critical",
project_id=project_id,
usage_pct=float(usage_pct),
used_usd=float(used_usd),
limit_usd=float(limit_usd),
)
elif usage_pct >= BUDGET_ALERT_THRESHOLDS["warn"]:
logger.warning(
"budget_tenant_warn",
project_id=project_id,
usage_pct=float(usage_pct),
used_usd=float(used_usd),
limit_usd=float(limit_usd),
)
except BudgetExhaustedError:
raise
except Exception as exc:
logger.warning("budget_tenant_check_failed", project_id=project_id, error=str(exc))
async def _check_platform_budget(estimated_cost: Decimal) -> None:
"""Layer 1: Platform Budget（config 靜態上限 + Redis 累計）"""
platform_limit = getattr(settings, "PLATFORM_DAILY_BUDGET_USD", None)
if not platform_limit:
return # 未設定 → 放行
try:
from src.core.redis_client import get_redis
redis = get_redis()
used_raw = await redis.get(_PLATFORM_BUDGET_KEY)
used_usd = Decimal(used_raw.decode() if isinstance(used_raw, bytes) else used_raw or "0")
limit_usd = Decimal(str(platform_limit))
if used_usd + estimated_cost > limit_usd:
raise BudgetExhaustedError(
"E-BUDGET-003",
f"Platform budget exhausted: used ${used_usd:.4f} / ${limit_usd:.4f} — "
"all LLM calls suspended",
)
except BudgetExhaustedError:
raise
except Exception as exc:
logger.warning("budget_platform_check_failed", error=str(exc))
# ─────────────────────────────────────────────────────────────────────────────
# Post-call Accounting（ADR-120 D2 防線 2）
# ─────────────────────────────────────────────────────────────────────────────
async def record_token_usage(
*,
project_id: str,
model: str,
provider: str,
prompt_tokens: int,
completion_tokens: int,
agent_id: str | None = None,
run_id: str | None = None,
) -> Decimal:
"""
LLM call 完成後記帳。
1. 計算實際費用
2. INSERT budget_ledger
3. 更新 Redis budget cache（async，不阻擋回傳）
4. 觸發告警閾值通知
Returns:
actual_cost_usd
"""
import asyncio
from uuid import UUID
actual_cost = estimate_cost(prompt_tokens, completion_tokens, model)
# 寫入 budget_ledger（非阻擋）
asyncio.create_task(
_write_budget_ledger(
project_id=project_id,
agent_id=agent_id,
run_id=UUID(run_id) if run_id else None,
model=model,
provider=provider,
prompt_tokens=prompt_tokens,
completion_tokens=completion_tokens,
cost_usd=actual_cost,
),
name="budget_ledger_write",
)
# 更新 Redis cache（非阻擋）
asyncio.create_task(
_update_budget_cache(project_id, actual_cost),
name="budget_cache_update",
)
logger.info(
"token_usage_recorded",
project_id=project_id,
model=model,
prompt_tokens=prompt_tokens,
completion_tokens=completion_tokens,
cost_usd=float(actual_cost),
)
return actual_cost
async def _write_budget_ledger(
*,
project_id: str,
agent_id: str | None,
run_id, # UUID | None
model: str,
provider: str,
prompt_tokens: int,
completion_tokens: int,
cost_usd: Decimal,
) -> None:
"""INSERT budget_ledger（leWOOOgo: DB 寫入在 Service 層，非 Router）"""
try:
from sqlalchemy import text
from src.db.base import get_db_context
async with get_db_context(project_id) as db:
await db.execute(
text("""
INSERT INTO budget_ledger
(project_id, agent_id, run_id, model, provider,
prompt_tokens, completion_tokens, cost_usd)
VALUES
(:project_id, :agent_id, :run_id, :model, :provider,
:prompt_tokens, :completion_tokens, :cost_usd)
"""),
{
"project_id": project_id,
"agent_id": agent_id,
"run_id": run_id,
"model": model,
"provider": provider,
"prompt_tokens": prompt_tokens,
"completion_tokens": completion_tokens,
"cost_usd": cost_usd,
},
)
except Exception as exc:
logger.warning("budget_ledger_write_failed", project_id=project_id, error=str(exc))
async def _update_budget_cache(project_id: str, cost: Decimal) -> None:
"""用 Redis INCRBYFLOAT 更新 Tenant + Platform daily budget cache"""
try:
from src.core.redis_client import get_redis
redis = get_redis()
cost_f = float(cost)
# Tenant daily budget
tenant_key = f"{_TENANT_BUDGET_KEY_PREFIX}{project_id}"
await redis.incrbyfloat(tenant_key, cost_f)
await redis.expire(tenant_key, 86400) # 24h TTL（每日重置）
# Platform daily budget
await redis.incrbyfloat(_PLATFORM_BUDGET_KEY, cost_f)
await redis.expire(_PLATFORM_BUDGET_KEY, 86400)
except Exception as exc:
logger.warning("budget_cache_update_failed", project_id=project_id, error=str(exc))
# ─────────────────────────────────────────────────────────────────────────────
# Helper：從 DB 讀取 Tenant budget limit
# ─────────────────────────────────────────────────────────────────────────────
async def _get_tenant_budget_limit(project_id: str) -> Decimal | None:
"""從 awooop_projects.budget_limit_usd 讀取 Tenant 每日上限（允許 NULL = 無上限）"""
try:
from sqlalchemy import text
from src.db.base import get_db_context
async with get_db_context() as db:
row = await db.execute(
text("SELECT budget_limit_usd FROM awooop_projects WHERE project_id = :pid"),
{"pid": project_id},
)
result = row.scalar_one_or_none()
return Decimal(str(result)) if result is not None else None
except Exception as exc:
logger.warning("get_tenant_budget_limit_failed", project_id=project_id, error=str(exc))
return None
# ─────────────────────────────────────────────────────────────────────────────
# Emergency Kill Switch 管理（Admin 工具）
# ─────────────────────────────────────────────────────────────────────────────
async def activate_emergency_kill(reason: str = "") -> None:
"""啟動緊急停機 — SET platform:budget:emergency_kill"""
from src.core.redis_client import get_redis
redis = get_redis()
await redis.set(_EMERGENCY_KILL_KEY, reason or "activated", ex=86400 * 7)
logger.warning("budget_emergency_kill_activated", reason=reason)
async def deactivate_emergency_kill() -> None:
"""解除緊急停機"""
from src.core.redis_client import get_redis
redis = get_redis()
await redis.delete(_EMERGENCY_KILL_KEY)
logger.info("budget_emergency_kill_deactivated")
async def is_emergency_kill_active() -> bool:
"""查詢緊急停機狀態"""
try:
from src.core.redis_client import get_redis
redis = get_redis()
return bool(await redis.exists(_EMERGENCY_KILL_KEY))
except Exception:
return False

418
apps/api/src/services/channel_hub.py Normal file
View File

@@ -0,0 +1,418 @@
"""
Channel Hub — AwoooP 入站事件統一路由 + Progressive Feedback Policy
====================================================================
AwoooP Phase 7: ADR-106（channel_event family）
2026-05-04 ogt + Claude Sonnet 4.6
功能：
1. Telegram 入站事件鏡像（記錄到 awooop_conversation_event）
2. 建立 platform run（呼叫 platform_runtime.create_run）
3. Progressive Feedback Policy：
- run 進入 WAITING_TOOL 狀態 → 30 秒後若未 complete → 發 interim Telegram 訊息
- 訊息記錄到 awooop_outbound_message
4. Shadow Mode：不發任何 Telegram 訊息（只記錄到 outbound_message, status='shadow'）
Progressive Feedback Policy 設計（ADR-106 P2-03）：
- 用 asyncio.create_task 啟動 30s 計時器
- 30s 後查詢 run state：若仍在 WAITING_TOOL → 發 interim 訊息
- interim 訊息：「AI 正在分析中，請稍候...」（不洩漏 run 細節）
- Final reply 由 shadow_execute() 完成後觸發（Phase 8 實作）
與 legacy telegram_gateway.py 的關係：
- 完全獨立，不修改 legacy gateway
- legacy 繼續處理 legacy flow（signal_worker 觸發的 approval/notification）
- AwoooP run 只走本模組
"""
from __future__ import annotations
import asyncio
import hashlib
import json
from datetime import datetime, timezone
from typing import Any
from uuid import UUID
import structlog
from sqlalchemy import select, text
from sqlalchemy.ext.asyncio import AsyncSession
from src.db.awooop_models import AwoooPRunState
from src.services.audit_sink import _redact_string
from src.services.platform_runtime import create_run
logger = structlog.get_logger(__name__)
# Progressive Feedback Policy：等待超過此秒數才發 interim 訊息
_INTERIM_WAIT_SECONDS = 30
# ─────────────────────────────────────────────────────────────────────────────
# 入站事件記錄
# ─────────────────────────────────────────────────────────────────────────────
async def mirror_inbound_event(
db: AsyncSession,
*,
project_id: str,
channel_type: str,
provider_event_id: str,
platform_subject_id: str | None = None,
channel_user_id: str | None = None,
channel_chat_id: str | None = None,
content_type: str = "text",
raw_content: str | None = None,
attachment_sha256: str | None = None,
provider_ts: datetime | None = None,
run_id: UUID | None = None,
is_duplicate: bool = False,
) -> UUID:
"""
記錄入站 channel event 到 awooop_conversation_event。
raw_content 只用於計算 hash 和 preview，不入庫明文。
回傳 event_id。
"""
content_hash: str | None = None
content_preview: str | None = None
if raw_content is not None:
content_hash = hashlib.sha256(raw_content.encode()).hexdigest()
# preview：redact 後截取前 256 字元
redacted = _redact_string(raw_content)
content_preview = redacted[:256] if len(redacted) > 256 else redacted
result = await db.execute(
text("""
INSERT INTO awooop_conversation_event (
project_id, channel_type, provider_event_id,
platform_subject_id, channel_user_id, channel_chat_id,
run_id, content_type, content_hash, content_preview,
attachment_sha256, is_duplicate, provider_ts, received_at
) VALUES (
:project_id, :channel_type, :provider_event_id,
:platform_subject_id, :channel_user_id, :channel_chat_id,
:run_id, :content_type, :content_hash, :content_preview,
:attachment_sha256, :is_duplicate, :provider_ts, NOW()
)
ON CONFLICT (project_id, channel_type, provider_event_id) DO UPDATE SET
is_duplicate = TRUE,
run_id = COALESCE(EXCLUDED.run_id, awooop_conversation_event.run_id)
RETURNING event_id
"""),
{
"project_id": project_id,
"channel_type": channel_type,
"provider_event_id": provider_event_id,
"platform_subject_id": platform_subject_id,
"channel_user_id": channel_user_id,
"channel_chat_id": channel_chat_id,
"run_id": run_id,
"content_type": content_type,
"content_hash": content_hash,
"content_preview": content_preview,
"attachment_sha256": attachment_sha256,
"is_duplicate": is_duplicate,
"provider_ts": provider_ts,
},
)
row = result.fetchone()
event_id: UUID = row[0]
logger.info(
"channel_event_mirrored",
project_id=project_id,
channel_type=channel_type,
event_id=str(event_id),
is_duplicate=is_duplicate,
)
return event_id
# ─────────────────────────────────────────────────────────────────────────────
# 出站訊息記錄
# ─────────────────────────────────────────────────────────────────────────────
async def record_outbound_message(
db: AsyncSession,
*,
project_id: str,
run_id: UUID,
channel_type: str,
channel_chat_id: str,
message_type: str, # 'interim' | 'final' | 'error' | 'approval_request'
content: str | None = None,
provider_message_id: str | None = None,
send_status: str = "pending",
conversation_event_id: UUID | None = None,
triggered_by_state: str | None = None,
waiting_since: datetime | None = None,
is_shadow: bool = True,
) -> UUID:
"""
記錄出站訊息到 awooop_outbound_message。
is_shadow=True：status='shadow'（不實際發送，只記錄）
"""
content_hash: str | None = None
content_preview: str | None = None
if content is not None:
content_hash = hashlib.sha256(content.encode()).hexdigest()
redacted = _redact_string(content)
content_preview = redacted[:256]
actual_status = "shadow" if is_shadow else send_status
result = await db.execute(
text("""
INSERT INTO awooop_outbound_message (
project_id, run_id, conversation_event_id,
channel_type, channel_chat_id, message_type,
content_hash, content_preview, provider_message_id,
send_status, queued_at,
triggered_by_state, waiting_since
) VALUES (
:project_id, :run_id, :conversation_event_id,
:channel_type, :channel_chat_id, :message_type,
:content_hash, :content_preview, :provider_message_id,
:send_status, NOW(),
:triggered_by_state, :waiting_since
)
RETURNING message_id
"""),
{
"project_id": project_id,
"run_id": run_id,
"conversation_event_id": conversation_event_id,
"channel_type": channel_type,
"channel_chat_id": channel_chat_id,
"message_type": message_type,
"content_hash": content_hash,
"content_preview": content_preview,
"provider_message_id": provider_message_id,
"send_status": actual_status,
"triggered_by_state": triggered_by_state,
"waiting_since": waiting_since,
},
)
row = result.fetchone()
message_id: UUID = row[0]
logger.info(
"outbound_message_recorded",
project_id=project_id,
run_id=str(run_id),
message_type=message_type,
send_status=actual_status,
message_id=str(message_id),
)
return message_id
# ─────────────────────────────────────────────────────────────────────────────
# Progressive Feedback Policy
# ─────────────────────────────────────────────────────────────────────────────
async def schedule_interim_feedback(
*,
project_id: str,
run_id: UUID,
channel_type: str,
channel_chat_id: str,
conversation_event_id: UUID | None = None,
is_shadow: bool = True,
wait_seconds: int = _INTERIM_WAIT_SECONDS,
) -> None:
"""
Progressive Feedback Policy：
等待 wait_seconds 秒後，若 run 仍在 WAITING_TOOL → 發 interim 訊息。
Shadow Mode：記錄到 outbound_message(status='shadow')，不實際發 Telegram 訊息。
"""
asyncio.create_task(
_interim_feedback_task(
project_id=project_id,
run_id=run_id,
channel_type=channel_type,
channel_chat_id=channel_chat_id,
conversation_event_id=conversation_event_id,
is_shadow=is_shadow,
wait_seconds=wait_seconds,
),
name=f"interim_feedback_{str(run_id)[:8]}",
)
async def _interim_feedback_task(
*,
project_id: str,
run_id: UUID,
channel_type: str,
channel_chat_id: str,
conversation_event_id: UUID | None,
is_shadow: bool,
wait_seconds: int,
) -> None:
"""等待後查 run state，仍 waiting_tool 才發 interim"""
await asyncio.sleep(wait_seconds)
try:
from src.db.base import get_db_context
async with get_db_context(project_id) as db:
result = await db.execute(
select(AwoooPRunState.state, AwoooPRunState.is_shadow).where(
AwoooPRunState.run_id == run_id,
AwoooPRunState.project_id == project_id,
)
)
row = result.first()
if row is None:
logger.warning(
"interim_feedback_run_not_found",
run_id=str(run_id),
)
return
state, run_is_shadow = row
if state != "waiting_tool":
# run 已推進（complete/failed 等），不需要 interim
return
waiting_since = datetime.now(timezone.utc)
interim_content = "AI 正在分析中，請稍候... ⏳"
await record_outbound_message(
db,
project_id=project_id,
run_id=run_id,
channel_type=channel_type,
channel_chat_id=channel_chat_id,
message_type="interim",
content=interim_content,
send_status="pending",
conversation_event_id=conversation_event_id,
triggered_by_state="waiting_tool",
waiting_since=waiting_since,
is_shadow=is_shadow or run_is_shadow,
)
if not (is_shadow or run_is_shadow):
# Non-shadow：實際發 Telegram 訊息
await _send_telegram_interim(
channel_chat_id=channel_chat_id,
content=interim_content,
run_id=run_id,
)
logger.info(
"interim_feedback_sent",
project_id=project_id,
run_id=str(run_id),
is_shadow=is_shadow or run_is_shadow,
)
except Exception as exc:
logger.exception(
"interim_feedback_task_error",
run_id=str(run_id),
error=str(exc),
)
async def _send_telegram_interim(
*,
channel_chat_id: str,
content: str,
run_id: UUID,
) -> None:
"""實際發送 Telegram interim 訊息（non-shadow 專用）"""
try:
import os
import httpx
bot_token = os.environ.get("TELEGRAM_BOT_TOKEN")
if not bot_token:
logger.warning("interim_telegram_no_token", run_id=str(run_id))
return
async with httpx.AsyncClient(timeout=10) as client:
await client.post(
f"https://api.telegram.org/bot{bot_token}/sendMessage",
json={
"chat_id": channel_chat_id,
"text": content,
"parse_mode": "HTML",
},
)
except Exception as exc:
logger.warning(
"interim_telegram_send_failed",
run_id=str(run_id),
error=str(exc),
)
# ─────────────────────────────────────────────────────────────────────────────
# Channel Hub 主入口（Telegram inbound）
# ─────────────────────────────────────────────────────────────────────────────
async def handle_telegram_inbound(
db: AsyncSession,
*,
project_id: str,
agent_id: str,
message_id: str,
user_id: str,
chat_id: str,
text: str | None = None,
is_shadow: bool = True,
) -> dict[str, Any]:
"""
Telegram 入站訊息的統一處理入口：
1. mirror_inbound_event（記錄）
2. create_run（建立 platform run）
3. schedule_interim_feedback（Progressive Feedback）
4. 回傳 {event_id, run_id, is_duplicate}
"""
# Step 1: 嘗試建立 run（有冪等保護）
run_id, is_duplicate = await create_run(
project_id=project_id,
agent_id=agent_id,
trigger_type="channel_event",
trigger_ref=f"telegram:{message_id}",
input_payload={"chat_id": chat_id, "user_id": user_id},
channel_type="telegram",
provider_event_id=message_id,
)
# Step 2: Mirror event（含 run_id）
event_id = await mirror_inbound_event(
db,
project_id=project_id,
channel_type="telegram",
provider_event_id=message_id,
channel_user_id=user_id,
channel_chat_id=chat_id,
content_type="text" if text else "callback_query",
raw_content=text,
run_id=run_id,
is_duplicate=is_duplicate,
)
# Step 3: Progressive Feedback（30s 計時器）
if not is_duplicate:
await schedule_interim_feedback(
project_id=project_id,
run_id=run_id,
channel_type="telegram",
channel_chat_id=chat_id,
conversation_event_id=event_id,
is_shadow=is_shadow,
)
return {
"event_id": str(event_id),
"run_id": str(run_id),
"is_duplicate": is_duplicate,
}

449
apps/api/src/services/contract_service.py Normal file
View File

@@ -0,0 +1,449 @@
"""
Contract Lifecycle Service
===========================
AwoooP Phase 3: 合約生命週期管理（ADR-107/ADR-112）
2026-05-04 ogt + Claude Sonnet 4.6
生命週期狀態機：
draft → published → active → revoked
↑ ↓（新 active 把舊的設為 revoked）
操作：
draft() — 建立 draft revision（schema 驗證 + body_hash）
publish() — HMAC 簽章驗證後 draft → published
activate() — approval 確認後 published → active + outbox
get_active() — runtime 唯一讀取路徑（只返回 active revision）
安全機制：
- body_hash = sha256(canonical JSON)（ADR-112）
- publish() 需 HMAC 簽章（settings.CONTRACT_HMAC_KEY）
- activate() 需 Redis multi_sig 確認（ADR-112 approval workflow）
- 所有操作寫入 audit_log
"""
from __future__ import annotations
import hashlib
import hmac
import json
from datetime import datetime, timezone
from typing import Any
from uuid import UUID
import structlog
from pydantic import ValidationError
from src.core.config import settings
from src.db.awooop_models import AwoooPContractRevision
from src.models.awooop_contracts import validate_contract_body
from src.repositories import contract_repository
logger = structlog.get_logger(__name__)
# ─────────────────────────────────────────────────────────────────────────────
# 錯誤定義
# ─────────────────────────────────────────────────────────────────────────────
class ContractError(Exception):
"""合約操作基礎錯誤"""
def __init__(self, error_code: str, message: str) -> None:
self.error_code = error_code
super().__init__(f"[{error_code}] {message}")
class ContractSchemaError(ContractError):
"""body_json 不符合 schema"""
def __init__(self, family: str, details: str) -> None:
super().__init__("E-CONTRACT-001", f"Contract family={family} schema 驗證失敗: {details}")
class ContractSignatureError(ContractError):
"""HMAC 簽章驗證失敗"""
def __init__(self) -> None:
super().__init__("E-CONTRACT-002", "Contract publish 簽章驗證失敗")
class ContractStateError(ContractError):
"""非法狀態轉換"""
def __init__(self, from_state: str, to_state: str) -> None:
super().__init__(
"E-CONTRACT-003",
f"非法狀態轉換 {from_state!r} → {to_state!r}",
)
class ContractApprovalError(ContractError):
"""缺少必要的 activation approval"""
def __init__(self, revision_id: str) -> None:
super().__init__(
"E-CONTRACT-004",
f"revision {revision_id} 尚未取得足夠的 approval 簽核",
)
class ContractNotFoundError(ContractError):
"""Revision 不存在"""
def __init__(self, revision_id: str) -> None:
super().__init__(
"E-CONTRACT-005",
f"Revision {revision_id!r} 不存在或無權限存取",
)
# ─────────────────────────────────────────────────────────────────────────────
# Body hash（ADR-112 artifact integrity）
# ─────────────────────────────────────────────────────────────────────────────
def _compute_body_hash(body_json: dict[str, Any]) -> str:
"""
計算 body_json 的 SHA-256 hex digest。
使用 canonical JSON（sorted keys, no spaces）確保確定性。
"""
canonical = json.dumps(body_json, sort_keys=True, separators=(",", ":"), ensure_ascii=False)
return hashlib.sha256(canonical.encode("utf-8")).hexdigest()
def _verify_publish_signature(
revision_id: str,
body_hash: str,
publisher_id: str,
signature: str,
) -> bool:
"""
驗證 publish HMAC 簽章。
message = f"{revision_id}:{body_hash}:{publisher_id}"
secret = settings.CONTRACT_HMAC_KEY（base64 or hex）
"""
secret = getattr(settings, "CONTRACT_HMAC_KEY", "")
if not secret:
# 未設定 HMAC key → 開發環境放行（但記錄 warning）
logger.warning(
"contract_hmac_key_not_set",
warning="CONTRACT_HMAC_KEY 未設定，publish 簽章驗證跳過（非 production 行為）",
)
return True
message = f"{revision_id}:{body_hash}:{publisher_id}".encode("utf-8")
expected = hmac.new(
secret.encode("utf-8"), message, hashlib.sha256
).hexdigest()
return hmac.compare_digest(expected, signature)
# ─────────────────────────────────────────────────────────────────────────────
# Multi-sig approval（ADR-112 activation approval）
# ─────────────────────────────────────────────────────────────────────────────
_APPROVAL_KEY_PREFIX = "contract:approval:"
_APPROVAL_REQUIRED = 1 # Phase 3：1 人核准即可；Phase 5+ 升為 2
async def _check_activation_approval(revision_id: str, project_id: str) -> bool:
"""
檢查 Redis 中是否有足夠的 activation approval。
key = contract:approval:{project_id}:{revision_id}
value = JSON list of approver IDs
"""
try:
from src.core.redis_client import get_redis
redis = get_redis()
key = f"{_APPROVAL_KEY_PREFIX}{project_id}:{revision_id}"
raw = await redis.get(key)
if not raw:
return False
approvers = json.loads(raw.decode() if isinstance(raw, bytes) else raw)
return len(approvers) >= _APPROVAL_REQUIRED
except Exception as exc:
logger.warning("contract_approval_check_failed", revision_id=revision_id, error=str(exc))
return False
async def record_activation_approval(
revision_id: str,
project_id: str,
approver_id: str,
) -> int:
"""
記錄一個 approver 的核准簽名。
Returns: 目前收到的 approval 數。
"""
from src.core.redis_client import get_redis
redis = get_redis()
key = f"{_APPROVAL_KEY_PREFIX}{project_id}:{revision_id}"
raw = await redis.get(key)
approvers: list[str] = json.loads(raw.decode() if isinstance(raw, bytes) else raw or "[]")
if approver_id not in approvers:
approvers.append(approver_id)
await redis.set(key, json.dumps(approvers), ex=86400) # 24h TTL
logger.info(
"contract_approval_recorded",
revision_id=revision_id,
approver_id=approver_id,
total_approvals=len(approvers),
)
return len(approvers)
# ─────────────────────────────────────────────────────────────────────────────
# Core lifecycle operations
# ─────────────────────────────────────────────────────────────────────────────
async def draft(
*,
project_id: str,
contract_family: str,
contract_id: str,
version_major: int,
version_minor: int,
body_json: dict[str, Any],
body_schema_version: str = "v1.0",
) -> AwoooPContractRevision:
"""
Step 1: 建立 draft revision。
- 驗證 body_json 符合 contract_family 的 Pydantic schema
- 計算 body_hash（sha256 canonical JSON）
- 寫入 DB（lifecycle_status='draft'）
- 寫入 audit log
draft revision 不可被 runtime 讀取（get_active() 只返回 active）。
"""
# Schema 驗證
try:
validate_contract_body(contract_family, body_json)
except ValidationError as exc:
raise ContractSchemaError(contract_family, exc.json(indent=0)) from exc
except ValueError as exc:
raise ContractSchemaError(contract_family, str(exc)) from exc
body_hash = _compute_body_hash(body_json)
revision = await contract_repository.create_draft(
project_id=project_id,
contract_family=contract_family,
contract_id=contract_id,
version_major=version_major,
version_minor=version_minor,
body_json=body_json,
body_hash=body_hash,
body_schema_version=body_schema_version,
)
await _write_audit(
project_id=project_id,
action="contract.drafted",
resource_type="contract_revision",
resource_id=str(revision.revision_id),
details={
"contract_family": contract_family,
"contract_id": contract_id,
"version": f"{version_major}.{version_minor}",
"body_hash": body_hash,
},
)
return revision
async def publish(
*,
revision_id: UUID,
project_id: str,
publisher_id: str,
signature: str,
) -> AwoooPContractRevision:
"""
Step 2: draft → published。
- 讀取 revision（驗證 lifecycle_status='draft'）
- HMAC 簽章驗證（publisher_id + body_hash + revision_id）
- 更新 lifecycle_status='published'
- 寫入 audit log
"""
revision = await contract_repository.get_revision(revision_id, project_id)
if revision is None:
raise ContractNotFoundError(str(revision_id))
if revision.lifecycle_status != "draft":
raise ContractStateError(revision.lifecycle_status, "published")
if not _verify_publish_signature(
str(revision_id), revision.body_hash, publisher_id, signature
):
raise ContractSignatureError()
published_at = datetime.now(timezone.utc)
revision = await contract_repository.mark_published(
revision_id=revision_id,
project_id=project_id,
publisher_id=publisher_id,
publish_signature=signature,
published_at=published_at,
)
await _write_audit(
project_id=project_id,
action="contract.published",
resource_type="contract_revision",
resource_id=str(revision_id),
details={
"publisher_id": publisher_id,
"published_at": published_at.isoformat(),
"body_hash": revision.body_hash,
},
)
return revision
async def activate(
*,
revision_id: UUID,
project_id: str,
activator_id: str,
bypass_approval: bool = False,
) -> AwoooPContractRevision:
"""
Step 3: published → active。
- 讀取 revision（驗證 lifecycle_status='published'）
- 確認 Redis approval（除非 bypass_approval=True）
- 更新 active pointer（UPSERT awooop_active_revisions）
- 舊 active revision → revoked
- 寫入 outbox event（ADR-113）
- 寫入 audit log
"""
revision = await contract_repository.get_revision(revision_id, project_id)
if revision is None:
raise ContractNotFoundError(str(revision_id))
if revision.lifecycle_status != "published":
raise ContractStateError(revision.lifecycle_status, "active")
if not bypass_approval:
approved = await _check_activation_approval(str(revision_id), project_id)
if not approved:
raise ContractApprovalError(str(revision_id))
# 找舊 active revision（如果有）
old_revision = await contract_repository.get_active_revision(
project_id=project_id,
contract_family=revision.contract_family,
contract_id=revision.contract_id,
)
old_revision_id = old_revision.revision_id if old_revision else None
revision = await contract_repository.mark_active(
revision_id=revision_id,
project_id=project_id,
contract_family=revision.contract_family,
contract_id=revision.contract_id,
old_revision_id=old_revision_id,
)
await _write_audit(
project_id=project_id,
action="contract.activated",
resource_type="contract_revision",
resource_id=str(revision_id),
details={
"activator_id": activator_id,
"old_revision_id": str(old_revision_id) if old_revision_id else None,
"contract_family": revision.contract_family,
"contract_id": revision.contract_id,
},
)
return revision
async def get_active(
*,
project_id: str,
contract_family: str,
contract_id: str,
verify_hash: bool = True,
) -> AwoooPContractRevision | None:
"""
Runtime 讀取路徑：只返回 active revision。
verify_hash=True（預設）：從 DB 讀取後驗證 body_hash，
確保 body_json 未被竄改（ADR-112 artifact integrity）。
"""
revision = await contract_repository.get_active_revision(
project_id=project_id,
contract_family=contract_family,
contract_id=contract_id,
)
if revision is None:
return None
if verify_hash:
computed = _compute_body_hash(revision.body_json)
if computed != revision.body_hash:
logger.error(
"contract_hash_mismatch",
revision_id=str(revision.revision_id),
expected=revision.body_hash,
computed=computed,
)
raise ContractError(
"E-CONTRACT-006",
f"revision {revision.revision_id} body_hash 不符（資料可能被竄改）",
)
return revision
async def get_active_body(
*,
project_id: str,
contract_family: str,
contract_id: str,
) -> dict[str, Any] | None:
"""
便利方法：直接返回 body_json（含 hash 驗證）。
None = 沒有 active revision。
"""
revision = await get_active(
project_id=project_id,
contract_family=contract_family,
contract_id=contract_id,
)
return revision.body_json if revision else None
# ─────────────────────────────────────────────────────────────────────────────
# Audit log helper
# ─────────────────────────────────────────────────────────────────────────────
async def _write_audit(
*,
project_id: str,
action: str,
resource_type: str,
resource_id: str,
details: dict[str, Any],
) -> None:
"""寫入 audit_log（非阻擋，失敗只 warning）"""
try:
from sqlalchemy import text as sa_text
from src.db.base import get_db_context
async with get_db_context(project_id) as db:
await db.execute(
sa_text("""
INSERT INTO audit_logs
(project_id, action, resource_type, resource_id, details)
VALUES
(:project_id, :action, :resource_type, :resource_id, :details::jsonb)
"""),
{
"project_id": project_id,
"action": action,
"resource_type": resource_type,
"resource_id": resource_id,
"details": json.dumps(details),
},
)
except Exception as exc:
logger.warning(
"contract_audit_write_failed",
action=action,
resource_id=resource_id,
error=str(exc),
)

375
apps/api/src/services/platform_runtime.py Normal file
View File

@@ -0,0 +1,375 @@
"""
Platform Runtime（Shadow Mode Shell）
======================================
AwoooP Phase 4: 第一個 runtime shell，只跑 shadow，不改 legacy 行為（ADR-106）
2026-05-04 ogt + Claude Sonnet 4.6
Shadow Mode 保證：
1. 0 user-visible response（不發送 Telegram/Slack 任何訊息）
2. 0 destructive tool call（is_destructive=true 的工具全部攔截）
3. 所有執行記錄寫入 awooop_run_state + step_journal（可觀測）
4. budget_service hard kill 同樣作用（防止 shadow 跑出超額費用）
Idempotency（ADR-114）：
(project_id, channel_type, provider_event_id) 複合唯一
Redis NX 先攔（快），PG constraint 最後防（準確）
"""
from __future__ import annotations
import hashlib
import json
import uuid
from datetime import datetime, timedelta, timezone
from typing import Any
import structlog
from sqlalchemy import select
from sqlalchemy.dialects.postgresql import insert as pg_insert
from src.db.awooop_models import AwoooPRunIdempotency, AwoooPRunState, AwoooPRunStepJournal
from src.db.base import get_db_context
from src.services.run_state_machine import LEASE_TTL_SECONDS, transition
logger = structlog.get_logger(__name__)
# Shadow mode 設定
_SHADOW_ENABLED = True # Phase 4 固定 True；Phase 6+ 由 migration_mode 控制
_DESTRUCTIVE_TOOL_KEYWORDS = frozenset({
"delete", "drop", "remove", "kill", "terminate", "destroy",
"rollback", "revert", "patch", "apply", "exec", "execute",
"restart", "scale", "cordon", "drain",
})
# ─────────────────────────────────────────────────────────────────────────────
# UUID v7（時間有序）
# ─────────────────────────────────────────────────────────────────────────────
def _uuid7() -> uuid.UUID:
"""
生成 UUID v7（時間有序，適合資料庫 PK）。
格式：48-bit Unix timestamp ms + version(7) + 74-bit random
"""
now_ms = int(datetime.now(timezone.utc).timestamp() * 1000)
rand_bits = int.from_bytes(uuid.uuid4().bytes[6:], "big") & 0x3FFFFFFFFFFFFFFF
val = (now_ms << 80) | (0x7 << 76) | (0x8 << 72) | rand_bits
return uuid.UUID(int=val)
# ─────────────────────────────────────────────────────────────────────────────
# W3C traceparent 生成
# ─────────────────────────────────────────────────────────────────────────────
def _new_trace_id() -> str:
"""生成 W3C traceparent-compatible trace_id"""
trace_id_hex = uuid.uuid4().hex + uuid.uuid4().hex[:16] # 128-bit
span_id_hex = uuid.uuid4().hex[:16] # 64-bit
return f"00-{trace_id_hex}-{span_id_hex}-01"
# ─────────────────────────────────────────────────────────────────────────────
# Idempotency
# ─────────────────────────────────────────────────────────────────────────────
_IDEMPOTENCY_REDIS_PREFIX = "awooop:run:idem:"
_IDEMPOTENCY_REDIS_TTL = 86400 # 24h
async def check_idempotency(
project_id: str,
channel_type: str,
provider_event_id: str,
) -> uuid.UUID | None:
"""
檢查 (project_id, channel_type, provider_event_id) 是否已有對應 run_id。
Layer 1：Redis NX（快速攔截，TTL 24h）
Layer 2：PostgreSQL awooop_run_idempotency（準確）
Returns: 既有 run_id，或 None（尚未處理）
"""
idem_key = f"{_IDEMPOTENCY_REDIS_PREFIX}{project_id}:{channel_type}:{provider_event_id}"
# Layer 1: Redis
try:
from src.core.redis_client import get_redis
redis = get_redis()
cached = await redis.get(idem_key)
if cached:
run_id_str = cached.decode() if isinstance(cached, bytes) else cached
logger.info(
"idempotency_hit_redis",
project_id=project_id,
provider_event_id=provider_event_id,
run_id=run_id_str,
)
return uuid.UUID(run_id_str)
except Exception as exc:
logger.warning("idempotency_redis_check_failed", error=str(exc))
# Layer 2: PostgreSQL
try:
async with get_db_context(project_id) as db:
result = await db.execute(
select(AwoooPRunIdempotency.run_id).where(
AwoooPRunIdempotency.project_id == project_id,
AwoooPRunIdempotency.channel_type == channel_type,
AwoooPRunIdempotency.provider_event_id == provider_event_id,
)
)
row = result.fetchone()
if row:
return uuid.UUID(str(row[0]))
except Exception as exc:
logger.warning("idempotency_pg_check_failed", error=str(exc))
return None
async def _register_idempotency(
project_id: str,
channel_type: str,
provider_event_id: str,
run_id: uuid.UUID,
) -> None:
"""寫入 idempotency 記錄（Redis + PostgreSQL）"""
idem_key = f"{_IDEMPOTENCY_REDIS_PREFIX}{project_id}:{channel_type}:{provider_event_id}"
run_id_str = str(run_id)
# Redis NX（若已有其他 worker 寫入，NX 失敗，無害）
try:
from src.core.redis_client import get_redis
redis = get_redis()
await redis.set(idem_key, run_id_str, ex=_IDEMPOTENCY_REDIS_TTL, nx=True)
except Exception as exc:
logger.warning("idempotency_redis_write_failed", error=str(exc))
# PostgreSQL（INSERT OR IGNORE）
try:
async with get_db_context(project_id) as db:
stmt = pg_insert(AwoooPRunIdempotency).values(
project_id=project_id,
channel_type=channel_type,
provider_event_id=provider_event_id,
run_id=run_id,
).on_conflict_do_nothing(constraint="uix_run_idempotency_key")
await db.execute(stmt)
except Exception as exc:
logger.warning("idempotency_pg_write_failed", error=str(exc))
# ─────────────────────────────────────────────────────────────────────────────
# Shadow destructive tool check
# ─────────────────────────────────────────────────────────────────────────────
def is_destructive_tool(tool_name: str, is_destructive_flag: bool = False) -> bool:
"""
判斷 tool call 是否為破壞性操作。
Shadow mode 下一律攔截。
判斷邏輯：
1. MCP Gateway contract 的 is_destructive=True flag
2. tool_name 包含破壞性關鍵字（fallback，無 contract 時使用）
"""
if is_destructive_flag:
return True
tool_lower = tool_name.lower()
return any(kw in tool_lower for kw in _DESTRUCTIVE_TOOL_KEYWORDS)
# ─────────────────────────────────────────────────────────────────────────────
# Run 建立
# ─────────────────────────────────────────────────────────────────────────────
async def create_run(
*,
project_id: str,
agent_id: str,
trigger_type: str,
trigger_ref: str | None = None,
input_payload: dict[str, Any] | None = None,
channel_type: str | None = None,
provider_event_id: str | None = None,
timeout_seconds: int = 600,
) -> tuple[uuid.UUID, bool]:
"""
建立新 run（或返回既有 run，若重複事件）。
Returns:
(run_id, is_duplicate) — is_duplicate=True 表示冪等命中
Shadow mode：is_shadow=True，不產生 user response。
"""
# Idempotency 檢查
if channel_type and provider_event_id:
existing_run_id = await check_idempotency(project_id, channel_type, provider_event_id)
if existing_run_id:
logger.info(
"run_creation_idempotent",
project_id=project_id,
channel_type=channel_type,
provider_event_id=provider_event_id,
existing_run_id=str(existing_run_id),
)
return existing_run_id, True
run_id = _uuid7()
trace_id = _new_trace_id()
timeout_at = datetime.now(timezone.utc) + timedelta(seconds=timeout_seconds)
# 計算 input_sha256
input_sha256 = None
if input_payload:
canonical = json.dumps(input_payload, sort_keys=True, separators=(",", ":"))
input_sha256 = hashlib.sha256(canonical.encode()).hexdigest()
async with get_db_context(project_id) as db:
run = AwoooPRunState(
run_id=run_id,
project_id=project_id,
agent_id=agent_id,
state="pending",
trace_id=trace_id,
trigger_type=trigger_type,
trigger_ref=trigger_ref,
is_shadow=_SHADOW_ENABLED,
input_sha256=input_sha256,
timeout_at=timeout_at,
)
db.add(run)
# 寫入 idempotency 記錄
if channel_type and provider_event_id:
await _register_idempotency(project_id, channel_type, provider_event_id, run_id)
logger.info(
"run_created",
run_id=str(run_id),
project_id=project_id,
agent_id=agent_id,
is_shadow=_SHADOW_ENABLED,
trace_id=trace_id,
trigger_type=trigger_type,
)
return run_id, False
# ─────────────────────────────────────────────────────────────────────────────
# Shadow Execution（Phase 4 主邏輯）
# ─────────────────────────────────────────────────────────────────────────────
async def shadow_execute(run: AwoooPRunState) -> None:
"""
Shadow mode 執行一個 run。
Phase 4 行為：
- 解析 agent contract（get_active()）
- 執行 tool calls（全部攔截，不實際執行）
- 記錄 step_journal
- 完成後 COMPLETED（無 user response）
Phase 6+ 才接真實 LLM + channel adapter。
"""
run_id = run.run_id
project_id = run.project_id
agent_id = run.agent_id
logger.info(
"shadow_execute_start",
run_id=str(run_id),
project_id=project_id,
agent_id=agent_id,
)
try:
# 解析 agent contract（取得工具清單）
from src.services.contract_service import get_active_body
agent_contract = await get_active_body(
project_id=project_id,
contract_family="agent",
contract_id=agent_id,
)
tools = agent_contract.get("tools", []) if agent_contract else []
# Shadow step journal：記錄每個工具會被攔截
step_seq = 0
async with get_db_context(project_id) as db:
for tool in tools:
tool_name = tool.get("tool_name", "unknown")
blocked = is_destructive_tool(tool_name)
step = AwoooPRunStepJournal(
run_id=run_id,
project_id=project_id,
step_seq=step_seq,
tool_name=tool_name,
mcp_gateway_id=tool.get("mcp_gateway_id"),
result_status="success" if not blocked else "pending",
was_blocked=blocked,
block_reason="shadow_mode_destructive_blocked" if blocked else None,
)
db.add(step)
step_seq += 1
# 完成 run（shadow mode：無 user response）
await transition(
run_id=run_id,
project_id=project_id,
to_state="completed",
step_count_delta=step_seq,
)
logger.info(
"shadow_execute_completed",
run_id=str(run_id),
steps=step_seq,
)
except Exception as exc:
logger.exception(
"shadow_execute_failed",
run_id=str(run_id),
error=str(exc),
)
await transition(
run_id=run_id,
project_id=project_id,
to_state="failed",
error_code="E-RUN-001",
error_detail=str(exc)[:500],
)
async def get_run_status(run_id: uuid.UUID, project_id: str) -> dict[str, Any] | None:
"""
查詢單一 run 的 FSM 狀態。回傳 None 表示不存在。
Router 層透過此 service 函數存取，不直接操作 DB。
"""
async with get_db_context(project_id) as db:
result = await db.execute(
select(AwoooPRunState).where(
AwoooPRunState.run_id == run_id,
AwoooPRunState.project_id == project_id,
)
)
run = result.scalar_one_or_none()
if run is None:
return None
return {
"run_id": str(run.run_id),
"project_id": run.project_id,
"agent_id": run.agent_id,
"state": run.state,
"is_shadow": run.is_shadow,
"trace_id": run.trace_id,
"attempt_count": run.attempt_count,
"cost_usd": float(run.cost_usd),
"step_count": run.step_count,
"error_code": run.error_code,
"created_at": run.created_at.isoformat() if run.created_at else None,
"started_at": run.started_at.isoformat() if run.started_at else None,
"completed_at": run.completed_at.isoformat() if run.completed_at else None,
}

240
apps/api/src/services/provider_proxy.py Normal file
View File

@@ -0,0 +1,240 @@
"""
Provider Proxy Adapter — EwoooC AwoooP Envelope 注入
=====================================================
AwoooP Phase 6: ADR-115 D3
2026-05-04 ogt + Claude Sonnet 4.6
功能：
EwoooC（或任何外部 tenant）的請求在進入 AwoooP 前，
必須注入完整的 platform envelope，確保：
- project_id 正確（budget/audit/RLS 有效）
- agent_id 存在（Gate 2 通過）
- trace_id / run_id 有 W3C traceparent format
- platform_subject_id 已建立（channel user 身份映射）
使用方式：
from src.services.provider_proxy import ProviderProxy
proxy = ProviderProxy(project_id="ewoooc", db=db)
envelope = await proxy.build_envelope(
agent_id="openclaw-biz",
channel_type="telegram",
channel_user_id="123456789",
channel_chat_id="123456789",
)
# envelope 可直接作為 GatewayContext 的初始化參數
設計原則（ADR-115 D3）：
- Proxy 只做 envelope 注入（<1ms），不做額外複雜 IO
- platform_subject upsert 是唯一 DB write（auto-provisioning）
- run_id 由 platform_runtime.create_run() 分配，Proxy 不自行生成
- 每個 tenant 有獨立的 budget partition 和 RLS 隔離
"""
from __future__ import annotations
import hashlib
import os
import re
import struct
import time
import uuid
from dataclasses import dataclass, field
from datetime import datetime, timezone
from typing import Any
from uuid import UUID
import structlog
from sqlalchemy import select, text
from sqlalchemy.ext.asyncio import AsyncSession
from src.db.awooop_models import AwoooPPlatformSubject, AwoooPProject
logger = structlog.get_logger(__name__)
# ─────────────────────────────────────────────────────────────────────────────
# Platform Envelope
# ─────────────────────────────────────────────────────────────────────────────
@dataclass
class PlatformEnvelope:
"""
AwoooP Platform Envelope — 每個 EwoooC 請求注入的 metadata。
下游（Gateway / Budget / Audit）都依賴這個 envelope。
"""
project_id: str
agent_id: str
trace_id: str # W3C traceparent
platform_subject_id: str # "{project_id}:{channel_type}:{channel_user_id}"
channel_type: str
channel_user_id: str
channel_chat_id: str | None = None
run_id: UUID | None = None # 由 create_run() 填入
policy_revision_id: str | None = None # active policy contract revision
tags: dict[str, Any] = field(default_factory=dict)
def as_dict(self) -> dict[str, Any]:
return {
"project_id": self.project_id,
"agent_id": self.agent_id,
"trace_id": self.trace_id,
"platform_subject_id": self.platform_subject_id,
"channel_type": self.channel_type,
"channel_user_id": self.channel_user_id,
"channel_chat_id": self.channel_chat_id,
"run_id": str(self.run_id) if self.run_id else None,
"policy_revision_id": self.policy_revision_id,
}
# ─────────────────────────────────────────────────────────────────────────────
# W3C traceparent 生成
# ─────────────────────────────────────────────────────────────────────────────
def _new_trace_id() -> str:
"""生成 W3C traceparent 格式 trace_id。格式：00-{32hex}-{16hex}-01"""
trace_id = uuid.uuid4().hex # 32 hex chars = 128 bits
span_id = uuid.uuid4().hex[:16] # 16 hex chars = 64 bits
return f"00-{trace_id}-{span_id}-01"
# ─────────────────────────────────────────────────────────────────────────────
# platform_subject_id 格式
# ─────────────────────────────────────────────────────────────────────────────
def build_platform_subject_id(project_id: str, channel_type: str, channel_user_id: str) -> str:
"""
格式：{project_id}:{channel_type}:{channel_user_id}
例：ewoooc:telegram:123456789
"""
return f"{project_id}:{channel_type}:{channel_user_id}"
# ─────────────────────────────────────────────────────────────────────────────
# ProviderProxy
# ─────────────────────────────────────────────────────────────────────────────
class ProviderProxy:
"""
AwoooP Provider Proxy Adapter（ADR-115 D3）。
職責：
1. 驗證 project 存在且不是 legacy mode
2. upsert platform_subject（auto-provisioning）
3. 生成 trace_id（W3C traceparent）
4. 返回 PlatformEnvelope 供下游使用
"""
def __init__(self, project_id: str, db: AsyncSession) -> None:
self.project_id = project_id
self._db = db
async def build_envelope(
self,
*,
agent_id: str,
channel_type: str,
channel_user_id: str,
channel_chat_id: str | None = None,
display_name: str | None = None,
extra_tags: dict[str, Any] | None = None,
) -> PlatformEnvelope:
"""
建立 PlatformEnvelope：
1. 驗證 project_id（不是 legacy mode）
2. upsert platform_subject（auto-provisioning）
3. 生成 trace_id
4. 返回 envelope
"""
await self._validate_project()
await self._upsert_platform_subject(
channel_type=channel_type,
channel_user_id=channel_user_id,
channel_chat_id=channel_chat_id,
display_name=display_name,
)
platform_subject_id = build_platform_subject_id(
self.project_id, channel_type, channel_user_id
)
trace_id = _new_trace_id()
logger.info(
"provider_proxy_envelope_built",
project_id=self.project_id,
agent_id=agent_id,
channel_type=channel_type,
platform_subject_id=platform_subject_id,
trace_id=trace_id[:32] + "...", # 只 log 前 32 字元
)
return PlatformEnvelope(
project_id=self.project_id,
agent_id=agent_id,
trace_id=trace_id,
platform_subject_id=platform_subject_id,
channel_type=channel_type,
channel_user_id=channel_user_id,
channel_chat_id=channel_chat_id,
tags=extra_tags or {},
)
async def _validate_project(self) -> None:
"""project 必須存在且不是 legacy_awoooi_default mode"""
result = await self._db.execute(
select(AwoooPProject).where(
AwoooPProject.project_id == self.project_id,
AwoooPProject.migration_mode != "legacy_awoooi_default",
)
)
project = result.scalar_one_or_none()
if project is None:
raise ValueError(
f"project '{self.project_id}' 不存在或 migration_mode=legacy_awoooi_default"
"（EwoooC 接入需要至少 migration_mode='shadow'）"
)
async def _upsert_platform_subject(
self,
*,
channel_type: str,
channel_user_id: str,
channel_chat_id: str | None,
display_name: str | None,
) -> None:
"""
Auto-provisioning：第一次看到這個 channel user 就建立 platform_subject。
後續請求更新 last_seen_at。
"""
platform_subject_id = build_platform_subject_id(
self.project_id, channel_type, channel_user_id
)
now = datetime.now(timezone.utc)
await self._db.execute(
text("""
INSERT INTO awooop_platform_subjects (
project_id, channel_type, channel_user_id, channel_chat_id,
platform_subject_id, display_name, roles,
first_seen_at, last_seen_at
) VALUES (
:project_id, :channel_type, :channel_user_id, :channel_chat_id,
:platform_subject_id, :display_name, '["viewer"]'::jsonb,
:now, :now
)
ON CONFLICT (project_id, channel_type, channel_user_id) DO UPDATE SET
last_seen_at = :now,
channel_chat_id = COALESCE(EXCLUDED.channel_chat_id, awooop_platform_subjects.channel_chat_id),
display_name = COALESCE(EXCLUDED.display_name, awooop_platform_subjects.display_name)
"""),
{
"project_id": self.project_id,
"channel_type": channel_type,
"channel_user_id": channel_user_id,
"channel_chat_id": channel_chat_id,
"platform_subject_id": platform_subject_id,
"display_name": display_name,
"now": now,
},
)

304
apps/api/src/services/run_state_machine.py Normal file
View File

@@ -0,0 +1,304 @@
"""
Run State Machine
==================
AwoooP Phase 4: Run FSM 轉換規則 + Worker Lease（ADR-114/ADR-119）
2026-05-04 ogt + Claude Sonnet 4.6
狀態機：
PENDING → RUNNING（worker 取得 lease）
RUNNING → WAITING_TOOL（等待 tool call 完成）
RUNNING → WAITING_APPROVAL（等待人工審核）
RUNNING → COMPLETED / FAILED / CANCELLED
WAITING_TOOL → RUNNING（tool call 完成）
WAITING_TOOL → FAILED（tool call 失敗 + 超過 max_attempts）
WAITING_APPROVAL → RUNNING（核准）
WAITING_APPROVAL → CANCELLED（拒絕/超時）
* → TIMEOUT（lease_until 過期且超過 max_attempts）
SKIP LOCKED：
Worker 以 SELECT ... FOR UPDATE SKIP LOCKED 取單，防 double-pickup。
Lease TTL = 60 秒；Heartbeat 每 15 秒更新。
Stale run reaper：
每分鐘掃描 lease_until < NOW() 的 running run：
attempt_count < max_attempts → 重設 PENDING
attempt_count >= max_attempts → 標記 FAILED(E-RUN-002)
"""
from __future__ import annotations
import socket
import uuid
from datetime import datetime, timedelta, timezone
from typing import TYPE_CHECKING
import structlog
from sqlalchemy import select, text, update
from src.db.awooop_models import AwoooPRunState
from src.db.base import get_db_context
if TYPE_CHECKING:
from uuid import UUID
logger = structlog.get_logger(__name__)
# Worker lease TTL（秒）
LEASE_TTL_SECONDS = 60
HEARTBEAT_INTERVAL_SECONDS = 15
STALE_REAPER_INTERVAL_SECONDS = 60
# 有效的 FSM 轉換表
# key: from_state, value: set of valid to_states
_VALID_TRANSITIONS: dict[str, frozenset[str]] = {
"pending": frozenset({"running", "cancelled"}),
"running": frozenset({"waiting_tool", "waiting_approval", "completed", "failed", "cancelled", "timeout"}),
"waiting_tool": frozenset({"running", "failed", "cancelled"}),
"waiting_approval": frozenset({"running", "cancelled", "timeout"}),
"completed": frozenset(), # terminal
"failed": frozenset(), # terminal
"cancelled": frozenset(), # terminal
"timeout": frozenset(), # terminal
}
TERMINAL_STATES = frozenset({"completed", "failed", "cancelled", "timeout"})
_WORKER_ID = f"{socket.gethostname()}:{uuid.uuid4().hex[:8]}"
# ─────────────────────────────────────────────────────────────────────────────
# FSM 驗證
# ─────────────────────────────────────────────────────────────────────────────
class InvalidStateTransitionError(Exception):
def __init__(self, from_state: str, to_state: str) -> None:
self.from_state = from_state
self.to_state = to_state
super().__init__(f"非法 FSM 轉換: {from_state!r} → {to_state!r}")
def validate_transition(from_state: str, to_state: str) -> None:
"""驗證 FSM 轉換是否合法，非法則拋出 InvalidStateTransitionError"""
valid_targets = _VALID_TRANSITIONS.get(from_state, frozenset())
if to_state not in valid_targets:
raise InvalidStateTransitionError(from_state, to_state)
# ─────────────────────────────────────────────────────────────────────────────
# Worker Lease（SKIP LOCKED）
# ─────────────────────────────────────────────────────────────────────────────
async def acquire_pending_run(
project_id: str,
worker_id: str = _WORKER_ID,
) -> AwoooPRunState | None:
"""
以 SKIP LOCKED 取得一筆 PENDING run，並設定 lease。
同時只有一個 worker 可取得同一筆 run（PostgreSQL SKIP LOCKED 保證）。
Returns None 表示目前沒有待處理的 run。
"""
lease_until = datetime.now(timezone.utc) + timedelta(seconds=LEASE_TTL_SECONDS)
now = datetime.now(timezone.utc)
async with get_db_context(project_id) as db:
# SKIP LOCKED：其他 worker 已鎖定的 row 直接跳過
result = await db.execute(
text("""
SELECT run_id FROM awooop_run_state
WHERE project_id = :project_id
AND state = 'pending'
AND (lease_until IS NULL OR lease_until < NOW())
ORDER BY created_at ASC
LIMIT 1
FOR UPDATE SKIP LOCKED
"""),
{"project_id": project_id},
)
row = result.fetchone()
if row is None:
return None
run_id = row[0]
# 更新 lease + 轉為 RUNNING
await db.execute(
update(AwoooPRunState)
.where(
AwoooPRunState.run_id == run_id,
AwoooPRunState.project_id == project_id,
)
.values(
state="running",
lease_until=lease_until,
heartbeat_at=now,
worker_id=worker_id,
started_at=now,
attempt_count=AwoooPRunState.attempt_count + 1,
)
)
# 重新讀取完整 record
result2 = await db.execute(
select(AwoooPRunState).where(AwoooPRunState.run_id == run_id)
)
run = result2.scalar_one()
logger.info(
"run_lease_acquired",
run_id=str(run_id),
project_id=project_id,
worker_id=worker_id,
attempt_count=run.attempt_count,
)
return run
async def heartbeat(run_id: "UUID", project_id: str) -> None:
"""更新 run 的 heartbeat + 延長 lease TTL"""
new_lease = datetime.now(timezone.utc) + timedelta(seconds=LEASE_TTL_SECONDS)
async with get_db_context(project_id) as db:
await db.execute(
update(AwoooPRunState)
.where(
AwoooPRunState.run_id == run_id,
AwoooPRunState.state == "running",
)
.values(
heartbeat_at=datetime.now(timezone.utc),
lease_until=new_lease,
)
)
async def transition(
run_id: "UUID",
project_id: str,
to_state: str,
*,
error_code: str | None = None,
error_detail: str | None = None,
output_sha256: str | None = None,
cost_usd_delta: float = 0.0,
step_count_delta: int = 0,
) -> None:
"""
執行 FSM 狀態轉換（含驗證）。
先從 DB 讀取 current state，驗證轉換合法性，再 UPDATE。
terminal state 同時寫入 completed_at。
"""
async with get_db_context(project_id) as db:
result = await db.execute(
select(AwoooPRunState.state).where(
AwoooPRunState.run_id == run_id,
AwoooPRunState.project_id == project_id,
)
)
row = result.fetchone()
if row is None:
raise ValueError(f"run {run_id} 不存在或無 RLS 權限")
from_state = row[0]
validate_transition(from_state, to_state)
values: dict = {"state": to_state}
if error_code:
values["error_code"] = error_code
if error_detail:
values["error_detail"] = error_detail
if output_sha256:
values["output_sha256"] = output_sha256
if cost_usd_delta:
values["cost_usd"] = AwoooPRunState.cost_usd + cost_usd_delta
if step_count_delta:
values["step_count"] = AwoooPRunState.step_count + step_count_delta
if to_state in TERMINAL_STATES:
values["completed_at"] = datetime.now(timezone.utc)
values["lease_until"] = None
values["worker_id"] = None
await db.execute(
update(AwoooPRunState)
.where(AwoooPRunState.run_id == run_id)
.values(**values)
)
logger.info(
"run_state_transition",
run_id=str(run_id),
from_state=from_state,
to_state=to_state,
error_code=error_code,
)
# ─────────────────────────────────────────────────────────────────────────────
# Stale Run Reaper
# ─────────────────────────────────────────────────────────────────────────────
async def reap_stale_runs(project_id: str) -> int:
"""
掃描 lease_until < NOW() 的 RUNNING run。
- attempt_count < max_attempts → 重設 PENDING（retry）
- attempt_count >= max_attempts → FAILED(E-RUN-002)
Returns: 處理的 stale run 數
"""
now = datetime.now(timezone.utc)
reaped = 0
async with get_db_context(project_id) as db:
# 找所有 stale RUNNING runs
result = await db.execute(
select(AwoooPRunState).where(
AwoooPRunState.project_id == project_id,
AwoooPRunState.state == "running",
AwoooPRunState.lease_until < now,
)
)
stale_runs = list(result.scalars().all())
for run in stale_runs:
if run.attempt_count < run.max_attempts:
# Retry：重設為 PENDING
await db.execute(
update(AwoooPRunState)
.where(AwoooPRunState.run_id == run.run_id)
.values(
state="pending",
lease_until=None,
worker_id=None,
heartbeat_at=None,
)
)
logger.warning(
"stale_run_requeued",
run_id=str(run.run_id),
attempt_count=run.attempt_count,
max_attempts=run.max_attempts,
)
else:
# 超過最大重試次數 → FAILED
await db.execute(
update(AwoooPRunState)
.where(AwoooPRunState.run_id == run.run_id)
.values(
state="failed",
error_code="E-RUN-002",
error_detail=f"max_attempts={run.max_attempts} 超過，stale run 已廢棄",
completed_at=now,
lease_until=None,
worker_id=None,
)
)
logger.error(
"stale_run_failed",
run_id=str(run.run_id),
attempt_count=run.attempt_count,
)
reaped += 1
if reaped:
logger.info("stale_run_reaper_done", project_id=project_id, reaped=reaped)
return reaped

262
apps/api/src/services/schema_validator.py Normal file
View File

@@ -0,0 +1,262 @@
"""
LLM Output Schema Validator
=============================
AwoooP Phase 3.3: LLM 輸出 → schema 驗證 → retry 機制（ADR-112）
2026-05-04 ogt + Claude Sonnet 4.6
設計原則：
- LLM 輸出必須通過 Pydantic schema 驗證才能到達 channel adapter
- 驗證失敗 → 自動 retry（最多 3 次，含 retry prompt）
- 3 次全部失敗 → 拋出 SchemaValidationError（E-SCHEMA-001）
- 支援六合約家族 + 自訂 Pydantic model
位置：介於 LLM response 和 channel adapter 之間
呼叫方：任何需要結構化 LLM 輸出的 service（playbook_generator, decision_manager 等）
"""
from __future__ import annotations
import json
import re
from typing import Any, TypeVar
import structlog
from pydantic import BaseModel, ValidationError
logger = structlog.get_logger(__name__)
T = TypeVar("T", bound=BaseModel)
_MAX_RETRIES = 3
_JSON_EXTRACT_RE = re.compile(r"```(?:json)?\s*(\{[\s\S]*?\})\s*```|(\{[\s\S]*\})", re.DOTALL)
# ─────────────────────────────────────────────────────────────────────────────
# 錯誤定義
# ─────────────────────────────────────────────────────────────────────────────
class SchemaValidationError(Exception):
"""LLM 輸出連續 3 次 schema 驗證失敗"""
error_code: str = "E-SCHEMA-001"
def __init__(self, model_name: str, attempts: int, last_error: str) -> None:
self.model_name = model_name
self.attempts = attempts
self.last_error = last_error
super().__init__(
f"[E-SCHEMA-001] LLM 輸出 {attempts} 次驗證失敗 "
f"(model={model_name}): {last_error}"
)
# ─────────────────────────────────────────────────────────────────────────────
# JSON 萃取（容錯解析）
# ─────────────────────────────────────────────────────────────────────────────
def extract_json_from_llm_output(raw: str) -> dict[str, Any] | None:
"""
從 LLM 原始輸出中萃取 JSON。
策略：
1. 直接 json.loads（最常見：LLM 直接回傳 JSON）
2. 從 ```json ... ``` 程式碼區塊萃取
3. 找第一個 { ... } 區塊嘗試解析
"""
raw = raw.strip()
# 策略 1：直接解析
try:
obj = json.loads(raw)
if isinstance(obj, dict):
return obj
except json.JSONDecodeError:
pass
# 策略 2 + 3：正則萃取
for match in _JSON_EXTRACT_RE.finditer(raw):
candidate = match.group(1) or match.group(2)
if candidate:
try:
obj = json.loads(candidate)
if isinstance(obj, dict):
return obj
except json.JSONDecodeError:
continue
return None
# ─────────────────────────────────────────────────────────────────────────────
# Retry prompt builder
# ─────────────────────────────────────────────────────────────────────────────
def build_retry_prompt(
original_prompt: str,
failed_output: str,
validation_error: str,
model_name: str,
attempt: int,
) -> str:
"""
建立包含錯誤回饋的 retry prompt。
讓 LLM 知道上次輸出哪裡出錯，引導修正。
"""
return (
f"{original_prompt}\n\n"
f"---\n"
f"[SCHEMA VALIDATION RETRY {attempt}/{_MAX_RETRIES}]\n"
f"上次回應未通過結構驗證（{model_name}），請修正以下問題後重新回應：\n\n"
f"驗證錯誤：\n{validation_error}\n\n"
f"上次回應（供參考）：\n{failed_output[:500]}...\n"
f"---\n\n"
f"請只回傳符合格式的 JSON 物件，不要包含任何額外說明。"
)
# ─────────────────────────────────────────────────────────────────────────────
# Core validator
# ─────────────────────────────────────────────────────────────────────────────
async def validate_llm_output(
*,
raw_output: str,
model_cls: type[T],
llm_caller: Any, # Callable[[str], Awaitable[str]] — 供 retry 使用
original_prompt: str,
context: dict[str, Any] | None = None,
) -> T:
"""
驗證 LLM 輸出是否符合 Pydantic model。
Args:
raw_output: LLM 第一次回傳的原始字串
model_cls: 目標 Pydantic model class
llm_caller: async callable(prompt: str) -> str，用於 retry
original_prompt: 原始 prompt（retry 時附加錯誤回饋）
context: 額外 logging context
Returns:
驗證成功的 model instance
Raises:
SchemaValidationError: 連續 3 次失敗後拋出
"""
model_name = model_cls.__name__
ctx = context or {}
current_output = raw_output
last_error = ""
for attempt in range(1, _MAX_RETRIES + 1):
# 1. 萃取 JSON
parsed = extract_json_from_llm_output(current_output)
if parsed is None:
last_error = "無法從 LLM 輸出中萃取 JSON 物件"
logger.warning(
"schema_validator_no_json",
model_name=model_name,
attempt=attempt,
output_preview=current_output[:200],
**ctx,
)
else:
# 2. Pydantic 驗證
try:
instance = model_cls.model_validate(parsed)
logger.info(
"schema_validator_passed",
model_name=model_name,
attempt=attempt,
**ctx,
)
return instance
except ValidationError as exc:
last_error = exc.json(indent=None)
logger.warning(
"schema_validator_failed",
model_name=model_name,
attempt=attempt,
error=last_error[:500],
**ctx,
)
# 3. Retry（如果不是最後一次）
if attempt < _MAX_RETRIES:
retry_prompt = build_retry_prompt(
original_prompt=original_prompt,
failed_output=current_output,
validation_error=last_error,
model_name=model_name,
attempt=attempt,
)
try:
current_output = await llm_caller(retry_prompt)
except Exception as exc:
logger.warning(
"schema_validator_llm_retry_failed",
model_name=model_name,
attempt=attempt,
error=str(exc),
**ctx,
)
# LLM 呼叫本身失敗，保留上次 output，繼續嘗試（或直接結束）
break
# 3 次全失敗
logger.error(
"schema_validator_exhausted",
model_name=model_name,
total_attempts=_MAX_RETRIES,
last_error=last_error[:500],
**ctx,
)
raise SchemaValidationError(model_name, _MAX_RETRIES, last_error)
# ─────────────────────────────────────────────────────────────────────────────
# 便利方法：從 contract family 名稱驗證（不需知道具體 model class）
# ─────────────────────────────────────────────────────────────────────────────
async def validate_llm_output_by_family(
*,
raw_output: str,
contract_family: str,
llm_caller: Any,
original_prompt: str,
context: dict[str, Any] | None = None,
) -> BaseModel:
"""
依 contract_family 自動選擇 model class 並驗證。
適合 generic pipeline 呼叫（不知道具體 model）。
"""
from src.models.awooop_contracts import CONTRACT_FAMILY_MODELS, VALID_CONTRACT_FAMILIES
model_cls = CONTRACT_FAMILY_MODELS.get(contract_family)
if model_cls is None:
raise ValueError(
f"未知 contract_family: {contract_family!r}，"
f"合法值：{sorted(VALID_CONTRACT_FAMILIES)}"
)
return await validate_llm_output(
raw_output=raw_output,
model_cls=model_cls,
llm_caller=llm_caller,
original_prompt=original_prompt,
context=context,
)
# ─────────────────────────────────────────────────────────────────────────────
# 同步版本（非 LLM retry，只做一次驗證）— 供測試和非 LLM 路徑使用
# ─────────────────────────────────────────────────────────────────────────────
def validate_once(raw: str | dict[str, Any], model_cls: type[T]) -> T:
"""
單次驗證，不做 retry。
適合：已知格式正確的內部資料、測試 fixture 驗證。
"""
if isinstance(raw, str):
parsed = extract_json_from_llm_output(raw)
if parsed is None:
raise SchemaValidationError(model_cls.__name__, 1, "無法萃取 JSON")
return model_cls.model_validate(parsed)
return model_cls.model_validate(raw)

196
apps/api/src/workers/platform_worker.py Normal file
View File

@@ -0,0 +1,196 @@
"""
Platform Worker
================
AwoooP Phase 4: SKIP LOCKED worker + stale run reaper（ADR-114）
2026-05-04 ogt + Claude Sonnet 4.6
功能：
1. Worker Loop：以 SKIP LOCKED 從 awooop_run_state 取 PENDING run 並執行
2. Stale Run Reaper：每 60 秒掃描 lease 過期的 RUNNING run
3. Shadow Mode Enforcer：所有 Phase 4 run 強制 is_shadow=True
Worker 設計：
- 啟動時以 asyncio.create_task 掛入 main.py lifespan
- 多個 worker 安全並行（SKIP LOCKED 保證每 run 只被一個 worker 取得）
- Heartbeat 每 15 秒更新 lease（防 stale reaper 誤殺）
- 優雅停機：收到 stop signal 後完成當前 run 再退出
與 legacy 的關係：
- 完全獨立，不碰任何既有 signal_worker.py / aider_event_processor.py
- 只處理 awooop_run_state 表（legacy signal 不寫入此表）
"""
from __future__ import annotations
import asyncio
from datetime import datetime, timezone
import structlog
from src.services.platform_runtime import shadow_execute
from src.services.run_state_machine import (
HEARTBEAT_INTERVAL_SECONDS,
STALE_REAPER_INTERVAL_SECONDS,
acquire_pending_run,
heartbeat,
reap_stale_runs,
)
logger = structlog.get_logger(__name__)
# Phase 4 固定處理 awoooi 租戶（Phase 6+ 改為多租戶掃描）
_DEFAULT_PROJECT_ID = "awoooi"
_WORKER_POLL_INTERVAL_SECONDS = 5 # 無任務時的等待間隔
_WORKER_CONCURRENCY = 2 # 同時最多幾個 run 並行
class PlatformWorker:
"""
Platform Worker：SKIP LOCKED + shadow execution + stale reaper。
Usage（在 main.py lifespan 中）：
worker = PlatformWorker()
asyncio.create_task(worker.run_loop())
asyncio.create_task(worker.reaper_loop())
"""
def __init__(self, project_id: str = _DEFAULT_PROJECT_ID) -> None:
self.project_id = project_id
self._stop_event = asyncio.Event()
self._active_runs: set[str] = set()
def stop(self) -> None:
"""優雅停機信號"""
self._stop_event.set()
async def run_loop(self) -> None:
"""
主 worker loop：
1. 取一筆 PENDING run（SKIP LOCKED）
2. 執行 shadow_execute（不產生 user response）
3. Heartbeat（每 15 秒）
4. 等待 5 秒後重新掃描
"""
logger.info("platform_worker_started", project_id=self.project_id)
while not self._stop_event.is_set():
try:
# 控制並行度
if len(self._active_runs) >= _WORKER_CONCURRENCY:
await asyncio.sleep(1)
continue
run = await acquire_pending_run(self.project_id)
if run is None:
await asyncio.sleep(_WORKER_POLL_INTERVAL_SECONDS)
continue
run_id_str = str(run.run_id)
self._active_runs.add(run_id_str)
# 每個 run 獨立 task，不阻塞 loop
asyncio.create_task(
self._execute_with_heartbeat(run),
name=f"platform_run_{run_id_str[:8]}",
)
except asyncio.CancelledError:
break
except Exception as exc:
logger.exception("platform_worker_loop_error", error=str(exc))
await asyncio.sleep(_WORKER_POLL_INTERVAL_SECONDS)
logger.info("platform_worker_stopped", project_id=self.project_id)
async def _execute_with_heartbeat(self, run: object) -> None:
"""
在 shadow_execute 執行期間，同步 heartbeat 防 stale reaper 誤殺。
"""
from src.db.awooop_models import AwoooPRunState
assert isinstance(run, AwoooPRunState)
run_id_str = str(run.run_id)
# Heartbeat task（每 15 秒更新 lease）
heartbeat_task = asyncio.create_task(
self._heartbeat_loop(run.run_id, self.project_id),
name=f"heartbeat_{run_id_str[:8]}",
)
try:
await shadow_execute(run)
except Exception as exc:
logger.exception(
"platform_run_execution_error",
run_id=run_id_str,
error=str(exc),
)
finally:
heartbeat_task.cancel()
self._active_runs.discard(run_id_str)
async def _heartbeat_loop(self, run_id: object, project_id: str) -> None:
"""每 HEARTBEAT_INTERVAL_SECONDS 秒更新 lease，直到被 cancel"""
import uuid as _uuid
while True:
await asyncio.sleep(HEARTBEAT_INTERVAL_SECONDS)
try:
await heartbeat(run_id, project_id) # type: ignore[arg-type]
except Exception as exc:
logger.warning(
"platform_heartbeat_failed",
run_id=str(run_id),
error=str(exc),
)
async def reaper_loop(self) -> None:
"""
Stale run reaper：每 60 秒掃描 lease 過期的 RUNNING run。
lease < NOW() + attempt < max → PENDING（retry）
lease < NOW() + attempt >= max → FAILED(E-RUN-002)
"""
logger.info("stale_run_reaper_started", project_id=self.project_id)
while not self._stop_event.is_set():
try:
await asyncio.sleep(STALE_REAPER_INTERVAL_SECONDS)
reaped = await reap_stale_runs(self.project_id)
if reaped:
logger.info(
"stale_run_reaper_cycle",
project_id=self.project_id,
reaped=reaped,
ts=datetime.now(timezone.utc).isoformat(),
)
except asyncio.CancelledError:
break
except Exception as exc:
logger.exception("stale_run_reaper_error", error=str(exc))
logger.info("stale_run_reaper_stopped", project_id=self.project_id)
# ─────────────────────────────────────────────────────────────────────────────
# Singleton（掛入 lifespan 用）
# ─────────────────────────────────────────────────────────────────────────────
_platform_worker: PlatformWorker | None = None
def get_platform_worker() -> PlatformWorker:
global _platform_worker
if _platform_worker is None:
_platform_worker = PlatformWorker()
return _platform_worker
async def start_platform_worker() -> None:
"""在 main.py lifespan 中呼叫此函數啟動 worker"""
worker = get_platform_worker()
asyncio.create_task(worker.run_loop(), name="platform_worker_run_loop")
asyncio.create_task(worker.reaper_loop(), name="platform_worker_reaper_loop")
logger.info("platform_worker_tasks_started")
async def stop_platform_worker() -> None:
"""在 main.py lifespan 關閉時呼叫"""
worker = get_platform_worker()
worker.stop()
logger.info("platform_worker_stop_requested")

191
apps/api/tests/test_mcp_credential_isolation.py Normal file
View File

@@ -0,0 +1,191 @@
"""
MCP Credential Isolation 迴歸測試
==================================
AwoooP Phase 5.5：防止 2026-04-18 Secret Leak 事故再現
2026-05-04 ogt + Claude Sonnet 4.6
覆蓋：
1. credential_resolver 格式驗證（bad ref 拒絕）
2. dev fallback 正確返回 (value, masked, sha256)
3. secret value 不洩漏到 redaction_middleware output
4. _mcp_audit key 在 redact_mcp_input 中被移除（不送 provider）
5. AuditedMCPToolProvider.__provider 不可從外部直接存取（name mangling）
"""
from __future__ import annotations
import hashlib
import json
import os
import pytest
class TestCredentialResolverFormat:
"""credential_resolver 格式驗證"""
def test_bad_ref_raises(self):
"""格式錯誤的 ref 應拋出 CredentialResolutionError"""
import asyncio
import sys
sys.path.insert(0, ".")
from src.plugins.mcp.credential_resolver import (
CredentialResolutionError,
resolve_k8s_secret,
)
bad_refs = [
"no-slash",
"namespace/secret", # 缺 #key
"NAMESPACE/secret#key", # namespace 大寫不符格式
"ns/secret#", # key 為空
"",
]
for ref in bad_refs:
with pytest.raises(CredentialResolutionError):
asyncio.run(resolve_k8s_secret(ref))
def test_dev_fallback_resolves(self, monkeypatch):
"""AWOOOP_DEV_SECRETS_JSON 設定後應正確解析"""
import asyncio
import sys
sys.path.insert(0, ".")
dev_secrets = {"awoooi/telegram-bot#TELEGRAM_BOT_TOKEN": "test-token-value-1234"}
monkeypatch.setenv("AWOOOP_DEV_SECRETS_JSON", json.dumps(dev_secrets))
from src.plugins.mcp.credential_resolver import resolve_k8s_secret
value, masked, sha256 = asyncio.run(
resolve_k8s_secret("awoooi/telegram-bot#TELEGRAM_BOT_TOKEN")
)
assert value == "test-token-value-1234"
assert "test" in masked # 前 4 字元
assert "***" in masked
assert sha256 == hashlib.sha256("test-token-value-1234".encode()).hexdigest()
def test_dev_fallback_missing_key_raises(self, monkeypatch):
"""dev secrets 中找不到 key 應拋出錯誤"""
import asyncio
import sys
sys.path.insert(0, ".")
monkeypatch.setenv("AWOOOP_DEV_SECRETS_JSON", json.dumps({}))
from src.plugins.mcp.credential_resolver import (
CredentialResolutionError,
resolve_k8s_secret,
)
with pytest.raises(CredentialResolutionError):
asyncio.run(resolve_k8s_secret("awoooi/some-secret#key"))
class TestRedactionMiddlewareSecretLeak:
"""2026-04-18 Secret Leak 迴歸：secret value 不得進入 output"""
def test_pg_dsn_redacted_from_output(self):
"""PG DSN 在 output redaction 後不可見"""
import sys
sys.path.insert(0, ".")
from src.plugins.mcp.redaction_middleware import redact_mcp_output
output = {
"connection": "postgresql+asyncpg://admin:supersecret@10.0.1.5/prod",
"status": "connected",
}
cleaned = redact_mcp_output(output)
assert "supersecret" not in json.dumps(cleaned), "PG DSN 密碼不得出現在 output"
assert "[REDACTED:PG_DSN]" in cleaned["connection"]
def test_telegram_token_redacted_from_output(self):
"""Telegram token 在 output redaction 後不可見"""
import sys
sys.path.insert(0, ".")
from src.plugins.mcp.redaction_middleware import redact_mcp_output
output = "Bot token: 1234567890:ABCDEFGHIJKLMNOPabcdefghijklmno12345678"
cleaned = redact_mcp_output(output)
assert "ABCDEFGHIJKLMNO" not in cleaned, "Telegram token 不得出現在 output"
assert "[REDACTED:TELEGRAM_TOKEN]" in cleaned
def test_internal_ip_redacted(self):
"""GCP 內網 IP 在 output redaction 後不可見"""
import sys
sys.path.insert(0, ".")
from src.plugins.mcp.redaction_middleware import redact_mcp_output
output = {"host": "10.0.1.5", "port": 5432}
cleaned = redact_mcp_output(output)
assert "10.0.1.5" not in json.dumps(cleaned), "內網 IP 不得出現在 output"
assert "[REDACTED:INTERNAL_IP]" in cleaned["host"]
def test_mcp_audit_key_removed_from_input(self):
"""_mcp_audit key 在 redact_mcp_input 後應被移除"""
import sys
sys.path.insert(0, ".")
from src.plugins.mcp.redaction_middleware import redact_mcp_input
params = {
"_mcp_audit": {"session_id": "abc123", "run_id": "xyz"},
"namespace": "default",
}
cleaned = redact_mcp_input(params)
assert "_mcp_audit" not in cleaned, "_mcp_audit 應在送 provider 前移除"
assert cleaned["namespace"] == "default"
def test_k8s_value_credential_isolation(self):
"""k8s_value 欄位應被 credential isolation 攔截"""
import sys
sys.path.insert(0, ".")
from src.plugins.mcp.redaction_middleware import redact_mcp_input
params = {
"k8s_value": "actual-secret-credential",
"tool": "some-tool",
}
cleaned = redact_mcp_input(params)
assert "actual-secret-credential" not in json.dumps(cleaned)
assert cleaned["k8s_value"] == "[REDACTED:CREDENTIAL_ISOLATION]"
class TestNameManglingEncapsulation:
"""AuditedMCPToolProvider.__provider name mangling 封裝驗證"""
def test_single_underscore_not_accessible(self):
"""_provider（單底線）應不存在於 AuditedMCPToolProvider 實例"""
import sys
sys.path.insert(0, ".")
from src.plugins.mcp.interfaces import MCPToolProvider, MCPTool, MCPToolResult
from src.plugins.mcp.registry import AuditedMCPToolProvider
class DummyProvider(MCPToolProvider):
@property
def name(self): return "dummy"
async def list_tools(self): return []
async def execute(self, tool_name, parameters):
return MCPToolResult(success=True, execution_id="t")
wrapped = AuditedMCPToolProvider(DummyProvider())
assert not hasattr(wrapped, "_provider"), (
"_provider 不應可直接存取（name mangling 防止直接存取 inner provider）"
)
def test_double_underscore_mangled(self):
"""__provider 應被 Python name mangling 重命名"""
import sys
sys.path.insert(0, ".")
from src.plugins.mcp.interfaces import MCPToolProvider, MCPTool, MCPToolResult
from src.plugins.mcp.registry import AuditedMCPToolProvider
class DummyProvider(MCPToolProvider):
@property
def name(self): return "dummy"
async def list_tools(self): return []
async def execute(self, tool_name, parameters):
return MCPToolResult(success=True, execution_id="t")
wrapped = AuditedMCPToolProvider(DummyProvider())
# Python name mangling: __provider → _AuditedMCPToolProvider__provider
assert hasattr(wrapped, "_AuditedMCPToolProvider__provider"), (
"name mangling 後的屬性應可被內部存取"
)
assert wrapped.name == "dummy"