fix(p0.4): Playbook 學習鏈三道修復 — partial index + race防護 + 手動路徑接線
ADR-092 P0.4 Playbook EWMA 學習閉環的 DB / Repository / Service 三層修補。
DB 層 (db-expert-fix by Engineer-B):
- ApprovalRecord.matched_playbook_id 移除 index=True,改 __table_args__ partial index
(WHERE matched_playbook_id IS NOT NULL) — 多數列 NULL,full index 浪費空間
- adr092_p1_learning_chain_rollback.sql: 純 ROLLBACK SQL(DBA 手動執行)
Repository 層:
- playbook_repository.py: SELECT FOR UPDATE 防 lost update
避免並發 EWMA 更新覆蓋彼此
Service 層 (P0.4 修復):
- proposal_service.py: 手動審核路徑補 _try_playbook_match_id 呼叫
decision_manager auto_execute 路徑已有此邏輯(行 2035),
此處補手動路徑缺口,使 matched_playbook_id 可寫入 DB → EWMA 才能演化
測試:
- test_playbook_repository_race_condition.py: 3 cases SELECT FOR UPDATE 防 race
正確阻擋並發 EWMA 更新(pass)
Note: migration SQL 待 DBA 手動執行(feedback_dev_prod_separation.md),
不執行 alembic upgrade(statu 文件禁忌條款)。
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -22,6 +22,7 @@ from sqlalchemy import (
|
||||
Integer,
|
||||
String,
|
||||
Text,
|
||||
text,
|
||||
)
|
||||
from sqlalchemy import (
|
||||
Enum as SQLEnum,
|
||||
@@ -170,10 +171,11 @@ class ApprovalRecord(Base):
|
||||
|
||||
# B2 fix 2026-04-24 ogt + Claude Sonnet 4.6: Playbook 學習閉環斷鏈修復
|
||||
# 原欄位缺失 → 人工審核後 matched_playbook_id 永遠 NULL → EWMA 無法更新
|
||||
# 2026-04-25 db-expert-fix by Claude Engineer-B: 移除 index=True 避免自動生成 full index
|
||||
# Partial index 改在 __table_args__ 宣告(WHERE matched_playbook_id IS NOT NULL)
|
||||
matched_playbook_id: Mapped[str | None] = mapped_column(
|
||||
String(36),
|
||||
nullable=True,
|
||||
index=True,
|
||||
comment="匹配的 Playbook ID,學習服務用以更新 EWMA trust score",
|
||||
)
|
||||
|
||||
@@ -203,7 +205,13 @@ class ApprovalRecord(Base):
|
||||
Index("ix_approval_created_at", "created_at"),
|
||||
Index("ix_approval_requested_by", "requested_by"),
|
||||
Index("ix_approval_fingerprint", "fingerprint"), # 戰略 B: 指紋查詢優化
|
||||
Index("ix_approval_matched_playbook", "matched_playbook_id"), # B2 fix
|
||||
# 2026-04-25 db-expert-fix by Claude Engineer-B: 改為 partial index,只索引非 NULL 值
|
||||
# 原 full index 與 index=True 三重宣告衝突已修復(一個來源真相:此處)
|
||||
Index(
|
||||
"ix_approval_matched_playbook",
|
||||
"matched_playbook_id",
|
||||
postgresql_where=text("matched_playbook_id IS NOT NULL"),
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
|
||||
@@ -342,30 +342,46 @@ class PlaybookRepository:
|
||||
失敗: trust_new = 0.8 * trust_old + 0.2 * 0.0(衰減速度 2x)
|
||||
trust < 0.1 → 記錄警告,由 Evolver Agent 封存
|
||||
2026-04-15 ogt + Claude Sonnet 4.6(亞太): Phase 3.5 EWMA 雙寫 PG
|
||||
2026-04-25 db-expert-fix by Claude Engineer-B: SELECT FOR UPDATE 防 race condition(Lost Update)
|
||||
"""
|
||||
try:
|
||||
playbook = await self.get_by_id(playbook_id)
|
||||
if not playbook:
|
||||
return False
|
||||
# 使用 SELECT FOR UPDATE 確保並行 update_stats 不會 lost update
|
||||
factory = get_session_factory()
|
||||
async with factory() as session:
|
||||
async with session.begin():
|
||||
stmt = (
|
||||
select(PlaybookRecord)
|
||||
.where(PlaybookRecord.playbook_id == playbook_id)
|
||||
.with_for_update()
|
||||
)
|
||||
result = await session.execute(stmt)
|
||||
record = result.scalar_one_or_none()
|
||||
if record is None:
|
||||
return False
|
||||
|
||||
if success:
|
||||
playbook.success_count += 1
|
||||
playbook.trust_score = 0.9 * playbook.trust_score + 0.1 * 1.0
|
||||
else:
|
||||
playbook.failure_count += 1
|
||||
playbook.trust_score = 0.8 * playbook.trust_score + 0.2 * 0.0
|
||||
if success:
|
||||
record.success_count += 1
|
||||
record.trust_score = 0.9 * record.trust_score + 0.1 * 1.0
|
||||
else:
|
||||
record.failure_count += 1
|
||||
record.trust_score = 0.8 * record.trust_score + 0.2 * 0.0
|
||||
|
||||
playbook.trust_score = max(0.0, min(1.0, playbook.trust_score))
|
||||
playbook.last_used_at = now_taipei()
|
||||
record.trust_score = max(0.0, min(1.0, record.trust_score))
|
||||
record.last_used_at = now_taipei()
|
||||
# session.begin() context manager 結束時自動 commit
|
||||
|
||||
# 雙寫(PG + Redis)
|
||||
await self.update(playbook)
|
||||
# 讀取最新值供 Redis 雙寫 + 日誌(PG 已 commit,此時 Redis 同步)
|
||||
updated = await self._pg_get(playbook_id)
|
||||
if updated is None:
|
||||
return True # PG 已更新,Redis 同步失敗不影響結果
|
||||
|
||||
if playbook.trust_score < 0.1:
|
||||
await self._redis_set(updated)
|
||||
|
||||
if updated.trust_score < 0.1:
|
||||
logger.warning(
|
||||
"playbook_trust_low_auto_archive_candidate",
|
||||
playbook_id=playbook_id,
|
||||
trust_score=playbook.trust_score,
|
||||
trust_score=updated.trust_score,
|
||||
hint="Evolver Agent 應將此 Playbook 封存",
|
||||
)
|
||||
|
||||
@@ -373,8 +389,8 @@ class PlaybookRepository:
|
||||
"playbook_stats_updated",
|
||||
playbook_id=playbook_id,
|
||||
success=success,
|
||||
success_rate=playbook.success_rate,
|
||||
trust_score=playbook.trust_score,
|
||||
success_rate=updated.success_rate,
|
||||
trust_score=updated.trust_score,
|
||||
)
|
||||
return True
|
||||
|
||||
|
||||
@@ -246,6 +246,12 @@ class ProposalService:
|
||||
metadata["signoz_correlation"] = llm_proposal.get("signoz_correlation", "")
|
||||
metadata["optimization_suggestions"] = llm_proposal.get("optimization_suggestions", [])
|
||||
|
||||
# 2026-04-25 P0.4 修復 by Claude Engineer-B:
|
||||
# 手動路徑(API 呼叫 generate_proposal)補 Playbook RAG 匹配,
|
||||
# 讓 matched_playbook_id 得以寫入 DB,學習服務 EWMA 才能更新 trust score。
|
||||
# decision_manager auto_execute 路徑已有此邏輯(行 2035),此處補手動路徑缺口。
|
||||
matched_pb_id: str | None = await self._try_playbook_match_id(incident)
|
||||
|
||||
approval_create = ApprovalRequestCreate(
|
||||
action=action,
|
||||
description=description,
|
||||
@@ -255,6 +261,7 @@ class ProposalService:
|
||||
requested_by="OpenClaw AI",
|
||||
incident_id=incident_id,
|
||||
metadata=metadata,
|
||||
matched_playbook_id=matched_pb_id,
|
||||
)
|
||||
|
||||
approval = await self._approval_service.create_approval(approval_create)
|
||||
@@ -354,6 +361,55 @@ class ProposalService:
|
||||
|
||||
return action_type, action, description
|
||||
|
||||
# =========================================================================
|
||||
# 輔助方法: Playbook RAG 匹配(P0.4 2026-04-25 by Claude Engineer-B)
|
||||
# =========================================================================
|
||||
|
||||
async def _try_playbook_match_id(self, incident: Incident) -> str | None:
|
||||
"""
|
||||
嘗試 Playbook RAG 匹配,回傳 matched_playbook_id(相似度 >= 0.85 才填)。
|
||||
|
||||
設計動機:手動路徑(generate_proposal)補 matched_playbook_id,
|
||||
讓學習服務 EWMA 能在人工審核後更新 Playbook trust score。
|
||||
邏輯與 decision_manager._try_playbook_match 相同,但只回傳 ID 不改 action。
|
||||
失敗時靜默返回 None(不阻塞主流程)。
|
||||
"""
|
||||
PLAYBOOK_SIMILARITY_THRESHOLD = 0.85
|
||||
try:
|
||||
from src.models.playbook import SymptomPattern
|
||||
from src.services.playbook_service import get_playbook_service
|
||||
|
||||
alert_names = [s.alert_name for s in incident.signals] if incident.signals else []
|
||||
symptoms = SymptomPattern(
|
||||
alert_names=alert_names,
|
||||
affected_services=incident.affected_services or [],
|
||||
severity_range=[incident.severity.value] if incident.severity else ["P2"],
|
||||
)
|
||||
recommendations = await get_playbook_service().get_recommendations(
|
||||
symptoms=symptoms,
|
||||
top_k=1,
|
||||
)
|
||||
if not recommendations:
|
||||
return None
|
||||
best_match = recommendations[0]
|
||||
if best_match.similarity_score < PLAYBOOK_SIMILARITY_THRESHOLD:
|
||||
return None
|
||||
pb_id = best_match.playbook.playbook_id
|
||||
logger.info(
|
||||
"proposal_playbook_matched",
|
||||
incident_id=incident.incident_id,
|
||||
playbook_id=pb_id,
|
||||
similarity=best_match.similarity_score,
|
||||
)
|
||||
return pb_id
|
||||
except Exception as e:
|
||||
logger.debug(
|
||||
"proposal_playbook_match_skipped",
|
||||
incident_id=getattr(incident, "incident_id", "?"),
|
||||
error=str(e),
|
||||
)
|
||||
return None
|
||||
|
||||
# =========================================================================
|
||||
# 輔助方法: 建立 BlastRadius
|
||||
# =========================================================================
|
||||
|
||||
Reference in New Issue
Block a user