Files
awoooi/docs/superpowers/plans/2026-04-04-p1-knowledge-auto-harvesting.md
OG T 0b41df45d6 docs(plans): 三方向實作計畫 P0/P1/P2
- P0: DIAGNOSE Privacy-First Routing(local chain 隔離 + REJECT 保護)
- P1: Knowledge Auto-Harvesting(Anti-Pattern 閉環 + Runbook 生成)
- P2: Config Drift Detection(GitOps 守門員 + Nemotron 意圖分析)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-04 12:31:36 +08:00

43 KiB
Raw Blame History

P1Knowledge Auto-Harvesting 實作計畫

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: 修復成功後 Nemotron 自動生成完整 9 段 RunbookDRAFT 待審核),修復失敗自動生成 ANTI_PATTERN直接發布執行前加入 Anti-Pattern 閉環攔截,阻止在同一個坑重複摔倒。

Architecture: 三個整合點:(1) SymptomPattern.compute_hash() — O(1) 確定性比對;(2) AutoRepairService.evaluate_auto_repair() — 執行前查 KB anti-pattern 閘門;(3) AutoRepairService.execute_auto_repair() — 執行後背景異步呼叫 NemotronRunbookGenerator。Runbook 生成完全異步,不阻塞主流程。

Tech Stack: Python 3.11, asyncio, structlog, Pydantic v2, existing NemotronProvider / KnowledgeService / TelegramGateway, pytest-asyncio


File Map

動作 檔案 變更內容
修改 apps/api/src/models/knowledge.py 新增 ANTI_PATTERN / AUTO_RUNBOOK EntryTypeKnowledgeEntry 新增 symptoms_hash 欄位
修改 apps/api/src/models/playbook.py SymptomPattern 新增 compute_hash() 方法
修改 apps/api/src/services/knowledge_service.py 新增 check_anti_pattern(symptoms_hash, days) 方法
新增 apps/api/src/services/runbook_generator.py NemotronRunbookGenerator — 呼叫 NemotronProvider 生成 Runbook
修改 apps/api/src/services/auto_repair_service.py evaluate_auto_repair() 加 anti_pattern gateexecute_auto_repair() 結束後背景觸發生成
新增 apps/api/migrations/phase8_knowledge_symptoms_hash.sql knowledge 表新增 symptoms_hash VARCHAR(16) + index
新增 apps/api/tests/test_p1_knowledge_auto_harvesting.py 完整測試套件

Task 1DB Migration — 新增 symptoms_hash 欄位

Files:

  • Create: apps/api/migrations/phase8_knowledge_symptoms_hash.sql

  • Step 1建立 migration 檔案

-- phase8_knowledge_symptoms_hash.sql
-- Knowledge Auto-Harvesting: 新增 symptoms_hash 欄位
-- 建立時間: 2026-04-04 (台北時區)
-- 建立者: Claude Code (P1 Knowledge Auto-Harvesting)

-- 新增 symptoms_hash 欄位(可為 NULL既有資料不受影響
ALTER TABLE knowledge_entries
ADD COLUMN IF NOT EXISTS symptoms_hash VARCHAR(16) NULL;

-- 建立 index 供 check_anti_pattern() O(1) 查詢使用
CREATE INDEX IF NOT EXISTS idx_knowledge_entries_symptoms_hash
    ON knowledge_entries (symptoms_hash)
    WHERE symptoms_hash IS NOT NULL;

-- 確認
SELECT column_name, data_type, is_nullable
FROM information_schema.columns
WHERE table_name = 'knowledge_entries'
  AND column_name = 'symptoms_hash';
  • Step 2確認 migration 語法
cat apps/api/migrations/phase8_knowledge_symptoms_hash.sql
  • Step 3Commitmigration 先 commit後續程式碼與之對應
git add apps/api/migrations/phase8_knowledge_symptoms_hash.sql
git commit -m "feat(migration): knowledge_entries 新增 symptoms_hash 欄位 + index (P1)"

Task 2Model 更新 — EntryType + KnowledgeEntry + SymptomPattern

Files:

  • Modify: apps/api/src/models/knowledge.py

  • Modify: apps/api/src/models/playbook.py

  • Test: apps/api/tests/test_p1_knowledge_auto_harvesting.py

  • Step 1寫失敗測試

建立 apps/api/tests/test_p1_knowledge_auto_harvesting.py

"""
P1 Knowledge Auto-Harvesting Tests
=====================================
測試 SymptomPattern.compute_hash()、Anti-Pattern 閉環、Runbook 生成

建立時間: 2026-04-04 (台北時區)
建立者: Claude Code (P1 Knowledge Auto-Harvesting)
"""

import os
os.environ.setdefault("MOCK_MODE", "true")

import hashlib
import json
import pytest


class TestSymptomPatternHash:
    """SymptomPattern.compute_hash() — 確定性 hash"""

    def test_same_symptoms_same_hash(self):
        """相同症狀永遠產生相同 hash"""
        from src.models.playbook import SymptomPattern

        sp1 = SymptomPattern(
            alert_names=["HighCPU", "PodCrash"],
            affected_services=["awoooi-api"],
            label_patterns={"namespace": "awoooi-prod"},
        )
        sp2 = SymptomPattern(
            alert_names=["PodCrash", "HighCPU"],  # 順序不同
            affected_services=["awoooi-api"],
            label_patterns={"namespace": "awoooi-prod"},
        )

        assert sp1.compute_hash() == sp2.compute_hash()

    def test_different_symptoms_different_hash(self):
        """不同症狀產生不同 hash"""
        from src.models.playbook import SymptomPattern

        sp1 = SymptomPattern(alert_names=["HighCPU"], affected_services=["api"])
        sp2 = SymptomPattern(alert_names=["OOMKilled"], affected_services=["api"])

        assert sp1.compute_hash() != sp2.compute_hash()

    def test_hash_is_16_chars(self):
        """hash 長度固定為 16"""
        from src.models.playbook import SymptomPattern

        sp = SymptomPattern(alert_names=["Test"])
        assert len(sp.compute_hash()) == 16

    def test_empty_symptoms_has_valid_hash(self):
        """空症狀不 crash產生合法 hash"""
        from src.models.playbook import SymptomPattern

        sp = SymptomPattern()
        h = sp.compute_hash()
        assert len(h) == 16
        assert isinstance(h, str)


class TestKnowledgeEntryTypes:
    """EntryType 新增 ANTI_PATTERN / AUTO_RUNBOOK"""

    def test_anti_pattern_entry_type_exists(self):
        from src.models.knowledge import EntryType
        assert EntryType.ANTI_PATTERN == "anti_pattern"

    def test_auto_runbook_entry_type_exists(self):
        from src.models.knowledge import EntryType
        assert EntryType.AUTO_RUNBOOK == "auto_runbook"

    def test_knowledge_entry_accepts_symptoms_hash(self):
        """KnowledgeEntry 接受 symptoms_hash 欄位"""
        from src.models.knowledge import KnowledgeEntry, EntryType, EntrySource, EntryStatus

        entry = KnowledgeEntry(
            id="test-001",
            title="測試 Runbook",
            content="內容",
            entry_type=EntryType.AUTO_RUNBOOK,
            category="auto-generated",
            source=EntrySource.AI,
            status=EntryStatus.DRAFT,
            symptoms_hash="abc123def456789a",
        )
        assert entry.symptoms_hash == "abc123def456789a"

    def test_knowledge_entry_symptoms_hash_optional(self):
        """symptoms_hash 為 Optional不傳入不 crash"""
        from src.models.knowledge import KnowledgeEntry, EntryType, EntrySource

        entry = KnowledgeEntry(
            id="test-002",
            title="手動 Runbook",
            content="內容",
            entry_type=EntryType.RUNBOOK,
            category="manual",
            source=EntrySource.HUMAN,
        )
        assert entry.symptoms_hash is None
  • Step 2執行確認失敗
cd apps/api && python -m pytest tests/test_p1_knowledge_auto_harvesting.py::TestSymptomPatternHash tests/test_p1_knowledge_auto_harvesting.py::TestKnowledgeEntryTypes -v

預期FAILcompute_hash 不存在,ANTI_PATTERN 不存在)

  • Step 3修改 models/knowledge.py — 新增 EntryType 和 symptoms_hash 欄位

找到 EntryType class約 L30在最後一個成員後新增

    ANTI_PATTERN = "anti_pattern"    # 失敗案例,自動分類隔離,直接發布
    AUTO_RUNBOOK = "auto_runbook"    # Nemotron 自動生成,待人工審核

找到 KnowledgeEntry class約 L80related_playbook_id 欄位後新增:

    symptoms_hash: str | None = Field(
        default=None,
        description="症狀特徵 hash供 Anti-Pattern O(1) 精確比對16 chars SHA256 前綴)",
    )
  • Step 4修改 models/playbook.py — 新增 compute_hash() 方法

找到 SymptomPattern class約 L66model_config 上方新增:

    def compute_hash(self) -> str:
        """
        確定性 hashalert_names + affected_services + label_patterns
        目的O(1) 精確比對,避免純語意搜尋的模糊性
        順序無關(使用 sorted跨平台一致json.dumps sort_keys=True

        2026-04-04 Claude Code (P1): Anti-Pattern 閉環查詢鍵
        """
        import hashlib
        import json as _json
        key = (
            "|".join(sorted(self.alert_names)) + "||" +
            "|".join(sorted(self.affected_services)) + "||" +
            _json.dumps(self.label_patterns, sort_keys=True)
        )
        return hashlib.sha256(key.encode()).hexdigest()[:16]
  • Step 5執行測試
cd apps/api && python -m pytest tests/test_p1_knowledge_auto_harvesting.py::TestSymptomPatternHash tests/test_p1_knowledge_auto_harvesting.py::TestKnowledgeEntryTypes -v

預期:全部 PASS

  • Step 6確認既有測試不受影響
cd apps/api && python -m pytest tests/test_playbook_service.py -v

預期:全部 PASS

  • Step 7Commit
git add apps/api/src/models/knowledge.py apps/api/src/models/playbook.py apps/api/tests/test_p1_knowledge_auto_harvesting.py
git commit -m "feat(models): 新增 ANTI_PATTERN/AUTO_RUNBOOK EntryType + symptoms_hash + SymptomPattern.compute_hash() (P1)"

Task 3KnowledgeService 新增 check_anti_pattern()

Files:

  • Modify: apps/api/src/services/knowledge_service.py

  • Step 1新增測試

tests/test_p1_knowledge_auto_harvesting.py 新增:

class TestCheckAntiPattern:
    """KnowledgeService.check_anti_pattern() — 7 天內 ANTI_PATTERN 查詢"""

    @pytest.mark.asyncio
    async def test_returns_empty_when_no_anti_pattern(self):
        """無 ANTI_PATTERN 記錄時回傳空 list"""
        from src.services.knowledge_service import KnowledgeService
        from unittest.mock import AsyncMock, MagicMock

        svc = KnowledgeService.__new__(KnowledgeService)

        mock_repo = AsyncMock()
        mock_repo.find_by_symptoms_hash = AsyncMock(return_value=[])
        svc._repo = mock_repo
        svc._embed_svc = MagicMock()
        svc._pending_tasks = set()

        result = await svc.check_anti_pattern("abc123def456789a", days=7)
        assert result == []

    @pytest.mark.asyncio
    async def test_returns_anti_patterns_within_days(self):
        """7 天內有 ANTI_PATTERN → 回傳該記錄"""
        from src.services.knowledge_service import KnowledgeService
        from src.models.knowledge import KnowledgeEntry, EntryType, EntrySource, EntryStatus
        from unittest.mock import AsyncMock, MagicMock
        from datetime import datetime, timezone, timedelta

        svc = KnowledgeService.__new__(KnowledgeService)

        recent_entry = KnowledgeEntry(
            id="ap-001",
            title="Pod OOM 修復失敗",
            content="失敗原因: ...",
            entry_type=EntryType.ANTI_PATTERN,
            category="auto-generated",
            source=EntrySource.AI,
            status=EntryStatus.PUBLISHED,
            symptoms_hash="abc123def456789a",
            created_at=datetime.now(timezone.utc) - timedelta(days=3),
        )

        mock_repo = AsyncMock()
        mock_repo.find_by_symptoms_hash = AsyncMock(return_value=[recent_entry])
        svc._repo = mock_repo
        svc._embed_svc = MagicMock()
        svc._pending_tasks = set()

        result = await svc.check_anti_pattern("abc123def456789a", days=7)
        assert len(result) == 1
        assert result[0].id == "ap-001"
  • Step 2執行確認失敗
cd apps/api && python -m pytest tests/test_p1_knowledge_auto_harvesting.py::TestCheckAntiPattern -v

預期FAILcheck_anti_pattern 不存在)

  • Step 3knowledge_service.py 新增方法

KnowledgeService class 中,create_entry() 方法之後新增:

    async def check_anti_pattern(
        self,
        symptoms_hash: str,
        days: int = 7,
    ) -> list:
        """
        查詢近期相同症狀的失敗案例ANTI_PATTERN

        P1 2026-04-04 Claude Code: Anti-Pattern 閉環攔截查詢鍵
        時間範圍: days 天內(預設 7 天)
        回傳: 符合條件的 KnowledgeEntry list通常 0-3 筆)

        Args:
            symptoms_hash: SymptomPattern.compute_hash() 產生的 16 char hash
            days: 往回查幾天(預設 7

        Returns:
            list[KnowledgeEntry]: ANTI_PATTERN 記錄,空 list 表示無紀錄
        """
        from datetime import datetime, timezone, timedelta
        from src.models.knowledge import EntryType

        cutoff = datetime.now(timezone.utc) - timedelta(days=days)

        try:
            async with get_db_context() as db:
                repo = KnowledgeDBRepository(db)
                entries = await repo.find_by_symptoms_hash(
                    symptoms_hash=symptoms_hash,
                    entry_type=EntryType.ANTI_PATTERN,
                    created_after=cutoff,
                )
                return entries
        except Exception as e:
            logger.error(
                "check_anti_pattern_error",
                symptoms_hash=symptoms_hash,
                error=str(e),
            )
            return []  # 查詢失敗時回傳空 list不阻斷主流程
  • Step 4knowledge_repository.py 新增 find_by_symptoms_hash()
grep -n "class KnowledgeDBRepository\|async def find\|async def search" apps/api/src/repositories/knowledge_repository.py | head -15

KnowledgeDBRepository class 中新增:

    async def find_by_symptoms_hash(
        self,
        symptoms_hash: str,
        entry_type,
        created_after,
    ) -> list:
        """
        依 symptoms_hash + entry_type + 時間範圍查詢

        P1 2026-04-04 Claude Code: Anti-Pattern 閉環查詢
        """
        from sqlalchemy import select, and_
        from src.db.models import KnowledgeEntryORM

        stmt = select(KnowledgeEntryORM).where(
            and_(
                KnowledgeEntryORM.symptoms_hash == symptoms_hash,
                KnowledgeEntryORM.entry_type == entry_type.value,
                KnowledgeEntryORM.created_at >= created_after,
            )
        ).order_by(KnowledgeEntryORM.created_at.desc())

        result = await self._db.execute(stmt)
        rows = result.scalars().all()
        return [KnowledgeEntry.model_validate(row) for row in rows]
  • Step 5確認 KnowledgeEntryORM 有 symptoms_hash 欄位
grep -n "symptoms_hash\|class KnowledgeEntryORM" apps/api/src/db/models.py | head -10

若無,在 KnowledgeEntryORM class 中新增(在 related_playbook_id 後):

    symptoms_hash: Mapped[str | None] = mapped_column(String(16), nullable=True, index=True)
  • Step 6執行測試使用 mock不依賴真實 DB
cd apps/api && python -m pytest tests/test_p1_knowledge_auto_harvesting.py::TestCheckAntiPattern -v

預期PASS

  • Step 7Commit
git add apps/api/src/services/knowledge_service.py apps/api/src/repositories/knowledge_repository.py apps/api/src/db/models.py apps/api/tests/test_p1_knowledge_auto_harvesting.py
git commit -m "feat(knowledge): check_anti_pattern() + find_by_symptoms_hash() + ORM symptoms_hash 欄位 (P1)"

Task 4AutoRepairService — 執行前 Anti-Pattern 閘門

Files:

  • Modify: apps/api/src/services/auto_repair_service.py

  • Step 1新增閘門測試

tests/test_p1_knowledge_auto_harvesting.py 新增:

class TestAntiPatternGate:
    """AutoRepairService.evaluate_auto_repair() — Anti-Pattern 閉環閘門"""

    @pytest.mark.asyncio
    async def test_blocked_when_anti_pattern_exists(self):
        """7 天內有 ANTI_PATTERN → 阻斷自動修復,強制 HITL"""
        from src.services.auto_repair_service import AutoRepairService
        from src.models.incident import Incident, IncidentStatus, Severity, Signal
        from src.models.playbook import SymptomPattern
        from src.models.knowledge import KnowledgeEntry, EntryType, EntrySource, EntryStatus
        from unittest.mock import AsyncMock, MagicMock, patch
        from src.utils.timezone import now_taipei

        # 建立 mock knowledge service回傳一筆 ANTI_PATTERN
        mock_knowledge_svc = AsyncMock()
        anti_pattern_entry = MagicMock(spec=KnowledgeEntry)
        anti_pattern_entry.title = "Pod OOM 修復失敗(已知無效)"
        mock_knowledge_svc.check_anti_pattern = AsyncMock(return_value=[anti_pattern_entry])

        svc = AutoRepairService(knowledge_service=mock_knowledge_svc)

        incident = Incident(
            incident_id="INC-001",
            title="Pod OOMKilled",
            status=IncidentStatus.OPEN,
            severity=Severity.P2,
            signals=[Signal(name="OOMKilled", value="1", source="prometheus")],
            created_at=now_taipei(),
        )

        decision = await svc.evaluate_auto_repair(incident)

        assert decision.can_auto_repair is False
        assert decision.blocked_by == "ANTI_PATTERN"
        assert "Pod OOM 修復失敗" in decision.reason

    @pytest.mark.asyncio
    async def test_proceeds_when_no_anti_pattern(self):
        """無 ANTI_PATTERN 記錄 → 繼續原有評估流程"""
        from src.services.auto_repair_service import AutoRepairService
        from src.models.incident import Incident, IncidentStatus, Severity, Signal
        from unittest.mock import AsyncMock, MagicMock
        from src.utils.timezone import now_taipei

        mock_knowledge_svc = AsyncMock()
        mock_knowledge_svc.check_anti_pattern = AsyncMock(return_value=[])

        mock_playbook_svc = AsyncMock()
        mock_playbook_svc.get_recommendations = AsyncMock(return_value=[])

        svc = AutoRepairService(
            knowledge_service=mock_knowledge_svc,
            playbook_service=mock_playbook_svc,
        )

        incident = Incident(
            incident_id="INC-002",
            title="高 CPU",
            status=IncidentStatus.OPEN,
            severity=Severity.P2,
            signals=[Signal(name="HighCPU", value="95", source="prometheus")],
            created_at=now_taipei(),
        )

        decision = await svc.evaluate_auto_repair(incident)

        # 無 Playbook match → can_auto_repair=False但 blocked_by 不是 ANTI_PATTERN
        assert decision.blocked_by != "ANTI_PATTERN"
        mock_knowledge_svc.check_anti_pattern.assert_called_once()
  • Step 2執行確認失敗
cd apps/api && python -m pytest tests/test_p1_knowledge_auto_harvesting.py::TestAntiPatternGate -v

預期FAILAutoRepairService 不接受 knowledge_service 參數)

  • Step 3修改 auto_repair_service.py — 注入 KnowledgeService + 加閘門

找到 AutoRepairService.__init__() 方法(約 L140新增 knowledge_service 參數:

    def __init__(
        self,
        playbook_service: IPlaybookService | None = None,
        knowledge_service=None,   # P1 2026-04-04 Claude Code: Anti-Pattern gate
    ) -> None:
        self._playbook_service = playbook_service or get_playbook_service()
        # P1 2026-04-04 Claude Code: 延遲 import 避免循環依賴
        self._knowledge_service = knowledge_service

找到 evaluate_auto_repair() 方法中,symptoms = self._extract_symptoms(incident) 這行(約 L197在其後、get_recommendations() 呼叫前插入:

        # P1 2026-04-04 Claude Code: Anti-Pattern 閉環閘門
        # 查詢 7 天內相同症狀的失敗案例,避免重複踩坑
        symptoms_hash = symptoms.compute_hash()
        try:
            if self._knowledge_service is None:
                from src.services.knowledge_service import get_knowledge_service
                self._knowledge_service = get_knowledge_service()

            anti_patterns = await self._knowledge_service.check_anti_pattern(
                symptoms_hash=symptoms_hash,
                days=7,
            )
            if anti_patterns:
                logger.warning(
                    "auto_repair_blocked_anti_pattern",
                    incident_id=incident.incident_id,
                    symptoms_hash=symptoms_hash,
                    anti_pattern_title=anti_patterns[0].title,
                )
                return AutoRepairDecision(
                    can_auto_repair=False,
                    blocked_by="ANTI_PATTERN",
                    reason=f"過去 7 天有失敗案例: {anti_patterns[0].title}",
                )
        except Exception as e:
            # 查詢失敗不阻斷,記錄 warning 後繼續
            logger.warning(
                "auto_repair_anti_pattern_check_failed",
                incident_id=incident.incident_id,
                error=str(e),
            )
  • Step 4執行測試
cd apps/api && python -m pytest tests/test_p1_knowledge_auto_harvesting.py::TestAntiPatternGate -v

預期PASS

  • Step 5確認既有 auto_repair 測試不受影響
cd apps/api && python -m pytest tests/test_auto_repair_service.py -v

預期:全部 PASS

  • Step 6Commit
git add apps/api/src/services/auto_repair_service.py apps/api/tests/test_p1_knowledge_auto_harvesting.py
git commit -m "feat(auto-repair): Anti-Pattern 閉環閘門 — 7 天內失敗案例直接阻斷 (P1)"

Task 5NemotronRunbookGenerator 服務

Files:

  • Create: apps/api/src/services/runbook_generator.py

  • Step 1新增測試

tests/test_p1_knowledge_auto_harvesting.py 新增:

class TestNemotronRunbookGenerator:
    """NemotronRunbookGenerator — 生成 Runbook / Anti-Pattern 條目"""

    @pytest.mark.asyncio
    async def test_generate_success_creates_auto_runbook(self):
        """修復成功 → 生成 AUTO_RUNBOOKstatus=DRAFT"""
        from src.services.runbook_generator import NemotronRunbookGenerator
        from src.models.incident import Incident, IncidentStatus, Severity, Signal
        from src.models.playbook import Playbook, PlaybookStatus, SymptomPattern
        from src.services.auto_repair_service import AutoRepairResult
        from src.models.knowledge import EntryType, EntryStatus
        from src.services.ai_providers.interfaces import AIResult
        from unittest.mock import AsyncMock, MagicMock, patch
        from src.utils.timezone import now_taipei

        mock_ai_router = AsyncMock()
        mock_ai_router.execute = AsyncMock(return_value=AIResult(
            raw_response='{"symptoms_description":"Pod 記憶體不足","root_cause_analysis":"OOM","execution_steps":["kubectl rollout restart"],"verification_steps":["kubectl get pods"],"precautions":["備份資料"],"impact_scope":"awoooi-api","related_incident_ids":["INC-001"],"prevention_suggestions":["增加記憶體限制"],"applicable_conditions":["OOMKilled 告警"]}',
            success=True,
            provider="nemotron",
        ))

        mock_knowledge_svc = AsyncMock()
        created_entry = MagicMock()
        created_entry.id = "kb-001"
        mock_knowledge_svc.create_entry = AsyncMock(return_value=created_entry)

        gen = NemotronRunbookGenerator(
            ai_router=mock_ai_router,
            knowledge_service=mock_knowledge_svc,
        )

        incident = Incident(
            incident_id="INC-001",
            title="Pod OOMKilled",
            status=IncidentStatus.RESOLVED,
            severity=Severity.P2,
            signals=[Signal(name="OOMKilled", value="1", source="prometheus")],
            created_at=now_taipei(),
        )
        playbook = Playbook(
            playbook_id="PB-001",
            name="OOM Restart",
            status=PlaybookStatus.ACTIVE,
            symptom_pattern=SymptomPattern(alert_names=["OOMKilled"]),
            repair_steps=[],
        )
        repair_result = AutoRepairResult(
            success=True,
            playbook_id="PB-001",
            incident_id="INC-001",
            executed_steps=["kubectl rollout restart deployment/awoooi-api"],
        )

        entry = await gen.generate(incident, playbook, repair_result)

        assert entry is not None
        mock_knowledge_svc.create_entry.assert_called_once()
        call_args = mock_knowledge_svc.create_entry.call_args[0][0]
        assert call_args.entry_type == EntryType.AUTO_RUNBOOK
        assert call_args.status == EntryStatus.DRAFT

    @pytest.mark.asyncio
    async def test_generate_anti_pattern_creates_published_entry(self):
        """修復失敗 → 生成 ANTI_PATTERNstatus=PUBLISHED直接發布"""
        from src.services.runbook_generator import NemotronRunbookGenerator
        from src.models.incident import Incident, IncidentStatus, Severity, Signal
        from src.models.playbook import Playbook, PlaybookStatus, SymptomPattern
        from src.services.auto_repair_service import AutoRepairResult
        from src.models.knowledge import EntryType, EntryStatus
        from src.services.ai_providers.interfaces import AIResult
        from unittest.mock import AsyncMock, MagicMock
        from src.utils.timezone import now_taipei

        mock_ai_router = AsyncMock()
        mock_ai_router.execute = AsyncMock(return_value=AIResult(
            raw_response='{"failure_reason":"kubectl 超時","ineffective_steps":["kubectl rollout restart"],"alternative_suggestions":["手動 drain node"],"applicable_conditions":["節點磁碟壓力時此方法無效"]}',
            success=True,
            provider="nemotron",
        ))

        mock_knowledge_svc = AsyncMock()
        created_entry = MagicMock()
        created_entry.id = "kb-002"
        mock_knowledge_svc.create_entry = AsyncMock(return_value=created_entry)

        gen = NemotronRunbookGenerator(
            ai_router=mock_ai_router,
            knowledge_service=mock_knowledge_svc,
        )

        incident = Incident(
            incident_id="INC-002",
            title="Pod OOMKilled",
            status=IncidentStatus.OPEN,
            severity=Severity.P2,
            signals=[Signal(name="OOMKilled", value="1", source="prometheus")],
            created_at=now_taipei(),
        )
        playbook = Playbook(
            playbook_id="PB-001",
            name="OOM Restart",
            status=PlaybookStatus.ACTIVE,
            symptom_pattern=SymptomPattern(alert_names=["OOMKilled"]),
            repair_steps=[],
        )
        repair_result = AutoRepairResult(
            success=False,
            playbook_id="PB-001",
            incident_id="INC-002",
            executed_steps=[],
            error="kubectl timeout",
        )

        entry = await gen.generate_anti_pattern(incident, playbook, repair_result)

        assert entry is not None
        call_args = mock_knowledge_svc.create_entry.call_args[0][0]
        assert call_args.entry_type == EntryType.ANTI_PATTERN
        assert call_args.status == EntryStatus.PUBLISHED
  • Step 2執行確認失敗
cd apps/api && python -m pytest tests/test_p1_knowledge_auto_harvesting.py::TestNemotronRunbookGenerator -v

預期FAILrunbook_generator 不存在)

  • Step 3建立 services/runbook_generator.py
"""
Runbook Generator Service - P1 Knowledge Auto-Harvesting
=========================================================
修復成功/失敗後,由 Nemotron 自動生成 Runbook / Anti-Pattern 條目

設計原則:
- 完全異步,不阻塞 AutoRepairService 主流程
- 生成失敗不影響修復結果(防禦性工程)
- SUCCESS → AUTO_RUNBOOK (DRAFT) → 人工審核
- FAILURE → ANTI_PATTERN (PUBLISHED) → 直接發布

建立時間: 2026-04-04 (台北時區)
建立者: Claude Code (P1 Knowledge Auto-Harvesting)
"""

from __future__ import annotations

import json
from typing import TYPE_CHECKING, Any

import structlog

from src.models.knowledge import EntrySource, EntryStatus, EntryType, KnowledgeEntryCreate

if TYPE_CHECKING:
    from src.models.incident import Incident
    from src.models.playbook import Playbook
    from src.services.auto_repair_service import AutoRepairResult

logger = structlog.get_logger(__name__)

# =============================================================================
# Prompts
# =============================================================================

_SUCCESS_PROMPT_TEMPLATE = """你是 SRE 知識管理系統。根據以下修復案例,生成一份完整的 Runbook。

## 事件資訊
- Incident ID: {incident_id}
- 標題: {incident_title}
- 嚴重度: {severity}
- 告警: {alert_names}
- 受影響服務: {affected_services}

## 執行的 Playbook
- Playbook ID: {playbook_id}
- Playbook 名稱: {playbook_name}

## 執行結果
- 狀態: 成功
- 執行步驟: {executed_steps}

請以 JSON 格式回應,包含以下欄位:
{{
  "symptoms_description": "症狀描述",
  "root_cause_analysis": "根因分析",
  "execution_steps": ["步驟1", "步驟2"],
  "verification_steps": ["驗證1", "驗證2"],
  "precautions": ["注意1", "注意2"],
  "impact_scope": "影響範圍",
  "related_incident_ids": ["{incident_id}"],
  "prevention_suggestions": ["預防1", "預防2"],
  "applicable_conditions": ["條件1", "條件2"]
}}"""

_FAILURE_PROMPT_TEMPLATE = """你是 SRE 知識管理系統。根據以下修復失敗案例,生成一份 Anti-Pattern 記錄。

## 事件資訊
- Incident ID: {incident_id}
- 標題: {incident_title}
- 嚴重度: {severity}
- 告警: {alert_names}

## 嘗試的 Playbook
- Playbook ID: {playbook_id}
- Playbook 名稱: {playbook_name}

## 失敗資訊
- 錯誤: {error}
- 已執行步驟: {executed_steps}

請以 JSON 格式回應,包含以下欄位:
{{
  "failure_reason": "失敗原因",
  "ineffective_steps": ["無效步驟1"],
  "alternative_suggestions": ["替代方案1"],
  "applicable_conditions": ["此方法不適用的條件"]
}}"""


# =============================================================================
# NemotronRunbookGenerator
# =============================================================================


class NemotronRunbookGenerator:
    """
    Nemotron 驅動的 Runbook 生成器

    職責: 接收修復結果 → 呼叫 Nemotron → 寫入 KB
    不負責: Telegram 推送(由 caller 負責)、主流程錯誤處理
    """

    def __init__(
        self,
        ai_router=None,
        knowledge_service=None,
    ) -> None:
        self._ai_router = ai_router
        self._knowledge_service = knowledge_service

    def _get_ai_router(self):
        if self._ai_router is None:
            from src.services.ai_router import get_ai_router
            self._ai_router = get_ai_router()
        return self._ai_router

    def _get_knowledge_service(self):
        if self._knowledge_service is None:
            from src.services.knowledge_service import get_knowledge_service
            self._knowledge_service = get_knowledge_service()
        return self._knowledge_service

    async def generate(
        self,
        incident: Incident,
        playbook: Playbook,
        repair_result: AutoRepairResult,
    ):
        """
        修復成功後生成完整 Runbook → KB (AUTO_RUNBOOK, DRAFT)

        Returns:
            KnowledgeEntry | None: 建立成功的條目,失敗時回傳 None不 raise
        """
        symptoms_hash = playbook.symptom_pattern.compute_hash()
        prompt = _SUCCESS_PROMPT_TEMPLATE.format(
            incident_id=incident.incident_id,
            incident_title=incident.title,
            severity=incident.severity.value,
            alert_names=", ".join(playbook.symptom_pattern.alert_names),
            affected_services=", ".join(playbook.symptom_pattern.affected_services),
            playbook_id=playbook.playbook_id,
            playbook_name=playbook.name,
            executed_steps="\n".join(repair_result.executed_steps),
        )

        try:
            router = self._get_ai_router()
            result = await router.execute(
                prompt=prompt,
                provider_order=["nemotron", "gemini"],
                context={"task_type": "runbook_generation"},
            )

            if not result.success:
                logger.error("runbook_generation_ai_failed", incident_id=incident.incident_id)
                return None

            content = self._build_success_content(result.raw_response)
            title = f"[AUTO] {incident.title} — Runbook"

            svc = self._get_knowledge_service()
            entry = await svc.create_entry(KnowledgeEntryCreate(
                title=title,
                content=content,
                entry_type=EntryType.AUTO_RUNBOOK,
                category="auto-generated",
                tags=playbook.symptom_pattern.alert_names[:3],
                source=EntrySource.AI,
                status=EntryStatus.DRAFT,
                related_incident_id=incident.incident_id,
                related_playbook_id=playbook.playbook_id,
                symptoms_hash=symptoms_hash,
            ))

            logger.info(
                "runbook_generated",
                incident_id=incident.incident_id,
                entry_id=entry.id,
                symptoms_hash=symptoms_hash,
            )
            return entry

        except Exception as e:
            logger.error(
                "runbook_generation_error",
                incident_id=incident.incident_id,
                error=str(e),
            )
            return None

    async def generate_anti_pattern(
        self,
        incident: Incident,
        playbook: Playbook,
        repair_result: AutoRepairResult,
    ):
        """
        修復失敗後生成 Anti-Pattern → KB (ANTI_PATTERN, PUBLISHED)

        Returns:
            KnowledgeEntry | None: 建立成功的條目,失敗時回傳 None不 raise
        """
        symptoms_hash = playbook.symptom_pattern.compute_hash()
        prompt = _FAILURE_PROMPT_TEMPLATE.format(
            incident_id=incident.incident_id,
            incident_title=incident.title,
            severity=incident.severity.value,
            alert_names=", ".join(playbook.symptom_pattern.alert_names),
            playbook_id=playbook.playbook_id,
            playbook_name=playbook.name,
            error=repair_result.error or "未知錯誤",
            executed_steps="\n".join(repair_result.executed_steps),
        )

        try:
            router = self._get_ai_router()
            result = await router.execute(
                prompt=prompt,
                provider_order=["nemotron", "gemini"],
                context={"task_type": "anti_pattern_generation"},
            )

            if not result.success:
                logger.error("anti_pattern_generation_ai_failed", incident_id=incident.incident_id)
                return None

            content = self._build_failure_content(result.raw_response)
            title = f"[ANTI-PATTERN] {incident.title} — 已知無效方案"

            svc = self._get_knowledge_service()
            entry = await svc.create_entry(KnowledgeEntryCreate(
                title=title,
                content=content,
                entry_type=EntryType.ANTI_PATTERN,
                category="auto-generated",
                tags=playbook.symptom_pattern.alert_names[:3],
                source=EntrySource.AI,
                status=EntryStatus.PUBLISHED,
                related_incident_id=incident.incident_id,
                related_playbook_id=playbook.playbook_id,
                symptoms_hash=symptoms_hash,
            ))

            logger.info(
                "anti_pattern_recorded",
                incident_id=incident.incident_id,
                entry_id=entry.id,
                symptoms_hash=symptoms_hash,
            )
            return entry

        except Exception as e:
            logger.error(
                "anti_pattern_generation_error",
                incident_id=incident.incident_id,
                error=str(e),
            )
            return None

    def _build_success_content(self, raw_response: str) -> str:
        """將 Nemotron JSON 回應轉為 Markdown 格式 Runbook"""
        try:
            data = json.loads(raw_response)
        except json.JSONDecodeError:
            return raw_response  # fallback: 原始文字

        sections = [
            f"## 症狀描述\n{data.get('symptoms_description', '')}",
            f"## 根因分析\n{data.get('root_cause_analysis', '')}",
            "## 執行步驟\n" + "\n".join(f"{i+1}. {s}" for i, s in enumerate(data.get('execution_steps', []))),
            "## 驗證步驟\n" + "\n".join(f"- {s}" for s in data.get('verification_steps', [])),
            "## 注意事項\n" + "\n".join(f"- {s}" for s in data.get('precautions', [])),
            f"## 影響範圍\n{data.get('impact_scope', '')}",
            "## 相關 Incident\n" + "\n".join(f"- {i}" for i in data.get('related_incident_ids', [])),
            "## 下次預防建議\n" + "\n".join(f"- {s}" for s in data.get('prevention_suggestions', [])),
            "## 適用條件\n" + "\n".join(f"- {s}" for s in data.get('applicable_conditions', [])),
        ]
        return "\n\n".join(sections)

    def _build_failure_content(self, raw_response: str) -> str:
        """將 Nemotron JSON 回應轉為 Markdown 格式 Anti-Pattern"""
        try:
            data = json.loads(raw_response)
        except json.JSONDecodeError:
            return raw_response

        sections = [
            f"## 失敗原因\n{data.get('failure_reason', '')}",
            "## 無效步驟\n" + "\n".join(f"- {s}" for s in data.get('ineffective_steps', [])),
            "## 替代方案建議\n" + "\n".join(f"- {s}" for s in data.get('alternative_suggestions', [])),
            "## 不適用條件\n" + "\n".join(f"- {s}" for s in data.get('applicable_conditions', [])),
        ]
        return "\n\n".join(sections)


# =============================================================================
# Singleton
# =============================================================================

_runbook_generator: NemotronRunbookGenerator | None = None


def get_runbook_generator() -> NemotronRunbookGenerator:
    global _runbook_generator
    if _runbook_generator is None:
        _runbook_generator = NemotronRunbookGenerator()
    return _runbook_generator
  • Step 4執行測試
cd apps/api && python -m pytest tests/test_p1_knowledge_auto_harvesting.py::TestNemotronRunbookGenerator -v

預期PASS

  • Step 5Commit
git add apps/api/src/services/runbook_generator.py apps/api/tests/test_p1_knowledge_auto_harvesting.py
git commit -m "feat(runbook-generator): NemotronRunbookGenerator — SUCCESS→AUTO_RUNBOOK / FAILURE→ANTI_PATTERN (P1)"

Task 6AutoRepairService — 執行後背景觸發生成

Files:

  • Modify: apps/api/src/services/auto_repair_service.py

  • Step 1execute_auto_repair() 結束後加入背景觸發

找到 execute_auto_repair()return AutoRepairResult(success=True, ...) 的行(約 L330return 前插入:

            # P1 2026-04-04 Claude Code: 背景異步生成 Runbook不阻塞主流程
            self._schedule_runbook_generation(incident, playbook, result=AutoRepairResult(
                success=True,
                playbook_id=playbook.playbook_id,
                incident_id=incident.incident_id,
                executed_steps=executed_steps,
                execution_time_ms=execution_time,
            ))

找到 return AutoRepairResult(success=False, ...) 的行(約 L350return 前插入:

            # P1 2026-04-04 Claude Code: 背景異步生成 Anti-Pattern不阻塞主流程
            failure_result = AutoRepairResult(
                success=False,
                playbook_id=playbook.playbook_id,
                incident_id=incident.incident_id,
                executed_steps=executed_steps,
                error=str(e),
                execution_time_ms=execution_time,
            )
            self._schedule_anti_pattern_generation(incident, playbook, failure_result)

AutoRepairService class 末尾新增兩個私有方法:

    def _schedule_runbook_generation(
        self,
        incident,
        playbook,
        result,
    ) -> None:
        """
        背景排程 Runbook 生成fire-and-forget不阻塞主流程

        P1 2026-04-04 Claude Code: 異步不阻塞設計
        """
        import asyncio

        async def _generate():
            try:
                from src.services.runbook_generator import get_runbook_generator
                gen = get_runbook_generator()
                entry = await gen.generate(incident, playbook, result)
                if entry:
                    logger.info(
                        "runbook_generation_scheduled_done",
                        incident_id=incident.incident_id,
                        entry_id=entry.id,
                    )
            except Exception as e:
                logger.error(
                    "runbook_generation_scheduled_error",
                    incident_id=incident.incident_id,
                    error=str(e),
                )

        task = asyncio.create_task(_generate())
        # 防止 GC 提前回收 task
        self._background_tasks = getattr(self, "_background_tasks", set())
        self._background_tasks.add(task)
        task.add_done_callback(self._background_tasks.discard)

    def _schedule_anti_pattern_generation(
        self,
        incident,
        playbook,
        result,
    ) -> None:
        """
        背景排程 Anti-Pattern 生成fire-and-forget不阻塞主流程

        P1 2026-04-04 Claude Code: 異步不阻塞設計
        """
        import asyncio

        async def _generate():
            try:
                from src.services.runbook_generator import get_runbook_generator
                gen = get_runbook_generator()
                entry = await gen.generate_anti_pattern(incident, playbook, result)
                if entry:
                    logger.info(
                        "anti_pattern_generation_scheduled_done",
                        incident_id=incident.incident_id,
                        entry_id=entry.id,
                    )
            except Exception as e:
                logger.error(
                    "anti_pattern_generation_scheduled_error",
                    incident_id=incident.incident_id,
                    error=str(e),
                )

        task = asyncio.create_task(_generate())
        self._background_tasks = getattr(self, "_background_tasks", set())
        self._background_tasks.add(task)
        task.add_done_callback(self._background_tasks.discard)
  • Step 2執行全部 P1 測試
cd apps/api && python -m pytest tests/test_p1_knowledge_auto_harvesting.py -v

預期:全部 PASS

  • Step 3確認既有測試不受影響
cd apps/api && python -m pytest tests/test_auto_repair_service.py tests/test_playbook_service.py -v

預期:全部 PASS

  • Step 4Commit
git add apps/api/src/services/auto_repair_service.py
git commit -m "feat(auto-repair): 執行後背景觸發 Runbook/Anti-Pattern 生成(異步不阻塞)(P1)"

驗收標準

# 全部 P1 測試通過
cd apps/api && python -m pytest tests/test_p1_knowledge_auto_harvesting.py -v

# 既有測試未破壞
cd apps/api && python -m pytest tests/test_auto_repair_service.py tests/test_playbook_service.py tests/test_smart_router.py -v

# Migration 語法驗證
psql -h localhost -U postgres -d awoooi_dev -f apps/api/migrations/phase8_knowledge_symptoms_hash.sql

Co-Authored-By: Claude Sonnet 4.6 noreply@anthropic.com