wooo/awoooi
1
1
Fork 0
Code Issues 2 Pull Requests 1 Actions 2 Packages Projects Releases Wiki Activity
5,784 Commits 203 Branches 2 Tags
e1f232b1535c6887b5b910aa352ea49dcdcee39b
Commit Graph

39 Commits

Author SHA1 Message Date
Your Name
ff30c61c4c fix(rls): 收斂 API DB access context
All checks were successful
Code Review / ai-code-review (push) Successful in 21s
Details
CD Pipeline / tests (push) Successful in 1m20s
Details
CD Pipeline / build-and-deploy (push) Successful in 4m15s
Details
CD Pipeline / post-deploy-checks (push) Successful in 1m58s
Details
2026-05-12 19:55:13 +08:00
Your Name
2ef54ccc94 fix(ai): enforce ollama first for drift governance
All checks were successful
Code Review / ai-code-review (push) Successful in 16s
Details
CD Pipeline / tests (push) Successful in 1m17s
Details
CD Pipeline / build-and-deploy (push) Successful in 4m54s
Details
CD Pipeline / post-deploy-checks (push) Successful in 3m10s
Details
2026-05-06 19:26:09 +08:00
Your Name
4111ea4f9f fix(ai): remove 188 ollama provider
All checks were successful
Code Review / ai-code-review (push) Successful in 12s
Details
CD Pipeline / tests (push) Successful in 1m13s
Details
CD Pipeline / build-and-deploy (push) Successful in 3m36s
Details
CD Pipeline / post-deploy-checks (push) Successful in 1m20s
Details
2026-05-06 14:34:48 +08:00
Your Name
2aa31c205a fix(ai): require 111 before alert cloud fallback
All checks were successful
CD Pipeline / tests (push) Successful in 54s
Details
Code Review / ai-code-review (push) Successful in 10s
Details
CD Pipeline / build-and-deploy (push) Successful in 3m21s
Details
CD Pipeline / post-deploy-checks (push) Successful in 2m2s
Details
2026-05-06 00:05:51 +08:00
Your Name
ee5e3bc94f fix(openclaw): gate alert cloud fallback behind flag
Some checks failed
Code Review / ai-code-review (push) Successful in 27s
Details
CD Pipeline / tests (push) Successful in 5m17s
Details
CD Pipeline / build-and-deploy (push) Failing after 5m35s
Details
CD Pipeline / post-deploy-checks (push) Has been skipped
Details
2026-05-05 20:54:47 +08:00
Your Name
2b2359e367 fix(ai-router): ADR-110 GCP 三層容災 — 修復 Ollama 直跳 Gemini 根因
All checks were successful
Code Review / ai-code-review (push) Successful in 55s
Details
run-migration / migrate (push) Successful in 41s
Details 
根因（所有告警 Ollama 失敗直接跳 Gemini 的原因）：
AIProviderEnum 缺少 ollama_gcp_a / ollama_gcp_b / ollama_local
→ AIProviderEnum("ollama_gcp_a") 拋 ValueError
→ fallback chain 清空（所有 GCP 端點轉換全失敗）
→ failover_fallback = []（空 list，非 None）
→ fallback_chain 被覆寫為 [] 而非走 Gemini 備援
→ AIProviderRegistry.get("ollama_gcp_a") 回傳 None → not_registered → 跳過
→ 整條 Ollama 鏈（GCP-A → GCP-B → 111）全部略過，直接跳 Gemini

修復：
1. AIProviderEnum 新增 OLLAMA_GCP_A / OLLAMA_GCP_B / OLLAMA_LOCAL
2. PROVIDER_LATENCY_BUDGET 補齊三個新 enum
3. ollama.py 新增 OllamaGcpBProvider（OLLAMA_SECONDARY_URL = GCP-B 34.21.145.224）
4. _init_registry() 補登：
   - "ollama_gcp_a" alias → OllamaProvider（GCP-A，OLLAMA_URL）
   - OllamaGcpBProvider（"ollama_gcp_b"，OLLAMA_SECONDARY_URL）
   - "ollama_local" alias → Ollama188Provider（111，OLLAMA_FALLBACK_URL）

修復後路由順序：GCP-A → GCP-B → Local(111) → Gemini → Claude

2026-05-04 ogt + Claude Sonnet 4.6

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-05-04 13:49:32 +08:00
Your Name
61f5a6a419 fix(telegram): route alerts to SRE war room
Some checks failed
CD Pipeline / tests (push) Has been cancelled
Details
CD Pipeline / build-and-deploy (push) Has been cancelled
Details
CD Pipeline / post-deploy-checks (push) Has been cancelled
Details
Code Review / ai-code-review (push) Has been cancelled
Details
2026-04-30 15:01:23 +08:00
Your Name
fb0c72db42 feat(ai-router): 推翻 A2 鐵律 — DIAGNOSE primary 改 Ollama 本地優先
Some checks failed
CD Pipeline / build-and-deploy (push) Failing after 2m26s
Details 
統帥鐵律 2026-04-29：「主要優先用 111 主機的 Ollama」
+ feedback_ai_autonomous_direction.md：以本地免費 LLM 為主
+ feedback_ollama_111_only.md：Ollama 唯一主機 = 111

## 推翻 A2 (2026-04-27 INC-20260425) 的事實基礎

**舊事實**：Ollama = CPU-only deepseek-r1:14b @ 238s（不可用）
**新事實**：prod Ollama 111 = M1 Pro Apple Silicon GPU + qwen2.5:7b-instruct
           VRAM 8.2GB 全載入，ctx 32k，實測 hi prompt 0.54s

**雲端全死**（2026-04-29 prod log 證據）：
- OpenClaw 188:8088 → 500 Internal Server Error
- Gemini → 429 Too Many Requests（配額爆）
- Claude → 404 Not Found（model claude-3-haiku-20240307 過期）

**不推翻 A2 → 100% incident llm_failed → AI 自動修復永遠不啟動**

## 修改範圍（最小、安全、可驗證）

### ai_router.py
- `_diagnose_fallback_chain`: OLLAMA 第一順位（取代「永久排除」舊註解）
  順序：[OLLAMA, OPENCLAW_NEMO, GEMINI, CLAUDE]
- `_intent_provider_overrides[DIAGNOSE]`: OPENCLAW_NEMO → OLLAMA
- 不動 _full_fallback_chain（避免影響 RESTART/SCALE/CONFIG/DELETE）
- 不動 _tool_calling_fallback_chain
- 不動 complexity_map（critic M2 留待後續）

### openclaw.py
- 注入 task_type="diagnose" 到 alert_context（critic C2 真根因）
- 修復 ai_providers/ollama.py:77 timeout 對齊問題：
  - 有 task_type → OLLAMA_DIAGNOSE_TIMEOUT_SECONDS=200s
  - 沒有 → OPENCLAW_TIMEOUT=30s（不夠 qwen2.5:7b 推理）
- prod log 看到 latency_ms=120014 的根因
- 用 dict(alert_context) 複製，不污染原 context

## Regression Test 同步更新（5 個）

A2 鐵律守門 test 全部反映新鐵律：

- test_p0_diagnose_routing.py::test_diagnose_override_is_ollama
  （原 test_diagnose_override_is_openclaw_nemo）
- test_ai_router_diagnose_fallback.py::test_diagnose_fallback_chain_ollama_primary
  （原 test_diagnose_fallback_chain_no_ollama）
- test_ai_router_diagnose_fallback.py::test_diagnose_route_primary_is_ollama
  （原 test_diagnose_route_fallback_chain_excludes_ollama）
- test_ai_router_diagnose_fallback.py::test_diagnose_route_sync_primary_is_ollama
  （原 test_diagnose_route_sync_fallback_chain_excludes_ollama）
- test_ai_router_diagnose_fallback.py::test_build_fallback_chain_for_intent_diagnose_with_ollama_primary
  （原 test_build_fallback_chain_for_intent_diagnose_no_ollama）
- test_ai_router_failover_integration.py::test_router_uses_failover_for_diagnose_ollama_primary
  （原 test_router_does_not_use_failover_for_openclaw_nemo）

每個 test docstring 都記載歷史脈絡 + 推翻原因。

## 驗證

- 1608 unit tests 全綠
- LLM 路徑 16 個 test 全綠（含 6 個 A2 守門 test 更新版）
- complexity_scorer / failover_manager / intent_classifier 不受影響

## 期望 prod 行為（部署後驗證）

incident 進入 → DIAGNOSE intent → primary OLLAMA (qwen2.5:7b on M1 Pro GPU)
  失敗才 fallback → OpenClaw 188 → Gemini → Claude
  Ollama 用 200s timeout（之前 30s 不夠）
  → AI 自動修復終於可以啟動，不再 100% llm_failed

## 已知債（後續處理）

- models.json:21 ollama.default 仍是 deepseek-r1:14b（critic C1，但 prod 已自動 route 到實載 model）
- complexity 4/5 仍寫死 gemini/claude（critic M2）
- Gemini API key 在 prod log 明文（需輪換 + sanitize）

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-29 11:39:36 +08:00
Your Name
143c15f052 feat(wave2+km): LLM 動態按鈕啟用 + KM 自動寫入 + AI Router dead code 標記
All checks were successful
CD Pipeline / build-and-deploy (push) Successful in 9m52s
Details 
- ConfigMap: USE_LLM_DYNAMIC_BUTTONS=true（B2/B3/B4 handler 全就緒）
- decision_manager: auto_execute 失敗路徑補 KM fire-and-forget 寫入
- ai_router: _build_fallback_chain 標記 DEPRECATED 2026-04-28
- tests: test_golden_regression.py 新增 172 行 golden 回歸測試

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-04-28 15:27:33 +08:00
Your Name
9908fdf50d feat(p3.1-t2-patha): DiagnosisAggregator 路徑 A + Solver F4 critical reject + 對齊測試
Some checks failed
CD Pipeline / build-and-deploy (push) Failing after 1m59s
Details 
Wave 8 P3.1-T2 PathA 啟用 + Solver F4 安全強化 + test 對齊：

PathA — DiagnosisAggregator 信號分類層補 PDI:
- ENABLE_DIAGNOSIS_AGGREGATOR default=False → True
  · PathA 純信號分類層（OOMKilled/CrashLoop 等業務邏輯）
  · 不重複呼叫 K8s/SignOz API（只取 PDI 已收集的 raw 資料）
  · 安全 default on — 純邏輯處理，無外部依賴重疊
- diagnosis_aggregator.py +155 行（PathA 實作）
- pre_decision_investigator.py 已接 (commit 3a2cd151)

F4 — Solver critical risk reject:
- solver_agent.py: _validate_recommended_action 拒絕 risk=critical
  · 鐵律：critical 動作必須走人工審批，不可變 Telegram 按鈕
  · log warning + return None（被 _extract 過濾掉）
- _extract_recommended_actions 改返回 (list, status_str) tuple
  · status="ok"/"empty"/"all_invalid" 供呼叫端決策
- protocol.py +16 / metrics.py +9 / ai_router.py +18 — 配套 metric + protocol field

測試對齊:
- test_solver_recommended_actions.py 拆 test_all_valid → low/medium/high accepted +
  test_critical_rejected
- result tuple unpack: result, _ = _extract_recommended_actions(...)
- test_diagnosis_aggregator_stub.py: feature flag default 改 True 對齊 PathA

Tests: 51 passed (solver 28 + aggregator 16 + router fallback 8)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Multiple Engineers (Wave 8 P3.1-T2 PathA + F4) <noreply@anthropic.com>
 2026-04-27 14:42:29 +08:00
Your Name
595629c013 fix(inc-20260425): A1 三段 Agent timeout 拆分 + A2 DIAGNOSE 移除 Ollama
Some checks failed
CD Pipeline / build-and-deploy (push) Has been cancelled
Details 
INC-20260425-8D17BB / 3B6C39 兩則告警 AI 信心降到 20% 根因雙修（統帥批准 A+B）：

A1 — 三段 Agent step timeout 拆分（北極星 §1.2 Observable by Default）:
- diagnostician_agent.py: PHASE2_STEP_TIMEOUT_SEC=20.0 共用值 → 拆三段
  · AGENT_DIAGNOSTICIAN_TIMEOUT_SEC=30.0（NIM 主吃口，最大 prompt + 多假設）
  · AGENT_SOLVER_TIMEOUT_SEC=20.0（後續 commit 接線）
  · AGENT_CRITIC_TIMEOUT_SEC=15.0（後續 commit 接線）
  · env override 支援，K8s ConfigMap 動態調整不需 rebuild
  · 保留 PHASE2_STEP_TIMEOUT_SEC alias（DEPRECATED，下 sprint 移除）
- observability/agent_step_metrics.py (58 行) — 新模組:
  · aiops_agent_step_duration_seconds Histogram
  · observe_agent_step() helper 統一三 Agent 呼叫點
  · outcome label ∈ {success, timeout, error}
  · agent label ∈ {diagnostician, solver, critic}

A2 — ai_router DIAGNOSE chain 移除 Ollama:
- ai_router.py v4.4 by Claude Sonnet 4.6
  · 新增 _diagnose_fallback_chain: NEMO → GEMINI → CLAUDE
  · Ollama 永久排除於此 chain（CPU-only 實測 238s，二次 timeout 必爆）
  · 新增 aiops_diagnose_fallback_total Prometheus metric
- 根因: NIM timeout 後 fallback 到 Ollama deepseek-r1:14b CPU 238s
  → 二次 timeout → degraded confidence=0.2

Wave8-X2 整合測試補正:
- test_ollama_failover_manager.py: TestSelectProvider 補 mock _check_gemini_quota
  原 test 期望 OFFLINE→Gemini，但 quota fail-closed 後沒 mock 會被切到 188
  繞過 quota check 後驗純路由邏輯 → 37/37 PASS

Tests: 37 passed (test_ollama_failover_manager 全部)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 (Wave 8 INC-20260425) <noreply@anthropic.com>
 2026-04-27 08:15:10 +08:00
Your Name
02362eddcf feat(wave4-5): P1.3+P1.4 真接線 + Ollama_188 provider 註冊 + quota atomic 修復
Some checks failed
CD Pipeline / build-and-deploy (push) Failing after 2m0s
Details 
3 個 engineers 在限額前的 Wave 4/5 完成工作（補 commit）：

Engineer-B3 — Wave 4 P1.3+P1.4 真飛輪閉環（auto_repair_service.py 才是正確接線位置）:
- execute_auto_repair 成功後 fire-and-forget 啟動 PostExecutionVerifier
- record_verification_result 觸發 EWMA trust_score 演化
- snapshot=None（不依賴 EvidenceSnapshot，避免我之前 webhooks.py 補丁的 B2 bug）
- _pending_tasks 管理生命週期，Lifespan shutdown 時等任務完成

Engineer-A4 — Wave 5 B1-fix Ollama188Provider 註冊:
- ai_providers/ollama.py: 新增 Ollama188Provider(OllamaProvider) 子類
  - name="ollama_188", is_enabled 看 ENABLE_OLLAMA_188 + OLLAMA_FALLBACK_URL
  - analyze() 用 OLLAMA_FALLBACK_URL（192.168.0.188:11434）作為推理端點
- ai_router.py:_init_registry 補 registry.register(Ollama188Provider())
- 修復 BLOCKER：原本 failover_manager 決策返回 "ollama_188"，但 executor 查不到
  → not_registered → 188 從未被打到。Wave 2 P1.1 整套容災系統前段卡住。

Engineer-A4 — Wave 5 B3-fix Gemini quota TOCTOU 修復:
- ollama_failover_manager.py:_check_gemini_quota 改用 redis.pipeline()
  原 GET → 判斷 → INCR → EXPIRE 四步分離，並行請求在 GET/INCR 間競爭超發
  修法：SET NX(首次設 TTL) + INCR atomic pipeline，用 INCR 後新值判斷

Engineer-B3 — test_learning_chain_e2e.py（377 行 No-Mock 整合測試）:
- 純 Python Stub + monkeypatch（feedback_no_mock_testing.md 合規）
- execute_auto_repair 成功 → verifier 被呼叫 ✓
- execute_auto_repair 失敗 → verifier 不被呼叫 ✓
- matched_playbook_id=None → log warning 不 crash ✓
- verifier 拋例外 → 修復回傳成功，trust 不更新 ✓

Tests: 42 passed (failover_manager + ai_router_failover_integration 全綠)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Engineer-A4 + Engineer-B3 (上 session) <noreply@anthropic.com>
 2026-04-26 20:44:19 +08:00
Your Name
55c6b4e2d9 feat(p1): Ollama 多層容災系統 — P1.1 健康檢測 + P1.2 ai_router 整合 + P1.5 容災告警 
ADR-092 P1 飛輪閉環的 Ollama 失敗轉移子系統，全部 Engineer-A2/C/C2 補上。

新服務 (1581 行)：
- ollama_health_monitor.py (356)：3 層健康檢測（TCP/HTTP/推理）
- ollama_failover_manager.py (571)：111→188 自動切換 + Redis 持久化 + recovery callback
- ollama_auto_recovery.py (436)：30s 背景監控 + 連續 3 次 HEALTHY → 切回 + clear_cache
- failover_alerter.py (218)：P1.5 Telegram 容災告警

服務整合：
- ai_router.py: AIProviderEnum.OLLAMA_188 + 120s budget + failover fallback chain
- main.py lifespan: 啟動時 wire callback + start recovery，關閉時優雅 stop
- config.py: OLLAMA_FALLBACK_URL / OLLAMA_HEALTH_CHECK_MODEL / GEMINI_DAILY_QUOTA（帳單熔斷）

K8s 配置：
- 04-configmap.yaml.patch-188-fallback：注入 OLLAMA_FALLBACK_URL=http://192.168.0.188:11434

測試 (2082 行)：
- test_ollama_health_monitor.py (402)
- test_ollama_failover_manager.py (707)
- test_ollama_auto_recovery.py (580)
- test_ai_router_failover_integration.py (257)
- test_lifespan_failover_wiring.py (136)

依賴鏈：service 三件套 + ai_router + main.py 一起 commit，缺一就 ImportError。

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-26 20:18:33 +08:00
Your Name
0d81b28b1b fix(aiops): bound phase2 timeout and repair incident links
All checks were successful
E2E Health Check / e2e-health (push) Successful in 52s
Details
CD Pipeline / build-and-deploy (push) Successful in 9m24s
Details
2026-04-24 23:53:56 +08:00
Your Name
479f8d8971 refactor(tests): 技術債清零 — 移除 FakeRepo/FakeSession Mock DB 違規
Some checks failed
CD Pipeline / build-and-deploy (push) Failing after 35s
Details 
## ai_router.py
- 抽取 _aggregate_feedback_stats() 純函數，feedback_from_aider_events 呼叫它

## aider_event_processor.py
- _process_one 加 _session_factory=None DI 參數（預設 get_session_factory()）
- 可注入測試 factory，不改既有生產邏輯

## test_ai_router_feedback.py（完全重寫）
- 移除 FakeRepo/FakeSession，改為直接測試 _aggregate_feedback_stats 純函數
- 新增 test_feedback_skips_missing_model 邊界條件
- DB 失敗降級行為 test 保留（只 patch get_session_factory，無 FakeRepo）

## test_aider_event_processor.py（完全重寫）
- 移除 FakeRepo/FakeSession，改用真實 PostgreSQL（real_factory fixture）
- Redis xack + IncidentEngine 保留 mock（外部 broker/AI 服務，符合例外）
- 每個測試後 rollback，不污染 dev DB

## setup_test_schema.sql
- 補入 aider_events_payload_gin GIN index（與 adr091 生產 migration 一致）

## integration/conftest.py
- 補注解說明密碼名稱 awoooi_prod_2026 的歷史混淆
- 修正 assert 邏輯：檢查 DB 名稱而非 URL 字串，避免密碼含 prod 觸發誤判

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-04-22 01:33:30 +08:00
Your Name
40771cda6d feat(ai_router): feedback_from_aider_events read-only hook (Phase 24 A8) 2026-04-20 19:40:01 +08:00
OG T
d4fed639f6 fix(ai-router): DIAGNOSE 恢復 OPENCLAW_NEMO 路由，修復全部 timeout 降級問題
Some checks failed
CD Pipeline / build-and-deploy (push) Failing after 2m16s
Details 
根因: 2026-04-12 patch 把 DIAGNOSE 改為 None（複雜度路由）
→ 落入 Rule 6 → Ollama deepseek-r1:14b (CPU 238s) → timeout
→ 降級 20% 信心 + 「待分析」→ 全部 unknown

修復:
1. ai_router.py: DIAGNOSE → OPENCLAW_NEMO（via 188:8088 NVIDIA NIM, 2-27s）
2. ai_router.py: 移除 Rule 6 的 DIAGNOSE deepseek 特殊case（已無用）
3. 04-configmap.yaml: AI_FALLBACK_ORDER 改為 nvidia 優先
   gemini→ollama→nvidia（舊）→ nvidia→gemini→ollama（新）

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-04-16 00:03:04 +08:00
OG T
3ce5025ca7 fix(alerts): 3 個飛輪沉默節點 — DIAGNOSE routing + 心跳停用 + 通知格式
Some checks failed
CD Pipeline / build-and-deploy (push) Has been cancelled
Details 
1. openclaw.py: DIAGNOSE 移除 require_local=True
   - v4.3 已決定 NIM 為主力且無隱私問題
   - require_local=True 導致所有 provider 被 privacy_skip → 告警永遠失敗
   - 修後 DIAGNOSE 走 _full_fallback_chain（NIM → Gemini → Claude）

2. ai_router.py: require_local 失敗通知改為 ADR-075 TYPE-1 格式
   - 禁止純文字 raw notification（統帥鐵律：所有訊息必須符合格式模板）
   - 改用 ├─ / └─ 樹狀結構 + 語義化標籤

3. main.py: 停用 Telegram 心跳監控
   - 心跳已轉發到另一個 Telegram 群組，不需在此頻道重複發送

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-04-15 19:49:43 +08:00
OG T
8b7e9cbfb8 fix(BLOCKER): LLM 連續失敗 — 4 個違反設計處全部修復
All checks were successful
CD Pipeline / build-and-deploy (push) Successful in 14m21s
Details 
統帥盤點發現飛輪沉默真因：4 個違反既定架構設計的 bug 同時撞車。

P0a — Ollama timeout 違反 GAP-B4 設計
  config.py:OPENCLAW_TIMEOUT 從 120s 改 30s
  原 120s 違反 ADR-052 GAP-B4 (LLM 25s hard timeout) 設計
  致 Ollama 過載時 thread 飢餓 120s 才降級

P0b — AI Router silent skip 觀測性修復
  ai_router.py: not_registered/circuit_open/rate_limit/privacy_skip
  全部累積到 errors 陣列，log all_providers_failed 時可知為何 skip
  原本 errors=["ollama: Timeout"] 但 tried=4 個，無法診斷

P1a — send_text 方法不存在 bug
  ai_router.py:1005 tg.send_text() → tg.send_notification(parse_mode=HTML)
  TelegramGateway 只有 send_notification 沒 send_text
  致 fallback 失敗通知本身失敗（雙重靜默）

P1b — resend_stale_ready_tokens 並發爆炸
  decision_manager.py: 加 asyncio.Semaphore(5) + 200ms throttle
  原本 fire_and_forget N 個 task 同時跑，N=108 時 Ollama embedding
  全部 timeout，包括我打的 live-fire 也被擠爆
  改：max 5 並發 + 每完成喘 200ms

CD 流程審查 (Blocker 1): 完全符合 ADR-039 設計，10-15 min 是預期
不需修，是設計就需要這時間。

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
 2026-04-14 19:37:03 +08:00
OG T
0c88f6702e fix(ai-router): DIAGNOSE 強制用 deepseek-r1:14b，不用 gemma3:4b
Some checks failed
CD Pipeline / build-and-deploy (push) Has been cancelled
Details 
gemma3:4b (summary model, complexity≤1) 不輸出結構化 JSON
→ _parse_llm_response 無法提取 confidence → confidence=0.0

deepseek-r1:14b (default model) 已驗證可輸出 confidence=0.8

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-04-12 22:43:49 +08:00
OG T
09134f5c47 fix(openclaw): 修復 incident.title + DIAGNOSE→NEMOTRON confidence=0.0
Some checks failed
CD Pipeline / build-and-deploy (push) Failing after 2m10s
Details 
1. telegram_gateway.py:1169 — classify_notification() 仍用 incident.title
   改用 alertname + signal annotations 組合 (同 decision_manager.py 修法)

2. ai_router.py — DIAGNOSE 路由暫停 NEMOTRON
   NIM tool_call 返回無 confidence → openclaw_analysis_complete confidence=0.0
   改為 None (複雜度路由)，讓 Gemini/openclaw_nemo 處理 DIAGNOSE

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-04-12 22:12:55 +08:00
OG T
73577f7c5d chore(ai-router): v4.3 版本號同步 (trigger CD push event)
Some checks failed
CD Pipeline / Deploy Prometheus Alert Rules (push) Has been cancelled
Details
CD Pipeline / build-and-deploy (push) Has been cancelled
Details
2026-04-05 12:03:15 +08:00
OG T
2243a21b96 fix(ai-router): v4.3 NIM 保護 — timeout 不計 CB 失敗，每次先跑 NIM 才切 Gemini
Some checks failed
CD Pipeline / build-and-deploy (push) Failing after 20s
Details 
需求: NIM 必須等到有回應才切換，不能因為慢就被 CB 封鎖走 Gemini

變更:
- Timeout exception 不累積 CB failure（只有真實連線錯誤才計）
- NIM CB: failure_threshold=10, recovery_timeout=30s（比預設寬鬆）
- 設計文件 v4.3: 更新方向二，移除錯誤假設

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-04-05 01:51:12 +08:00
OG T
5ad403b287 fix(p0): v4.3 — 實測確認 Ollama CPU-only 不可用，DIAGNOSE 統一走 NIM 
實測依據 (2026-04-05):
- Ollama llama3.2:3b CPU-only: 238s 回 {"ok":true}，生產不可用
- Nemotron NIM: 2.2s~27.3s，avg 10.6s，一直是主力（Phase 22 起）
- NIM 從未有隱私問題，Incident 資料一直送雲端 GPU

變更:
- ai_router.py: _local_fallback_chain 廢棄（空 list）
- ai_router.py: DIAGNOSE route/route_sync 改回 _full_fallback_chain
- config.py: 更新 timeout 說明反映實測結果
- test_p0_diagnose_routing.py: 更新 docstring

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-04-05 01:49:06 +08:00
OG T
96d5e18924 fix(p0): 實測修正 — timeout 依 benchmark 調整，_local_fallback_chain 移除雲端 Nemotron 
- config.py: NEMOTRON_DIAGNOSE_TIMEOUT_SECONDS=60s (NIM 實測 11-45s + 15s buffer)
- config.py: OLLAMA_DIAGNOSE_TIMEOUT_SECONDS=200s (Ollama 實測 ~173s + 27s buffer)
- ollama.py: 新增 per-task timeout (diagnose/force_local 用 200s)
- ai_router.py: _local_fallback_chain 移除 Nemotron (NIM=雲端，不可進 local chain)
- ai_router.py: v4.2 — Option C 分情境路由正式確立

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-04-05 00:29:09 +08:00
OG T
a562db4048 fix(phase25): 首席架構師 Review C1/C2/I1/I3 修正
Some checks failed
CD Pipeline / build-and-deploy (push) Failing after 57s
Details 
C1: NemotronProvider.privacy_level cloud→local
    NIM 部署在 192.168.0.188 內網，非官方雲端 API
    可納入 DIAGNOSE _local_fallback_chain 隱私邊界

C2: adopt() 端點暫停，返回 501
    API Pod 執行 git add -A 有安全風險
    ADR-057 起草後改用 Gitea PR API 實作

I1: timeout log 修正，記錄實際套用的 timeout 值
    原本永遠記錄 NEMOTRON_TIMEOUT_SECONDS=45
    現在記錄依 task_type 選擇的正確值

I3: route_sync() 補 DIAGNOSE 隱私邊界
    async route() 已有 _local_fallback_chain
    sync 版本遺漏，此次補齊

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-04-04 18:00:05 +08:00
OG T
c4eafd2a5b fix(ai-router): fallback_models 排除 selected_model 避免重複
All checks were successful
CD Pipeline / build-and-deploy (push) Successful in 6m58s
Details 
DIAGNOSE intent 路由至 Nemotron 後，fallback_chain 中的 OPENCLAW_NEMO
也使用相同 model string，導致 fallback_models 包含已選模型。

修正: 過濾掉與 selected_model 相同的 model string。

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-04-04 17:43:44 +08:00
OG T
8056be5847 feat(ai-router): DIAGNOSE intent override 升級至 Nemotron (P0) 2026-04-04 17:41:45 +08:00
OG T
3455044457 feat(phase25): Nemotron 主動防禦三方向 P0+P1+P2 完整實作
Some checks failed
CD Pipeline / build-and-deploy (push) Failing after 38s
Details
Type Sync Check / check-type-sync (push) Failing after 35s
Details 
P0 - DIAGNOSE Privacy-First Routing:
- ai_router.py: _local_fallback_chain [NEMOTRON→OLLAMA→REJECT]
- DIAGNOSE 意圖 override 改為 NEMOTRON (原 OLLAMA)
- DIAGNOSE fallback 使用 local-only 鏈，不觸碰雲端
- 全部失敗時 REJECT + Telegram 通知
- config.py: NEMOTRON_DIAGNOSE_TIMEOUT_SECONDS=30, OLLAMA_DIAGNOSE_TIMEOUT_SECONDS=60
- nemotron.py: 根據 context[task_type] 選擇 timeout

P1 - Knowledge Auto-Harvesting:
- models/knowledge.py: EntryType.AUTO_RUNBOOK + ANTI_PATTERN + symptoms_hash
- EntryStatus.PUBLISHED (ANTI_PATTERN 直接發布，無需審核)
- models/playbook.py: SymptomPattern.compute_hash() (16字元確定性 hash)
- services/runbook_generator.py: NemotronRunbookGenerator (v1.1)
  - generate_runbook() → AUTO_RUNBOOK (DRAFT) + Telegram 審核 card
  - generate_anti_pattern() → ANTI_PATTERN (PUBLISHED) + Telegram 通知
  - 使用 nvidia.chat() (正確介面)，Nemotron 超時時 Minimal fallback
- knowledge_service.py: check_anti_pattern(symptoms_hash, days=7)
- db/models.py: symptoms_hash VARCHAR(16) + ix_knowledge_symptoms_hash
- repositories/knowledge_repository.py: create() 支援 symptoms_hash + status
- auto_repair_service.py: anti_pattern_gate 在 decide() + runbook hook 在 execute()
- migrations/phase8_symptoms_hash.sql: ALTER TABLE + partial index + PUBLISHED constraint

P2 - Config Drift Detection:
- models/drift.py: DriftItem/DriftReport/DriftLevel/DriftIntent/DriftStatus
- services/drift_detector.py: GitStateReader + K8sStateReader + DriftDetector
- services/drift_analyzer.py: 白名單過濾 + DriftLevel 分級
- services/drift_interpreter.py: NemotronDriftInterpreter（意圖分析，不生成修復指令）
- services/drift_remediator.py: rollback(kubectl apply) + adopt(git push gitea)
- api/v1/drift.py: POST /scan, GET /reports, POST /rollback, POST /adopt
- migrations/phase9_drift_reports.sql: drift_reports 表
- k8s/drift-cronjob.yaml: 每小時自動掃描 CronJob

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-04-04 12:35:05 +08:00
OG T
b225c23ad8 fix(ai_router): DIAGNOSE/ALERT_TRIAGE 改用 llama3.2:3b 避免 90秒 timeout
All checks were successful
CD Pipeline / build-and-deploy (push) Successful in 7m5s
Details 
qwen2.5:7b-instruct 在 prod 需要 >90s，導致 DIAGNOSE intent 全鏈路失敗。
llama3.2:3b (summary model) 實測 4s 回應，適合 triage 類快速判斷。

規則 3 新增特判: DIAGNOSE/ALERT_TRIAGE/QUERY → ollama summary model
不影響其他 intent 的 model 選擇邏輯。

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-04-03 12:32:01 +08:00
OG T
97d86861ed fix(ai_router): C1 修復 — AIProviderEnum 對齊 Registry 實際 Provider 名稱
Some checks failed
CD Pipeline / build-and-deploy (push) Failing after 37s
Details 
問題: AIProviderEnum.NVIDIA = "nvidia" 在 Registry 無對應 Provider
      OpenClawNemoProvider.name = "openclaw_nemo"
      NemotronProvider.name = "nemotron"
      → 高複雜度/Tool Calling 路由永遠 skip，靜默 fallback 到 Gemini/Ollama

修復:
- 新增 OPENCLAW_NEMO = "openclaw_nemo" (一般推理, via .188 → NVIDIA NIM)
- 新增 NEMOTRON = "nemotron" (Tool Calling, direct NVIDIA NIM)
- 移除 NVIDIA = "nvidia" (Registry 無對應)
- 規則 4 (複雜度>=4/HIGH風險): NVIDIA → OPENCLAW_NEMO
- route_tool_calling: NVIDIA → NEMOTRON
- Rate Limiter check: "nvidia" → "openclaw_nemo"
- _full_fallback_chain: OPENCLAW_NEMO 首位
- _tool_calling_fallback_chain: NEMOTRON 首位

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-04-02 23:31:31 +08:00
OG T
5a8aae89c4 fix(phase24): 首席架構師 Review C1/C2/C3/I4 修復
All checks were successful
CD Pipeline / build-and-deploy (push) Successful in 7m12s
Details
E2E Health Check / e2e-health (push) Successful in 18s
Details 
C1 (P0): AIRouterExecutor.execute() 補 Langfuse Trace (D5)
  - 建立 langfuse_trace("ai_router_execute") 包住整個執行鏈
  - 成功時記錄 generation (model/input/output/tokens/cost)
  - prod 所有 AI 呼叫現在有 LLMOps 追蹤

C2 (P0): 絞殺者改為呼叫 AIRouter.route() 智慧路由
  - 先取得 RoutingDecision (意圖分類 + 複雜度評分)
  - provider_order 從 selected_provider + fallback_chain 動態生成
  - D1 意圖路由矩陣、D7 隱私保護 (DIAGNOSE 強制 local) 生效

C3 (P1): 型別標注 typo 修復
  - AIProviderEnumEnum → AIProviderEnum
  - AIProviderEnumProtocol → AIProviderProtocol

I4 (P1): interfaces.py AIProvider Protocol 補 close() 定義

S1: ai_router.py 模組版本標頭更新至 v4.0

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-04-02 21:47:06 +08:00
OG T
3ad7b60f68 fix(ai): Phase 24 R1+R2 首席架構師 Review 修復 (C1-C3 + I1-I5)
Some checks failed
E2E Health Check / e2e-health (push) Successful in 18s
Details
CD Pipeline / build-and-deploy (push) Has been cancelled
Details 
Critical 修復:
- C1: AIProvider Enum 改名為 AIProviderEnum (避免與 Protocol 同名衝突)
- C2: 共用 Circuit Breaker → per-provider _SimpleCircuitBreaker
  (避免 Gemini 掛掉時 Ollama 也被擋)
- C3: cache_key 移到 try 外面 (避免 UnboundLocalError)

Important 修復:
- I1: Claude hardcode model → 用 get_model_registry()
- I2: Claude 追蹤 tokens/cost (input_tokens + output_tokens)
- I3: Ollama 追蹤 tokens (eval_count + prompt_eval_count)
- I4: Gemini temperature → 用 model_registry
- I5: AIProviderRegistry.close_all() shutdown hook

2026-04-02 ogt: Phase 24 首席架構師審查通過後修復

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-02 13:40:58 +08:00
OG T
73e8f8ab77 feat(ai): Phase 24-A+B1 — AI Provider Registry + 絞殺者包裝 (ADR-052)
Some checks failed
E2E Health Check / e2e-health (push) Successful in 16s
Details
CD Pipeline / build-and-deploy (push) Has been cancelled
Details 
Brain Layer 雙軌 Registry 架構:
- 新建 src/services/ai_providers/ 目錄 (interfaces + 4 providers)
  - OllamaProvider (local, rca/chat/code_review)
  - GeminiProvider (cloud, rca/chat)
  - ClaudeProvider (cloud, rca/chat/code_review)
  - OpenClawNemoProvider (cloud, rca — 委派 188→NIM)
- 擴展 ai_router.py 加入:
  - AIProviderRegistry (動態註冊/啟停)
  - AIRouterExecutor (Cache + 閘門 CB/RL/Sem + 執行)
- openclaw.py 絞殺者包裝: USE_AI_ROUTER=true 走新路徑
- config.py + ConfigMap 加入 USE_AI_ROUTER=false (安全預設)
- ADR-052 正式文件 (14 項決策 D1-D14)
- HARD_RULES v1.7 加入 AI Router 規範

安全: USE_AI_ROUTER=false 預設不啟用，需手動開啟觀察
回滾: kubectl set env deployment/awoooi-api USE_AI_ROUTER=false

2026-04-02 ogt: Phase 24 首批實作

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-02 13:16:09 +08:00
OG T
1f9e94e78d refactor(ai-router): 新增 IAIRouter Protocol (P1 修復) 
首席架構師審查 P1 修復:
- 新增 IAIRouter Protocol 支援 DI 測試替換
- 參考 IModelRegistry, IComplexityScorer 實作模式
- 包含 route(), route_sync(), route_tool_calling() 方法簽名

審查評分: 78/100 → 85/100

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
 2026-03-31 21:23:07 +08:00
OG T
000533d32e feat(ai): promote Nvidia nemotron as default arbitrator for high complexity/risk incidents
All checks were successful
CD Pipeline / build-and-deploy (push) Successful in 4m35s
Details
E2E Health Check / e2e-health (push) Successful in 16s
Details
2026-03-30 00:26:53 +08:00
OG T
b77e151387 feat(ai): ADR-036 NVIDIA Nemotron Tool Calling 整合 
Phase 20 - 提升 Tool Calling 精準度 50% → 83.3%

新增:
- src/models/nvidia.py: Pydantic Schema
- src/services/nvidia_provider.py: NvidiaProvider 類別
- tests/test_nvidia_provider.py: 15 項單元測試 (全部通過)

修改:
- ai_router.py: AIProvider.NVIDIA + route_tool_calling()
- ai_rate_limiter.py: NVIDIA 限制 (5 RPM, 100/day)
- models.json: NVIDIA 配置
- cd.yaml: Secrets 注入 NVIDIA_API_KEY

路由策略:
- Tool Calling: Nemotron → Gemini → Claude
- 一般對話: Ollama → Gemini → Claude (不變)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
 2026-03-29 00:00:08 +08:00
OG T
579da38b8b feat(api): Phase 13 智能路由 + CI/CD 整合 (#74-88) 
Phase 13.1 CI/CD Integration:
- #76 workflow_run handler for CI failure diagnosis
- #77 SignOz log query (query_logs, error_logs_summary MCP)
- #78 CIAutoRepairService with risk-based execution decisions

Phase 13.3 Smart Routing:
- #85 Intent Classifier v2.0 (rule engine + LLM fallback)
- #86 Complexity Scorer (9-dimension scoring)
- #87 AI Router v3.0 (routing decision matrix)
- #88 Token Counter (OTEL + Langfuse integration)

New files:
- services/ci_auto_repair.py (risk stratification)
- services/model_registry.py (centralized model config)
- services/token_counter.py (677 lines)
- Skill 08: Model Router Expert
- Skill 09: Strangler Pattern Expert
- ADR-023: Smart Routing Architecture
- ADR-024: API Layer Architecture

Tests:
- phase11-conversational.spec.ts (E2E tests)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
 2026-03-26 15:32:52 +08:00
OG T
58b4004a18 feat(api): Phase 13.3 智能路由 (#85-87) 
- IntentClassifier: 意圖分類 (告警/部署/查詢/維運/審查)
- ComplexityScorer: 複雜度評分 (1-5 分)
- AIRouter: 動態模型選擇 (整合 Intent + Complexity)
- 測試: 完整單元測試覆蓋

Phase 13.3 設計: project_phase13_3_smart_router.md

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
 2026-03-26 10:01:04 +08:00