fix(BLOCKER): LLM 連續失敗 — 4 個違反設計處全部修復
All checks were successful
CD Pipeline / build-and-deploy (push) Successful in 14m21s
All checks were successful
CD Pipeline / build-and-deploy (push) Successful in 14m21s
統帥盤點發現飛輪沉默真因:4 個違反既定架構設計的 bug 同時撞車。 P0a — Ollama timeout 違反 GAP-B4 設計 config.py:OPENCLAW_TIMEOUT 從 120s 改 30s 原 120s 違反 ADR-052 GAP-B4 (LLM 25s hard timeout) 設計 致 Ollama 過載時 thread 飢餓 120s 才降級 P0b — AI Router silent skip 觀測性修復 ai_router.py: not_registered/circuit_open/rate_limit/privacy_skip 全部累積到 errors 陣列,log all_providers_failed 時可知為何 skip 原本 errors=["ollama: Timeout"] 但 tried=4 個,無法診斷 P1a — send_text 方法不存在 bug ai_router.py:1005 tg.send_text() → tg.send_notification(parse_mode=HTML) TelegramGateway 只有 send_notification 沒 send_text 致 fallback 失敗通知本身失敗(雙重靜默) P1b — resend_stale_ready_tokens 並發爆炸 decision_manager.py: 加 asyncio.Semaphore(5) + 200ms throttle 原本 fire_and_forget N 個 task 同時跑,N=108 時 Ollama embedding 全部 timeout,包括我打的 live-fire 也被擠爆 改:max 5 並發 + 每完成喘 200ms CD 流程審查 (Blocker 1): 完全符合 ADR-039 設計,10-15 min 是預期 不需修,是設計就需要這時間。 Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
This commit is contained in:
@@ -895,16 +895,20 @@ class AIRouterExecutor:
|
||||
for provider_name in provider_order:
|
||||
provider = self._registry.get(provider_name)
|
||||
if not provider:
|
||||
# 2026-04-14 Claude Sonnet 4.6: silent skip 改 errors 累積(觀測性)
|
||||
errors.append(f"{provider_name}: not_registered")
|
||||
continue
|
||||
|
||||
# 隱私過濾 (D7)
|
||||
if require_local and provider.privacy_level != "local":
|
||||
errors.append(f"{provider_name}: privacy_skip(non_local)")
|
||||
continue
|
||||
|
||||
# 閘門 1: Circuit Breaker (per-provider, C2 修復)
|
||||
cb = self._get_circuit_breaker(provider_name)
|
||||
if cb.is_open():
|
||||
logger.debug("ai_router_circuit_open", provider=provider_name)
|
||||
errors.append(f"{provider_name}: circuit_open")
|
||||
logger.warning("ai_router_circuit_open", provider=provider_name)
|
||||
continue
|
||||
|
||||
# 閘門 2: Rate Limiter
|
||||
@@ -915,6 +919,7 @@ class AIRouterExecutor:
|
||||
rate_limiter = get_ai_rate_limiter()
|
||||
allowed, reason = await rate_limiter.check_and_increment(provider_name)
|
||||
if not allowed:
|
||||
errors.append(f"{provider_name}: rate_limit({reason})")
|
||||
logger.info("ai_router_rate_limited", provider=provider_name, reason=reason)
|
||||
continue
|
||||
except Exception as e:
|
||||
@@ -1001,11 +1006,13 @@ class AIRouterExecutor:
|
||||
from src.services.telegram_gateway import get_telegram_gateway
|
||||
tg = get_telegram_gateway()
|
||||
import asyncio as _asyncio
|
||||
# 2026-04-14 Claude Sonnet 4.6: send_text 方法不存在,改 send_notification
|
||||
_asyncio.create_task(
|
||||
tg.send_text(
|
||||
tg.send_notification(
|
||||
"⚠️ <b>DIAGNOSE 本地 Provider 不可用</b>\n"
|
||||
f"已嘗試: {', '.join(provider_order)}\n"
|
||||
"需要人工介入,雲端 Provider 不會被呼叫(隱私邊界)。"
|
||||
"需要人工介入,雲端 Provider 不會被呼叫(隱私邊界)。",
|
||||
parse_mode="HTML",
|
||||
)
|
||||
)
|
||||
except Exception as _tg_e:
|
||||
|
||||
Reference in New Issue
Block a user