fix(ai-router): v4.3 NIM 保護 — timeout 不計 CB 失敗,每次先跑 NIM 才切 Gemini
Some checks failed
CD Pipeline / build-and-deploy (push) Failing after 20s

需求: NIM 必須等到有回應才切換,不能因為慢就被 CB 封鎖走 Gemini

變更:
- Timeout exception 不累積 CB failure(只有真實連線錯誤才計)
- NIM CB: failure_threshold=10, recovery_timeout=30s(比預設寬鬆)
- 設計文件 v4.3: 更新方向二,移除錯誤假設

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
OG T
2026-04-05 01:51:12 +08:00
parent 5ad403b287
commit 2243a21b96
2 changed files with 46 additions and 89 deletions

View File

@@ -787,7 +787,16 @@ class AIRouterExecutor:
def _get_circuit_breaker(self, name: str) -> "_SimpleCircuitBreaker":
"""取得 Provider 的 Circuit Breaker (per-provider, lazy init)"""
if name not in self._circuit_breakers:
self._circuit_breakers[name] = _SimpleCircuitBreaker(name)
# 2026-04-05 Claude Code: v4.3 — NIM 使用更寬鬆的 CB 參數
# 每次都先跑 NIM只有真正連線錯誤非 timeout才累積失敗
# failure_threshold=10: 需要 10 次真實錯誤才 OPENtimeout 不計)
# recovery_timeout=30: 30s 後進入 half-open立即重試 NIM
if name == "nemotron":
self._circuit_breakers[name] = _SimpleCircuitBreaker(
name, failure_threshold=10, recovery_timeout=30.0
)
else:
self._circuit_breakers[name] = _SimpleCircuitBreaker(name)
return self._circuit_breakers[name]
@staticmethod
@@ -961,7 +970,12 @@ class AIRouterExecutor:
except Exception as e:
errors.append(f"{provider_name}: {e}")
logger.warning("ai_router_provider_exception", provider=provider_name, error=str(e))
cb.record_failure()
# 2026-04-05 Claude Code: v4.3 — Timeout 不計 CB 失敗
# NIM 偶爾 GPU 忙碌導致 27stimeout 不代表 NIM 故障
# 只有明確連線錯誤(非 timeout才累積 CB 失敗次數
import httpx as _httpx
if not isinstance(e, _httpx.TimeoutException):
cb.record_failure()
# 全部失敗
logger.error("ai_router_all_providers_failed", tried=provider_order, errors=errors)