fix(ai-router): v4.3 NIM 保護 — timeout 不計 CB 失敗,每次先跑 NIM 才切 Gemini
Some checks failed
CD Pipeline / build-and-deploy (push) Failing after 20s
Some checks failed
CD Pipeline / build-and-deploy (push) Failing after 20s
需求: NIM 必須等到有回應才切換,不能因為慢就被 CB 封鎖走 Gemini 變更: - Timeout exception 不累積 CB failure(只有真實連線錯誤才計) - NIM CB: failure_threshold=10, recovery_timeout=30s(比預設寬鬆) - 設計文件 v4.3: 更新方向二,移除錯誤假設 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -787,7 +787,16 @@ class AIRouterExecutor:
|
||||
def _get_circuit_breaker(self, name: str) -> "_SimpleCircuitBreaker":
|
||||
"""取得 Provider 的 Circuit Breaker (per-provider, lazy init)"""
|
||||
if name not in self._circuit_breakers:
|
||||
self._circuit_breakers[name] = _SimpleCircuitBreaker(name)
|
||||
# 2026-04-05 Claude Code: v4.3 — NIM 使用更寬鬆的 CB 參數
|
||||
# 每次都先跑 NIM,只有真正連線錯誤(非 timeout)才累積失敗
|
||||
# failure_threshold=10: 需要 10 次真實錯誤才 OPEN(timeout 不計)
|
||||
# recovery_timeout=30: 30s 後進入 half-open,立即重試 NIM
|
||||
if name == "nemotron":
|
||||
self._circuit_breakers[name] = _SimpleCircuitBreaker(
|
||||
name, failure_threshold=10, recovery_timeout=30.0
|
||||
)
|
||||
else:
|
||||
self._circuit_breakers[name] = _SimpleCircuitBreaker(name)
|
||||
return self._circuit_breakers[name]
|
||||
|
||||
@staticmethod
|
||||
@@ -961,7 +970,12 @@ class AIRouterExecutor:
|
||||
except Exception as e:
|
||||
errors.append(f"{provider_name}: {e}")
|
||||
logger.warning("ai_router_provider_exception", provider=provider_name, error=str(e))
|
||||
cb.record_failure()
|
||||
# 2026-04-05 Claude Code: v4.3 — Timeout 不計 CB 失敗
|
||||
# NIM 偶爾 GPU 忙碌導致 27s,timeout 不代表 NIM 故障
|
||||
# 只有明確連線錯誤(非 timeout)才累積 CB 失敗次數
|
||||
import httpx as _httpx
|
||||
if not isinstance(e, _httpx.TimeoutException):
|
||||
cb.record_failure()
|
||||
|
||||
# 全部失敗
|
||||
logger.error("ai_router_all_providers_failed", tried=provider_order, errors=errors)
|
||||
|
||||
Reference in New Issue
Block a user