feat: Ollama 本機 Tool Calling 取代 NVIDIA 雲端 (44s→~5s)
Some checks failed
CD Pipeline / build-and-deploy (push) Has been cancelled
Some checks failed
CD Pipeline / build-and-deploy (push) Has been cancelled
- nvidia_provider.py: 新增 OllamaToolProvider - 實作 INvidiaProvider protocol,打 Ollama /v1/chat/completions - 模型: llama3.1:8b (tool calling 最穩定的 8B) - 延遲: 44s → ~5s(本機 M1 Pro 192.168.0.111) - get_nvidia_provider() 根據 USE_OLLAMA_TOOL_CALLING 切換 - config.py: USE_OLLAMA_TOOL_CALLING=True (預設開啟), OLLAMA_TOOL_MODEL=llama3.1:8b - 回退: USE_OLLAMA_TOOL_CALLING=False → 恢復 NvidiaProvider 雲端 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -261,6 +261,15 @@ class Settings(BaseSettings):
|
||||
default="",
|
||||
description="NVIDIA NIM API key for Nemotron Tool Calling (ADR-036)",
|
||||
)
|
||||
# 2026-04-09 Claude Sonnet 4.6: Ollama Tool Calling — 替代 NVIDIA 雲端,本機推理
|
||||
USE_OLLAMA_TOOL_CALLING: bool = Field(
|
||||
default=True,
|
||||
description="使用 Ollama 本機做 Tool Calling,取代 NVIDIA NIM 雲端 (44s→5s)",
|
||||
)
|
||||
OLLAMA_TOOL_MODEL: str = Field(
|
||||
default="llama3.1:8b",
|
||||
description="Ollama Tool Calling 模型 (支援 function calling 格式)",
|
||||
)
|
||||
|
||||
@field_validator("AI_FALLBACK_ORDER", mode="before")
|
||||
@classmethod
|
||||
|
||||
Reference in New Issue
Block a user