feat: Ollama 本機 Tool Calling 取代 NVIDIA 雲端 (44s→~5s)
Some checks failed
CD Pipeline / build-and-deploy (push) Has been cancelled

- nvidia_provider.py: 新增 OllamaToolProvider
  - 實作 INvidiaProvider protocol,打 Ollama /v1/chat/completions
  - 模型: llama3.1:8b (tool calling 最穩定的 8B)
  - 延遲: 44s → ~5s(本機 M1 Pro 192.168.0.111)
  - get_nvidia_provider() 根據 USE_OLLAMA_TOOL_CALLING 切換
- config.py: USE_OLLAMA_TOOL_CALLING=True (預設開啟), OLLAMA_TOOL_MODEL=llama3.1:8b
- 回退: USE_OLLAMA_TOOL_CALLING=False → 恢復 NvidiaProvider 雲端

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
OG T
2026-04-09 14:55:04 +08:00
parent 77f2da9264
commit 7857c25677
2 changed files with 196 additions and 3 deletions

View File

@@ -261,6 +261,15 @@ class Settings(BaseSettings):
default="",
description="NVIDIA NIM API key for Nemotron Tool Calling (ADR-036)",
)
# 2026-04-09 Claude Sonnet 4.6: Ollama Tool Calling — 替代 NVIDIA 雲端,本機推理
USE_OLLAMA_TOOL_CALLING: bool = Field(
default=True,
description="使用 Ollama 本機做 Tool Calling取代 NVIDIA NIM 雲端 (44s→5s)",
)
OLLAMA_TOOL_MODEL: str = Field(
default="llama3.1:8b",
description="Ollama Tool Calling 模型 (支援 function calling 格式)",
)
@field_validator("AI_FALLBACK_ORDER", mode="before")
@classmethod