feat(ollama): ADR-110 GCP 三層容災架構(GCP-A → GCP-B → Local → Gemini)
Some checks failed
Code Review / ai-code-review (push) Successful in 50s
CD Pipeline / tests (push) Failing after 1m14s
CD Pipeline / build-and-deploy (push) Has been skipped
CD Pipeline / post-deploy-checks (push) Has been skipped

## 變更摘要
- Primary: http://34.143.170.20:11434 (GCP-A SSD, 9x 載速 + 2x 推理)
- Secondary: http://34.21.145.224:11434 (GCP-B SSD)
- Fallback: http://192.168.0.111:11434 (M1 Pro Local HDD,最後防線)
- 廢止 ADR-105「111 唯一鐵律」,新建 ADR-110

## 核心改動
- config.py: 新增 OLLAMA_SECONDARY_URL;validator 加 GCP IP 白名單(34.143.170.20, 34.21.145.224)
- ollama_failover_manager.py: 三層 Ollama 決策矩陣;並行健康檢查三台;health_111 → health_gcp_a
- ollama_health_monitor.py: host label 萃取改為通用版(支援 GCP 公網 IP)
- failover_alerter.py: 故障/恢復主機動態顯示,不再硬編碼「Ollama 111 (GPU)」
- ollama_auto_recovery.py: notify_recovery 改為 ollama_gcp_a;recovered_host 動態
- k8s/awoooi-prod: configmap + deployment + network-policy 同步更新(egress 加 GCP /32)
- 服務層: 10 個服務檔案硬編碼 192.168.0.111 改為讀 settings.OLLAMA_URL
- 測試: URL 常數更新,新增三層容災場景,GCP IP 白名單驗證測試

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Your Name
2026-05-03 22:49:23 +08:00
parent e45b055e0e
commit b1ef05fa8c
28 changed files with 816 additions and 355 deletions

View File

@@ -285,32 +285,39 @@ class Settings(BaseSettings):
# ==========================================================================
# External Services - Four Host Architecture
# ==========================================================================
# 2026-05-03 ogt: GCP 三層容災ADR-110GCP-A → GCP-B → Local → Gemini
OLLAMA_URL: str = Field(
default="http://192.168.0.111:11434", # 2026-04-08 ogt: 切換至 M1 Pro (40+ tok/s vs 0.45 tok/s)
description="Ollama LLM service URL",
default="http://34.143.170.20:11434", # 2026-05-03 ogt: 切換至 GCP-A SSD 主機9x 載速 + 2x 推理)
description="Ollama LLM service URL (GCP-A Primary)",
)
# 2026-04-25 Claude Engineer-C (P1.1): Ollama 188 CPU-only 備援 (方案 C)
# 若空字串則 OllamaFailoverManager 僅使用 OLLAMA_URL單節點模式
# 2026-05-03 ogt: GCP-B SSD 備援ADR-110 三層容災第二層)
OLLAMA_SECONDARY_URL: str = Field(
default="http://34.21.145.224:11434", # 2026-05-03 ogt: GCP-B SSD 備援
description="Ollama LLM secondary URL (GCP-B Secondary)",
)
# 2026-05-03 ogt: Local HDD 最後防線(原 2026-04-08 M1 Pro 主機降為第三層)
OLLAMA_FALLBACK_URL: str = Field(
default="",
description="Ollama CPU-only fallback URL (188 備援P1.1),空字串=停用",
default="http://192.168.0.111:11434", # 2026-05-03 ogt: M1 Pro Local HDD 最後防線
description="Ollama local fallback URL (Local HDD, 最後防線)",
)
# 2026-04-27 Wave8-X2 by Claude — vuln #1 URL endpoint poisoning 修復
# 攻擊情境:攻擊者改 ConfigMap OLLAMA_FALLBACK_URL=http://attacker.com:11434
# → ai_router 盲信 → C&C 通道。修法:啟動時拒絕非私網/loopback 的外部 URL。
@field_validator("OLLAMA_URL", "OLLAMA_FALLBACK_URL")
# 2026-05-03 ogt: 擴充 validator 覆蓋 OLLAMA_SECONDARY_URL新增 GCP IP 白名單ADR-110
@field_validator("OLLAMA_URL", "OLLAMA_SECONDARY_URL", "OLLAMA_FALLBACK_URL")
@classmethod
def _validate_ollama_url(cls, v: str) -> str:
"""
Ollama URL 安全校驗:拒絕非 private/loopback IP 或非已知服務名稱的 URL。
允許:
- 空字串(未設定OLLAMA_FALLBACK_URL 預設空字串
- 空字串(未設定)
- 已知 Kubernetes Service hostname 白名單
- 私網 IPRFC 1918或 loopback127.x.x.x
- GCP 核准公網 IP 白名單ADR-110 GCP-A / GCP-B
拒絕:
- 公網 IP8.8.8.8
- 非白名單公網 IP8.8.8.8
- 外部域名attacker.com
"""
if not v:
@@ -337,6 +344,16 @@ class Settings(BaseSettings):
if host in _ALLOWED_HOSTNAMES:
return v
# GCP 核准公網 IP 白名單ADR-1102026-05-03 ogt
# GCP-A: 34.143.170.20SSD, 9x 載速)
# GCP-B: 34.21.145.224SSD, 9x 載速)
_ALLOWED_PUBLIC_IPS: frozenset[str] = frozenset({
"34.143.170.20", # GCP-A Ollama Primary (SSD)
"34.21.145.224", # GCP-B Ollama Secondary (SSD)
})
if host in _ALLOWED_PUBLIC_IPS:
return v
# 否則必須是 private/loopback IP
try:
ip = ipaddress.ip_address(host)
@@ -348,7 +365,7 @@ class Settings(BaseSettings):
)
if not (ip.is_private or ip.is_loopback):
raise ValueError(
f"OLLAMA URL 必須是私網/loopback IP已知 K8s SVC"
f"OLLAMA URL 必須是私網/loopback IP已知 K8s SVC 或 GCP 白名單 IP"
f"收到公網 IP {host!r}{v!r}),可能是端點中毒攻擊"
)
return v