Your Name
|
4111ea4f9f
|
fix(ai): remove 188 ollama provider
Code Review / ai-code-review (push) Successful in 12s
CD Pipeline / tests (push) Successful in 1m13s
CD Pipeline / build-and-deploy (push) Successful in 3m36s
CD Pipeline / post-deploy-checks (push) Successful in 1m20s
|
2026-05-06 14:34:48 +08:00 |
|
Your Name
|
b1ef05fa8c
|
feat(ollama): ADR-110 GCP 三層容災架構（GCP-A → GCP-B → Local → Gemini）
Code Review / ai-code-review (push) Successful in 50s
CD Pipeline / tests (push) Failing after 1m14s
CD Pipeline / build-and-deploy (push) Has been skipped
CD Pipeline / post-deploy-checks (push) Has been skipped
## 變更摘要
- Primary: http://34.143.170.20:11434 (GCP-A SSD, 9x 載速 + 2x 推理)
- Secondary: http://34.21.145.224:11434 (GCP-B SSD)
- Fallback: http://192.168.0.111:11434 (M1 Pro Local HDD，最後防線)
- 廢止 ADR-105「111 唯一鐵律」，新建 ADR-110
## 核心改動
- config.py: 新增 OLLAMA_SECONDARY_URL；validator 加 GCP IP 白名單（34.143.170.20, 34.21.145.224）
- ollama_failover_manager.py: 三層 Ollama 決策矩陣；並行健康檢查三台；health_111 → health_gcp_a
- ollama_health_monitor.py: host label 萃取改為通用版（支援 GCP 公網 IP）
- failover_alerter.py: 故障/恢復主機動態顯示，不再硬編碼「Ollama 111 (GPU)」
- ollama_auto_recovery.py: notify_recovery 改為 ollama_gcp_a；recovered_host 動態
- k8s/awoooi-prod: configmap + deployment + network-policy 同步更新（egress 加 GCP /32）
- 服務層: 10 個服務檔案硬編碼 192.168.0.111 改為讀 settings.OLLAMA_URL
- 測試: URL 常數更新，新增三層容災場景，GCP IP 白名單驗證測試
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
2026-05-03 22:49:23 +08:00 |
|
Your Name
|
2c57b71db9
|
feat(wave5-p2): GovernanceAgent 4 項自檢 + Ollama 健康告警規則 + Prometheus metrics 整合
CD Pipeline / build-and-deploy (push) Successful in 10m45s
MASTER plan_complete_v3.md Wave 5 P2.2 + P2.3 完成（multiple engineers 在限額前完成代碼，補 commit）：
P2.2 — GovernanceAgent 4 項自檢:
- governance_agent.py (342 行) — 每 1 小時自檢循環:
· trust_drift（信任度漂移檢測）
· knowledge_degradation（知識退化檢測）
· llm_hallucination（LLM 幻覺檢測）
· execution_blast_radius（執行爆炸半徑檢測）
- main.py lifespan: asyncio.create_task(run_governance_loop()) 啟動
try/except 包裹，schedule 失敗不阻斷主流程
- failover_alerter.py: alert_governance(event_type, payload) 1h dedup
四類事件 → Telegram MarkdownV2 告警
P2.3 — Ollama 健康規則 + Prometheus Metrics:
- ops/monitoring/ollama_health_rules.yaml (148 行):
· OllamaHealthDegraded / OllamaPrimaryDown
· OllamaFailoverTriggered / GeminiQuotaExceeded
· 補 Prometheus 取資料的 alert rules
- core/metrics.py (57 行):
· GEMINI_DAILY_CALL_COUNT / GEMINI_DAILY_QUOTA Gauge
· OLLAMA_FAILOVER_TRIGGERED_TOTAL Counter
· OLLAMA_CURRENT_PRIMARY_IS_OLLAMA Gauge
- ollama_failover_manager.py:
· _check_gemini_quota: 每次 check 同步更新 Gauge（讓 Prometheus 取最新值）
· select_provider: failover 時 inc Counter + 切 Primary Gauge
· try/except 包裹，metric 失敗不阻斷主路由
E2E 測試:
- test_failover_e2e_dispatch.py (365 行)
完整 dispatch 路徑：health check → failover decide → alerter → metrics
Tests: 54 passed (e2e_dispatch + failover_manager + failover_alerter)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Multiple Engineers (上 session Wave 5) <noreply@anthropic.com>
|
2026-04-26 20:56:19 +08:00 |
|