226 lines
5.1 KiB
Markdown
226 lines
5.1 KiB
Markdown
# GCP Ollama Nginx Proxy 部署指南
|
||
|
||
> ADR-110 三層容災 — 啟用 GCP Ollama 的關鍵步驟
|
||
>
|
||
> 2026-05-05 修正：此 runbook 只保留為過渡 / rollback bridge。正式方案改用
|
||
> ADR-125 的 WireGuard private mesh 與 AwoooP Inference Gateway。新部署不得把
|
||
> GCP `11434/tcp` 對 `0.0.0.0/0` 長期開放。
|
||
|
||
---
|
||
|
||
## 背景
|
||
|
||
GCP Ollama (34.143.170.20 / 34.21.145.224) 已部署完成。
|
||
最初透過 192.168.0.110 (DevOps 金庫) 架設 nginx 反向代理，讓 K3s Pod 走內網連線 GCP Ollama。
|
||
|
||
2026-05-06 現況：K3s Pod 已可直連 GCP-A/GCP-B `11434/tcp`，且 production
|
||
暫時改用 direct endpoint，避開 110 上舊 `conf.d/ollama-gcp-proxy.conf`
|
||
的 120s `proxy_read_timeout`。正式長期方案仍是 ADR-125 WireGuard mesh。
|
||
|
||
---
|
||
|
||
## 部署檔案
|
||
|
||
| 檔案 | 用途 |
|
||
|-----|------|
|
||
| `infra/ansible/roles/nginx/templates/110-ollama-proxy.conf.j2` | nginx 配置模板 |
|
||
| `infra/ansible/playbooks/nginx-sync.yml` | Ansible Playbook |
|
||
|
||
---
|
||
|
||
## 執行部署
|
||
|
||
```bash
|
||
# 1. 進入 Ansible 目錄
|
||
cd /Users/ogt/awoooi/infra/ansible
|
||
|
||
# 2. 部署到 110 (Dry-run 先驗證)
|
||
ansible-playbook -i inventory/hosts.yml playbooks/nginx-sync.yml --tags 110 --check
|
||
|
||
# 3. 正式部署
|
||
ansible-playbook -i inventory/hosts.yml playbooks/nginx-sync.yml --tags 110
|
||
```
|
||
|
||
---
|
||
|
||
## 驗證部署
|
||
|
||
### 從 110 本機驗證
|
||
|
||
```bash
|
||
# 測試 GCP-A proxy
|
||
curl http://127.0.0.1:11435/api/tags
|
||
|
||
# 測試 GCP-B proxy
|
||
curl http://127.0.0.1:11436/api/tags
|
||
```
|
||
|
||
### 從 K3s Node 驗證
|
||
|
||
```bash
|
||
# 進入 K3s node (120 或 121)
|
||
ssh wooo@192.168.0.120
|
||
|
||
# 測試連線 110 proxy
|
||
curl http://192.168.0.110:11435/api/tags
|
||
curl http://192.168.0.110:11436/api/tags
|
||
```
|
||
|
||
### 從 K8s Pod 驗證
|
||
|
||
```bash
|
||
# 進入 API Pod
|
||
kubectl exec -it -n awoooi-prod deployment/awoooi-api -- bash
|
||
|
||
# 測試連線
|
||
apt-get update && apt-get install -y curl
|
||
curl http://192.168.0.110:11435/api/tags
|
||
```
|
||
|
||
---
|
||
|
||
## 啟用 GCP Ollama
|
||
|
||
代理部署完成後，修改 ConfigMap 啟用 GCP 端點：
|
||
|
||
```bash
|
||
# 編輯 ConfigMap
|
||
kubectl edit configmap -n awoooi-prod awoooi-config
|
||
```
|
||
|
||
若使用 nginx bridge，修改以下欄位：
|
||
|
||
```yaml
|
||
# 修改前
|
||
OLLAMA_URL: "http://192.168.0.111:11434"
|
||
OLLAMA_SECONDARY_URL: "http://192.168.0.110:11435"
|
||
OLLAMA_FALLBACK_URL: "http://192.168.0.110:11436"
|
||
|
||
# 修改後 (啟用 GCP-A 作為 Primary)
|
||
OLLAMA_URL: "http://192.168.0.110:11435" # GCP-A via proxy
|
||
OLLAMA_SECONDARY_URL: "http://192.168.0.110:11436" # GCP-B via proxy
|
||
OLLAMA_FALLBACK_URL: "http://192.168.0.111:11434" # Local GPU 最後防線
|
||
```
|
||
|
||
重啟 Deployment：
|
||
|
||
```bash
|
||
kubectl rollout restart deployment/awoooi-api -n awoooi-prod
|
||
```
|
||
|
||
若需要繞過 110 bridge timeout，使用 direct GCP endpoint：
|
||
|
||
```yaml
|
||
OLLAMA_URL: "http://34.143.170.20:11434"
|
||
OLLAMA_SECONDARY_URL: "http://34.21.145.224:11434"
|
||
OLLAMA_FALLBACK_URL: "http://192.168.0.111:11434"
|
||
OLLAMA_DIAGNOSE_TIMEOUT_SECONDS: "300"
|
||
INCIDENT_LLM_TIMEOUT_SECONDS: "360"
|
||
```
|
||
|
||
---
|
||
|
||
## 確認模型已載入
|
||
|
||
GCP Ollama 必須已載入以下模型：
|
||
|
||
```bash
|
||
# GCP-A 檢查
|
||
curl http://34.143.170.20:11434/api/tags | jq '.models[].name'
|
||
|
||
# 必須包含:
|
||
# - bge-m3:latest (embedding)
|
||
# - qwen2.5:7b-instruct (health check)
|
||
# - qwen3:14b (RCA analysis)
|
||
# - hermes3:latest (tool calling)
|
||
# - deepseek-r1:14b (reasoning)
|
||
```
|
||
|
||
若模型未載入，SSH 到 GCP 主機執行：
|
||
|
||
```bash
|
||
ollama pull bge-m3:latest
|
||
ollama pull qwen2.5:7b-instruct
|
||
ollama pull qwen3:14b
|
||
ollama pull hermes3:latest
|
||
ollama pull deepseek-r1:14b
|
||
```
|
||
|
||
---
|
||
|
||
## 部署檢查清單
|
||
|
||
- [ ] Ansible playbook 執行成功 (110)
|
||
- [ ] 110:11435 監聽確認 (`ss -tlnp | grep 11435`)
|
||
- [ ] 110:11436 監聽確認 (`ss -tlnp | grep 11436`)
|
||
- [ ] K3s node 可連線 110:11435/11436
|
||
- [ ] K8s Pod 可連線 110:11435/11436
|
||
- [ ] GCP-A/B 模型已載入
|
||
- [ ] ConfigMap 已修改
|
||
- [ ] Deployment 已重啟
|
||
- [ ] API Pod 啟動無錯誤
|
||
- [ ] 推理測試成功 (檢查 latency < 10s)
|
||
|
||
---
|
||
|
||
## 常見問題
|
||
|
||
### 1. K3s Pod 連線被拒絕
|
||
|
||
檢查 NetworkPolicy：
|
||
```bash
|
||
kubectl describe networkpolicy -n awoooi-prod allow-required-egress
|
||
```
|
||
|
||
確認包含：
|
||
```yaml
|
||
- to:
|
||
- ipBlock:
|
||
cidr: 192.168.0.110/32
|
||
ports:
|
||
- protocol: TCP
|
||
port: 11435
|
||
- protocol: TCP
|
||
port: 11436
|
||
```
|
||
|
||
### 2. nginx 無法連線 GCP
|
||
|
||
檢查 110 外網連線：
|
||
```bash
|
||
curl -v http://34.143.170.20:11434/api/tags
|
||
```
|
||
|
||
若失敗，只允許短時間確認 GCP 防火牆是否對 110 的固定出口 IP 開放
|
||
`11434/tcp`。不得把 `0.0.0.0/0:11434` 當成正式設定。
|
||
|
||
正式切換請改走 [GCP-OLLAMA-WIREGUARD-MESH.md](GCP-OLLAMA-WIREGUARD-MESH.md)。
|
||
|
||
### 3. 模型載入但推理失敗
|
||
|
||
檢查 GCP VM 記憶體/CPU 使用率：
|
||
```bash
|
||
# GCP Console → Compute Engine → VM 執行個體 → 監控
|
||
```
|
||
|
||
若記憶體不足，升級機型或減少同時載入模型數量。
|
||
|
||
---
|
||
|
||
## 相關文件
|
||
|
||
- ADR-110: GCP 三層容災架構
|
||
- ADR-125: GCP Ollama Private Mesh and AwoooP Inference Gateway
|
||
- `k8s/awoooi-prod/04-configmap.yaml`
|
||
- `k8s/awoooi-prod/02-network-policy.yaml`
|
||
- `docs/runbooks/RUNBOOK-OLLAMA-FAILOVER.md`
|
||
- `docs/runbooks/GCP-OLLAMA-WIREGUARD-MESH.md`
|
||
- `docs/runbooks/AWOOOP-INFERENCE-GATEWAY.md`
|
||
|
||
---
|
||
|
||
## 負責人
|
||
|
||
- 建立: Claude Sonnet 4.6 — 2026-05-04
|
||
- 審查: 首席架構師 ogt
|