wooo/ewoooc
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
1,456 Commits 22 Branches 0 Tags
d0b14577cbbc58d8ec2a079d5d855ea4ce7c7fa4
Commit Graph

37 Commits

Author SHA1 Message Date
OoO
ba5fe06b13 fix: update ollama primary host
Some checks failed
CD Pipeline / deploy (push) Has been cancelled
Details
2026-06-18 14:24:55 +08:00
OoO
bd942e9427 fix: skip unhealthy direct ollama probes
All checks were successful
CD Pipeline / deploy (push) Successful in 1m10s
Details
2026-06-18 13:46:30 +08:00
OoO
ee50e440ce fix: rescue gcp ollama through proxy
All checks were successful
CD Pipeline / deploy (push) Successful in 1m9s
Details
2026-06-18 12:13:38 +08:00
OoO
9bafa73ffc fix: defer embedding work during gcp backoff
All checks were successful
CD Pipeline / deploy (push) Successful in 1m10s
Details
2026-06-18 12:06:24 +08:00
OoO
3b9448b776 V10.576 修正 GCP-only Ollama retry
All checks were successful
CD Pipeline / deploy (push) Successful in 1m5s
Details
2026-06-04 11:20:52 +08:00
OoO
66f2c50964 V10.556 補 GCP-B 模型 fallback 防 111 過載 2026-06-01 14:55:13 +08:00
OoO
a00f34ce87 加入 Ollama GCP failover 診斷與 unhealthy skip
All checks were successful
CD Pipeline / deploy (push) Successful in 1m5s
Details
2026-05-25 14:16:51 +08:00
OoO
44ef5a70a1 校準 GCP-B embedding timeout
All checks were successful
CD Pipeline / deploy (push) Successful in 1m6s
Details
2026-05-25 13:43:10 +08:00
OoO
ac5539a8ef 調整背景 embedding 熔斷告警語意
All checks were successful
CD Pipeline / deploy (push) Successful in 1m5s
Details
2026-05-25 12:35:56 +08:00
OoO
0ade55469e 穩定 Ollama embedding GCP 失敗熔斷
All checks were successful
CD Pipeline / deploy (push) Successful in 1m5s
Details
2026-05-25 12:28:44 +08:00
OoO
3fd349c830 Keep embedding retry on GCP when 111 fallback disabled
All checks were successful
CD Pipeline / deploy (push) Successful in 1m5s
Details
2026-05-25 09:59:00 +08:00
OoO
5ae28083d9 V10.427 add 111 Ollama circuit breaker 2026-05-24 15:55:12 +08:00
OoO
353e565e52 V10.417 protect embedding fallback routing
All checks were successful
CD Pipeline / deploy (push) Successful in 1m4s
Details
2026-05-24 14:53:43 +08:00
OoO
b73dc6df3f V10.415 protect Hermes fallback routing
All checks were successful
CD Pipeline / deploy (push) Successful in 1m5s
Details
2026-05-24 14:25:22 +08:00
OoO
106c1935f4 收緊 111 Ollama fallback 資源上限
Some checks failed
CD Pipeline / deploy (push) Failing after 11m7s
Details
2026-05-21 18:13:50 +08:00
OoO
c329d96dff 限制 111 fallback context 大小
All checks were successful
CD Pipeline / deploy (push) Successful in 1m10s
Details
2026-05-21 12:44:33 +08:00
OoO
00a808518e 將 111 Ollama fallback 收斂到輕量模型 2026-05-21 12:39:23 +08:00
OoO
d6ae216c8c 限制 111 Ollama fallback 資源占用
All checks were successful
CD Pipeline / deploy (push) Successful in 1m11s
Details
2026-05-21 12:24:49 +08:00
OoO
1cf1fd01b1 修正 Ollama fallback 與 PPT vision payload
All checks were successful
CD Pipeline / deploy (push) Successful in 1m30s
Details
2026-05-18 21:32:15 +08:00
OoO
2234488378 fix: route special ollama fallbacks through cascade
All checks were successful
CD Pipeline / deploy (push) Successful in 57s
Details
2026-05-13 21:41:43 +08:00
OoO
ed29e66fde fix: route hermes through ollama cascade
All checks were successful
CD Pipeline / deploy (push) Successful in 57s
Details
2026-05-13 21:21:05 +08:00
OoO
bd6310365e 停用 CICD 舊叢集副作用
Some checks failed
CD Pipeline / deploy (push) Has been cancelled
Details
2026-05-13 19:19:15 +08:00
OoO
ce208921af 限制 Ollama host 只能走核准節點
All checks were successful
CD Pipeline / deploy (push) Successful in 58s
Details
2026-05-13 11:58:45 +08:00
OoO
7a10d27d61 feat(p53): K8s Nginx Proxy 支援 — host_label/provider_tag 補完
Some checks failed
CD Pipeline / deploy (push) Has been cancelled
Details 
問題：
K8s 內網無法直連 GCP 公網 11434，所以 110 跳板架了 Nginx Proxy
轉發 11435/11436 到 GCP-A/GCP-B。但 services/ollama_service.py 的
get_host_label() 只看 IP substring（34.143.170.20 / 34.21.145.224），
K8s 環境會 fallback 到「未知」造成觀測台主機標籤錯亂。

修補：
- services/ollama_service.py::get_host_label
  新增：
    192.168.0.110:11435 → "GCP-SSD（via Nginx 110）"
    192.168.0.110:11436 → "GCP-SSD-2（via Nginx 110）"
  保留：直連 GCP / 111 / 188 / localhost 既有判斷

- services/ollama_service.py::get_provider_tag（新函式）
  統一 provider 標籤判斷（之前散落 code_review_pipeline 等多處重寫）：
    GCP 直連 + Nginx 11435 → 'gcp_ollama'
    GCP-B 直連 + Nginx 11436 → 'ollama_secondary'
    111 → 'ollama_111'
    其他 → 'ollama_other'
  跨環境統一：ai_calls.provider 在 docker-compose / K8s 都記同一 tag，
  跨環境統計不分裂。

- services/code_review_pipeline_service.py:233
  改用統一 get_provider_tag()，移除原本 hardcode 的 if/else IP 判斷。

- k8s/02-configmap.yaml（user 已改）
  OLLAMA_HOST_PRIMARY = http://192.168.0.110:11435 (Nginx → GCP-A)
  OLLAMA_HOST_SECONDARY = http://192.168.0.110:11436 (Nginx → GCP-B)
  OLLAMA_HOST_FALLBACK = http://192.168.0.111:11434 (內網)

driver test:
  http://34.143.170.20:11434 → GCP-SSD / gcp_ollama
  http://192.168.0.110:11435 → GCP-SSD（via Nginx 110）/ gcp_ollama
  http://192.168.0.111:11434 → 111 備援 / ollama_111

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-05 01:09:56 +08:00
OoO
1da7564567 feat(p22+23): caller 整合 model_router (sales_copy) + cost_throttle (claude)
Some checks failed
CD Pipeline / deploy (push) Has been cancelled
Details 
Operation Ollama-First v5.0 / Phase 22.1 + Phase 23 — 路由治理整合

Phase 22.1: services/ollama_service.py generate_sales_copy
- 短文 < 100 字（estimated_length = product_name × 5）→ 走 model_router
- model_router 推算 → 短文 gemma3:4b / 長文 llama3.1:8b
- MODEL_ROUTER_ENABLED=false 時 select_model 直接回 self.model（向下相容）
- router 失敗 swallow（fallback self.model 不影響主流程）

Phase 23: services/anthropic_service.py is_available()
- 新增 cost_throttle 整合：claude provider throttle 時 is_available=False
- caller 看到 False 自動走 Gemini fallback，不送 Claude 請求
- COST_THROTTLE_ENABLED=false 時不影響（戰役預設 OFF）
- cost_throttle 不可用時 try/except 不阻擋（向下相容）

行為對照：
  戰前：sales_copy 永遠用 self.model (llama3.1:8b)
  戰後（flag OFF）：完全相同
  戰後（flag ON）：商品名 < 20 字 → gemma3:4b 短文，提速 50%

  戰前：Claude 燒到月底超預算才告警，無自動節流
  戰後（throttle flag OFF）：完全相同
  戰後（throttle flag ON）：claude 月底推估 > 110% → is_available=False
                              → caller fallback Gemini 自動省錢

regression: 既有 retry_chain / anthropic / model_router test 全綠

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-04 11:00:04 +08:00
OoO
942193db2a feat(p13): OllamaResponse token 補欄 + COST_TABLE 補新模型 + retry 鏈 unit test
All checks were successful
CD Pipeline / deploy (push) Successful in 2m41s
Details 
Operation Ollama-First v5.0 / Phase 13 補強

(A) services/ollama_service.py — OllamaResponse 加 input_tokens/output_tokens
- A4 Phase 1 已知 limitation 修補：openclaw_bot_main token=0 假數據誤導日報
- generate() 解 prompt_eval_count + eval_count 寫 OllamaResponse
- 影響：ai_call_logger 收到正確 token 數，token 日報 Ollama 占比準確

(B) services/ai_call_logger.py — COST_TABLE 補 GCP 已拉/候選模型
- qwen2.5:7b-instruct (Phase 3 A7 OpenClaw Q&A 預設)
- qwen3:14b (Phase 3 A9 Nemotron + A7 升級候選)
- qwen2.5-coder:32b (Phase 8 候選)
- qwen2-vl:7b (Phase 13+ PPT vision 候選)
- deepseek-r1:14b / gemma3:4b (推理增強 / 輕量)
- 全部 cost=0（Ollama 自架）
- 解 logger.warning「unknown model cost」誤報

(J) tests/test_ollama_retry_chain.py (10 unit tests) — 驗 hotfix e862a90/6572d52
- T1 self.host @property lazy resolve
- T2 explicit host 凍結不 retry
- T3 generate 第一台 timeout → 第二台成功（核心 retry 鏈）
- T4 三主機都失敗 → success=False
- T5 cache 卡同主機 → break 不無限迴圈
- T6 Phase 13 token 解析驗證
- T7-T9 generate_embedding 同類驗證
- T10 mark_unhealthy 清 resolve cache

regression: 全戰役 14 test 檔仍 zero regression

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-04 10:07:33 +08:00
OoO
6572d521ba fix(embed): generate_embedding 三主機 retry — 修同類「111 死則全死」bug
All checks were successful
CD Pipeline / deploy (push) Successful in 2m42s
Details 
承前 commit e862a90（generate retry）的同類修補：
generate_embedding 之前邏輯：
  target_host = host or env or resolve  # 一次解析
  try: post → mark_unhealthy + return []  # 失敗無 retry

修補後：
  caller 顯式 host → 凍結不 retry（向下相容）
  caller 走 lazy → 三主機 retry 鏈：
    每次 self.host 走 resolve_ollama_host()
    失敗 mark_unhealthy + cache 失效 + 取新主機
    最多 3 次（避免同主機無限迴圈）

影響範圍：KM embedding worker / RAG query embedding / openclaw_learning

regression: 57 unit tests 全綠

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-04 08:56:38 +08:00
OoO
e862a9040c fix(ollama): 解「111 關機 → GCP 也斷」三主機 retry 鏈 + lazy host
All checks were successful
CD Pipeline / deploy (push) Successful in 9m14s
Details 
統帥 2026-05-04 反饋：「111 關機後，兩台 GCP Ollama 也跟著斷線不可用」

根因（雙 bug 連動）：
  1. OllamaService.__init__ 凍結 self.host：
     `self.host = host or resolve_ollama_host()` 是 instance 級凍結
     容器啟動時若 GCP cold start 觸發 fallback 111 → self.host 永遠卡 111
     即使 cache 過期，instance 不會重新 resolve
  2. generate() 失敗無 retry 鏈：
     原邏輯只 mark_unhealthy(self.host) + return failure
     沒有「換下一台主機再試」邏輯 → 111 死則全死

修補（雙管齊下）：
  A. self.host 改 @property：每次存取走 resolve_ollama_host()
     - caller 顯式指定（_explicit_host）才凍結
     - 內部 _RESOLVE_TTL 120s cache 控制 HTTP probe 成本
  B. generate() 三主機 retry 鏈：
     for attempt in range(3):
         current_host = self.host  # property lazy resolve
         if attempted → break (避免無限迴圈)
         post → success ? return : mark_unhealthy + retry
     mark_unhealthy 自動清空 resolve cache，下次 self.host 取新主機

行為對比：
  戰前：GCP cold start 卡 111 → 111 關機 → 全部失敗
  戰後：GCP cold start 卡 111 → 111 關機 → mark_unhealthy(111) →
        self.host 重 resolve → GCP Primary（已暖機）→ 成功

generate_embedding 同類修補延後（caller 多走 explicit host 路徑風險較低）

regression: 36 unit tests 全綠（test_ollama_resolve + test_ai_call_logger）

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-04 08:42:52 +08:00
OoO
078bf2683c fix(adr-027): Phase 2 — ADR-027 4 破洞修補 + 移除寫死 111 
config.py — B1+B2 lazy resolve
- get_ollama_host() 取代 import-time freeze 的 OLLAMA_HOST
- get_embedding_host() 取代 EMBEDDING_HOST
- 主機切換時不需重啟 Python 進程

services/ollama_service.py — B3+B4 三主機級聯
- resolve_ollama_host(primary, secondary, fallback) 三主機級聯
  - Primary:   34.143.170.20 (SSD) — GCP 主主機
  - Secondary: 34.21.145.224 (SSD) — 同等效能備援
  - Fallback:  192.168.0.111 (HDD) — 最後一道防線
- _is_reachable: HTTP /api/version probe 取代 TCP socket（防 process 卡死假活）
- mark_unhealthy(host) 即時失效 cache，30s 內跳過該主機
- 14 unit tests 全綠

services/aider_heal_executor.py — N2
- 移除寫死 192.168.0.111，改用 get_ollama_host()
- AiderHeal 終於遵循 ADR-027 GCP 優先策略

Operation Ollama-First v5.0 / Phase 2 A6

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-03 23:05:11 +08:00
OoO
15800a29ac [V-New] Ollama 主機切換：GCP 優先 / 111 自動備援架構 | services/ollama_service.py, docker-compose.yml
All checks were successful
CD Pipeline / deploy (push) Successful in 12m19s
Details
2026-05-03 16:03:00 +08:00
OoO
72fa166729 fix(ai): 遷移 Ollama embedding 至 api embed
All checks were successful
CD Pipeline / deploy (push) Successful in 1m46s
Details
2026-04-30 10:24:15 +08:00
OoO
b49b704e82 fix(post-3.5g): restore generate_embedding for KM dual-write (CRIT-1) 
從 4349db2~1 撈回 OllamaService.generate_embedding，補齊被誤刪的方法。

問題：
- services/openclaw_learning_service.py:67 仍呼叫 ollama_service.generate_embedding(...)
- 4349db2 大改時把這個方法刪掉，導致每次 NemoTron 寫入學習資料時觸發
  AttributeError: 'OllamaService' object has no attribute 'generate_embedding'
- pgvector KM 因此完全停寫，違反 ADR-007 雙寫鐵律

修法：
- 把 method paste 回 OllamaService 末端（line 508）
- 對齊現代 config：os 已在檔案頂部 import，移除方法內重複 import
- embedding 走 EMBEDDING_HOST（Hermes 主機，內網免認證）
- model 預設 bge-m3:latest（ADR-003 對齊）

驗證：
- AST parse OK
- grep 'def generate_embedding' 已存在

Critic finding: CRIT-1
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-28 14:40:22 +08:00
OoO
1ecec162dd fix: increase Ollama health check timeout to prevent false offline status
All checks were successful
CD Pipeline / deploy (push) Successful in 1m18s
Details
2026-04-28 12:35:58 +08:00
OoO
30fc7609df fix: 將預設 Ollama 模型改為 111 主機已有的 llama3.1:8b
All checks were successful
CD Pipeline / deploy (push) Successful in 1m17s
Details
2026-04-28 12:00:57 +08:00
OoO
4349db2015 feat: AiderHeal 支援 ssh 與 Ollama 設為首選 AI 引擎
All checks were successful
CD Pipeline / deploy (push) Successful in 8m40s
Details
2026-04-28 11:41:12 +08:00
ogt
676c711e7a feat: AI 治理完備 V10.3 — 技術債清零 + DB 備份機制 + 備份 AI 監控
Some checks are pending
CD Pipeline / deploy (push) Waiting to run
Details 
技術債清零 (2026-04-19):
- migrations/010: ai_insights 補 decay_exempt/avg_quality/status/ai_model/feedback 欄位
- migrations/011: embedding_retry_queue 持久化表 (ADR-009)
- migrations/012: backup_log 備份記錄表
- services/openclaw_learning_service: 記憶體 Queue → DB retry queue，時間衰減 RAG
- services/nemoton_dispatcher_service: 三個 tool 強制雙寫 ai_insights (_sink_insight_to_km)
- services/import_service: Excel 前置欄位防禦（商品名稱類 + 業績金額類）
- services/ollama_service: generate_embedding 新增 EMBEDDING_HOST env，embedding 永遠走 192.168.0.111
- SYSTEM_VERSION: V9.4 → V10.3

DB 備份機制:
- scripts/pg_backup.sh: host-level pg_dump 備份腳本，cron 每日 02:00，保留 7 天，Telegram 通知
- services/db_backup_service.py: Python 備份 service，寫入 backup_log
- scheduler: run_db_backup_task (02:00) + run_backup_monitor_task (每 6h AI Agent 監控)
- Dockerfile: 加入 postgresql-client

文件:
- CLAUDE.md: 環境架構依 ADR-008 實地重寫，含完整 SSH/Docker 部署 SOP
- PROJECT_CONSTITUTION.md: 內容已整合入 CLAUDE.md，刪除重複檔案

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-04-19 02:03:45 +08:00
ogt
1b4f3a7bbe feat: EwoooC 初始化 — 完整專案推版至 Gitea
Some checks failed
CD Pipeline / deploy (push) Failing after 59s
Details 
- 建立 Gitea Actions CD pipeline (.gitea/workflows/cd.yaml)
- 部署模式: rsync Python 檔案至 188 → docker restart (volume mount)
- Dockerfile/requirements 變動時自動重建 Docker image
- 部署通知: Telegram (開始/成功/失敗)
- 健康檢查: https://mo.wooo.work/health (最多 5 次重試)
- 同步最新 CLAUDE.md / ADR-008 / memory (2026-04-19)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-04-19 01:21:13 +08:00