Commit Graph

56 Commits

Author SHA1 Message Date
OG T
85d4857d1b fix(monitoring): RedisMemoryHigh 誤報 — max_bytes=0 除以零修正
Some checks are pending
CD Pipeline / build-and-deploy (push) Has started running
Deploy Alert Rules / Deploy Prometheus Alert Rules (push) Successful in 37s
- 加入 redis_memory_max_bytes > 0 前置條件
- 防止 Redis 未設 maxmemory 時除以零產生 +Inf 永遠觸發
- 影響: alerts-unified.yml + database-alerts.yaml

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 11:41:10 +08:00
OG T
9799a14f54 feat(monitoring): Plan C 外部網站告警 — 4個網站 + SSL憑證預警
All checks were successful
Deploy Alert Rules / Deploy Prometheus Alert Rules (push) Successful in 34s
新增 external_website_alerts 群組:
- MoWoooWorkDown (mo.wooo.work, 188, momo-app)
- TsenyangWebsiteDown (tsenyang.com, 188, tsenyang-website)
- StockWoooWorkDown (stock.wooo.work, 110, stock-platform)
- BitanWoooWorkDown (bitan.wooo.work, 188, bitan-app)
- ExternalSiteSSLExpiringSoon (14天預警, auto_repair:false)

blackbox-http 已涵蓋全部目標,此為結構化告警規則。

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 08:53:08 +08:00
OG T
3c6807d79c ops(monitoring): 觸發 deploy-alerts — database_detail_alerts 6條規則補部署
All checks were successful
Deploy Alert Rules / Deploy Prometheus Alert Rules (push) Successful in 39s
d9e0fab 新增了 6 條 DB 詳細告警規則但 deploy-alerts 因 pyyaml 未安裝失敗
0f86c5c 已修復 workflow,此 commit 觸發重新部署

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-08 21:17:26 +08:00
OG T
d9e0fab3fe feat(monitoring): Sprint 5.2 Plan B — 資料庫詳細告警規則
Some checks failed
Deploy Alert Rules / Deploy Prometheus Alert Rules (push) Failing after 17s
新增 database_detail_alerts 規則群組:
  PostgreSQL:
    - PostgreSQLSlowQueries: 慢查詢 >60s
    - PostgreSQLDeadlocks: 死鎖發生
    - PostgreSQLTooManyConnections: 連接數 >50
  Redis:
    - RedisKeyEviction: Key 驅逐
    - RedisConnectionsHigh: 連接數 >100
    - RedisCommandLatencyHigh: 命令延遲 >10ms

前置: postgres-exporter:9187 + redis-exporter:9121 已在 188 部署 
Prometheus scrape 已更新 

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-08 18:19:03 +08:00
OG T
0847fa3a60 feat(sprint5.1): L2-2 — alerts-unified.yml 補 DockerContainerUnhealthy/Exited 規則
Some checks failed
Deploy Alert Rules / Deploy Prometheus Alert Rules (push) Failing after 19s
新增 docker_health_alerts group:
  - DockerContainerUnhealthy: container_health_status==0, for 2m, auto_repair=true
  - DockerContainerExited:    container_running_status==0, for 1m, auto_repair=true

標籤 auto_repair=true 讓 AWOOOI API 進入 Guardrail 決策鏈路,
實際修復動作由 Service Registry 分級(ADR-062)決定,
docker-health-monitor.sh(純感知層)送 webhook 後由此規則補充 Prometheus 路徑。

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-08 16:40:44 +08:00
OG T
dc27f8f811 ops(monitoring): 統一 Prometheus 告警規則 — 40+條含統一 layer 標籤
修正:
- ClawBotDown → OpenClawDown (舊命名廢棄)
- 加入 SentryDown/HarborDown/GiteaDown/AlertmanagerDown
- 所有規則補齊 layer/component/host/auto_repair 統一標籤
- 整合 k8s/monitoring/*.yaml → ops/monitoring/alerts-unified.yml

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-05 02:26:18 +08:00