fix(observability): 同步 .188 實際部署調整至 repo
Some checks failed
CD Pipeline / build-and-deploy (push) Has been cancelled
E2E Health Check / e2e-health (push) Has been cancelled

與原始計畫的差異:

1. MinIO Bearer Token 認證
   - 原計畫: MINIO_PROMETHEUS_AUTH_TYPE=public (此版本不支援)
   - 實際: mc admin prometheus generate 產生 Bearer Token
   - 更新: prometheus-config-phase-o.yaml 加入 bearer_token

2. remote_write 廢棄 → OTEL Collector Prometheus scrape
   - 原計畫: Prometheus remote_write → SigNoz OTEL /api/v1/write
   - 實際: SigNoz OTEL Collector 不支援 Prometheus remote_write 格式 (404)
   - 改用: OTEL Collector prometheus receiver 直接 scrape node-exporter + kube-state-metrics
   - 新增: ops/signoz/otel-collector-config-phase-o.yaml (版本控管副本)

3. ADR-053 驗收清單更新為實際結果

Co-Authored-By: Claude Code <noreply@anthropic.com>
This commit is contained in:
OG T
2026-04-02 21:23:47 +08:00
parent 93e3aa6811
commit 3f339110dd
4 changed files with 227 additions and 26 deletions

View File

@@ -43,9 +43,13 @@
→ ClickHouse (統一儲存)
→ SigNoz UI (統一查詢)
Prometheus (指標收集)
remote_write (白名單過濾 ~50 series)
SigNoz ClickHouse (長期 90 天)
SigNoz OTEL Collector (Prometheus Receiver)
直接 scrape node-exporter (172.28.0.1:9100)
直接 scrape kube-state-metrics (192.168.0.121:30888)
→ SigNoz ClickHouse (長期儲存)
注意: 原計畫 Prometheus remote_write 因 SigNoz 不支援 Protobuf 格式而廢棄
改用 OTEL Collector 內建 prometheus receiver 直接 scrape 關鍵指標
```
---
@@ -110,12 +114,14 @@ Warning/Error Event 全量保留。Normal/Scheduled/Pulling/Pulled/Created/Start
- [x] OTEL Collector 2 Pod Running (mon + mon1)
- [x] Event Exporter 1 Pod Running
- [x] Descheduler CronJob 正常執行 (Completed)
- [x] MinIO + Kali 告警規則已加入 Prometheus
- [x] Alert Chain Smoke Test Script 完成
- [x] MinIO 監控 up (Bearer Token 認證mc admin prometheus generate)
- [x] Kali Scanner TCP probe up
- [x] MinIO/Kali 告警規則已加入 Prometheus (追加至 alerts.yml7 groups)
- [x] SigNoz 指標流入 (OTEL Collector prometheus receiver: node + kube-state)
- [x] Alert Chain Smoke Test 7/8 PASSED (1 non-critical: 指標剛啟動)
- [x] CD Pipeline 整合 Alert Chain Smoke Test + Sentry Token 注入
- [ ] ClickHouse TTL 設定 (待 .188 操作)
- [ ] Prometheus remote_write 部署 (待 .188 操作)
- [ ] SignOz 告警規則部署 (待 .188 操作)
- [ ] ClickHouse TTL 設定 (待 .188 操作: signoz_logs 30天 / signoz_metrics 90天)
- [x] ~~Prometheus remote_write~~ → 改用 OTEL Collector federation scrape (SigNoz 不支援 remote_write 格式)
---