diff --git a/HOST-REBOOT-AI-AUTOMATION-SOP.md b/HOST-REBOOT-AI-AUTOMATION-SOP.md index 87abaf693..069c92438 100644 --- a/HOST-REBOOT-AI-AUTOMATION-SOP.md +++ b/HOST-REBOOT-AI-AUTOMATION-SOP.md @@ -146,8 +146,9 @@ AwoooI SRE Telegram command loop: - Operators and project AI Agents can issue commands in the SRE Telegram group with `/agent99 ...`. - Agent99 records these as `source=telegram-sre-war-room`. -- Completed automated repairs send `operator_command_result` back to Telegram. -- Results include source, instruction, mode, exit code, evidence path, problem count, and KM path. +- Failed or escalated automated repairs send `operator_command_result` back to Telegram. +- Successful automated repairs are evidence-first: they write queue, evidence, problem count, KM, and RAG receipts, then suppress success-only Telegram noise. +- Failure results include source, failure instruction, mode, exit code, evidence path, and problem count. - The command path must not depend on Codex quota. - The command path must not depend only on the default `TELEGRAM_CHAT_ID`, because that can point to a private chat. Agent99 also accepts command updates from Telegram chats whose title matches `AwoooI SRE|AWOOOI SRE|SRE`. - When the command comes from a matched Telegram group, the completion report is sent back to that same Telegram origin; stored evidence redacts the raw chat id as `telegram-origin`. @@ -172,7 +173,8 @@ Producer-side Agent99 bridge: - Fallback transport: when `AGENT99_SRE_ALERT_INBOX_PATH` is configured and relay URL is empty, the bridge writes sanitized alert JSON directly to the Agent99 SRE inbox contract. - If relay/path is empty or unavailable, the bridge fails open and logs; it must not cause Alertmanager retries. - Relay safety: only LAN/local source prefixes are accepted by default; optional `AGENT99_SRE_RELAY_TOKEN` / `AGENT99_SRE_ALERT_RELAY_TOKEN` values stay in env/Secret and are never stored in repo or evidence. -- Provider freshness bridge output must require provider window, last seen, direct reference, and verifier readback before dashboard green is trusted. +- Provider freshness bridge output must run direct readback, then require provider window, last seen, direct reference, and verifier readback before dashboard green is trusted. +- Provider freshness no-false-green rule: route 200 or dashboard up is not enough. The Agent99 verifier must read `https://awoooi.wooo.work/api/v1/platform/events/dossier/recurrence?project_id=awoooi&limit=20` and record `providerWindowMinutes`, `lastSeen`, `directReference`, and `verifierReadback`. Approved controlled actions for the 99 agent: @@ -257,7 +259,8 @@ Recovery is complete only when all of the following are captured: - Current 110 runner action requires `load_per_core_critical`. - Memory/disk pressure alerts do not execute the runner pause action. - Telegram audit: - - Agent99 emits Telegram attempts for all operation start/complete events, warnings, critical events, and load-shed actions. + - Agent99 emits Telegram attempts for warnings, critical events, failed or escalated operator commands, and load-shed actions. + - Successful operator commands are not Telegram noise; they are recorded in evidence, problem counts, KM, and RAG. - Agent99 supports local env aliases and an existing-token relay. - Current relay: 110 `stockplatform-v2-api-1`, using that container's existing `TELEGRAM_BOT_TOKEN` / `TELEGRAM_CHAT_ID`. - 2026-07-09 17:27 verification: `Wooo-Agent99-Performance-Guard` `LastTaskResult=0`; latest evidence `agent99-Perf-20260709-172712.json`; `tgSent=2`; `relayOk=True`. @@ -359,8 +362,8 @@ Recovery is complete only when all of the following are captured: - Group matching uses `allowedCommandChatTitlePattern=AwoooI SRE|AWOOOI SRE|SRE`. - Loop prevention: - Agent99's own `Agent99 Alert` / `operator_command_result` messages are ignored by monitoring auto-ingest. - - Operator command result Telegram reports bypass dedupe so every completed repair is reported. - - Provider freshness signals are not passive dashboard health; they create a read-only freshness candidate and require provider window, last seen, direct reference, and verifier readback. + - Successful operator command results are suppressed from Telegram and kept in evidence/KM/RAG; failed or escalated command results remain alertable. + - Provider freshness signals are not passive dashboard health; they create a read-only freshness candidate, run direct AWOOI readback, and require provider window, last seen, direct reference, and verifier readback. - Provider freshness guardrail: do not switch provider, call paid model, edit env, or reload. - Synthetic monitoring alert: - `Alertmanager FIRING critical public_route_502 service=awoooi host=192.168.0.120 HTTP 502` @@ -389,6 +392,37 @@ Recovery is complete only when all of the following are captured: - problem key: `agent99-sre-alert-inbox_awoooi_192.168.0.120_public_route_502` - problem occurrences: `2` +## Agent99 Alert Noise And Provider Freshness Readback Receipt + +2026-07-10 10:00 Asia/Taipei: + +- Runtime patch deployed on 99: + - Script: `C:\Wooo\Agent99\bin\agent99-control-plane.ps1`. + - Parser readback: `parse_ok`. + - Source commit: `0a7a0115 fix(agent99): reduce alert noise and verify freshness`. +- Provider freshness behavior: + - Evidence: `C:\Wooo\Agent99\evidence\agent99-ProviderFreshness-20260710-100028.json`. + - Status: `direct_readback_ok_no_source_events`. + - Missing fields after automatic readback: `lastSeen` only. + - `providerWindowMinutes=120`. + - `directReference=https://awoooi.wooo.work/api/v1/platform/events/dossier/recurrence?project_id=awoooi&limit=20`. + - `verifierReadback.ok=True`, `httpStatus=200`, `sourceEventTotal=0`. +- Telegram noise control: + - Evidence log: `C:\Wooo\Agent99\evidence\agent99-ProviderFreshness-20260710-100028.log`. + - Same provider-freshness state was suppressed: `telegram_dedupe_skip type=provider_freshness_triage severity=warning ageMinutes=5.83 cooldown=30`. + - Evidence log: `C:\Wooo\Agent99\evidence\agent99-ControlTick-20260710-100027.log`. + - Successful operator command was evidence-only: `telegram_operator_success_suppressed type=operator_command_result mode=ProviderFreshness id=operator-suppress-smoke-20260710-100026`. + - Control evidence: `C:\Wooo\Agent99\evidence\agent99-ControlTick-20260710-100027.json`, `processedCount=1`, `pendingCount=0`, `telegram=[]`. +- Scheduled entrypoints verified on 99: + - `Wooo-Agent99-Control-Loop`. + - `Wooo-Agent99-SRE-Alert-Inbox`. + - `Wooo-Agent99-Telegram-Inbox`. + - `Wooo-Agent99-SRE-Alert-Relay` is running. +- Operational rule: + - Do not treat repeated success receipts as user-facing alerts. + - Send user-facing Telegram only for warnings, criticals, unresolved blockers, failed/escalated operator commands, and controlled-remediation actions. + - Keep successful automation in evidence, problem counters, KM, and RAG so repeated incidents trend down without flooding the SRE group. + ## OpenClaw Cold-Start Readback 2026-07-09 17:38 Asia/Taipei: