Files
awoooi/docs/operations/AGENT99-HOST112-CONTROLLED-RECOVERY-CONTRACT.md
ogt c9bb4bbfc6
Some checks failed
CD Pipeline / workflow-shape (push) Successful in 0s
CD Pipeline / cancel-stale-cd (push) Has been skipped
CD Pipeline / tests (push) Successful in 2m35s
CD Pipeline / build-and-deploy (push) Failing after 7m35s
CD Pipeline / post-deploy-checks (push) Has been skipped
AWOOOI Harbor 110 Local Repair / workflow-shape (push) Successful in 1s
AWOOOI Harbor 110 Local Repair / harbor-110-local-repair (push) Successful in 16s
fix(recovery): bind host112 closure artifacts
2026-07-14 12:37:28 +08:00

3.5 KiB

Agent99 Host112 controlled recovery contract

Host112 recovery is a bounded, correlated controlled-apply lane. Agent99 uses the Windows99 agent99_ed25519 identity directly to kali@192.168.0.112; it does not fall back through the 110 jump route for this lane.

Candidate separation

  • The general guest candidate covers systemd, console/LightDM, VMware Tools, SSH, Wazuh Indexer, and the recovery timer.
  • The Wazuh manager candidate is evaluated separately. A healthy manager, a manager cooldown, or a manager resource block does not suppress a valid general guest repair candidate.
  • When the manager was already verified healthy, a general-only apply must retain manager_attempt_count=0 and manager_runtime_write_performed=0.

Apply closure

Before any apply, Agent99 runs the correlated sudo --dry-run with the same trace_id, run_id, and work_item_id. The dry-run must have a working direct transport, matching identity and boot, a readable preflight result, and no runtime or receipt write.

A manager recovery is verified only when all of the following are true for the same apply:

  1. The direct apply transport completed and the executor returned healthy.
  2. The complete guest verifier (console, SSH, recovery timer, Wazuh services, manager process and ports) passes before the apply emits terminal=verified_healthy. A manager-only success cannot freeze a green receipt while the rest of the guest is degraded.
  3. The apply emits a new immutable receipt and immutable readback with matching identity, boot, terminal, receipt path, receipt SHA-256, and readback SHA-256. A second correlated sudo --dry-run reloads and verifies that exact receipt/readback pair; it does not manufacture a current-state substitute.
  4. A final ordinary --check independently verifies current guest and manager runtime health on the same boot.

The last check is evidence only. A timer or external process that makes the manager healthy after a failed apply cannot replace the apply transport or its immutable receipt and therefore cannot turn that run green.

All mutating systemctl calls are conservatively classified as a write attempt before execution. A timeout or non-zero return therefore still produces a failure receipt instead of entering the no-write path. TERM/INT/HUP handling remains active through the complete verifier; the final receipt/readback pair is committed in one short signal-masked section.

Timer artifact bound

The 60-second systemd timer does not create per-invocation immutable files when it observes no runtime mutation. Repeated no-write observations update one atomic timer-observation.latest aggregate with a state fingerprint and repeat count. Timer runs that attempt a runtime mutation use a 128-slot rolling ledger containing at most 128 receipt files and 128 linked readback files. Each slot records the exact run identity, receipt hash, terminal and explicit rolling retention policy. Agent99-triggered correlated applies remain immutable and are not part of this rolling timer ledger.

Timeout chain

  • Host executor absolute deadline: 420 seconds.
  • Agent99 apply transport: 510 seconds, leaving 90 seconds for transport teardown and receipt delivery.
  • Worst-case Host112 chain, including five 90-second SSH lock waits: 1080 seconds.
  • Queue child outer deadline: 1680 seconds, leaving 600 seconds after the Host112 chain.
  • Dispatch client deadline: 1800 seconds, leaving 120 seconds after the queue child deadline.

The control plane validates these reserves before attempting Host112 recovery.