fix(automation): enforce durable alert closure
All checks were successful
CD Pipeline / workflow-shape (push) Successful in 0s
CD Pipeline / cancel-stale-cd (push) Has been skipped
CD Pipeline / tests (push) Successful in 2m32s
CD Pipeline / build-and-deploy (push) Successful in 8m2s
CD Pipeline / post-deploy-checks (push) Successful in 2m22s

Restore D037 durable incident readback without weakening database failure semantics.

Fence Agent99 dispatch and terminal learning to canonical identities, durable leases, verifier receipts, and reconciler-owned checkpoints.

Drain backup and restore legacy receipts in bounded cohorts with strict transport, claim, projection, and no-false-green controls.

Keep SSH service refusal visible while separating it from broker network-policy reachability.
This commit is contained in:
ogt
2026-07-14 09:40:53 +08:00
parent 72a167a322
commit 6cf8429d17
56 changed files with 15100 additions and 227 deletions

View File

@@ -329,6 +329,8 @@ class AwoooPRunState(Base):
),
Index("idx_run_state_pending", "project_id", "created_at",
postgresql_where=text("state = 'pending' AND lease_until IS NULL")),
Index("idx_run_state_retry_due", "project_id", "next_attempt_at",
postgresql_where=text("state = 'pending' AND next_attempt_at IS NOT NULL")),
Index("idx_run_state_stale", "lease_until",
postgresql_where=text("state = 'running' AND lease_until IS NOT NULL")),
Index("idx_run_state_project_timeline", "project_id", "created_at"),
@@ -343,6 +345,7 @@ class AwoooPRunState(Base):
agent_id: Mapped[str] = mapped_column(String(128), nullable=False)
state: Mapped[str] = mapped_column(String(32), nullable=False, default="pending")
lease_until: Mapped[datetime | None] = mapped_column(nullable=True)
next_attempt_at: Mapped[datetime | None] = mapped_column(nullable=True)
heartbeat_at: Mapped[datetime | None] = mapped_column(nullable=True)
worker_id: Mapped[str | None] = mapped_column(String(128), nullable=True)
attempt_count: Mapped[int] = mapped_column(SmallInteger, nullable=False, default=0)

View File

@@ -368,6 +368,22 @@ async def _run_init_db_ddl(conn: AsyncConnection) -> None:
else:
raise
# Agent99 dispatch retry scheduling is read by the worker immediately
# after ``init_db`` returns. Normal CD does not apply arbitrary migration
# files, so this nullable compatibility column and bounded partial index
# must commit before any serving/background query can reference them.
# Keep the audited up/down migration as the explicit schema record.
async with conn.begin():
await conn.execute(text("""
ALTER TABLE awooop_run_state
ADD COLUMN IF NOT EXISTS next_attempt_at TIMESTAMP NULL;
"""))
await conn.execute(text(
"CREATE INDEX IF NOT EXISTS idx_run_state_retry_due "
"ON awooop_run_state(project_id, next_attempt_at) "
"WHERE state = 'pending' AND next_attempt_at IS NOT NULL;"
))
async with conn.begin():
# 2026-04-02 Claude Code: 確保 risklevel enum 包含 'high' 值
# Phase 23 新增,避免舊 DB 缺少此值導致 InvalidTextRepresentation