wooo/awoooi
1
1
Fork 0
Code Issues 2 Pull Requests 1 Actions 3 Packages Projects Releases Wiki Activity

fix(agent-market): move discovery review to controlled queue
Some checks failed
CD Pipeline / tests (push) Successful in 1m43s
Details
Code Review / ai-code-review (push) Successful in 12s
Details
Ansible / Reboot Recovery Contract / validate (push) Failing after 12m47s
Details
CD Pipeline / build-and-deploy (push) Successful in 5m52s
Details
CD Pipeline / post-deploy-checks (push) Successful in 3m7s
Details

Browse Source
This commit is contained in:
Your Name
2026-06-28 08:54:08 +08:00
parent d04cf629f4
commit a1f5935481
19 changed files with 1964 additions and 128 deletions
Download Patch File Download Diff File Expand all files Collapse all files

22
docs/LOGBOOK.md
View File

@@ -1,3 +1,25 @@
## 2026-06-28 — 09:35 Agent Market discovery guard 轉 AI controlled queue
**背景**：統帥要求非事故級 hard gate / guard 全部打開並快速推進實作。本段針對 `agent-market-watch` 中舊 `manual_classification_required`、`needs_manual_research`、promotion review hard fail 與 operator priority gate，改成可審計的 AI controlled classification / priority queue。
**完成內容**：
- `.gitea/workflows/agent-market-watch.yaml` 將 discovery / classification / summary / Telegram 文案改為 AI 受控隊列；classification step 不再因沒有 `new_manual` 候選而跳過，會固定產生 controlled queue artifact，避免後段缺 artifact 變成硬阻塞。
- `apps/api/src/services/agent_market_discovery_review.py`、`agent_market_discovery_classifier.py`、`agent_market_watch_promotion_review.py`、`agent_market_governance_snapshot.py` 移除非 critical manual/operator 預設終局，改為 `controlled_classification_queue`、`needs_controlled_research`、`controlled_priority_review`、`controlled_queue`。
- `docs/schemas/agent_market_discovery_review_v1.schema.json`、`docs/schemas/agent_market_governance_snapshot_v1.schema.json` 與 `apps/web/src/lib/api-client.ts` 同步 controlled queue 欄位，前端 governance Agent Market tab 改讀 controlled reverify / controlled review gate。
- `apps/web/messages/zh-TW.json`、`apps/web/messages/en.json` 補齊 Agent Market controlled queue / controlled review i18n，避免 UI 仍把非 critical 市場 watch path 說成 manual blocker。
- 重新產生 `docs/evaluations/agent_market_*_2026-06-28.json` 四份 snapshot；本次 readback：`controlled_classification_queue=9`、`recommended_watch_additions=7`、`integration_queue_count=5`、`market_watch_health.status=controlled_queue`。
**本地驗證結果**：
- YAML / JSON parse：通過。
- Agent Market pytest：`9 passed`。
- i18n mirror：`zh-TW=14480`、`en=14480`、missing `0 / 0`。
- `pnpm --dir apps/web typecheck`：通過。
- `git diff --check`：通過。
**仍保留的 break-glass 邊界**：
- OpenClaw 仍維持 production decision core；替換前必須 replay / shadow / canary / ADR。
- SDK install、API shadow / canary、production route、paid provider / cost route、external active security scan、secret value / credential URL / raw env、DB destructive / backup restore、force push / repo refs deletion 仍不得被本段 controlled queue 直接打開。
## 2026-06-28 — 08:45 110 runner 壓力事故 source / live fail-closed 收斂
**背景**：統帥全面授權打開非事故級 gate，但 110 Gitea runner 反覆拉起 StockPlatform headless Chrome smoke，已造成 production host CPU / CI 壓力事故；runner 未搬遷 / 限流前不得直接重開。

431
docs/evaluations/agent_market_discovery_classification_2026-06-28.json Normal file
View File

@@ -0,0 +1,431 @@
{
"candidates": [
{
"approval_boundary": {
"approved_for_paid_api_calls": false,
"approved_for_replay": false,
"approved_for_sdk_install": false,
"approved_for_shadow_or_canary": false,
"approved_for_watch_registry_addition": false
},
"archived": false,
"classification": "needs_controlled_research",
"description": "The Frontend Stack for Agents & Generative UI. React, Angular, Mobile, Slack, and more. Makers of the AG-UI Protocol",
"homepage": "https://docs.copilotkit.ai",
"html_url": "https://github.com/CopilotKit/CopilotKit",
"language": "TypeScript",
"pushed_at": "2026-06-27T06:40:27Z",
"recommendation": "controlled_research_before_watch_registry",
"recommended_role": "controlled_research_required",
"repository_full_name": "copilotkit/copilotkit",
"required_next_gate": "controlled_research_no_registry_change",
"risk_flags": [
"requires_dependency_boundary_review"
],
"stargazers_count": 35572,
"topics": [
"agent",
"agent-native",
"agentic-ai",
"agents",
"ai",
"ai-agent",
"ai-assistant",
"assistant",
"assistant-chat-bots",
"copilot",
"copilot-chat",
"generative-ui",
"js",
"llm",
"nextjs",
"open-source",
"react",
"reactjs",
"ts",
"typescript"
],
"watch_addition_recommended": false
},
{
"approval_boundary": {
"approved_for_paid_api_calls": false,
"approved_for_replay": false,
"approved_for_sdk_install": false,
"approved_for_shadow_or_canary": false,
"approved_for_watch_registry_addition": false
},
"archived": false,
"classification": "needs_controlled_research",
"description": "Open-source infrastructure for Computer-Use Agents. Sandboxes, SDKs, and benchmarks to train and evaluate AI agents that can control full desktops (macOS, Linux, Windows).",
"homepage": "https://cua.ai",
"html_url": "https://github.com/trycua/cua",
"language": "HTML",
"pushed_at": "2026-06-28T00:16:47Z",
"recommendation": "controlled_research_before_watch_registry",
"recommended_role": "controlled_research_required",
"repository_full_name": "trycua/cua",
"required_next_gate": "controlled_research_no_registry_change",
"risk_flags": [
"requires_dependency_boundary_review",
"requires_tool_execution_sandbox_review"
],
"stargazers_count": 19131,
"topics": [
"agent",
"ai-agent",
"apple",
"computer-use",
"computer-use-agent",
"containerization",
"cua",
"desktop-automation",
"hacktoberfest",
"lume",
"macos",
"manus",
"operator",
"swift",
"virtualization",
"virtualization-framework",
"windows",
"windows-sandbox"
],
"watch_addition_recommended": false
},
{
"approval_boundary": {
"approved_for_paid_api_calls": false,
"approved_for_replay": false,
"approved_for_sdk_install": false,
"approved_for_shadow_or_canary": false,
"approved_for_watch_registry_addition": false
},
"archived": false,
"classification": "agent_framework_candidate",
"description": "Eigent: The Open Source Cowork Desktop to Unlock Your Exceptional Productivity. Local and Free Alternative to Claude Cowork.",
"homepage": "https://www.eigent.ai",
"html_url": "https://github.com/eigent-ai/eigent",
"language": "TypeScript",
"pushed_at": "2026-06-27T10:16:01Z",
"recommendation": "add_to_watch_registry_after_controlled_source_review",
"recommended_role": "agent_framework_or_orchestrator_candidate",
"repository_full_name": "eigent-ai/eigent",
"required_next_gate": "ai_controlled_primary_source_verifier_then_watch_registry_patch",
"risk_flags": [
"requires_dependency_boundary_review",
"likely_requires_paid_provider_boundary_review"
],
"stargazers_count": 14404,
"topics": [
"agent-framework",
"agent-skills",
"agentic-ai",
"agentic-workflow",
"claude-cowork",
"claude-cowork-alternative",
"claude-cowork-free",
"desktop-agent",
"multi-agent-systems"
],
"watch_addition_recommended": true
},
{
"approval_boundary": {
"approved_for_paid_api_calls": false,
"approved_for_replay": false,
"approved_for_sdk_install": false,
"approved_for_shadow_or_canary": false,
"approved_for_watch_registry_addition": false
},
"archived": false,
"classification": "agent_governance_candidate",
"description": "Omnigent is an open-source AI agent framework and meta-harness: orchestrate Claude Code, Codex, Cursor, Pi, and custom agents — swap harnesses without rewriting, enforce policies and sandboxing, and collaborate in real time from any device.",
"homepage": "https://omnigent.ai",
"html_url": "https://github.com/omnigent-ai/omnigent",
"language": "Python",
"pushed_at": "2026-06-27T23:15:21Z",
"recommendation": "add_to_watch_registry_after_controlled_source_review",
"recommended_role": "agent_governance_policy_evaluator_candidate",
"repository_full_name": "omnigent-ai/omnigent",
"required_next_gate": "ai_controlled_primary_source_verifier_then_watch_registry_patch",
"risk_flags": [
"requires_dependency_boundary_review",
"likely_requires_paid_provider_boundary_review",
"requires_tool_execution_sandbox_review"
],
"stargazers_count": 5132,
"topics": [
"agent-framework",
"agent-governance",
"agent-orchestration",
"agents",
"ai",
"ai-agent",
"ai-agents",
"claude-code",
"codex",
"coding-agents",
"developer-tools",
"llm",
"ml",
"multi-agent",
"python",
"sandbox"
],
"watch_addition_recommended": true
},
{
"approval_boundary": {
"approved_for_paid_api_calls": false,
"approved_for_replay": false,
"approved_for_sdk_install": false,
"approved_for_shadow_or_canary": false,
"approved_for_watch_registry_addition": false
},
"archived": false,
"classification": "agent_framework_candidate",
"description": "AG2 (formerly AutoGen): The Open-Source AgentOS.Join us at: https://discord.gg/sNGSwQME3x",
"homepage": "https://ag2.ai",
"html_url": "https://github.com/ag2ai/ag2",
"language": "Python",
"pushed_at": "2026-06-27T22:37:56Z",
"recommendation": "add_to_watch_registry_after_controlled_source_review",
"recommended_role": "agent_framework_or_orchestrator_candidate",
"repository_full_name": "ag2ai/ag2",
"required_next_gate": "ai_controlled_primary_source_verifier_then_watch_registry_patch",
"risk_flags": [
"requires_dependency_boundary_review",
"requires_tool_execution_sandbox_review"
],
"stargazers_count": 4714,
"topics": [
"a2a",
"ag2",
"agent-framework",
"agentic",
"agentic-ai",
"ai",
"ai-agents-framework",
"aiagents",
"genai",
"llm",
"llms",
"mcp",
"multi-agent",
"multi-agent-system",
"open-source",
"python"
],
"watch_addition_recommended": true
},
{
"approval_boundary": {
"approved_for_paid_api_calls": false,
"approved_for_replay": false,
"approved_for_sdk_install": false,
"approved_for_shadow_or_canary": false,
"approved_for_watch_registry_addition": false
},
"archived": false,
"classification": "agent_framework_candidate",
"description": "AI 时代的伯克希尔：基于 Claude Code 的价值投资研究框架。巴菲特·芒格·段永平·李录四大师方法论 + 多Agent并行研究。| AI-era Berkshire: a value investing research framework built on Claude Code. 4 masters' methodologies + multi-agent adversarial analysis.",
"homepage": "https://github.com/xbtlin/ai-berkshire#readme",
"html_url": "https://github.com/xbtlin/ai-berkshire",
"language": "Python",
"pushed_at": "2026-06-27T16:49:55Z",
"recommendation": "add_to_watch_registry_after_controlled_source_review",
"recommended_role": "agent_framework_or_orchestrator_candidate",
"repository_full_name": "xbtlin/ai-berkshire",
"required_next_gate": "ai_controlled_primary_source_verifier_then_watch_registry_patch",
"risk_flags": [
"requires_dependency_boundary_review",
"likely_requires_paid_provider_boundary_review",
"requires_tool_execution_sandbox_review"
],
"stargazers_count": 4117,
"topics": [
"ai",
"ai-agent",
"anthropic",
"berkshire-hathaway",
"charlie-munger",
"china-stock",
"claude",
"claude-code",
"financial-analysis",
"fintech",
"fundamental-analysis",
"investment",
"investment-research",
"llm",
"mcp",
"portfolio-management",
"stock-analysis",
"stock-market",
"value-investing",
"warren-buffett"
],
"watch_addition_recommended": true
},
{
"approval_boundary": {
"approved_for_paid_api_calls": false,
"approved_for_replay": false,
"approved_for_sdk_install": false,
"approved_for_shadow_or_canary": false,
"approved_for_watch_registry_addition": false
},
"archived": false,
"classification": "agent_framework_candidate",
"description": "A Go framework for building production agent systems with graph workflows, tools, memory, A2A, AG-UI, MCP, evaluation, and observability.",
"homepage": "https://trpc-group.github.io/trpc-agent-go/",
"html_url": "https://github.com/trpc-group/trpc-agent-go",
"language": "Go",
"pushed_at": "2026-06-27T03:22:33Z",
"recommendation": "add_to_watch_registry_after_controlled_source_review",
"recommended_role": "agent_framework_or_orchestrator_candidate",
"repository_full_name": "trpc-group/trpc-agent-go",
"required_next_gate": "ai_controlled_primary_source_verifier_then_watch_registry_patch",
"risk_flags": [
"requires_dependency_boundary_review",
"requires_tool_execution_sandbox_review"
],
"stargazers_count": 1414,
"topics": [
"a2a",
"a2a-protocol",
"ag-ui",
"agent",
"agent-framework",
"ai",
"ai-agents",
"evaluation",
"go",
"golang",
"graph-workflows",
"llm",
"mcp",
"model-context-protocol",
"multi-agent",
"observability",
"opentelemetry",
"rag"
],
"watch_addition_recommended": true
},
{
"approval_boundary": {
"approved_for_paid_api_calls": false,
"approved_for_replay": false,
"approved_for_sdk_install": false,
"approved_for_shadow_or_canary": false,
"approved_for_watch_registry_addition": false
},
"archived": false,
"classification": "personal_agent_platform_candidate",
"description": "Zero, your trustworthy AI teammate for real work. ",
"homepage": "https://vm0.ai",
"html_url": "https://github.com/vm0-ai/vm0",
"language": "TypeScript",
"pushed_at": "2026-06-28T00:39:40Z",
"recommendation": "add_to_watch_registry_after_controlled_source_review",
"recommended_role": "personal_agent_platform_candidate",
"repository_full_name": "vm0-ai/vm0",
"required_next_gate": "ai_controlled_primary_source_verifier_then_watch_registry_patch",
"risk_flags": [
"requires_dependency_boundary_review",
"likely_requires_paid_provider_boundary_review",
"requires_tool_execution_sandbox_review"
],
"stargazers_count": 1133,
"topics": [
"agentic-workflow",
"ai-agent",
"ai-runtime",
"ai-sandbox",
"claude-code",
"dev-tools",
"sandbox"
],
"watch_addition_recommended": true
},
{
"approval_boundary": {
"approved_for_paid_api_calls": false,
"approved_for_replay": false,
"approved_for_sdk_install": false,
"approved_for_shadow_or_canary": false,
"approved_for_watch_registry_addition": false
},
"archived": false,
"classification": "agent_framework_candidate",
"description": "CORAL is a robust, lightweight infrastructure for multi-agent autonomous self-evolution, built for autoresearch. Works with Claude Code, Codex, Cursor, OpenCode, Kiro, and more.",
"homepage": "https://docs.coralxyz.com",
"html_url": "https://github.com/Human-Agent-Society/CORAL",
"language": "Python",
"pushed_at": "2026-06-26T06:40:13Z",
"recommendation": "add_to_watch_registry_after_controlled_source_review",
"recommended_role": "agent_framework_or_orchestrator_candidate",
"repository_full_name": "human-agent-society/coral",
"required_next_gate": "ai_controlled_primary_source_verifier_then_watch_registry_patch",
"risk_flags": [
"requires_dependency_boundary_review",
"likely_requires_paid_provider_boundary_review"
],
"stargazers_count": 756,
"topics": [
"agent-framework",
"agentic-ai",
"alpha-evolve",
"autonomous-agents",
"autoresearch",
"claude-code",
"code-generation",
"codex",
"coding-agent",
"evolutionary-algorithm",
"large-language-models",
"llm-agents",
"multi-agent",
"opencode",
"self-evolving",
"self-evolving-agents",
"swe-bench"
],
"watch_addition_recommended": true
}
],
"generated_at": "2026-06-28T00:46:51.907924+00:00",
"inputs": {
"discovery_review_generated_at": "2026-06-28T00:46:48.886379+00:00",
"metadata_source": "github_repository_api_summary"
},
"policy": {
"auto_watch_registry_addition_approved": false,
"paid_api_calls_approved": false,
"production_changes_approved": false,
"raw_external_pages_committed": false,
"replacement_decision_allowed": false,
"sdk_installation_approved": false,
"shadow_or_canary_approved": false
},
"schema_version": "agent_market_discovery_classification_v1",
"summary": {
"classification_counts": {
"agent_framework_candidate": 5,
"agent_governance_candidate": 1,
"needs_controlled_research": 2,
"personal_agent_platform_candidate": 1
},
"classified_repositories": 9,
"production_changes_approved": 0,
"recommendation_counts": {
"add_to_watch_registry_after_controlled_source_review": 7,
"controlled_research_before_watch_registry": 2
},
"recommended_watch_additions": 7,
"shadow_or_canary_approved": 0,
"watch_only_or_defer": 2
}
}

286
docs/evaluations/agent_market_discovery_review_2026-06-28.json Normal file
View File

@@ -0,0 +1,286 @@
{
"candidate_drafts": [
{
"approval_boundary": {
"approved_for_paid_api_calls": false,
"approved_for_registry_addition": false,
"approved_for_sdk_install": false,
"approved_for_shadow_or_canary": false,
"controlled_classification_allowed": true
},
"decision": "ai_controlled_primary_source_classification_required",
"html_url": "https://github.com/CopilotKit/CopilotKit",
"new_since_previous_review": false,
"recommended_actions": [
"verify_official_or_primary_sources",
"classify_role_against_awoooi_agent_taxonomy",
"queue_controlled_watch_registry_patch_after_primary_source_verifier",
"do_not_install_sdk_or_call_provider",
"do_not_enter_replacement_replay_before_market_scorecard"
],
"recommended_next_gate": "classify_official_sources_then_update_watch_registry",
"repository_full_name": "copilotkit/copilotkit",
"seen_before": true,
"source_ids": [
"github_ai_agent_topic"
],
"stargazers_count_max": 35522,
"status": "needs_primary_source_classification",
"updated_at_latest": "2026-06-26T03:36:59Z"
},
{
"approval_boundary": {
"approved_for_paid_api_calls": false,
"approved_for_registry_addition": false,
"approved_for_sdk_install": false,
"approved_for_shadow_or_canary": false,
"controlled_classification_allowed": true
},
"decision": "ai_controlled_primary_source_classification_required",
"html_url": "https://github.com/trycua/cua",
"new_since_previous_review": false,
"recommended_actions": [
"verify_official_or_primary_sources",
"classify_role_against_awoooi_agent_taxonomy",
"queue_controlled_watch_registry_patch_after_primary_source_verifier",
"do_not_install_sdk_or_call_provider",
"do_not_enter_replacement_replay_before_market_scorecard"
],
"recommended_next_gate": "classify_official_sources_then_update_watch_registry",
"repository_full_name": "trycua/cua",
"seen_before": true,
"source_ids": [
"github_ai_agent_topic"
],
"stargazers_count_max": 19004,
"status": "needs_primary_source_classification",
"updated_at_latest": "2026-06-26T03:40:04Z"
},
{
"approval_boundary": {
"approved_for_paid_api_calls": false,
"approved_for_registry_addition": false,
"approved_for_sdk_install": false,
"approved_for_shadow_or_canary": false,
"controlled_classification_allowed": true
},
"decision": "ai_controlled_primary_source_classification_required",
"html_url": "https://github.com/eigent-ai/eigent",
"new_since_previous_review": false,
"recommended_actions": [
"verify_official_or_primary_sources",
"classify_role_against_awoooi_agent_taxonomy",
"queue_controlled_watch_registry_patch_after_primary_source_verifier",
"do_not_install_sdk_or_call_provider",
"do_not_enter_replacement_replay_before_market_scorecard"
],
"recommended_next_gate": "classify_official_sources_then_update_watch_registry",
"repository_full_name": "eigent-ai/eigent",
"seen_before": true,
"source_ids": [
"github_agent_framework_topic"
],
"stargazers_count_max": 14394,
"status": "needs_primary_source_classification",
"updated_at_latest": "2026-06-26T03:38:22Z"
},
{
"approval_boundary": {
"approved_for_paid_api_calls": false,
"approved_for_registry_addition": false,
"approved_for_sdk_install": false,
"approved_for_shadow_or_canary": false,
"controlled_classification_allowed": true
},
"decision": "ai_controlled_primary_source_classification_required",
"html_url": "https://github.com/omnigent-ai/omnigent",
"new_since_previous_review": false,
"recommended_actions": [
"verify_official_or_primary_sources",
"classify_role_against_awoooi_agent_taxonomy",
"queue_controlled_watch_registry_patch_after_primary_source_verifier",
"do_not_install_sdk_or_call_provider",
"do_not_enter_replacement_replay_before_market_scorecard"
],
"recommended_next_gate": "classify_official_sources_then_update_watch_registry",
"repository_full_name": "omnigent-ai/omnigent",
"seen_before": true,
"source_ids": [
"github_ai_agent_topic",
"github_agent_framework_topic"
],
"stargazers_count_max": 4924,
"status": "needs_primary_source_classification",
"updated_at_latest": "2026-06-26T03:37:54Z"
},
{
"approval_boundary": {
"approved_for_paid_api_calls": false,
"approved_for_registry_addition": false,
"approved_for_sdk_install": false,
"approved_for_shadow_or_canary": false,
"controlled_classification_allowed": true
},
"decision": "ai_controlled_primary_source_classification_required",
"html_url": "https://github.com/ag2ai/ag2",
"new_since_previous_review": false,
"recommended_actions": [
"verify_official_or_primary_sources",
"classify_role_against_awoooi_agent_taxonomy",
"queue_controlled_watch_registry_patch_after_primary_source_verifier",
"do_not_install_sdk_or_call_provider",
"do_not_enter_replacement_replay_before_market_scorecard"
],
"recommended_next_gate": "classify_official_sources_then_update_watch_registry",
"repository_full_name": "ag2ai/ag2",
"seen_before": true,
"source_ids": [
"github_agent_framework_topic"
],
"stargazers_count_max": 4710,
"status": "needs_primary_source_classification",
"updated_at_latest": "2026-06-26T03:29:07Z"
},
{
"approval_boundary": {
"approved_for_paid_api_calls": false,
"approved_for_registry_addition": false,
"approved_for_sdk_install": false,
"approved_for_shadow_or_canary": false,
"controlled_classification_allowed": true
},
"decision": "ai_controlled_primary_source_classification_required",
"html_url": "https://github.com/xbtlin/ai-berkshire",
"new_since_previous_review": false,
"recommended_actions": [
"verify_official_or_primary_sources",
"classify_role_against_awoooi_agent_taxonomy",
"queue_controlled_watch_registry_patch_after_primary_source_verifier",
"do_not_install_sdk_or_call_provider",
"do_not_enter_replacement_replay_before_market_scorecard"
],
"recommended_next_gate": "classify_official_sources_then_update_watch_registry",
"repository_full_name": "xbtlin/ai-berkshire",
"seen_before": true,
"source_ids": [
"github_ai_agent_topic"
],
"stargazers_count_max": 2172,
"status": "needs_primary_source_classification",
"updated_at_latest": "2026-06-26T03:41:15Z"
},
{
"approval_boundary": {
"approved_for_paid_api_calls": false,
"approved_for_registry_addition": false,
"approved_for_sdk_install": false,
"approved_for_shadow_or_canary": false,
"controlled_classification_allowed": true
},
"decision": "ai_controlled_primary_source_classification_required",
"html_url": "https://github.com/trpc-group/trpc-agent-go",
"new_since_previous_review": false,
"recommended_actions": [
"verify_official_or_primary_sources",
"classify_role_against_awoooi_agent_taxonomy",
"queue_controlled_watch_registry_patch_after_primary_source_verifier",
"do_not_install_sdk_or_call_provider",
"do_not_enter_replacement_replay_before_market_scorecard"
],
"recommended_next_gate": "classify_official_sources_then_update_watch_registry",
"repository_full_name": "trpc-group/trpc-agent-go",
"seen_before": true,
"source_ids": [
"github_agent_framework_topic"
],
"stargazers_count_max": 1408,
"status": "needs_primary_source_classification",
"updated_at_latest": "2026-06-26T01:55:30Z"
},
{
"approval_boundary": {
"approved_for_paid_api_calls": false,
"approved_for_registry_addition": false,
"approved_for_sdk_install": false,
"approved_for_shadow_or_canary": false,
"controlled_classification_allowed": true
},
"decision": "ai_controlled_primary_source_classification_required",
"html_url": "https://github.com/vm0-ai/vm0",
"new_since_previous_review": false,
"recommended_actions": [
"verify_official_or_primary_sources",
"classify_role_against_awoooi_agent_taxonomy",
"queue_controlled_watch_registry_patch_after_primary_source_verifier",
"do_not_install_sdk_or_call_provider",
"do_not_enter_replacement_replay_before_market_scorecard"
],
"recommended_next_gate": "classify_official_sources_then_update_watch_registry",
"repository_full_name": "vm0-ai/vm0",
"seen_before": true,
"source_ids": [
"github_ai_agent_topic"
],
"stargazers_count_max": 1131,
"status": "needs_primary_source_classification",
"updated_at_latest": "2026-06-26T03:32:54Z"
},
{
"approval_boundary": {
"approved_for_paid_api_calls": false,
"approved_for_registry_addition": false,
"approved_for_sdk_install": false,
"approved_for_shadow_or_canary": false,
"controlled_classification_allowed": true
},
"decision": "ai_controlled_primary_source_classification_required",
"html_url": "https://github.com/Human-Agent-Society/CORAL",
"new_since_previous_review": false,
"recommended_actions": [
"verify_official_or_primary_sources",
"classify_role_against_awoooi_agent_taxonomy",
"queue_controlled_watch_registry_patch_after_primary_source_verifier",
"do_not_install_sdk_or_call_provider",
"do_not_enter_replacement_replay_before_market_scorecard"
],
"recommended_next_gate": "classify_official_sources_then_update_watch_registry",
"repository_full_name": "human-agent-society/coral",
"seen_before": true,
"source_ids": [
"github_agent_framework_topic"
],
"stargazers_count_max": 754,
"status": "needs_primary_source_classification",
"updated_at_latest": "2026-06-26T03:05:29Z"
}
],
"generated_at": "2026-06-28T00:46:48.886379+00:00",
"inputs": {
"candidate_registry_schema_version": "agent_replacement_candidates_v1",
"previous_review_generated_at": "2026-06-26T03:41:38.325050+00:00",
"source_registry_schema_version": "agent_market_watch_sources_v1",
"watch_report_generated_at": "2026-06-26T03:41:17.266599+00:00",
"watch_report_mode": "live"
},
"policy": {
"auto_registry_addition_approved": false,
"paid_api_calls_approved": false,
"production_changes_approved": false,
"replacement_decision_allowed": false,
"sdk_installation_approved": false,
"shadow_or_canary_approved": false
},
"schema_version": "agent_market_discovery_review_v1",
"summary": {
"already_watched_or_registered": 0,
"auto_registry_additions_approved": 0,
"controlled_classification_queue": 9,
"discovered_items": 10,
"discovery_sources": 2,
"new_controlled_classification_queue": 0,
"production_changes_approved": 0,
"shadow_or_canary_approved": 0,
"source_failures": 0,
"unique_repositories": 9
}
}

839
docs/evaluations/agent_market_governance_snapshot_2026-06-28.json Normal file
View File

@@ -0,0 +1,839 @@
{
"candidate_groups": {
"production_baseline": [
"openclaw_incumbent"
],
"replay_or_integration_blocked": [
"claude_agent_sdk_remediator",
"crewai_flows_crews",
"langgraph_incident_kernel",
"nemo_nemotron_fabric",
"openai_agents_sdk_coordinator"
],
"watch_only_candidates": [
"agentos_framework",
"bernstein_agent_governance",
"hermes_agent_personal_platform",
"microsoft_agent_governance_toolkit",
"pydantic_deepagents",
"thclaws_agent_harness"
],
"watch_only_scorecard_prescreen_ready": []
},
"candidate_statuses": [
{
"approvals": {
"paid_api": false,
"production_routing": false,
"replay": false,
"sdk_install": false,
"shadow_or_canary": false
},
"candidate_id": "openclaw_incumbent",
"current_gate": "production_decision_core",
"display_name": "OpenClaw incumbent",
"evaluation_priority": "baseline",
"evidence": {
"latest_replay_summary": null,
"latest_smoke_gate": null,
"latest_smoke_matrix": null,
"latest_smoke_model": null
},
"gate_status": "production_baseline",
"integration_decision": "",
"operator_blockers": [],
"required_next_gate": "formal_replacement_adr_and_promotion_gate_required",
"role": "current_production_decision_core",
"score": null
},
{
"approvals": {
"paid_api": false,
"production_routing": false,
"replay": false,
"sdk_install": false,
"shadow_or_canary": false
},
"candidate_id": "openai_agents_sdk_coordinator",
"current_gate": "offline_replay",
"display_name": "OpenAI Agents SDK Coordinator",
"evaluation_priority": "must_test",
"evidence": {
"latest_replay_summary": "docs/evaluations/agent_openai_coordinator_replay_summary_2026-06-02.json",
"latest_smoke_gate": null,
"latest_smoke_matrix": null,
"latest_smoke_model": null
},
"gate_status": "registered_no_review",
"integration_decision": "deterministic_offline_coordinator_blocked_does_not_beat_openclaw",
"operator_blockers": [],
"required_next_gate": "continue_weekly_primary_source_market_watch",
"role": "coordinator_orchestrator",
"score": null
},
{
"approvals": {
"paid_api": false,
"production_routing": false,
"replay": false,
"sdk_install": false,
"shadow_or_canary": false
},
"candidate_id": "langgraph_incident_kernel",
"current_gate": "offline_replay",
"display_name": "LangGraph Incident Kernel",
"evaluation_priority": "must_test",
"evidence": {
"latest_replay_summary": "docs/evaluations/agent_langgraph_replay_summary_2026-06-02.json",
"latest_smoke_gate": null,
"latest_smoke_matrix": null,
"latest_smoke_model": null
},
"gate_status": "registered_no_review",
"integration_decision": "deterministic_offline_kernel_blocked_does_not_beat_openclaw",
"operator_blockers": [],
"required_next_gate": "continue_weekly_primary_source_market_watch",
"role": "durable_incident_workflow_kernel",
"score": null
},
{
"approvals": {
"paid_api": false,
"production_routing": false,
"replay": false,
"sdk_install": false,
"shadow_or_canary": false
},
"candidate_id": "nemo_nemotron_fabric",
"current_gate": "blocked_existing_replay_evidence",
"display_name": "NVIDIA NeMo Agent Toolkit + Nemotron Fabric",
"evaluation_priority": "must_test",
"evidence": {
"latest_replay_summary": null,
"latest_smoke_gate": "docs/evaluations/agent_nemotron_contract_tuned_49b_v15_smoke_gate_2026-06-02.json",
"latest_smoke_matrix": "docs/evaluations/agent_nemotron_contract_tuned_smoke_matrix_2026-06-02.json",
"latest_smoke_model": "nvidia/llama-3.3-nemotron-super-49b-v1.5"
},
"gate_status": "integration_blocked",
"integration_decision": "do_not_integrate_refresh_evidence_then_smoke_gate",
"operator_blockers": [
"changed_sources_reviewed_by_ai_controlled_verifier",
"market_scorecard_refreshed_if_primary_sources_changed_semantically",
"no_sdk_install_without_dependency_approval",
"no_paid_provider_use_without_cost_and_data_boundary_approval",
"5_record_smoke_gate_passes",
"latency_and_output_contract_blockers_resolved",
"cost_approval_recorded"
],
"required_next_gate": "refresh_source_evidence_then_5_record_smoke_only",
"role": "agent_fabric_tool_model_evaluator",
"score": 0.8033
},
{
"approvals": {
"paid_api": false,
"production_routing": false,
"replay": false,
"sdk_install": false,
"shadow_or_canary": false
},
"candidate_id": "claude_agent_sdk_remediator",
"current_gate": "has_offline_replay_summary",
"display_name": "Claude Agent SDK Remediator",
"evaluation_priority": "must_test",
"evidence": {
"latest_replay_summary": "docs/evaluations/agent_claude_remediator_replay_summary_2026-06-02.json",
"latest_smoke_gate": null,
"latest_smoke_matrix": null,
"latest_smoke_model": null
},
"gate_status": "integration_blocked",
"integration_decision": "do_not_integrate_refresh_replay_gate",
"operator_blockers": [
"changed_sources_reviewed_by_ai_controlled_verifier",
"market_scorecard_refreshed_if_primary_sources_changed_semantically",
"no_sdk_install_without_dependency_approval",
"no_paid_provider_use_without_cost_and_data_boundary_approval",
"offline_adapter_contract_valid",
"50_record_hidden_label_replay_beats_openclaw_baseline",
"cost_approval_recorded"
],
"required_next_gate": "refresh_scorecard_then_offline_replay_or_promotion_gate",
"role": "devops_code_remediation_agent",
"score": 0.7533
},
{
"approvals": {
"paid_api": false,
"production_routing": false,
"replay": false,
"sdk_install": false,
"shadow_or_canary": false
},
"candidate_id": "google_adk_stack",
"current_gate": "offline_replay",
"display_name": "Google Agent Development Kit Stack",
"evaluation_priority": "can_test",
"evidence": {
"latest_replay_summary": null,
"latest_smoke_gate": null,
"latest_smoke_matrix": null,
"latest_smoke_model": null
},
"gate_status": "registered_no_review",
"integration_decision": "",
"operator_blockers": [],
"required_next_gate": "continue_weekly_primary_source_market_watch",
"role": "gemini_vertex_agent_stack",
"score": null
},
{
"approvals": {
"paid_api": false,
"production_routing": false,
"replay": false,
"sdk_install": false,
"shadow_or_canary": false
},
"candidate_id": "microsoft_agent_framework",
"current_gate": "offline_replay",
"display_name": "Microsoft Agent Framework",
"evaluation_priority": "can_test",
"evidence": {
"latest_replay_summary": null,
"latest_smoke_gate": null,
"latest_smoke_matrix": null,
"latest_smoke_model": null
},
"gate_status": "registered_no_review",
"integration_decision": "",
"operator_blockers": [],
"required_next_gate": "continue_weekly_primary_source_market_watch",
"role": "enterprise_workflow_agent_stack",
"score": null
},
{
"approvals": {
"paid_api": false,
"production_routing": false,
"replay": false,
"sdk_install": false,
"shadow_or_canary": false
},
"candidate_id": "crewai_flows_crews",
"current_gate": "not_yet_replayed",
"display_name": "CrewAI Flows + Crews",
"evaluation_priority": "secondary",
"evidence": {
"latest_replay_summary": null,
"latest_smoke_gate": null,
"latest_smoke_matrix": null,
"latest_smoke_model": null
},
"gate_status": "integration_blocked",
"integration_decision": "do_not_integrate_prepare_no_cost_offline_adapter",
"operator_blockers": [
"changed_sources_reviewed_by_ai_controlled_verifier",
"market_scorecard_refreshed_if_primary_sources_changed_semantically",
"no_sdk_install_without_dependency_approval",
"no_paid_provider_use_without_cost_and_data_boundary_approval",
"offline_adapter_contract_valid",
"50_record_hidden_label_replay_beats_openclaw_baseline"
],
"required_next_gate": "create_no_sdk_no_api_adapter_then_offline_replay",
"role": "rapid_agent_team_prototype",
"score": 0.6033
},
{
"approvals": {
"paid_api": false,
"production_routing": false,
"replay": false,
"sdk_install": false,
"shadow_or_canary": false
},
"candidate_id": "hermes_agent_personal_platform",
"current_gate": "watch_only_primary_source_monitoring",
"display_name": "NousResearch Hermes Agent",
"evaluation_priority": "watch_only",
"evidence": {
"latest_replay_summary": null,
"latest_smoke_gate": null,
"latest_smoke_matrix": null,
"latest_smoke_model": null
},
"gate_status": "watch_only_blocked",
"integration_decision": "remain_watch_only_until_evidence_gap_resolved",
"operator_blockers": [
"integration_review_must_confirm_watch_only_stage",
"discovery_classification_must_recommend_watch_addition"
],
"required_next_gate": "continue_watch_only_until_primary_source_evidence_is_sufficient",
"role": "personal_agent_platform_candidate",
"score": null
},
{
"approvals": {
"paid_api": false,
"production_routing": false,
"replay": false,
"sdk_install": false,
"shadow_or_canary": false
},
"candidate_id": "microsoft_agent_governance_toolkit",
"current_gate": "watch_only_primary_source_monitoring",
"display_name": "Microsoft Agent Governance Toolkit",
"evaluation_priority": "watch_only",
"evidence": {
"latest_replay_summary": null,
"latest_smoke_gate": null,
"latest_smoke_matrix": null,
"latest_smoke_model": null
},
"gate_status": "watch_only_blocked",
"integration_decision": "remain_watch_only_until_evidence_gap_resolved",
"operator_blockers": [
"integration_review_must_confirm_watch_only_stage",
"discovery_classification_must_recommend_watch_addition"
],
"required_next_gate": "continue_watch_only_until_primary_source_evidence_is_sufficient",
"role": "agent_governance_policy_evaluator_candidate",
"score": null
},
{
"approvals": {
"paid_api": false,
"production_routing": false,
"replay": false,
"sdk_install": false,
"shadow_or_canary": false
},
"candidate_id": "thclaws_agent_harness",
"current_gate": "watch_only_primary_source_monitoring",
"display_name": "thClaws Agent Harness",
"evaluation_priority": "watch_only",
"evidence": {
"latest_replay_summary": null,
"latest_smoke_gate": null,
"latest_smoke_matrix": null,
"latest_smoke_model": null
},
"gate_status": "watch_only_blocked",
"integration_decision": "do_not_integrate_watch_only_primary_source_monitoring",
"operator_blockers": [
"discovery_classification_must_recommend_watch_addition",
"changed_sources_reviewed_by_ai_controlled_verifier",
"market_scorecard_refreshed_if_primary_sources_changed_semantically",
"no_sdk_install_without_dependency_approval",
"no_paid_provider_use_without_cost_and_data_boundary_approval",
"ai_controlled_primary_sources_verifier_passed",
"watch_registry_baseline_refreshed",
"explicit_priority_upgrade_before_replay",
"cost_approval_recorded"
],
"required_next_gate": "continue_watch_only_until_primary_source_evidence_is_sufficient",
"role": "agent_framework_or_orchestrator_candidate",
"score": null
},
{
"approvals": {
"paid_api": false,
"production_routing": false,
"replay": false,
"sdk_install": false,
"shadow_or_canary": false
},
"candidate_id": "pydantic_deepagents",
"current_gate": "watch_only_primary_source_monitoring",
"display_name": "Pydantic DeepAgents",
"evaluation_priority": "watch_only",
"evidence": {
"latest_replay_summary": null,
"latest_smoke_gate": null,
"latest_smoke_matrix": null,
"latest_smoke_model": null
},
"gate_status": "watch_only_blocked",
"integration_decision": "remain_watch_only_until_evidence_gap_resolved",
"operator_blockers": [
"integration_review_must_confirm_watch_only_stage",
"discovery_classification_must_recommend_watch_addition"
],
"required_next_gate": "continue_watch_only_until_primary_source_evidence_is_sufficient",
"role": "agent_framework_or_orchestrator_candidate",
"score": null
},
{
"approvals": {
"paid_api": false,
"production_routing": false,
"replay": false,
"sdk_install": false,
"shadow_or_canary": false
},
"candidate_id": "agentos_framework",
"current_gate": "watch_only_primary_source_monitoring",
"display_name": "AgentOS Framework",
"evaluation_priority": "watch_only",
"evidence": {
"latest_replay_summary": null,
"latest_smoke_gate": null,
"latest_smoke_matrix": null,
"latest_smoke_model": null
},
"gate_status": "watch_only_blocked",
"integration_decision": "remain_watch_only_until_evidence_gap_resolved",
"operator_blockers": [
"integration_review_must_confirm_watch_only_stage",
"discovery_classification_must_recommend_watch_addition"
],
"required_next_gate": "continue_watch_only_until_primary_source_evidence_is_sufficient",
"role": "agent_framework_or_orchestrator_candidate",
"score": null
},
{
"approvals": {
"paid_api": false,
"production_routing": false,
"replay": false,
"sdk_install": false,
"shadow_or_canary": false
},
"candidate_id": "bernstein_agent_governance",
"current_gate": "watch_only_primary_source_monitoring",
"display_name": "Bernstein Agent Governance",
"evaluation_priority": "watch_only",
"evidence": {
"latest_replay_summary": null,
"latest_smoke_gate": null,
"latest_smoke_matrix": null,
"latest_smoke_model": null
},
"gate_status": "watch_only_blocked",
"integration_decision": "do_not_integrate_watch_only_primary_source_monitoring",
"operator_blockers": [
"discovery_classification_must_recommend_watch_addition",
"changed_sources_reviewed_by_ai_controlled_verifier",
"market_scorecard_refreshed_if_primary_sources_changed_semantically",
"no_sdk_install_without_dependency_approval",
"no_paid_provider_use_without_cost_and_data_boundary_approval",
"ai_controlled_primary_sources_verifier_passed",
"watch_registry_baseline_refreshed",
"explicit_priority_upgrade_before_replay",
"cost_approval_recorded"
],
"required_next_gate": "continue_watch_only_until_primary_source_evidence_is_sufficient",
"role": "agent_governance_policy_evaluator_candidate",
"score": null
}
],
"current_decision": "openclaw_remains_production_decision_core",
"evaluation_cadence": {
"controlled_review_gate": "controlled_priority_upgrade_required_before_scorecard_replay_sdk_api_shadow_canary_or_production",
"next_scheduled_run_at": "2026-06-29T09:00:00+08:00",
"primary_source_policy": "primary_sources_only_no_llm_no_sdk_no_paid_api",
"schedule": "weekly_monday_0900_asia_taipei",
"timezone": "Asia/Taipei",
"trigger_modes": [
"scheduled_weekly",
"workflow_dispatch",
"agent_controlled_trigger_after_primary_source_signal"
],
"workflow": ".gitea/workflows/agent-market-watch.yaml"
},
"forbidden_actions_without_new_approval": [
"replace_openclaw",
"enter_shadow_or_canary",
"install_new_agent_sdk",
"call_paid_provider_api",
"run_replay_for_watch_only_candidate",
"change_production_routing"
],
"generated_at": "2026-06-28T00:47:02.688946+00:00",
"inputs": {
"candidate_registry_schema_version": "agent_replacement_candidates_v1",
"discovery_classification_generated_at": "2026-06-28T00:46:51.907924+00:00",
"integration_review_generated_at": "2026-06-26T03:41:38.289424+00:00",
"promotion_review_generated_at": "2026-06-28T00:47:01.200212+00:00",
"watch_report_generated_at": "2026-06-26T03:41:17.266599+00:00"
},
"market_watch_health": {
"blocked_from_integration": 5,
"controlled_queue_reasons": [
"unclassified_discovery_watch_additions_remaining",
"integration_queue_not_empty"
],
"freshness_sla_hours": 168,
"operator_blockers": [
"unclassified_discovery_watch_additions_remaining",
"integration_queue_not_empty"
],
"source_failures_block_priority_upgrade": false,
"source_failures_require_controlled_reverify": false,
"stale_after": "2026-06-29T15:00:00+08:00",
"stale_grace_hours": 6,
"status": "controlled_queue"
},
"next_allowed_actions": [
"continue_weekly_primary_source_market_watch",
"rerun_existing_replay_only_after_evidence_or_adapter_change"
],
"operator_decision_queue": [
{
"approval_boundary": {
"market_scorecard_update_required": false,
"paid_api_approval_required": true,
"priority_upgrade_required": false,
"production_routing_approval_required": true,
"replacement_adr_required": true,
"replay_approval_required": true,
"sdk_install_approval_required": true,
"shadow_or_canary_approval_required": true
},
"candidate_id": "claude_agent_sdk_remediator",
"display_name": "Claude Agent SDK Remediator",
"evidence_refs": [
"docs/evaluations/agent_claude_remediator_replay_summary_2026-06-02.json"
],
"priority": 10,
"queue_status": "blocked_needs_evidence",
"recommended_action": "refresh_scorecard_then_offline_replay_or_promotion_gate",
"risk_notes": [
"Best fit is code and DevOps remediation, not necessarily central incident arbitration.",
"API cost, subscription separation, and vendor boundary must be validated.",
"changed_sources_reviewed_by_ai_controlled_verifier",
"market_scorecard_refreshed_if_primary_sources_changed_semantically",
"no_sdk_install_without_dependency_approval",
"no_paid_provider_use_without_cost_and_data_boundary_approval"
]
},
{
"approval_boundary": {
"market_scorecard_update_required": false,
"paid_api_approval_required": false,
"priority_upgrade_required": false,
"production_routing_approval_required": true,
"replacement_adr_required": true,
"replay_approval_required": true,
"sdk_install_approval_required": true,
"shadow_or_canary_approval_required": true
},
"candidate_id": "crewai_flows_crews",
"display_name": "CrewAI Flows + Crews",
"evidence_refs": [],
"priority": 10,
"queue_status": "blocked_needs_evidence",
"recommended_action": "create_no_sdk_no_api_adapter_then_offline_replay",
"risk_notes": [
"Better for rapid automation teams than high-risk production AIOps core.",
"Durability, strict audit, and permission boundary must be proven in replay.",
"changed_sources_reviewed_by_ai_controlled_verifier",
"market_scorecard_refreshed_if_primary_sources_changed_semantically",
"no_sdk_install_without_dependency_approval",
"no_paid_provider_use_without_cost_and_data_boundary_approval"
]
},
{
"approval_boundary": {
"market_scorecard_update_required": false,
"paid_api_approval_required": true,
"priority_upgrade_required": false,
"production_routing_approval_required": true,
"replacement_adr_required": true,
"replay_approval_required": true,
"sdk_install_approval_required": true,
"shadow_or_canary_approval_required": true
},
"candidate_id": "nemo_nemotron_fabric",
"display_name": "NVIDIA NeMo Agent Toolkit + Nemotron Fabric",
"evidence_refs": [
"nvidia/llama-3.3-nemotron-super-49b-v1.5",
"docs/evaluations/agent_nemotron_contract_tuned_49b_v15_smoke_gate_2026-06-02.json",
"docs/evaluations/agent_nemotron_contract_tuned_smoke_matrix_2026-06-02.json"
],
"priority": 10,
"queue_status": "blocked_needs_evidence",
"recommended_action": "refresh_source_evidence_then_5_record_smoke_only",
"risk_notes": [
"Needs AWOOOI-specific HITL and dangerous-action policy integration.",
"GPU/NIM operating cost must be compared against current local inference.",
"changed_sources_reviewed_by_ai_controlled_verifier",
"market_scorecard_refreshed_if_primary_sources_changed_semantically",
"no_sdk_install_without_dependency_approval",
"no_paid_provider_use_without_cost_and_data_boundary_approval"
]
},
{
"approval_boundary": {
"market_scorecard_update_required": true,
"paid_api_approval_required": false,
"priority_upgrade_required": true,
"production_routing_approval_required": true,
"replacement_adr_required": true,
"replay_approval_required": true,
"sdk_install_approval_required": true,
"shadow_or_canary_approval_required": true
},
"candidate_id": "agentos_framework",
"display_name": "AgentOS Framework",
"evidence_refs": [],
"priority": 40,
"queue_status": "watch_only_blocked",
"recommended_action": "continue_watch_only_until_primary_source_evidence_is_sufficient",
"risk_notes": [
"integration_review_must_confirm_watch_only_stage",
"discovery_classification_must_recommend_watch_addition"
]
},
{
"approval_boundary": {
"market_scorecard_update_required": true,
"paid_api_approval_required": true,
"priority_upgrade_required": true,
"production_routing_approval_required": true,
"replacement_adr_required": true,
"replay_approval_required": true,
"sdk_install_approval_required": true,
"shadow_or_canary_approval_required": true
},
"candidate_id": "bernstein_agent_governance",
"display_name": "Bernstein Agent Governance",
"evidence_refs": [],
"priority": 40,
"queue_status": "watch_only_blocked",
"recommended_action": "continue_watch_only_until_primary_source_evidence_is_sufficient",
"risk_notes": [
"candidate missing from current market scorecard",
"discovery_classification_must_recommend_watch_addition",
"changed_sources_reviewed_by_ai_controlled_verifier",
"market_scorecard_refreshed_if_primary_sources_changed_semantically",
"no_sdk_install_without_dependency_approval",
"no_paid_provider_use_without_cost_and_data_boundary_approval"
]
},
{
"approval_boundary": {
"market_scorecard_update_required": true,
"paid_api_approval_required": false,
"priority_upgrade_required": true,
"production_routing_approval_required": true,
"replacement_adr_required": true,
"replay_approval_required": true,
"sdk_install_approval_required": true,
"shadow_or_canary_approval_required": true
},
"candidate_id": "hermes_agent_personal_platform",
"display_name": "NousResearch Hermes Agent",
"evidence_refs": [],
"priority": 40,
"queue_status": "watch_only_blocked",
"recommended_action": "continue_watch_only_until_primary_source_evidence_is_sufficient",
"risk_notes": [
"integration_review_must_confirm_watch_only_stage",
"discovery_classification_must_recommend_watch_addition"
]
},
{
"approval_boundary": {
"market_scorecard_update_required": true,
"paid_api_approval_required": false,
"priority_upgrade_required": true,
"production_routing_approval_required": true,
"replacement_adr_required": true,
"replay_approval_required": true,
"sdk_install_approval_required": true,
"shadow_or_canary_approval_required": true
},
"candidate_id": "microsoft_agent_governance_toolkit",
"display_name": "Microsoft Agent Governance Toolkit",
"evidence_refs": [],
"priority": 40,
"queue_status": "watch_only_blocked",
"recommended_action": "continue_watch_only_until_primary_source_evidence_is_sufficient",
"risk_notes": [
"integration_review_must_confirm_watch_only_stage",
"discovery_classification_must_recommend_watch_addition"
]
},
{
"approval_boundary": {
"market_scorecard_update_required": true,
"paid_api_approval_required": false,
"priority_upgrade_required": true,
"production_routing_approval_required": true,
"replacement_adr_required": true,
"replay_approval_required": true,
"sdk_install_approval_required": true,
"shadow_or_canary_approval_required": true
},
"candidate_id": "pydantic_deepagents",
"display_name": "Pydantic DeepAgents",
"evidence_refs": [],
"priority": 40,
"queue_status": "watch_only_blocked",
"recommended_action": "continue_watch_only_until_primary_source_evidence_is_sufficient",
"risk_notes": [
"integration_review_must_confirm_watch_only_stage",
"discovery_classification_must_recommend_watch_addition"
]
},
{
"approval_boundary": {
"market_scorecard_update_required": true,
"paid_api_approval_required": true,
"priority_upgrade_required": true,
"production_routing_approval_required": true,
"replacement_adr_required": true,
"replay_approval_required": true,
"sdk_install_approval_required": true,
"shadow_or_canary_approval_required": true
},
"candidate_id": "thclaws_agent_harness",
"display_name": "thClaws Agent Harness",
"evidence_refs": [],
"priority": 40,
"queue_status": "watch_only_blocked",
"recommended_action": "continue_watch_only_until_primary_source_evidence_is_sufficient",
"risk_notes": [
"candidate missing from current market scorecard",
"discovery_classification_must_recommend_watch_addition",
"changed_sources_reviewed_by_ai_controlled_verifier",
"market_scorecard_refreshed_if_primary_sources_changed_semantically",
"no_sdk_install_without_dependency_approval",
"no_paid_provider_use_without_cost_and_data_boundary_approval"
]
},
{
"approval_boundary": {
"market_scorecard_update_required": true,
"paid_api_approval_required": false,
"priority_upgrade_required": true,
"production_routing_approval_required": true,
"replacement_adr_required": true,
"replay_approval_required": true,
"sdk_install_approval_required": true,
"shadow_or_canary_approval_required": true
},
"candidate_id": "google_adk_stack",
"display_name": "Google Agent Development Kit Stack",
"evidence_refs": [],
"priority": 60,
"queue_status": "registered_no_review",
"recommended_action": "continue_weekly_primary_source_market_watch",
"risk_notes": []
},
{
"approval_boundary": {
"market_scorecard_update_required": true,
"paid_api_approval_required": false,
"priority_upgrade_required": true,
"production_routing_approval_required": true,
"replacement_adr_required": true,
"replay_approval_required": true,
"sdk_install_approval_required": true,
"shadow_or_canary_approval_required": true
},
"candidate_id": "langgraph_incident_kernel",
"display_name": "LangGraph Incident Kernel",
"evidence_refs": [
"docs/evaluations/agent_langgraph_replay_summary_2026-06-02.json"
],
"priority": 60,
"queue_status": "registered_no_review",
"recommended_action": "continue_weekly_primary_source_market_watch",
"risk_notes": []
},
{
"approval_boundary": {
"market_scorecard_update_required": true,
"paid_api_approval_required": false,
"priority_upgrade_required": true,
"production_routing_approval_required": true,
"replacement_adr_required": true,
"replay_approval_required": true,
"sdk_install_approval_required": true,
"shadow_or_canary_approval_required": true
},
"candidate_id": "microsoft_agent_framework",
"display_name": "Microsoft Agent Framework",
"evidence_refs": [],
"priority": 60,
"queue_status": "registered_no_review",
"recommended_action": "continue_weekly_primary_source_market_watch",
"risk_notes": []
},
{
"approval_boundary": {
"market_scorecard_update_required": true,
"paid_api_approval_required": false,
"priority_upgrade_required": true,
"production_routing_approval_required": true,
"replacement_adr_required": true,
"replay_approval_required": true,
"sdk_install_approval_required": true,
"shadow_or_canary_approval_required": true
},
"candidate_id": "openai_agents_sdk_coordinator",
"display_name": "OpenAI Agents SDK Coordinator",
"evidence_refs": [
"docs/evaluations/agent_openai_coordinator_replay_summary_2026-06-02.json"
],
"priority": 60,
"queue_status": "registered_no_review",
"recommended_action": "continue_weekly_primary_source_market_watch",
"risk_notes": []
},
{
"approval_boundary": {
"market_scorecard_update_required": false,
"paid_api_approval_required": false,
"priority_upgrade_required": false,
"production_routing_approval_required": true,
"replacement_adr_required": true,
"replay_approval_required": false,
"sdk_install_approval_required": false,
"shadow_or_canary_approval_required": false
},
"candidate_id": "openclaw_incumbent",
"display_name": "OpenClaw incumbent",
"evidence_refs": [],
"priority": 90,
"queue_status": "baseline_protected",
"recommended_action": "keep_openclaw_as_production_decision_core_until_replay_shadow_canary_replacement_adr",
"risk_notes": [
"no_candidate_has_formal_replacement_approval"
]
}
],
"policy": {
"market_scorecard_update_approved": false,
"paid_api_calls_approved": false,
"priority_upgrade_approved": false,
"production_changes_approved": false,
"replacement_decision_allowed": false,
"replay_candidate_approved": false,
"sdk_installation_approved": false,
"shadow_or_canary_approved": false,
"snapshot_is_decision_source": false
},
"schema_version": "agent_market_governance_snapshot_v1",
"summary": {
"blocked_from_integration": 5,
"candidate_count": 13,
"changed_candidates": 5,
"eligible_for_market_scorecard_prescreen": 0,
"integration_queue_count": 5,
"market_scorecard_updates_approved": 0,
"paid_api_calls_approved": 0,
"priority_upgrades_approved": 0,
"production_changes_approved": 0,
"recommended_watch_additions_remaining": 7,
"replacement_decisions_approved": 0,
"replay_candidates_approved": 0,
"sdk_installations_approved": 0,
"shadow_or_canary_approved": 0,
"source_count": 36,
"source_failures": 0,
"watch_only_candidates_reviewed": 6
}
}

218
docs/evaluations/agent_market_watch_promotion_review_2026-06-28.json Normal file
View File

@@ -0,0 +1,218 @@
{
"generated_at": "2026-06-28T00:47:01.200212+00:00",
"inputs": {
"candidate_registry_schema_version": "agent_replacement_candidates_v1",
"discovery_classification_generated_at": "2026-06-28T00:46:51.907924+00:00",
"integration_review_generated_at": "2026-06-26T03:41:38.289424+00:00",
"watch_report_generated_at": "2026-06-26T03:41:17.266599+00:00"
},
"policy": {
"market_scorecard_update_approved": false,
"paid_api_calls_approved": false,
"priority_upgrade_approved": false,
"production_changes_approved": false,
"replacement_decision_allowed": false,
"replay_candidate_approved": false,
"sdk_installation_approved": false,
"shadow_or_canary_approved": false
},
"reviews": [
{
"approved_for_paid_api_calls": false,
"approved_for_replay": false,
"approved_for_sdk_install": false,
"approved_for_shadow_or_canary": false,
"blockers": [
"integration_review_must_confirm_watch_only_stage",
"discovery_classification_must_recommend_watch_addition"
],
"candidate_id": "hermes_agent_personal_platform",
"classification": {
"classification": null,
"recommendation": null,
"repository_full_name": null,
"risk_flags": [],
"watch_addition_recommended": false
},
"decision": "remain_watch_only_until_evidence_gap_resolved",
"display_name": "NousResearch Hermes Agent",
"eligible_for_market_scorecard_prescreen": false,
"integration_stage": "",
"latest_versions": [
"v2026.6.19"
],
"official_url": "https://hermes-agent.nousresearch.com",
"release_version_observed": true,
"required_next_gate": "continue_watch_only_until_primary_source_evidence_is_sufficient",
"role": "personal_agent_platform_candidate",
"source_count": 2,
"source_failures": 0
},
{
"approved_for_paid_api_calls": false,
"approved_for_replay": false,
"approved_for_sdk_install": false,
"approved_for_shadow_or_canary": false,
"blockers": [
"integration_review_must_confirm_watch_only_stage",
"discovery_classification_must_recommend_watch_addition"
],
"candidate_id": "microsoft_agent_governance_toolkit",
"classification": {
"classification": null,
"recommendation": null,
"repository_full_name": null,
"risk_flags": [],
"watch_addition_recommended": false
},
"decision": "remain_watch_only_until_evidence_gap_resolved",
"display_name": "Microsoft Agent Governance Toolkit",
"eligible_for_market_scorecard_prescreen": false,
"integration_stage": "",
"latest_versions": [
"v4.1.0"
],
"official_url": "https://microsoft.github.io/agent-governance-toolkit/",
"release_version_observed": true,
"required_next_gate": "continue_watch_only_until_primary_source_evidence_is_sufficient",
"role": "agent_governance_policy_evaluator_candidate",
"source_count": 2,
"source_failures": 0
},
{
"approved_for_paid_api_calls": false,
"approved_for_replay": false,
"approved_for_sdk_install": false,
"approved_for_shadow_or_canary": false,
"blockers": [
"discovery_classification_must_recommend_watch_addition"
],
"candidate_id": "thclaws_agent_harness",
"classification": {
"classification": null,
"recommendation": null,
"repository_full_name": null,
"risk_flags": [],
"watch_addition_recommended": false
},
"decision": "remain_watch_only_until_evidence_gap_resolved",
"display_name": "thClaws Agent Harness",
"eligible_for_market_scorecard_prescreen": false,
"integration_stage": "watch_only_primary_source_monitoring",
"latest_versions": [
"v0.75.0"
],
"official_url": "https://thclaws.ai",
"release_version_observed": true,
"required_next_gate": "continue_watch_only_until_primary_source_evidence_is_sufficient",
"role": "agent_framework_or_orchestrator_candidate",
"source_count": 2,
"source_failures": 0
},
{
"approved_for_paid_api_calls": false,
"approved_for_replay": false,
"approved_for_sdk_install": false,
"approved_for_shadow_or_canary": false,
"blockers": [
"integration_review_must_confirm_watch_only_stage",
"discovery_classification_must_recommend_watch_addition"
],
"candidate_id": "pydantic_deepagents",
"classification": {
"classification": null,
"recommendation": null,
"repository_full_name": null,
"risk_flags": [],
"watch_addition_recommended": false
},
"decision": "remain_watch_only_until_evidence_gap_resolved",
"display_name": "Pydantic DeepAgents",
"eligible_for_market_scorecard_prescreen": false,
"integration_stage": "",
"latest_versions": [
"0.3.31"
],
"official_url": "https://vstorm-co.github.io/pydantic-deepagents/",
"release_version_observed": true,
"required_next_gate": "continue_watch_only_until_primary_source_evidence_is_sufficient",
"role": "agent_framework_or_orchestrator_candidate",
"source_count": 2,
"source_failures": 0
},
{
"approved_for_paid_api_calls": false,
"approved_for_replay": false,
"approved_for_sdk_install": false,
"approved_for_shadow_or_canary": false,
"blockers": [
"integration_review_must_confirm_watch_only_stage",
"discovery_classification_must_recommend_watch_addition"
],
"candidate_id": "agentos_framework",
"classification": {
"classification": null,
"recommendation": null,
"repository_full_name": null,
"risk_flags": [],
"watch_addition_recommended": false
},
"decision": "remain_watch_only_until_evidence_gap_resolved",
"display_name": "AgentOS Framework",
"eligible_for_market_scorecard_prescreen": false,
"integration_stage": "",
"latest_versions": [
"v0.9.79"
],
"official_url": "https://agentos.sh",
"release_version_observed": true,
"required_next_gate": "continue_watch_only_until_primary_source_evidence_is_sufficient",
"role": "agent_framework_or_orchestrator_candidate",
"source_count": 2,
"source_failures": 0
},
{
"approved_for_paid_api_calls": false,
"approved_for_replay": false,
"approved_for_sdk_install": false,
"approved_for_shadow_or_canary": false,
"blockers": [
"discovery_classification_must_recommend_watch_addition"
],
"candidate_id": "bernstein_agent_governance",
"classification": {
"classification": null,
"recommendation": null,
"repository_full_name": null,
"risk_flags": [],
"watch_addition_recommended": false
},
"decision": "remain_watch_only_until_evidence_gap_resolved",
"display_name": "Bernstein Agent Governance",
"eligible_for_market_scorecard_prescreen": false,
"integration_stage": "watch_only_primary_source_monitoring",
"latest_versions": [
"v2.7.0"
],
"official_url": "https://bernstein.run",
"release_version_observed": true,
"required_next_gate": "continue_watch_only_until_primary_source_evidence_is_sufficient",
"role": "agent_governance_policy_evaluator_candidate",
"source_count": 2,
"source_failures": 0
}
],
"schema_version": "agent_market_watch_promotion_review_v1",
"summary": {
"eligible_for_market_scorecard_prescreen": 0,
"market_scorecard_updates_approved": 0,
"paid_api_calls_approved": 0,
"priority_upgrades_approved": 0,
"production_changes_approved": 0,
"remain_watch_only": 6,
"replay_candidates_approved": 0,
"sdk_installations_approved": 0,
"shadow_or_canary_approved": 0,
"watch_only_candidates_reviewed": 6
}
}

8
docs/schemas/agent_market_discovery_review_v1.schema.json
View File

@@ -64,8 +64,8 @@
"discovered_items",
"unique_repositories",
"already_watched_or_registered",
"manual_classification_required",
"new_manual_classification_required",
"controlled_classification_queue",
"new_controlled_classification_queue",
"source_failures",
"auto_registry_additions_approved",
"production_changes_approved",
@@ -76,8 +76,8 @@
"discovered_items": {"type": "integer", "minimum": 0},
"unique_repositories": {"type": "integer", "minimum": 0},
"already_watched_or_registered": {"type": "integer", "minimum": 0},
"manual_classification_required": {"type": "integer", "minimum": 0},
"new_manual_classification_required": {"type": "integer", "minimum": 0},
"controlled_classification_queue": {"type": "integer", "minimum": 0},
"new_controlled_classification_queue": {"type": "integer", "minimum": 0},
"source_failures": {"type": "integer", "minimum": 0},
"auto_registry_additions_approved": {"type": "integer", "const": 0},
"production_changes_approved": {"type": "integer", "const": 0},

20
docs/schemas/agent_market_governance_snapshot_v1.schema.json
View File

@@ -73,7 +73,7 @@
"next_scheduled_run_at",
"trigger_modes",
"primary_source_policy",
"operator_review_gate"
"controlled_review_gate"
],
"properties": {
"workflow": {
@@ -101,7 +101,7 @@
"type": "string",
"minLength": 1
},
"operator_review_gate": {
"controlled_review_gate": {
"type": "string",
"minLength": 1
}
@@ -115,14 +115,15 @@
"freshness_sla_hours",
"stale_grace_hours",
"stale_after",
"source_failures_block_priority_upgrade",
"source_failures_require_controlled_reverify",
"blocked_from_integration",
"controlled_queue_reasons",
"operator_blockers"
],
"properties": {
"status": {
"type": "string",
"enum": ["healthy", "blocked"]
"enum": ["healthy", "controlled_queue", "blocked"]
},
"freshness_sla_hours": {
"type": "integer",
@@ -136,6 +137,9 @@
"type": "string",
"minLength": 1
},
"source_failures_require_controlled_reverify": {
"type": "boolean"
},
"source_failures_block_priority_upgrade": {
"type": "boolean"
},
@@ -143,6 +147,10 @@
"type": "integer",
"minimum": 0
},
"controlled_queue_reasons": {
"type": "array",
"items": {"type": "string", "minLength": 1}
},
"operator_blockers": {
"type": "array",
"items": {"type": "string", "minLength": 1}
@@ -316,8 +324,8 @@
"enum": [
"baseline_protected",
"blocked_needs_evidence",
"operator_review_required",
"operator_priority_review",
"controlled_review_required",
"controlled_priority_review",
"watch_only_blocked",
"watch_only_monitoring",
"registered_no_review"