Improvement Orchestrator

Coordinates the full improvement pipeline: Generator → Discriminator → Evaluator → Executor → Gate.

When to Use

• Run a full improvement cycle on one or more skills
• Coordinate the 5-stage pipeline end-to-end (with optional evaluator)
• Retry failed improvements with trace-aware feedback (Ralph Wiggum loop)

When NOT to Use

• 只想检查 skill 质量评分 → use improvement-learner
• 只想手动给候选打分 → use improvement-discriminator
• 只想改一个文件 → use improvement-executor
• 只想查基准数据 → use benchmark-store

Pipeline

propose → discriminate → evaluate* → execute → gate (7-layer)
         ↻ Ralph Wiggum: fail → inject trace → retry (max N)
         * evaluate skipped if: no --task-suite, OR low-risk docs/reference/guardrail (adaptive complexity)

Adaptive Complexity Skip: candidates with risk_level=low AND category in (docs, reference, guardrail) skip the evaluator stage entirely. Other categories always run evaluator when --task-suite is provided.

Evaluator→Gate Forwarding: if evaluator produces an artifact, its path is forwarded to gate via --evaluation, enabling RegressionGate to check evaluator verdict.

Baseline Evaluation: when --task-suite is given, orchestrator first runs evaluator in --standalone mode on the current SKILL.md to discover which tasks fail, then injects those failures as --source feedback to the generator.

CLI

python3 scripts/orchestrate.py \
  --target /path/to/skill \        # REQUIRED: skill directory or file to improve
  --state-root /path/to/state \    # REQUIRED: where artifacts are written
  --source feedback.json \         # repeatable: memory/feedback/trace files
  --max-retries 3 \                # default 3: Ralph Wiggum retry attempts
  --task-suite tasks.yaml \        # enables evaluator stage (real LLM eval)
  --eval-mock                      # evaluator uses mock execution, no claude CLI

正确用法: 对一个 skill 运行全流程改进（含 evaluator）

$ python3 scripts/orchestrate.py --target /path/to/skill --state-root ./state --task-suite tasks.yaml

→ 0. Baseline evaluation: 发现 2 个 task 失败，注入 generator

→ 1. 生成候选 → 2. 多人盲审 → 3. 任务评估 → 4. 执行变更 → 5. 7层门禁

→ 失败时自动注入 trace 重试（最多 3 次）

→ stdout: /path/to/state/pipeline-summary.json

错误用法: 只想看评分却用了 orchestrator

$ python3 scripts/orchestrate.py --target /path/to/skill --state-root ./state

→ 会实际执行变更！应该用 improvement-learner 的 self_improve.py

Error Handling

• 每个 subprocess 有 1200s 超时，超时抛 RuntimeError
• evaluator 失败不中断流程（打印警告继续），但 evaluation_failure_trace 会注入下轮
• gate 返回 revert 时自动调用 extract_failure_trace() 写入 traces/trace-{run_id}.json
• pipeline-summary.json 最终输出到 {state-root}/pipeline-summary.json

Output

最终输出 pipeline-summary.json：

{"target": "/path/to/skill", "attempts": 2, "max_retries": 3,
 "final_decision": "keep", "final_candidate_id": "cand-01-docs",
 "final_artifact_path": "/state/receipts/gate-run001-cand-01.json"}

final_decision 取值: keep | revert | reject | pending_promote | no_candidates | no_accepted_candidates

Related Skills

• improvement-generator: Produces candidate proposals (stage 1) — orchestrator calls propose.py
• improvement-discriminator: Multi-reviewer panel scoring (stage 2) — orchestrator calls score.py
• improvement-evaluator: Task suite execution validation (stage 3) — called only when --task-suite provided; baseline failures injected as --source
• improvement-executor: Applies changes with backup/rollback (stage 4) — orchestrator calls execute.py
• improvement-gate: 7-layer quality gate (stage 5) — receives --evaluation artifact when evaluator ran