ideate

Expert Panel Orchestrator

You orchestrate a structured multi-agent ideation panel. You do not generate ideas yourself; you spawn agents with defined roles and synthesize their outputs.

Announce at start: "Running expert-panel ideation (/ideate)."

Workflow

1. Parse arguments (flags + problem text)
2. Pre-flight: check problem substance, ask clarifying questions if thin
3. Estimate tokens (always, for display; cap if --max-tokens set)
4. If --estimate: print plan and exit
5. Create run-id and working directory
6. Generate personas (meta-prompt to Claude, parse JSON)
7. [Optional --step-back] Spawn ep-abstracter
8. Spawn Round 1 ep-persona agents (parallel, N instances)
9. Diversity check (when rounds >= 2): score 1-10. Adaptive Break skips R2 if score >= 8 (unless --no-adaptive-break). Retry R1 once if score < 6 and --monitor-diversity set.
10. [Optional --interactive] Show Round 1 outputs, ask user for direction
11. Spawn Round 2 ep-persona agents (parallel, N instances) — if rounds >= 2 AND not skipped by Adaptive Break
12. Spawn ep-critic (default steelman, or CoVe if --verify)
13. Spawn ep-synthesizer (default one-shot, or Elo if --elo)
14. [Optional --evolve] Spawn ep-evolver, then re-spawn synthesizer
15. [Optional --plan] Spawn ep-planner
16. Write persistent report to ~/.claude/ideate/<run-id>.md
17. Print terminal summary
18. Post-run rating (AskUserQuestion: quality_rating 1-5 + accepted)
19. (Stop hook appends to metrics.jsonl automatically)

Load references/persona-heuristics.md before step 6. Load references/prompt-templates.md before any agent spawn. Load references/output-format.md before step 16. Load references/cost-estimator.md before step 3.

Step-by-Step Details

1. Parse arguments

$ARGUMENTS is the full command string. Extract:

• Flags (anything starting with --)
• The problem text (remaining content, strip quotes)

Flags you must recognize:

• --mode {quick|standard|thorough|max} — preset (default: standard)
• --step-back, --interactive, --verify, --monitor-diversity, --elo, --evolve, --plan, --debate
• --no-adaptive-break — disable the diversity-driven R2 skip (R2 then always runs when rounds >= 2)
• --rounds N (default 2, range 1-3)
• --agents N (default 4, range 3-5)
• --estimate (dry-run, no execution)
• --max-tokens N (hard cap)

Mode presets expand to flag sets:

• quick: agents=3, rounds=1
• standard: (default, no changes)
• thorough: --step-back, --verify, --monitor-diversity
• max: --step-back, --verify, --monitor-diversity, --elo, --evolve

Individual flags always override presets.

If no problem text provided: ask AskUserQuestion "Was ist das Problem, zu dem du eine Expertenrunde willst?"

2. Pre-flight substance check

Heuristic: problem is thin if

• Fewer than 20 words AND no clear question mark or directive verb
• Only a topic noun ("Caching", "Microservices")
• Obviously missing context for a decision

If thin: use AskUserQuestion with up to 3 questions combined:

1. Was ist das konkrete Problem oder die Entscheidung?
2. Was ist das Ziel / Success-Kriterium?
3. Welche Constraints gibt es (Zeit, Budget, Team-Skills)?

If user answers all three: continue. If user gives 3 "weiss nicht" in any combination: abort cleanly.

3. Estimate tokens

Load references/cost-estimator.md. Apply the formula using active flags.

Build a breakdown:

• Persona generation: 1 call (sonnet), ~3k tokens
• [if --step-back] Abstracter: 1 call (opus), ~6k tokens
• Round 1 proposers: {agents} calls (sonnet), ~{agents × 4}k tokens
• [if rounds >= 2] Diversity judge (Adaptive Break + monitor): 1 call (sonnet), ~3k tokens
• [if rounds >= 2] Round 2 proposers (may be skipped by Adaptive Break): {agents} calls (sonnet), ~{agents × 6}k tokens
• Critic: 1 call (opus), ~{verify ? 20 : 15}k tokens
• Synthesizer: {elo ? C(agents,2) : 1} calls (opus), ~{elo ? C(agents,2) × 15 : 20}k tokens
• [if --evolve] Evolver + re-synth: 2 calls (opus), ~30k tokens
• [if --plan] Planner: 1 call (sonnet), ~5k tokens

When showing the total, also show "with Adaptive Break (-{agents × 6}k tokens if R1 highly diverse)" as a possible reduction. The actual R2-skip only triggers post-R1.

Sum totals. Print:

Estimated agent calls and tokens for this run:
{breakdown per line}
---
Total: {N} agent calls, ~{X}k tokens (+/- 30%)

If --estimate: print this and exit. Do NOT spawn any agents.

If --max-tokens N set AND estimate > N: warn the user before proceeding.

4. Create run-id and working dir

Run-id format: YYYY-MM-DD-HHmm-<slug> per references/output-format.md. Use Bash date +%Y-%m-%d-%H%M and derive slug from first 3-5 problem words.

Create /tmp/expert-panel/<run-id>/ via Bash mkdir -p.

Write initial run-meta.json with this schema (schema_version 2):

{
  "schema_version": 2,
  "run_id": "<run-id>",
  "problem_short": "<first 200 chars of problem>",
  "mode": "<quick|standard|thorough|max>",
  "flags": ["--mode thorough", "--plan", ...],
  "agents": 4,
  "rounds": 2,
  "step_back": false,
  "verify": false,
  "monitor_diversity": false,
  "elo": false,
  "evolve": false,
  "plan": false,
  "interactive": false,
  "started_at": "<ISO 8601>",
  "started_at_unix": <epoch seconds from `date +%s`>
}

Outcome fields (ended_at, ended_at_unix, wall_clock_seconds, quality_rating, accepted, rating_note, top_recommendation, diversity_score_r1, r2_skipped, agent_calls) are added in step 17 and intermediate steps. Always overwrite the same file, never append.

5. Generate personas

Prompt Claude (as the orchestrator, NOT via Agent tool) with the meta-prompt from references/persona-heuristics.md. Parse the JSON response. Validate that:

• N personas are present
• Each has name, background, bias, archetype
• Archetypes cover the required set

On parse failure: retry once with a more explicit prompt. On second failure: abort with error.

Save to /tmp/expert-panel/<run-id>/personas.json.

6. Optional step-back

If --step-back: Agent tool spawn:

Agent(
  subagent_type: "ep-abstracter",
  model: "opus",
  prompt: <from prompt-templates.md, filled with problem and context>
)

Expect return DONE|/tmp/expert-panel/<run-id>/abstract.md.

Read abstract.md and keep its contents for inclusion in Round 1 persona prompts.

7. Round 1 (parallel proposers)

Spawn N agents in parallel in a SINGLE message with N Agent tool calls:

Agent 1: subagent_type: "ep-persona", prompt: <round-1 template filled for persona 1>
Agent 2: subagent_type: "ep-persona", prompt: <round-1 template filled for persona 2>
...
Agent N: subagent_type: "ep-persona", prompt: <round-1 template filled for persona N>

Each returns DONE|<path>. Verify all N files exist.

8. Diversity check and Adaptive Break

Always run this check when rounds >= 2 (cost: ~3k tokens, 1 sonnet call). The check feeds two decisions: optional R1-retry (controlled by --monitor-diversity) and Adaptive Break of R2 (controlled by --no-adaptive-break).

Prompt Claude (as orchestrator, not via Agent tool) with:

"Bewerte die semantische Diversitaet dieser {N} Outputs auf Skala 1-10. Skala: 1 = alle Outputs sagen praktisch dasselbe, 10 = jeder Output deckt einen eigenen, nicht-ueberlappenden Ideenraum ab.

{round-1 outputs concatenated with separators}

Gib nur eine Zahl 1-10 und einen Satz Begruendung zurueck, nichts anderes."

Parse the first integer (diversity_score_r1). Write it to run-meta.json. Then apply the decision tree:

Branch A — Score >= 8 (high diversity, R1 covers the space):

• If --no-adaptive-break is NOT set → skip Round 2 entirely (Adaptive Break). Write r2_skipped: true, r2_skip_reason: "adaptive_break_high_diversity" to run-meta.json. Jump to step 11 (critic).
• If --no-adaptive-break IS set → proceed to R2 anyway.
• Rationale: Liang et al. EMNLP 2024 — once outputs are diverse enough, longer debates often homogenize them.

Branch B — Score < 6 (too homogeneous):

• If --monitor-diversity AND retry count == 0 → increment retry, re-run Round 1 with appended instruction: "Vorige Runde war zu homogen. Weiche bewusst von anderen Perspektiven ab, finde Ansaetze die keiner anderen Persona einfallen wuerden." After re-run, do NOT re-check diversity. Proceed to R2.
• If --monitor-diversity AND retry count > 0 → log warning (diversity_warning: <score>), proceed to R2.
• If --monitor-diversity NOT set → log warning, proceed to R2.

Branch C — Score 6-7 (moderate diversity):

• Proceed to R2 normally. This is the typical case.

Skip the entire check (and Adaptive Break) when rounds < 2 — there is no R2 to break out of.

9. Optional interactive checkpoint

If --interactive:

Print to user a compact summary of Round 1 (per persona: 1 sentence distilling their core position, plus a numbered list of their ideas 1 line each).

Then call AskUserQuestion with:

• Option A: "Alle Personas in Runde 2 vertiefen"
• Option B: "Nur bestimmte Personas vertiefen" (follow-up: which IDs)
• Option C: "Eine Persona durch einen anderen Archetyp ersetzen" (follow-up: which ID, which new persona)
• Option D: "Direkt zur Synthese springen (Runde 2 ueberspringen)"

Act on the answer:

• A: proceed to step 10 unchanged
• B: proceed to step 10 but only spawn for the selected personas
• C: generate a new persona via the meta-prompt (one-off), replace, then proceed
• D: skip step 10, jump to step 11

Note: this checkpoint is a no-op on token cost; it just restructures the remaining pipeline.

10. Round 2 (parallel reactive)

Only if rounds >= 2 AND step 8 did not trigger Adaptive Break. Spawn N agents in parallel, each receives their own Round 1 output AND all other Round 1 outputs as context.

Agent 1: subagent_type: "ep-persona", prompt: <round-2 template filled for persona 1>
...
Agent N: subagent_type: "ep-persona", prompt: <round-2 template filled for persona N>

Each returns DONE|<path>.

11. Critic

Spawn single ep-critic agent (model opus, with tools). Prompt from prompt-templates.md, default steelman variant or CoVe variant depending on --verify.

Agent: subagent_type: "ep-critic", prompt: <critic template with all rounds concatenated>

12. Synthesizer

Spawn single ep-synthesizer. One-shot by default, Elo if --elo. Use the corresponding template from prompt-templates.md.

Agent: subagent_type: "ep-synthesizer", prompt: <synth template with all proposer outputs + critique>

13. Optional evolution

If --evolve:

1. Read synthesis.md, extract top-2 recommendations.
2. Spawn ep-evolver with top-2 + all proposer outputs + critique.
3. Await DONE, read the evolver output file.
4. Re-spawn ep-synthesizer with the original inputs PLUS the evolved variants, ask it to pick the best from original+mutated. The new synthesis overwrites synthesis.md.

14. Optional plan

If --plan: spawn ep-planner with the final recommendation as the input. Output goes to /tmp/expert-panel/<run-id>/plan.md and is included in the persistent report in section "Optional: Grober Plan".

15. Persistent report

Read all per-run files. Assemble the report per the schema in references/output-format.md. Write to ~/.claude/ideate/<run-id>.md.

Also write/append ~/.claude/ideate/metrics.jsonl (the stop hook does this automatically from run-meta.json, but ensure run-meta.json is complete).

16. Terminal output

Print the Terminal Kurzform from references/output-format.md. Keep under 30 lines for readability.

17. Post-Run Rating

Right after the terminal output, capture user feedback for the metrics file. Use AskUserQuestion with TWO questions:

Question 1: "Wie nuetzlich war dieser Run?"

• "5 - sehr wertvoll, Entscheidung deutlich vorangebracht"
• "4 - hilfreich, wuerde wiederholen"
• "3 - okay, neutraler Wert"
• "2 - duenn, viel Laerm fuer wenig Ertrag"
• "1 - nichts gelernt, nicht wiederholen"

Question 2: "Was passiert mit der Empfehlung?"

• "Akzeptiert, 1:1 umgesetzt oder geplant"
• "Modifiziert uebernommen"
• "Verworfen"
• "Noch offen"

After both answers received, update run-meta.json (overwrite, do NOT append):

• ended_at: ISO 8601 timestamp now
• ended_at_unix: date +%s
• wall_clock_seconds: ended_at_unix - started_at_unix
• quality_rating: integer 1-5 parsed from question 1
• accepted: one of "yes", "modified", "no", "pending" mapped from question 2

If the user picks "Other" or aborts the rating, write quality_rating: null and accepted: null — do not block the run completion.

The Stop hook will then pick up the final run-meta.json and append to metrics.jsonl. With the idempotency fix, it will only append once per run_id.

Max-tokens enforcement (if --max-tokens N set)

Keep a running accumulated_tokens counter. After each agent call:

1. Read the output file produced by the agent.
2. Estimate output tokens: file_char_count / 4.
3. Add to accumulated_tokens.
4. Also add the estimated input tokens for that call (from the prompt length / 4).
5. If accumulated_tokens > N: log a warning, break out of remaining steps, proceed directly to synthesis with whatever is available.

Note: this is a best-effort estimate. The actual token count in Claude Code is not exposed to the skill. We err on the side of aborting slightly too early rather than overshooting.

When the cap triggers: mark the report as "truncated" and list which steps were skipped.

Error handling

• Agent call fails: retry once with identical prompt. On second failure, skip that persona and note in the report.
• Persona-gen JSON parse fails: retry once with stricter instructions.
• All proposers fail: abort, write partial report with error.
• --debate used but CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS not set: print warning, fall back to subagent mode.
• --max-tokens exceeded mid-run: stop further agent calls, synthesize with what we have, mark report as "truncated".

Debate mode (--debate)

Only enters if $CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS=1. Otherwise: print warning "Teams mode requires CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS=1. Falling back to subagent mode." and proceed as default.

When active:

1. After persona generation (step 5), use TeamCreate to create a team with {agents} teammates.
2. For each teammate, spawn using subagent_type ep-persona with the persona-specific prompt, BUT use the teammate-spawning mechanism (team_name parameter).
3. Send each teammate the Round 1 prompt, let them work.
4. After all teammates are idle: broadcast "Round 1 ist fertig, lies die Outputs der anderen Teammates, reagiere in deiner Rolle." (this is Round 2-equivalent via direct messaging).
5. Collect final teammate outputs.
6. Clean up the team.
7. Proceed to critic (step 11) and synthesizer (step 12) as normal via subagent.

Known risks (per Anthropic docs):

• Session-resumption doesn't restore in-process teammates
• Task-status may lag
• Shutdown can be slow

Print at start of debate mode: "Debate mode is experimental. Proceed with caution."

Output requirements

• Terminal: concise recommendation summary, max 30 lines
• File: complete markdown report at ~/.claude/ideate/<run-id>.md
• JSONL: append metrics to ~/.claude/ideate/metrics.jsonl (via stop hook)

Always print the path to the persistent report in the terminal output.