scientific-figure

Scientific Figure Loop

The artifact is a scientific figure (the rendered image + the plot.py that produces it). Each iteration generates → critiques+grades: a generator authors a rendering script and renders the figure to land the frozen <goals> message; an adversarial critic grades it 0-100 against the fixed rubrics/rubric.md and decides pass; the generator then revises against the critic's concrete findings. The loop runs until the grade clears <pass_threshold> or the budget is hit. All work happens on copies inside a sandbox; the user's data is copied in read-only and never edited.

The cast (all in this folder):

• roles/generator.md — drafts/revises plot.py, renders figure.png by running <render_cmd>, optionally grounds domain content via <lit>; writes generation_notes.md.
• roles/critic.md — the adversarial grader: re-derives each rubric axis independently, spot-checks the figure's numbers against the data, optionally lit-checks domain completeness, and emits schemas/critique.schema.json (the grade + pass + executable findings).
• rubrics/rubric.md — the fixed grading rubric (the critic never edits it).
• schemas/critique.schema.json — the one validated output.

Spawn-or-degrade. On Claude Code, spawn the generator then the critic as real Agent subagents (sequential — the critic needs the generator's figure); otherwise adopt each role inline. You are the orchestrator.

Why the critic grades itself (the honesty problem)

The critic both critiques and grades, which under loop-termination pressure invites inflation and a generator that games the rubric. roles/critic.md + rubrics/rubric.md counter this: the critic (1) applies a fixed rubric it never edits, (2) re-derives each axis from the rendered figure + data + frozen <goals> rather than echoing the generator, (3) recomputes a sample of the figure's numbers itself instead of trusting "it's fixed", (4) holds a fixed, anchored bar with no credit for effort or elapsed iterations, and (5) applies hard gates (a figure value that contradicts the data, a misleading axis, or fabricated data presented as real fails the figure regardless of the average). The generator optimizes the concrete findings; the critic grades holistically against the frozen goal — so "address every finding" does not mechanically buy a pass. Because the two are separate agents, the critic never just rubber-stamps the generator's intent.

When to use

Use when scientific data (or a prompt describing it) exists and the user wants a polished figure pushed past a quality bar with adversarial critique and a graded rubric. Default: run the full generate→critique loop below. Escape hatch: if the user only wants one figure + a critique (no iterating), run one generate + critic pass and stop. Not for writing a paper or doing the analysis, and not for retouching an already-final image.

Setup

Resolve bindings interactively. If loop.run.yaml exists in the working dir, load it, confirm the values in one line, and skip to the loop. Otherwise: on Claude Code (the AskUserQuestion tool is available) infer a likely value for each binding and present it as the recommended option; on other hosts ask each as a quoted plain-text prompt. Then write loop.run.yaml (format: examples/run.example.yaml) and confirm every value — including the distilled <goals>, whether a journal spec applies, and the live/degraded literature tier — before creating any other files.

binding	meaning	default	how to infer
<brief>	the prompt describing the figure to create + the message/claim it must communicate (and, if data exists, what the data represents)	—	the user's request; if pasted as prose, save to <sandbox_root>/brief.md
<data_paths>	data file(s) the figure visualizes (CSV/TSV/parquet/JSON…); empty → an illustrative/schematic figure (the integrity axis then checks internal consistency, not data fidelity)	—	scan the working dir near the request; may be null
<goals>	the figure's communication objective(s), 1-3 bullets — frozen; the critic grades against these and the generator may never abandon them	—	distill from <brief> at setup, confirm with the user in one line
<render_cmd>	command/interpreter that runs the plot script the generator writes (it appends iter<N>/plot.py), in the user's env — the skill ships no plotting deps, the same contract as scientific-writer's <plot_command>	python3	pyproject.toml/.venv/README; e.g. uv run python or a venv python
<style>	optional aesthetic/style guide or a named target journal/venue — when a journal is named, its figure spec is fetched at setup (see below) and both roles conform to / grade against it	—	ask the user; check <brief> for a venue
<pass_threshold>	overall_score (0-100) the critic must reach (and no hard gate) to stop	85	a polished, publication-ready figure without demanding perfection
<budget>	max iterations	6	—
<patience>	stop after this many consecutive no-improvement iterations	2	—
<sandbox_root>	where the plot scripts, figures, critiques, and the ledger live	./sandbox	—

The domain axis is not a binding — the critic auto-detects whether the figure makes an external domain claim (a named pathway, gene set, canonical benchmark, taxonomy, mechanism, or a literature- established number) and activates the domain axis itself; no user toggle.

Literature toolchain (optional, S2 + arXiv only). Domain grounding goes through the sibling literature-search skill — resolve <lit_skill_dir> (it installs as a sibling, e.g. ~/.claude/skills/literature-search/), <lit_py> = python3, and <lit> = <lit_skill_dir>/tools/lit_search.py; append --cache-dir <sandbox_root>/literature/.cache after a subcommand to reuse the cache. Use only the keyless S2 + arXiv core (<lit> search default --source s2, snippet, cite, fulltext); do not use --source openalex|both, ask, or bgpt. Confirm <lit> --help works at setup; if the skill is absent, degrade all retrieval to WebSearch/WebFetch. Record the tier (presence only) in loop.run.yaml.

Reuse what you've already pulled — don't re-query every iteration. Every retrieval is cached under --cache-dir <sandbox_root>/literature/.cache, and each role appends the facts it establishes (claim → number/element → source) to <sandbox_root>/literature/sources.md. Both roles consult that record (and the cache) first and only fetch papers/snippets not already on hand; a value a prior iteration already verified is re-checked by re-reading its recorded source, not by re-searching from scratch. The point of the literature step is correctness, not call volume — once a paper is pulled, work from it.

Journal style sheets (separate path, via web search). When <style> names a journal/venue, fetch its figure guidelines once at setup via WebSearch/WebFetch → <sandbox_root>/style/journal_spec.md (column width in mm, minimum font size, fonts, line weights, color mode, panel-label convention, file requirements). Both roles read this single cached spec — the generator conforms, the critic anchors its aesthetic/clarity axes to it — so the two never grade against divergent specs. This is distinct from <lit>: web search finds the journal's style spec; <lit> (S2/arXiv) checks domain content.

Environment. The generator renders figures by running <render_cmd> in the user's own environment — that code needs third-party deps (matplotlib, pandas, …), so the skill ships none and never installs them; it shells out to <render_cmd> and reads the rendered figure.png. PNG is rendered so the critic can view the image (an SVG would be read as XML). The deliverable is figure.png plus its plot.py — the reproducible source the user re-renders to any vector format. Any helper code the skill writes stays stdlib-only.

Initialise the sandbox once bindings are confirmed (copy the data in read-only; never edit originals):

<sandbox_root>/
├── loop.run.yaml        ← resolved bindings + <goals> + literature_tiers
├── brief.md             ← <brief> (if pasted as prose)
├── ledger.tsv           ← header only (see Ledger)
├── data/                ← read-only COPY of <data_paths>   (omit if no data)
├── style/journal_spec.md ← fetched journal figure spec     (omit if no journal named)
├── literature/.cache/   ← lit_search on-disk cache
└── iter1/               ← created by the generator
    ├── plot.py
    ├── figure.png
    ├── generation_notes.md
    └── critique.json

The loop

<N> starts at 1. Unlike loops that grade an existing baseline, the generator runs first every iteration (there is no input figure to critique) — iteration 1 drafts from scratch, iterations 2+ revise. Re-grade fresh every iteration: the score comes only from a new critique of the current figure, never carried over. Surface-only changes won't move it.

Copy this checklist and tick items off:

• Generate — spawn generator (roles/generator.md) with <brief>, <data_paths>, <goals>, <style> (+ style/journal_spec.md), <render_cmd>, <lit>, and — on iter 2+ — iter<N-1>/critique.json. It writes/edits iter<N>/plot.py, runs <render_cmd> iter<N>/plot.py inside the sandbox to render iter<N>/figure.png, grounds any domain content via <lit>, and writes iter<N>/generation_notes.md.
• Critique + grade — spawn one fresh critic (roles/critic.md) over iter<N>/figure.png + the data + <goals>, applying rubrics/rubric.md: it re-derives each axis 1-5 independently, spot-checks the figure's numbers against the data, optionally lit-checks domain completeness, computes overall_score = 100 × Σscore / (5 × n_axes), applies hard gates → pass, and writes iter<N>/critique.json (validates against schemas/critique.schema.json).
• Log — append one ledger.tsv row (see Ledger).
• Stop check — critique.pass == true, or N == <budget>, or overall_score flat for <patience> iterations → stop (see Stops).
• N = N + 1 and repeat (back to Generate, which now revises against the critique).

A critique looks like (abridged; full shape in schemas/critique.schema.json):

{"iteration": 2, "summary": "Needs revision: honest now, but the MAPK panel omits ERK and the y-axis lacks units.",
 "axes": {"message": {"score": 4, "justification": "Up-regulation reads clearly."},
          "aesthetic": {"score": 3, "justification": "Palette not colorblind-safe (red/green)."},
          "clarity": {"score": 4, "justification": "Y-axis missing units."},
          "integrity": {"score": 4, "justification": "Bar heights match data/levels.csv."},
          "domain": {"score": 3, "justification": "MAPK cascade missing ERK node."}},
 "overall_score": 72.0, "pass": false, "gate_failures": [],
 "spotchecks": [{"target": "group-B bar = 2.4", "method": "recomputed from data/levels.csv", "result": "confirmed"}],
 "findings": [{"urgency": "must_fix", "action_type": "add", "area": "domain:incomplete",
   "finding": "MAPK cascade panel omits ERK1/2 downstream of MEK.", "proposed_action": "Add ERK node + MEK→ERK edge.",
   "target_artifact": "iter2/plot.py", "evidence": "lit snippet: canonical MAPK = RAF→MEK→ERK"}]}

Ledger

<sandbox_root>/ledger.tsv, tab-separated, never commas in free text:

iter	overall_score	pass	message	aesthetic	clarity	integrity	domain	top_fix	generation_summary
1	52.0	no	3	2	2	4	-	label axes + fix palette	baseline draft
2	74.0	no	4	3	4	4	3	add missing MAPK nodes (lit)	relabeled; colorblind palette; +ERK/MEK
3	88.0	yes	5	4	5	5	4	-	rebalanced panels; legend off-data

Use - in the domain column when the domain axis is inactive (n_axes=4). The per-iteration critique.json and generation_notes.md live in iter<N>/. Report the best-scoring iteration when stopping on budget/plateau, not necessarily the last. Leave the sandbox untracked.

Constraints

• Never edit or run anything outside <sandbox_root> — data is copied in read-only at setup; the generator's plot.py and <render_cmd> run from the sandbox; no ../ escapes.
• Never fabricate data, numbers, or domain elements. A figure presented as real data must render from <data_paths>; with no data, the figure must read as clearly illustrative/schematic, not a fake data plot. Domain content (genes, nodes, baselines, reported numbers) added from <lit> comes from a real retrieval that iteration, never invented.
• The grading bar is fixed and reproducible — the critic never relaxes a rubric anchor to let the loop finish; a confirmed hard gate fails the figure regardless of the average.
• Protect <goals> — the generator makes the same message prettier and clearer; it never drops or distorts the intended message to chase a higher score.
• One coherent revision batch per iteration, blocks/gates first, so score moves are attributable.
• No installs — the skill ships no plotting deps; <render_cmd> runs in the user's env, helper code is stdlib-only; literature is the keyless S2 + arXiv core only. Never print or commit API keys (keys.env stays gitignored).

Stops

The loop stops on the first of:

• Pass — critique.pass == true. Report the deliverable (iter<N>/figure.png + plot.py), the score, and the trajectory.
• Budget — N == <budget>. Report the best-scoring iteration as the deliverable.
• Plateau — overall_score flat for <patience> iterations. Report the best iteration + the standing gate_failures/must_fix blockers.

Always end with the deliverable (iter<N>/ path), its overall_score and pass/fail, the per-axis scores, the score trajectory from ledger.tsv, and — if it did not pass — the standing blockers (gate_failures + open must_fix) between the figure and the bar.