data-visualization

data-visualization

A standalone single-skill plugin (R63) that brainstorms, selects, and renders the most impactful visualisations for producer artifacts or ad-hoc data into self-contained (or interactive) reports. It is a rendering sibling: it emits presentation siblings (frontmatter-free, never a manifest target) governed by presentation-sibling-contract@1.

Read these three rules first — they bind every invocation. They live up top because violating any one collapses the presenter boundary. The operational detail is in references/boundary-and-visual-value-gate.md (consult it on every non-trivial run — manifest-first).

0. The three rules (front-and-centre, never buried)

0a. The presenter boundary — representation, never interpretation

This skill NEVER computes a number, ordering, or delta that is not already in the source. The operational test, stated once:

Does this operation produce a value, ordering, or delta NOT in the source? Yes → it is interpretation → refuse and route back to the producer / measurement-and-optimisation. No → it is representation → allowed.

• ALLOWED (representation — existing values only): copy/filter rows by existing values, sort by existing columns, keep/drop columns, rename labels, parse dates for display, format units, choose axes, choose scales (log axis, color ramp = visual re-encoding), choose marks, lay out. Render producer-supplied interpretations (findings, recommendations, scores-with-a-reading) faithfully.
• FORBIDDEN (producer computation — route back): group-by count/sum/mean, histograms (bin-count), percent-of-total, deltas, rankings, rolling averages, forecasts, anomaly flags, composite scores, thresholds, verdicts; draw conclusions; own a manifest entry; mutate status:.

scale is visual encoding (allowed). bin/group/aggregate/window are aggregation (forbidden unless the producer precomputed them). The ETL slice is mechanically enforced: ingest.normalise(...) raises BoundaryError on any compute=; build_chart_brief(...) raises on a data shape that would need a group-by (duplicate categories below the displayed grain). When required data is missing or an interpretation is needed → route back; never silently invent a number.

0b. The Vega-Lite spec allowlist (v1 — the single biggest doctrine trap)

Vega-Lite will quietly aggregate inside the chart engine. A generated spec MUST NOT contain aggregate, bin, window, or an aggregating timeUnit and also must not contain joinaggregate, impute, density, loess, regression, pivot, calculate, stack, data.url, href/url encoding channels, image marks, or HTML tooltip formatting. If the skill emitted such a spec while claiming it "only rendered", the chart engine/browser — not the producer — would be computing, fetching, or rendering outside the boundary. render_vegalite.render(...) defensively audits the built spec and raises BoundaryError on any forbidden transform or fetch/HTML surface. Allowed: inline data.values, field encodings, scale, ordinary non-fetching marks, axis/legend config, and sort over existing fields. If a chart genuinely needs an aggregate → route back for the pre-computed values and render those. Full table in references/boundary-and-visual-value-gate.md §1.

0c. The R68 bare-number rule

A bare number, grade, or colored gauge is never an acceptable output — it reads as a verdict, and verdicts are the producer's to make. Every rendered score carries its producer-supplied written interpretation beside it. No producer interpretation for a score → do not render a standalone gauge: route back for the interpretation, or render the underlying values as a chart that earns its keep (§0d). (Pairs with the SEO carve-out: keep scores — they are the expected idiom — but always pair them with interpretation; de-gate, don't de-score.)

0d. The visual-value gate — "earns its keep" (R29)

Run before selecting any chart:

Does the visual encoding (position · length · color · spatial · temporal) carry information a reader cannot get from the tabular values at a glance — a trend, distribution shape, outlier, correlation, density, temporal sequence (when/overlap/gaps), or coverage across two dimensions (which cells are empty)?

• Yes → the visual earns its keep. Proceed.
• No → the ask is markdown-equivalent. Refuse and say so; offer the table the data actually wants instead of a decorative chart.

Every output records a one-line visual_value_rationale naming the specific perceptual gain ("reveals the seasonal trend", "surfaces the long-tail outlier" — not "this needed a chart"). A generic rationale is itself a gate failure; viz_common.sibling_meta raises on an empty/boilerplate rationale.

1. The two phases (the user-facing behaviour)

This skill runs in two phases. Phase 1 plans (and never writes a durable artifact); Phase 2 renders the one durable output — the HTML sibling.

Phase 1 — Elicit · brainstorm · plan

1. Run §0a + §0d first. If the ask needs a number/ordering/delta not in the source → refuse + route back (don't proceed to plan). If the visual is markdown-equivalent → refuse + offer a table.
2. Project-surface + manifest-first. If a project exists, read MANIFEST.md to find the producer artifact to render (never glob the tree blindly). The sibling will ride beside that artifact (§7).
3. Ingest + detect the data shape. Use scripts/python/ingest.py — normalise() reshapes existing values only; detect_roles() is advisory (numeric IDs / years / ZIP / SKU / status enums are dimensions, not measures — override the guess with judgement).
4. Elicit goal + audience + delivery context with a few targeted questions: one-off or recurring? emailed or lives-on-a-drive? static or interactive? print-needed? data-sensitivity? who reads it? (the §4 scenario matrix in the spec drives the tech choice).
5. Brainstorm 2–3 options + recommend. Reason over references/chart-selection.md (simplest-truthful-first; time+measure→line, category+measure→bar, 2 measures→scatter, part-of-whole→bar by default, pie only as a narrow opt-in). Structural (no-measure) shapes also render: time+category (no measure)→timeline (a start+end pair → Gantt); two categories (no measure)→matrix (a presence/value grid). These plot existing dates / existing categorical cell values — no quantity is invented (a count over them is still producer-precomputed). State the reading path for each. The in-process plan object is build_chart_brief(...) — never a durable JSON artifact; the only durable output is the HTML sibling.
6. Optional interactive option-picker (Phase 3 — roadmapped). Instead of describing options in text, render the candidates and serve them to a local URL the user opens to pick (the brainstorming visual companion pattern). This is the principal use-case for the skill-managed ephemeral server escape-hatch (§3). Phase 1 of the build ships file-first only; the picker server is the Phase-3 roadmap item — point the user to it, don't fake it.

The plan is reviewable; rendering follows the user's nod (no autonomous greenlight — P8/R25).

Phase 2 — Render (per the plan)

1. ETL the source into the viz-ready shape on the fly (existing values only).
2. Render via scripts/python/render_vegalite.py → a self-contained HTML sibling. The spec is built only from the brief's encodings and audited against the §0b allowlist before it can be embedded.
3. The output is a presentation sibling: frontmatter-free, vendored/offline (no http:///https://), provenance-stamped (data-source-hash), riding beside the source (§7). Optionally request artifact-reviewer with role: presentation_sibling_reviewer for an independent review of provenance, presenter-boundary integrity, chart honesty, accessibility, portability, and impeccable craft. The local references/quality-rubric.md remains the producer self-check; the reviewer role is the cross-skill review home.

2. Script reference (helper scripts — for Claude, never the user)

All scripts self-locate (Path(__file__).resolve().parent) and self-bootstrap their own directory onto sys.path, so they import their siblings and resolve templates/ + vendor/ regardless of the caller's CWD — no cd required. They reshape + render existing values only; none computes a new number. Stdlib + Jinja2 (the template ships in-plugin).

Invocation — call them as modules from anywhere (the project root is fine):

python3 "${CLAUDE_SKILL_DIR}/scripts/python/render_vegalite.py"   # or import:
python3 -c "import sys; sys.path.insert(0, '${CLAUDE_SKILL_DIR}/scripts/python'); \
from render_vegalite import render; render(rows, goal, out_dir, source=src)"

Because each script inserts its own directory onto sys.path at import time, the bare-name sibling imports (from ingest import …, from build_chart_brief import …) resolve even when Claude runs python3 ${CLAUDE_SKILL_DIR}/scripts/python/render_vegalite.py from the project root — no ModuleNotFoundError, no PYTHONPATH export needed.

Script	Phase	Purpose / boundary
ingest.py	1	normalise(rows, *, sort_by=…, where=…, keep_columns=…) reshapes existing values; detect_roles(rows) is an advisory role guess (override it). normalise(..., compute=…) raises BoundaryError — any compute routes back. None-safe sort for sparse rows.
build_chart_brief.py	1	build_chart_brief(rows, goal, *, source=…, content=…, roles=…, prefer_pie=…) → the in-process plan dict (chart_type, encodings with no aggregating transform, reading_path, mandatory visual_value_rationale, source, source_hash). Raises on empty rows / a group-by-needing shape.
render_vegalite.py	2	render(rows, goal, out_dir, *, source=…, project_root=…, allow_outside_project=False, …) → the self-contained HTML sibling. Path-hardened (out_dir/source must resolve within project_root); writes via temp-file + os.replace and refuses symlinked destinations; displays project-relative source paths; caps self-contained rows/bytes and routes back for summarized artifacts; vendored libraries inlined (offline); spec audited against the §0b allowlist; context-specific escaping (autoescaped attrs + a JSON <script> island).
viz_common.py	both	sibling_path() (rides beside the source), sibling_meta() (the provenance stamp — real content hash, mandatory rationale, no manifest/status key), source_hash() (refuses to hash a missing source unless allow_missing/content).
vendor/	2	Pinned vega / vega-lite / vega-embed (offline). Versions in vendor/VERSIONS.txt.

References (loaded on demand — manifest-first):

Reference	Purpose
references/[email protected]	The sibling-emit contract (provenance, no manifest entry, combined-report rule).
references/boundary-and-visual-value-gate.md	The §0 rules in full: operational test, Vega-Lite spec allowlist, visual-value gate, R68, negative-eval list.
references/chart-selection.md	Data-type → chart-type selection rubric (R32-distilled).
references/quality-rubric.md	AI-reasoned reviewer rubric (chart-honesty / a11y / reading-path) — never a weighted score (R32).
references/[email protected]	The manifest contract for the multi-page combined-report engine (overview + drill-down + glossary) — what a producer fills.

2a. The combined report engine (report-manifest@1)

scripts/python/report_engine.py → render_report(manifest, out_dir, *, stem="report") is the manifest-driven multi-page renderer. A producer (seo-strategist, measurement, …) assembles a report manifest (see references/[email protected]) of pre-computed values + interpretations; the engine emits a self-contained, offline report: an overview (hero · KPI cards · charts · sortable/filterable tables with optional inline ⓘ explainers) + optional per-section drill-down detail pages (✗ Fehler / ⚠ Warnungen / ✓ Bestanden / n-a groups, each finding with its explainer) + an optional glossary, all cross-linked via a switcher. A KPI card carrying a drill slug becomes a link to that section's detail page. The presenter boundary still binds (§0a/§0b): the engine renders only what the manifest already contains — no aggregation, no new numbers. Reach for this whenever an audit/analysis wants the interactive, stakeholder-facing report; the producer supplies the manifest, the engine lays it out.

Grain gate (report path). The engine embeds each chart's Vega-Lite spec and renders it client-side via vegaEmbed, so the ingest compute-boundary cannot catch in-runtime aggregation. render_report therefore audits every chart spec with render_vegalite.assert_no_aggregating_transform before any page is written and routes back (raises ingest.BoundaryError) on the §0b forbidden compute/fetch/XSS surfaces (aggregate / bin / window / joinaggregate / impute / density / loess / regression / pivot / calculate / aggregating timeUnit / stack / data.url / href / url / image marks / HTML tooltip formatting) — the spec-structure half of the renderer-contract grain check (references/renderer-contract.md), applied at report altitude. The artifact-level mark↔row count assertions remain the Task-4 conformance gate's job. Combined reports also cap the inline DATA island and write pages atomically, refusing symlinked destinations.

2b. The renderer dispatcher (single-chart, pluggable — distinct from the engine)

scripts/python/render.py + scripts/python/renderers/ are a separate layer from the report engine: the single-chart, server-side sibling path. Build a chart-brief → select() a Renderer (the renderers.base.Renderer interface; selection is a registry walk over accepts(), never a weighted score — R32) → the renderer emits the presentation sibling. Today the sole registered renderer is the thin Vega-Lite adapter, which delegates to render_vegalite.render (no rewrite); the interface is what lets a future non-Vega renderer (d3/sankey) register without touching existing code, with the grain invariant (grain()) validated per the contract. The two layers coexist by design at different altitudes: the dispatcher emits one self-contained chart; report_engine lays out a whole manifest-driven report and embeds pre-built specs for client-side render — it does not route through the dispatcher and never server-renders a chart. See references/renderer-contract.md.

3. Output target — file-first; ephemeral server is the QA-gated escape-hatch

The default is a file that opens via file:// — a single self-contained HTML (possibly loading a sibling data.json for the dynamic "drop-a-file" scenario). Email/portable mode is the strict case: fully self-contained, vendored/inline, no network at all, with a visible data-sensitivity warning (the data is embedded). Huge inputs are capped by row count and inline-byte size; route back to the producer for a summarized/redacted artifact rather than embedding all rows.

A skill-managed ephemeral server is the sanctioned escape-hatch (Phase-3 roadmap) — used only when QA (orchestrator + rp-cli) judges it warranted, chiefly the interactive option-picker (§1 Phase 1) or a live-refresh view awkward to do client-side. The fixed doctrine lines: the server is ephemeral + activity-bound to the invocation (never a persistent daemon/scheduler — P9) and skill-managed (the user never runs npm run dev — P14). Persistent daemons and user-managed servers stay out. No client-side delta computation — a dynamic sibling re-renders the same encodings against fresh data but never computes cross-period deltas/aggregates in the browser (those arrive pre-computed from measurement — §0a view-time corollary).

Vendor/CDN policy is settled: every durable or interactive/dynamic render uses vendored, pinned libraries from the plugin. No CDN fallback is allowed. If a future ephemeral server needs a new browser library, vendor it into the plugin and update vendor/VERSIONS.txt; otherwise route back/defer.

4. Ownership boundary (stay in lane)

data-visualization is a rendering presenter — a sibling, not a fourth authority beside producer / reviewer / orchestrator (that would blur P11/R38). It runs after a producer has made an artifact and emits a presentation sibling of it. When a request crosses the line, route it to the owner:

Concern	Owner	This skill's role
New KPIs, deltas, rankings, forecasts, aggregates, "what changed"	producer / measurement-and-optimisation	renders the pre-computed values; never derives them
Conclusions, recommendations, verdicts, "which is best"	the producer / artifact-reviewer	renders producer-supplied interpretations faithfully; never makes one
Time-series persistence, period deltas, continuous monitoring	measurement-and-optimisation (owns the DATA)	renders what measurement computed; holds no state
The score's written interpretation (R68)	the producer	renders the score with its interpretation, or routes back
A manifest entry / status: mutation / greenlight	producer + orchestrator + user (P8/R25)	emits a frontmatter-free, un-indexed sibling; never gates a workflow
Brand voice / audience / positioning	brand-dna	consumes; never extracts

The line: representation, transformation, and visual encoding are this skill's; interpretation, computation, data-ownership, and workflow gating are not. If required data is missing or an interpretation is needed → surface the structured request and route back (the negative evals enforce this).

4a. Deferred/blocked closeout notes (P8MN)

• Measurement → presenter time-series handoff (skills-p8mn.4) remains blocked until measurement-contract@1 locks its v1 field list. Current contract status is a draft skeleton with seed fields explicitly marked non-authoritative; data-visualization must not invent a parallel time-series schema or own measurement state.
• Choropleth/geo renderer (skills-p8mn.10) is deliberately deferred. A safe implementation needs offline-vendored topology/GeoJSON licensing clarity, a region-grain invariant, and renderer conformance fixtures. Do not bolt this onto the Vega-Lite default path without that renderer decision.

5. Composability & contract

• A rendered artifact is a presentation sibling under presentation-sibling-contract@1: frontmatter-free, never a manifest target, no status:, carrying source provenance + content hash and the visual-value rationale. It rides beside the canonical artifact (e.g. seo/<instance>/report.html next to report.md) — never a new dashboards/ zone, never indexed (MANIFEST.yaml carries produces: []).
• Consumes (producer-agnostic): durable producer markdown+frontmatter deliverables, producer JSON envelopes in opaque workspace/ scratch, or ad-hoc data the user drops (CSV/JSON/tables). It reads them and emits a sibling — it never mutates them.
• Combined reports: a durable combined HTML report is allowed only when it rides beside a producer-owned envelope artifact (the producer owns the manifest entry; the .html is its un-indexed companion, listing all sources[]). With no owning envelope, a cross-producer view is ephemeral/preview only — regenerate-on-demand, never durable (references/[email protected] Rule 5).

End-of-run (P15 — conditional)

Standard runs (gate passed, chart selected, sibling rendered beside the source with a specific rationale + real provenance hash) end normally — no self-improvement prompt fires.

The prompt fires only on a data-visualization-specific deviation. Triggers and prompt shapes:

• The chart-selection default didn't fit and was overridden → "The rubric picked a <default> for <data shape> but you wanted a <chosen> because <reason>. Should references/chart-selection.md add this shape→chart mapping, or was this a one-off judgement?"
• A boundary call sat on the edge (allowed-vs-route-back) → "I treated <operation> as allowed reshaping / routed it back as interpretation, and you expected the other call. Should the operational test in references/boundary-and-visual-value-gate.md name this case explicitly?"
• detect_roles() mis-guessed a column → "I read <column> as a <role> but it's really a <role> (<reason> — e.g. a year/ID/postal code). Should the detector's advisory heuristics get a note, or was this an override-it-in-the-brief case?"
• A score arrived without a producer interpretation (R68) → "The <score> had no written interpretation from the producer, so I routed back rather than render a bare gauge. Should the §0c handoff surface this ask earlier to the producer?"
• A delivery scenario surfaced that the §3 / spec §4 matrix didn't cover → "You needed <scenario> (<reason>) and the file-first default / ephemeral server didn't fit. Should the matrix add this scenario, or was it a one-off?"
• A Vega-Lite spec needed a transform the §0b allowlist forbids → "The chart wanted a <transform>; I routed back to the producer for the pre-computed values rather than let the engine aggregate. Should anything in the allowlist note change, or was this the boundary working as intended?"

If a trigger fires, surface the specific deviation in context-specific prose (never a generic "any feedback?"). If the user confirms, update SKILL.md (or the relevant reference / script) inline before going idle. Standard runs end at sibling-rendered.

Language Handling

• Detect the conversation language; respond in the user's language unless explicitly asked otherwise.
• If MANIFEST.md declares an output_language: field, honor that declaration over inferred conversation language. The rendered sibling declares its language via render(..., lang=…) (P12 — the <html lang> attribute).
• Translate internal labels, chart titles, axis labels, captions, and the visual-value rationale prose naturally — do not force English headings into non-English output.
• Preserve legal, financial, and technical terms in their original language (Trennungsjahr, MwSt, BTW, Aufhebungsvertrag, etc.) — including in chart labels and data categories drawn from the source.
• Preserve Unicode characters natively (ü, ö, ä, ß, etc.) in titles, labels, and embedded data; never substitute transliterations (ue, oe, ae, ss). The renderer serializes data UTF-8-clean.
• Frontmatter field names and provenance keys (source, source_hash, produced_by, visual_value_rationale, …) stay English regardless of content language. Free-text values (the rationale, the title) may follow content language where natural.