auto-harness

auto-harness

A Claude Code plugin that automates integration of hook-driven test framework patterns into any project. The plugin scaffolds complete test infrastructure directly into target projects, enabling automated testing of Claude Code plugins, MCP tools, commands, skills, and hooks.

Overview

auto-harness is a generator/scaffolding plugin - it doesn't provide test commands itself, but rather generates test infrastructure INTO your project's .claude/ directory. This keeps test configuration project-scoped and portable.

What Gets Generated

When you run /harness:init, the plugin creates:

your-project/
├── .claude/
│   ├── commands/
│   │   ├── run-tests.md      # /run-tests command
│   │   ├── add-test.md       # /add-test command
│   │   └── test-report.md    # /test-report command
│   ├── hooks/
│   │   ├── hooks.json        # UserPromptSubmit hook config
│   │   └── test-wrapper.sh   # Hook script for test mode
│   └── test-state.json       # Runtime state (gitignored)
└── tests/
    └── functional/
        ├── runner.sh         # Core test orchestration
        ├── tests.yaml        # Test definitions (or .json)
        └── reports/          # Generated reports

Installation

Add to your Claude Code plugins:

# Via marketplace (when published)
claude plugins add auto-harness

# Or local development
claude --plugin-dir /path/to/auto-harness

Quick Start

1. Initialize test infrastructure in your project:

   /harness:init
   

   The setup wizard will guide you through:

   • Project type detection
   • Test format selection (YAML or JSON)
   • Initial test generation
   • Configuration verification

2. Start a test run:

   /run-tests
   

   This enters test mode where special commands become active.

3. Execute tests interactively:

   • Type next or n to get the next test
   • Claude executes the test action
   • Type validate <response> or v <response> to check results
   • Repeat until all tests complete

4. Generate a report:

   report
   

Commands

Plugin Commands

Command	Description
/harness:init	Initialize test infrastructure in current project
/harness:validate	Validate existing test setup

Generated Project Commands

After initialization, your project gets these commands:

Command	Description
/run-tests	Start test execution mode
/add-test	Add a new test interactively
/test-report	Generate test report

Test Mode Commands

When test mode is active, these prompts are intercepted:

Command	Shortcut	Description
next	n	Get next test to execute
skip	s	Skip current test
validate <response>	v <response>	Validate Claude's response
status	-	Show test progress
report	-	Generate report
vars	-	Show captured variables
abort	a	Stop test run
help	?	Show available commands

Test Definition Format

YAML Format

---
version: "1.0"
description: "My Project Test Suite"

tests:
  - id: smoke_basic
    description: Basic smoke test
    category: smoke
    action: >
      Confirm the test framework is operational by responding
      with 'Test framework operational'
    expect:
      - contains: "operational"
      - not_contains: "error"
    tags: [smoke, critical]

  - id: mcp_tool_test
    description: Test MCP tool functionality
    category: mcp-tools
    action: "Call mcp__server__tool_name with param: 'test'"
    expect:
      - contains: "success"
      - regex: "result:\\s+(.+)"
    save_as: tool_result
    tags: [mcp, integration]

  - id: dependent_test
    description: Test that uses captured variable
    category: integration
    action: "Process the result: ${tool_result}"
    depends_on: mcp_tool_test
    expect:
      - contains: "processed"
    tags: [integration]

JSON Format (Recommended)

JSON format is recommended as it requires no external dependencies (YAML requires PyYAML which may not be available on all systems).