By NVIDIA-TAO
Train, evaluate, export, and deploy NVIDIA TAO models for computer vision tasks including classification, detection, segmentation, pose estimation, depth estimation, OCR, and video analysis. Orchestrate AutoML hyperparameter optimization, DEFT iterative data improvement loops, and synthetic data generation. Submit GPU-accelerated training jobs across Docker, Kubernetes, SLURM, Brev, and DGX Cloud platforms.
Performs deep Root Cause Analysis (RCA) on NVIDIA TAO Visual ChangeNet classification experiments with image-evidence-driven investigation. Use when analyzing ChangeNet model failures, investigating poor recall / FAR / PASS-NO_PASS metrics, auditing visual inspection pipeline quality, or running an RCA report for an AOI defect-detection model. Trigger phrases include "RCA on my ChangeNet model", "why is my AOI model failing", "audit ChangeNet predictions", "investigate FAR regressions", "root cause analysis on visual-changenet".
Fine-tune any HuggingFace CV / VLM / LLM model on local NVIDIA GPUs inside an NGC PyTorch container. Use when the user wants to fine-tune a HuggingFace model (full or LoRA), train a vision / VLM / LLM model end-to-end, generate a reproducible HF training pipeline, smoke-test a HuggingFace model locally before scale-up, push a fine-tuned model to the HF Hub with a model card, or emit a self-contained rerun skill for an existing HuggingFace finetune. Supports image classification, object detection, semantic / instance / panoptic segmentation, depth estimation, image-text-to-text VLM (SFT / LoRA), and LLM SFT / DPO / GRPO. Six-step workflow: inspect and qualify, hardware and NGC image, research, generate and smoke, train + eval + infer, push and emit rerun skill.
Integrate a HuggingFace Computer Vision model into the NVIDIA TAO Toolkit ecosystem (tao-core config, tao-pytorch trainer, tao-deploy TensorRT pipeline). Use when the user asks to "integrate a HuggingFace model into TAO", "add an HF model to TAO Toolkit", "wire a HuggingFace ViT/DETR/ SegFormer into tao-pytorch", "build a TAO trainer + deploy pipeline for an HF CV model", or pastes a HuggingFace model URL/ID and wants it turned into a TAO model. Covers the full 7-phase loop: prerequisites check, HuggingFace inspection and validation, codebase exploration, tao-core configuration and native trainer implementation, ONNX export plus TensorRT deploy integration, packaging and L0 testing, container-based end-to-end validation, and (conditional) accuracy/latency tuning. Supports classification, object detection, semantic / instance / panoptic segmentation, zero-shot detection, and depth estimation.
Run the canonical NVIDIA AOI three-phase training pipeline — Phase 1 AutoML baseline (HPO), Phase 2 DEFT loop (RCA → SDG → mining → plain-train retrain), Phase 3 AutoML refinement on the DEFT-augmented dataset. Use when the user asks to "run the AOI workflow", "fine-tune my PCB AOI model end-to-end", "improve my AOI ChangeNet model", or "AOI workflow with AutoML" request — route here instead of tao-run-deft-aoi directly unless the user explicitly asks for the DEFT loop ONLY (e.g. "run JUST the DEFT loop", "skip AutoML, only DEFT"). Also handles the same three-phase pattern for non-AOI DEFT applications — AutoML baseline then DEFT loop warm-started from AutoML's winning HPs then post-DEFT AutoML refinement on the iteration-augmented dataset. Trigger phrases include "run the AOI workflow", "AOI end-to-end", "AutoML + DEFT", "AutoML then DEFT", "tune hyperparameters then DEFT", "DEFT with AutoML at both ends", "warm-start DEFT", "improve my AOI model".
Run AutoML / hyperparameter optimization (HPO) for NVIDIA TAO networks using AutoMLRunner. Handles algorithm selection (bayesian, hyperband, asha, bohb, llm, hybrid, autoresearch), WandB experiment tracking, job execution on any TAO SDK platform, result interpretation, and per-rec custom evaluation hooks. Use when the user mentions TAO AutoML, hyperparameter optimization, HPO, automl, automl_settings, AutoMLRunner, tao_automl, bayesian search, hyperband, ASHA, LLM-guided search, autoresearch, or wants to tune training hyperparameters for any TAO network. Platform-agnostic — runs on any SDK (Brev, SLURM, Kubernetes, Docker).
Own this plugin?
Verify ownership to unlock analytics, metadata editing, and a verified badge. GitHub access is read-only (username + org membership).
Sign in to claimOwn this plugin?
Verify ownership to unlock analytics, metadata editing, and a verified badge. GitHub access is read-only (username + org membership).
Sign in to claimBased on adoption, maintenance, documentation, and repository signals. Not a security audit or endorsement.
Portable agent skills for training, evaluating, and running inference on NVIDIA TAO models. Works with Claude Code, Codex, Gemini CLI, or any coding agent that speaks the Agent Skills open standard. Zero Python required for local docker workflows — install the plugin, install docker + nvidia-container-toolkit, and an agent can run every skill by constructing docker run commands directly. For advanced features (job tracking, multi-node, S3 I/O wrapping), an optional Python layer — the TAO Execution SDK — sits on top.
The skill bank works with both Claude Code and Codex. Pick the runtime you use.
In a Claude Code session, add the marketplace and install the plugin:
/plugin marketplace add [email protected]:NVIDIA-TAO/tao-skills-bank.git
/plugin install tao-skills@tao-skill-bank
That's it — no git clone, no pip install. The TAO Skill Bank plugin bundles all 56 skills (every model, data, platform, and application). The plugin's SessionStart hook loads the AGENTS.md identity at the start of every session.
Codex setup has two independent pieces — the plugin (which surfaces the skills to Codex) and AGENTS.md (which loads the agent identity). You need both for parity with Claude Code.
curl -fsSL https://raw.githubusercontent.com/NVIDIA-TAO/tao-skills-bank/main/scripts/install-codex-agents.sh | bash
…or, if you've already cloned or extracted the repo from a zip, run
scripts/install-codex-agents.sh from that directory. The script registers the
marketplace, installs the TAO Skill Bank plugin, and copies AGENTS.md to
~/.codex/AGENTS.md so the TAO identity loads in every Codex session. It's
idempotent and backs up any existing ~/.codex/AGENTS.md before overwriting.
Override the source with TAO_SKILL_BANK_MARKETPLACE=… and
TAO_SKILL_BANK_REF=… to use a fork, pinned ref, or local absolute path:
cd /absolute/path/to/tao-skills-external
TAO_SKILL_BANK_MARKETPLACE=/absolute/path/to/tao-skills-external \
scripts/install-codex-agents.sh
If you'd rather drive each step yourself:
1. Install the plugin. Either use the VS Code Codex extension's plugin UI (select TAO Skill Bank), or from the CLI:
codex plugin marketplace add [email protected]:NVIDIA-TAO/tao-skills-bank.git
codex plugin add tao-skill-bank@tao-local-plugins
This installs the bundle to ~/.codex/plugins/cache/tao-local-plugins/tao-skill-bank/<version>/ (the tao-local-plugins segment comes from the name field in .agents/plugins/marketplace.json).
For a local zip or clone, use the absolute path instead of the Git URL:
codex plugin marketplace add /absolute/path/to/tao-skills-external
codex plugin add tao-skill-bank@tao-local-plugins
2. Load the agent identity (AGENTS.md). The plugin install does not auto-load AGENTS.md — Codex's AGENTS.md discovery walks down from the project root, not into the plugin cache (see openai/codex#16430 for why plugin-bundled SessionStart hooks don't fix this yet). Pick one:
git clone this repo and launch codex from inside the clone. Codex auto-loads AGENTS.md from the project root per the agents.md cross-runtime spec.cp ~/.codex/plugins/cache/tao-local-plugins/tao-skill-bank/<version>/AGENTS.md ~/.codex/AGENTS.md. The identity then loads in every Codex session, anywhere.Once Codex starts honoring plugin-bundled hooks, the identity will install automatically alongside the plugin — until then, this manual step is needed.
The skill bank reads credentials from the session environment — export what you need in your shell before launching, and the session inherits them:
export NGC_KEY=... # nvcr.io image pulls
export HF_TOKEN=... # gated HuggingFace models
The vars each skill looks for (export only the ones your workflow needs):
| Var | Used for |
|---|---|
NGC_KEY | nvcr.io image pulls — required by almost everything |
HF_TOKEN | gated HuggingFace models / push_to_hub |
BREV_API_TOKEN | tao-run-on-brev (optional — brev login also works) |
ACCESS_KEY, SECRET_KEY, S3_BUCKET_NAME, S3_ENDPOINT_URL, CLOUD_REGION | S3 / object-storage I/O via script_runner |
WANDB_API_KEY, WANDB_PROJECT | WandB experiment logging (AutoML / HF fine-tune) |
The plugin does not create, load, or source any credentials file. On session start the hook reports which of these it detects in the environment (names only). The agent never reads credential values — it only checks presence.
npx claudepluginhub nvidia-tao/tao-skills-bank --plugin tao-daft-processA growing collection of Claude-compatible academic workflow bundles. Covers scientific figures, manuscript writing and polishing, reviewer assessment, citation retrieval, data availability, paper reading, literature search, response letters, paper-to-PPTX conversion, and evidence-grounded Chinese invention patent drafting. Rules are organized as reusable skill folders with explicit workflows and quality checks.
Persistent file-based planning for AI coding agents. Crash-proof markdown plans (task_plan.md, findings.md, progress.md) that survive context loss and /clear, with an opt-in completion gate and multi-agent shared state. Manus-style. Works with Claude Code, Codex CLI, Cursor, Kiro, OpenCode and 60+ agents via the SKILL.md standard. Includes Arabic, German, Spanish, and Chinese (Simplified and Traditional).
Core skills library for Claude Code: TDD, debugging, collaboration patterns, and proven techniques
Harness-native ECC plugin for engineering teams - 67 agents, 271 skills, 92 legacy command shims, reusable hooks, rules, MCP conventions, and operator workflows for Claude Code plus adjacent agent harnesses
Superpowers Plus core skills library for Claude Code: planning, execution routing, TDD, debugging, and collaboration workflows