By wshobson
Evaluate LLM application performance using automated metrics, human feedback, and AI-as-judge benchmarking. Defines and runs evaluation strategies for testing output quality, establishing benchmarks, and measuring AI application performance against defined criteria.
Own this plugin?
Verify ownership to unlock analytics, metadata editing, and a verified badge. GitHub access is read-only (username + org membership).
Sign in to claimOwn this plugin?
Verify ownership to unlock analytics, metadata editing, and a verified badge. GitHub access is read-only (username + org membership).
Sign in to claimBased on adoption, maintenance, documentation, and repository signals. Not a security audit or endorsement.
Dependency auditing, version management, and security vulnerability scanning
SAST analysis, dependency vulnerability scanning, OWASP Top 10 compliance, container security scanning, and automated security hardening
Database architecture, schema design, and SQL optimization for production systems
ML model training pipelines, hyperparameter tuning, model deployment automation, experiment tracking, and MLOps workflows
ETL pipeline construction, data warehouse design, batch processing workflows, and data-driven feature development
npx claudepluginhub p/wshobson-wshobson-llm-evaluation-plugins-llm-application-dev-skills-llm-evaluation26 Agent Skills (several with runnable, unit-tested scripts) for building, evaluating, securing, and monitoring reliable LLM & AI-agent apps.
Skills for adding DeepEval evaluations, tracing, datasets, Confident AI reports, and iterative improvement loops to AI applications.
Evaluate any LLM behind an OpenAI- or Anthropic-compatible endpoint across four dimensions — speed (TTFT + thinking-aware tokens/sec), concurrency/stability (success rate, p50/p90, breaking point), Anthropic protocol compliance (thinking-block trigger rate), and quality regression against your own accumulated use cases (blind-judge precision). Use to benchmark a model, verify a tokens-per-second claim, compare models head-to-head, or vet a newly released model before adopting it.
Master advanced prompt engineering techniques to maximize LLM performance, reliability, and controllability in production. Use when optimizing prompts, improving LLM outputs, or designing production prompt templates.
Comprehensive model evaluation with multiple metrics
Benchmark, evaluate, and optimize skills to ensure reliable performance across all LLMs