overfit

Overfit

Private LLMs, voice, RAG and agents — entirely in your .NET app. No Python, no cloud, no model server.

Overfit lets .NET teams add local AI features — chat, RAG, agents, and even voice (speech-to-text and text-to-speech) — without Python, Ollama, a model server, native binaries, or data leaving the process. It can also serve an OpenAI-compatible API from a tiny Native-AOT binary or a ~34 MB container.

Use it when you need AI inside an existing ASP.NET, WPF, Blazor, desktop, on-prem or air-gapped .NET product — especially when external LLM APIs are blocked by security, compliance, latency, deployment, or supply-chain constraints.

dotnet add package DevOnBike.Overfit            # the library
dotnet tool install -g DevOnBike.Overfit.Cli    # the CLI + OpenAI-compatible server (overfit serve)
dotnet new install DevOnBike.Overfit.Templates  # then: dotnet new overfit-chat → a local-LLM chat app in 60s

Overfit implements Microsoft.Extensions.AI (IChatClient / IEmbeddingGenerator), so you can drop a local model into the official .NET AI template or Semantic Kernel by swapping one line — no Ollama, no Docker, no cloud key: builder.Services.AddChatClient(overfit.AsChatClient()); (guide).

---

Where Overfit wins

The missing quadrant: .NET-native · on-prem · trainable · testable · auditable. Almost every LLM tool assumes Python, a GPU, a separate server, and data leaving your process. Overfit is the opposite — and that intersection is nearly empty on the market.

1. Private RAG & agents inside .NET — chat, RAG, tool-calling and schema-constrained JSON in-process; nothing leaves the box.
2. Replace a Python / Ollama / model-server sidecar — same capabilities, one .NET process, one deploy, one security surface.
3. Testable, auditable RAG for regulated teams — gate retrieval quality in CI, cite sources, reproduce answers bit-for-bit.
4. CPU QLoRA fine-tuning (advanced moat) — teach a model your private data on a CPU, no GPU, no Python.

→ Full market analysis and ten buyer cases: docs/use-cases-2026.md.

---

Maturity

Overfit covers a lot of surface; this is the honest tier for each capability so you know what to build a PoC on today versus what is still hardening. (Model-by-model support is in Supported model families.)

Tier	Capabilities
Stable — validated end-to-end, build a PoC on it	LocalAgent ASP.NET demo (chat · RAG · tools · JSON · metrics · Docker) · GGUF chat (Qwen 2/3, Llama, Mistral, Phi-3.5/4, Gemma-2, Mixtral, Bielik) · in-process RAG + VectorStore + persistent index (PersistentVectorStore) · guaranteed JSON (grammar and JSON-Schema subset) · OpenAI-compatible server (overfit serve, multi-session pool via --sessions N) · Microsoft.Extensions.AI adapter (IChatClient / IEmbeddingGenerator) · MCP tools profile · Whisper STT (tiny/base, EN + PL) · BERT embeddings (MiniLM / BGE / E5) · overfit doctor model inspector
Preview — works and tested, newer / heavier / advanced	CPU QLoRA fine-tuning (advanced moat) · local preset-voice TTS · large MoE models (Qwen1.5-MoE, Mixtral-8x7B) · serving benchmark (overfit bench) · interpretability hooks (activation capture + logit lens) · XGBoost tabular scoring (read-only predictor, parity-validated vs XGBoost 3.3.0, zero-alloc) · offline Q4_K weight repack (overfit repack) for ~1.6× faster prefill at no extra RAM
Experimental — opt-in / gated / incomplete	Voice cloning (consent + watermark gated) · whole-matrix Q4_K attention (OVERFIT_REPACK_ATTN, off by default) · multilingual / XLM-R (SentencePiece) embedders (not yet) · Qwen3-MoE & other new-arch loaders (not yet)

Every claim above maps to a runnable test, benchmark, demo, or CI guard — see docs/claim-to-test.md (the audit trail for regulated teams).

---

Start here: run the ASP.NET local agent demo

The fastest way to understand Overfit is to run the local ASP.NET agent demo.

dotnet run -c Release --project Demo/LocalAgentAspNetDemo

It exposes a private AI agent over HTTP: