From duende-skills
Covers reverse proxy config, data protection, health checks, distributed caching, OpenTelemetry, logging, and troubleshooting for production Duende IdentityServer deployments.
How this skill is triggered — by the user, by Claude, or both
Slash command
/duende-skills:identityserver-deploymentThe summary Claude sees in its skill listing — used to decide when to auto-load this skill
- Deploying IdentityServer behind a reverse proxy or load balancer
Docs: https://docs.duendesoftware.com/identityserver/deployment
IdentityServer is ASP.NET Core middleware. It can be hosted with the same diversity of technology as any ASP.NET Core application:
dotnet publish /t:PublishContainer)When IdentityServer runs behind a proxy that terminates TLS or changes the originating IP, the middleware sees incorrect request information. This causes:
.well-known/openid-configuration instead of HTTPSSecure attribute (breaks SameSite behavior)Most proxies set X-Forwarded-For and X-Forwarded-Proto headers. Configure ASP.NET Core to read them.
Set ASPNETCORE_FORWARDEDHEADERS_ENABLED=true. This automatically adds the middleware and accepts forwarded headers from any single proxy. Best for cloud-hosted environments and Kubernetes.
// Program.cs
builder.Services.Configure<ForwardedHeadersOptions>(options =>
{
options.ForwardedHeaders = ForwardedHeaders.XForwardedHost |
ForwardedHeaders.XForwardedProto;
// Add the IP address of your known proxy
options.KnownProxies.Add(IPAddress.Parse("203.0.113.42"));
// Or use a network range
// var network = new IPNetwork(IPAddress.Parse("198.51.100.0"), 24);
// options.KnownNetworks.Add(network);
// Number of proxies in front of the app
options.ForwardLimit = 1;
});
Important: The ForwardedHeaders middleware must run early in the pipeline, before IdentityServer middleware and ASP.NET authentication middleware.
By default, KnownNetworks and KnownProxies support localhost (127.0.0.1/8 and ::1). This is useful for local development or when the proxy and .NET host are on the same machine. In production, configure the actual proxy addresses.
Cross-cutting concern: Data protection is critical for all Duende products — both IdentityServer and BFF. See ASP.NET Core Data Protection for comprehensive guidance covering all Duende SDKs.
Data Protection is critical for IdentityServer. It encrypts and signs sensitive data including:
// Program.cs
builder.Services.AddDataProtection()
// Choose a persistence method
.PersistKeysToFoo() // PersistKeysToFileSystem, PersistKeysToDbContext,
// PersistKeysToAzureBlobStorage, PersistKeysToAWSSystemsManager,
// PersistKeysToStackExchangeRedis
// Choose a key protection method
.ProtectKeysWithBar() // ProtectKeysWithCertificate, ProtectKeysWithAzureKeyVault
// Set explicit application name
.SetApplicationName("My.IdentityServer");
.PersistKeysTo...() method.SetApplicationName() to prevent key isolation issuesIXmlEncryptor-based escrow| Aspect | Data Protection Keys | IdentityServer Signing Keys |
|---|---|---|
| Purpose | Encrypt/sign sensitive data at rest and in cookies | Sign JWT tokens (id_tokens, access tokens) |
| Cryptography | Symmetric (private key) | Asymmetric (public/private key pair) |
| Visibility | Internal to the application | Public keys published via discovery/JWKS |
| Managed by | ASP.NET Core framework | IdentityServer (automatic key management) |
| Storage | Configured via .PersistKeysTo...() | File system (default), EF operational store, or custom ISigningKeyStore |
Both are critical secrets. Losing either causes failures.
| Problem | Symptom | Solution |
|---|---|---|
| No shared keys in load-balanced environment | CryptographicException: key not found in key ring | Configure shared key persistence |
| Keys generated in dev included in build | Keys from wrong environment can't be read in production | Exclude ~/keys directory from source control and builds |
| Application name mismatch | Keys from one deployment can't be read by another | Set explicit SetApplicationName() consistently |
| IIS lacking permissions | Ephemeral keys generated every restart | Follow Microsoft's IIS Data Protection configuration |
| .NET 6 path normalization change | Keys break between .NET versions | Always set explicit application name (reverted in .NET 7+) |
CryptographicException in logsFor multi-instance deployments, configuration data must be shared:
| Scenario | Recommendation |
|---|---|
| Rarely changing configuration | In-memory stores loaded from config files (with redeploy for changes) |
| Dynamic configuration (SaaS) | Database via EF Core stores or custom stores |
Operational data must always be shared in multi-instance deployments:
ISigningKeyStore (EF operational store or custom)IServerSideSessionStoreUse Entity Framework Core or a persistent cache like Redis.
Some optional features require ASP.NET Core's IDistributedCache:
| Feature | Why It Needs Distributed Cache |
|---|---|
| OIDC state data formatter | Stores external provider state server-side instead of in URL |
| JWT replay cache | Prevents JWT client credentials replay |
| Device flow throttling | Rate-limits polling across instances |
| Authorization parameter store | Stores PAR request data |
Configure a distributed cache for multi-instance deployments:
// Program.cs — Example using Redis
builder.Services.AddStackExchangeRedisCache(options =>
{
options.Configuration = "localhost:6379";
});
Tests that IdentityServer can process requests and communicate with the configuration store:
public class DiscoveryHealthCheck : IHealthCheck
{
private readonly IEnumerable<Hosting.Endpoint> _endpoints;
private readonly IHttpContextAccessor _httpContextAccessor;
public DiscoveryHealthCheck(IEnumerable<Hosting.Endpoint> endpoints,
IHttpContextAccessor httpContextAccessor)
{
_endpoints = endpoints;
_httpContextAccessor = httpContextAccessor;
}
public async Task<HealthCheckResult> CheckHealthAsync(
HealthCheckContext context,
CancellationToken cancellationToken = default)
{
try
{
var endpoint = _endpoints.FirstOrDefault(
x => x.Name == IdentityServerConstants.EndpointNames.Discovery);
if (endpoint != null)
{
var handler = _httpContextAccessor.HttpContext.RequestServices
.GetRequiredService(endpoint.Handler) as IEndpointHandler;
if (handler != null)
{
var result = await handler.ProcessAsync(
_httpContextAccessor.HttpContext);
if (result is DiscoveryDocumentResult)
{
return HealthCheckResult.Healthy();
}
}
}
}
catch { }
return new HealthCheckResult(context.Registration.FailureStatus);
}
}
Tests that IdentityServer can access its signing keys:
public class DiscoveryKeysHealthCheck : IHealthCheck
{
private readonly IEnumerable<Hosting.Endpoint> _endpoints;
private readonly IHttpContextAccessor _httpContextAccessor;
public DiscoveryKeysHealthCheck(IEnumerable<Hosting.Endpoint> endpoints,
IHttpContextAccessor httpContextAccessor)
{
_endpoints = endpoints;
_httpContextAccessor = httpContextAccessor;
}
public async Task<HealthCheckResult> CheckHealthAsync(
HealthCheckContext context,
CancellationToken cancellationToken = default)
{
try
{
var endpoint = _endpoints.FirstOrDefault(
x => x.Name == IdentityServerConstants.EndpointNames.Jwks);
if (endpoint != null)
{
var handler = _httpContextAccessor.HttpContext.RequestServices
.GetRequiredService(endpoint.Handler) as IEndpointHandler;
if (handler != null)
{
var result = await handler.ProcessAsync(
_httpContextAccessor.HttpContext);
if (result is JsonWebKeysResult)
{
return HealthCheckResult.Healthy();
}
}
}
}
catch { }
return new HealthCheckResult(context.Registration.FailureStatus);
}
}
Note: Finding endpoints by name requires IdentityServer v6.3+.
IdentityServer emits traces, metrics, and logs via the .NET OpenTelemetry SDK (added in v6.1, expanded in v7.0).
dotnet add package OpenTelemetry
dotnet add package OpenTelemetry.Extensions.Hosting
dotnet add package OpenTelemetry.Instrumentation.AspNetCore
dotnet add package OpenTelemetry.Exporter.OpenTelemetryProtocol
// Program.cs
using OpenTelemetry.Resources;
// Add OpenTelemetry logging to correlate logs with traces
builder.Logging.AddOpenTelemetry();
var openTelemetry = builder.Services.AddOpenTelemetry();
openTelemetry.ConfigureResource(r => r
.AddService(builder.Environment.ApplicationName));
openTelemetry.WithMetrics(m => m
.AddMeter("Duende.IdentityServer") // Telemetry.ServiceName == "Duende.IdentityServer"
.AddPrometheusExporter());
openTelemetry.WithTracing(t => t
.AddSource(IdentityServerConstants.Tracing.Basic)
.AddSource(IdentityServerConstants.Tracing.Cache)
.AddSource(IdentityServerConstants.Tracing.Services)
.AddSource(IdentityServerConstants.Tracing.Stores)
.AddSource(IdentityServerConstants.Tracing.Validation)
.AddAspNetCoreInstrumentation()
.AddConsoleExporter());
// Add Prometheus scraping endpoint
app.UseOpenTelemetryPrometheusScrapingEndpoint();
| Source | What It Traces |
|---|---|
IdentityServerConstants.Tracing.Basic | High-level request processing (validators, response generators) |
IdentityServerConstants.Tracing.Cache | Cache operations |
IdentityServerConstants.Tracing.Services | Service-layer operations |
IdentityServerConstants.Tracing.Stores | Store operations (database calls) |
IdentityServerConstants.Tracing.Validation | Detailed validation operations |
In production, you may want only Basic tracing. Use all sources during development and troubleshooting.
The meter name is Duende.IdentityServer (accessible via Telemetry.ServiceName).
| Metric | Counter Name | Description |
|---|---|---|
| Operations | tokenservice.operation | Aggregated success/failure/internal_error counts |
| Active Requests | active_requests | Current requests being processed by endpoints |
| Token Issuance | tokenservice.token_issued | Successful/failed token issuance attempts |
| Client Auth | tokenservice.client.secret_validation | Client authentication success/failure |
| Introspection | tokenservice.introspection | Token introspection counts |
| Revocation | tokenservice.revocation | Token revocation counts |
| Metric | Counter Name | Tags |
|---|---|---|
| User Login | tokenservice.user_login | client, idp, error |
| User Logout | user_logout | idp |
| Consent | tokenservice.consent | client, scope, consent (granted/denied) |
IdentityServer uses ASP.NET Core's standard ILogger. Logs are written under the Duende.IdentityServer category.
| Level | Usage |
|---|---|
Trace | Sensitive data (tokens); never enable in production |
Debug | Internal flow and decisions; short-term debugging |
Information | General application flow; long-term value |
Warning | Abnormal or unexpected events |
Error | Failed validation, unhandled exceptions |
Critical | Missing store implementations, invalid key material |
{
"Logging": {
"LogLevel": {
"Default": "Information",
"Duende.IdentityServer": "Information"
}
}
}
In production, default to Warning to avoid excessive log volume.
builder.Services.AddIdentityServer(options =>
{
options.Logging.UnhandledExceptionLoggingFilter = (ctx, ex) =>
{
// Return false to suppress, true to log
if (ctx.RequestAborted.IsCancellationRequested && ex is OperationCanceledException)
return false; // Already the default
return true;
};
});
Logs written to ILogger in .NET 8+ can be exported to OpenTelemetry traces. Add builder.Logging.AddOpenTelemetry() to correlate logs with trace IDs.
Events provide higher-level structured data about operations, suitable for APM integration.
builder.Services.AddIdentityServer(options =>
{
options.Events.RaiseSuccessEvents = true;
options.Events.RaiseFailureEvents = true;
options.Events.RaiseErrorEvents = true;
options.Events.RaiseInformationEvents = true;
});
public async Task<IActionResult> Login(LoginInputModel model)
{
if (_users.ValidateCredentials(model.Username, model.Password))
{
var user = _users.FindByUsername(model.Username);
await _events.RaiseAsync(
new UserLoginSuccessEvent(user.Username, user.SubjectId, user.Username));
}
else
{
await _events.RaiseAsync(
new UserLoginFailureEvent(model.Username, "invalid credentials"));
}
}
public class SeqEventSink : IEventSink
{
private readonly Logger _log;
public SeqEventSink()
{
_log = new LoggerConfiguration()
.WriteTo.Seq("http://localhost:5341")
.CreateLogger();
}
public Task PersistAsync(Event evt)
{
if (evt.EventType == EventTypes.Success ||
evt.EventType == EventTypes.Information)
{
_log.Information("{Name} ({Id}), Details: {@details}",
evt.Name, evt.Id, evt);
}
else
{
_log.Error("{Name} ({Id}), Details: {@details}",
evt.Name, evt.Id, evt);
}
return Task.CompletedTask;
}
}
Events work well with structured logging stores like ELK, Seq, or Splunk.
| Item | Status | Notes |
|---|---|---|
| Data Protection keys persisted to durable storage | Required | .PersistKeysTo...() |
| Data Protection keys shared across instances | Required for multi-instance | Same storage for all instances |
| Explicit application name set | Required | .SetApplicationName("My.IdentityServer") |
| ForwardedHeaders configured (if behind proxy) | Required | Match your proxy's headers |
| Operational store configured with durable persistence | Required | EF Core or custom store |
| Token cleanup enabled | Recommended | EnableTokenCleanup = true |
| Configuration store cache enabled | Recommended | AddConfigurationStoreCache() |
| Distributed cache configured (if multi-instance) | Recommended | Redis, SQL, etc. |
| Health checks implemented | Recommended | Discovery + JWKS endpoints |
| OpenTelemetry configured | Recommended | Metrics + traces for monitoring |
| Events enabled | Recommended | For auditing and APM |
| Signing key store uses durable storage | Required for multi-instance | EF operational store or custom |
| Logging level set to Warning+ for production | Recommended | Avoid log bloat |
~/keys directory excluded from source control | Required if using file-based key store | Prevent dev keys in production |
❌ Deploying without configuring ForwardedHeaders behind a reverse proxy
✅ Always configure ForwardedHeaders when behind a proxy; test by checking the discovery document's issuer URL
❌ Using default (ephemeral) Data Protection keys in production
✅ Always persist keys to durable, shared storage with .PersistKeysTo...()
❌ Not setting SetApplicationName() causing key isolation between deployments
✅ Always set an explicit, consistent application name
❌ Using file-system signing key store in containerized/multi-instance deployments
✅ Use EF operational store or a shared ISigningKeyStore implementation
❌ Enabling Trace or Debug logging in production — exposes tokens and sensitive data
✅ Use Warning level in production; use Information temporarily for troubleshooting
❌ Not enabling token cleanup — database grows indefinitely
✅ Enable EnableTokenCleanup = true and configure appropriate intervals
Discovery document shows HTTP issuer: The most common deployment issue. Always configure ForwardedHeaders or the ASPNETCORE_FORWARDEDHEADERS_ENABLED environment variable when behind a TLS-terminating proxy.
CryptographicException on startup: Usually means Data Protection keys from one environment are being used in another. Check that keys are persisted correctly and the application name is consistent.
Signing keys not shared across instances: The default file-system key store is per-instance. Use AddOperationalStore() which includes ISigningKeyStore, or configure a custom shared store.
Redis losing Data Protection keys on restart: If using PersistKeysToStackExchangeRedis, configure Redis with persistence (RDB snapshots or AOF) to survive restarts.
IIS Data Protection permissions: IIS may lack permissions to persist Data Protection keys. Follow Microsoft's IIS-specific Data Protection documentation.
Multiple proxies in chain: If you have more than one proxy, set ForwardLimit to match the number of proxies, and add all proxy addresses to KnownProxies or KnownNetworks.
Cookie SameSite failures behind proxy: If the proxy strips HTTPS, cookies won't get the Secure attribute, causing SameSite=None cookies to be rejected by browsers. Fix the proxy configuration first.
OpenTelemetry trace source selection: In production, subscribing to all trace sources (Stores, Validation, etc.) can generate excessive trace data. Start with Basic and add more sources as needed for troubleshooting.
identityserver-hosting-setup — DI registration and middleware pipelineidentityserver-data-storage — EF Core stores, migrations, token cleanupidentityserver-aspire — orchestrating IdentityServer in Aspire AppHostBlocks Edit/Write/Bash actions until Claude investigates importers, data schemas, and user instructions. Improves output quality by forcing concrete facts before edits.
npx claudepluginhub duendesoftware/duende-skills --plugin duende-skills