---
This skill inherits all available tools. When active, it can use any tool Claude has access to.
references/brighter-resilience.mdreferences/circuit-breaker-config.mdreferences/dlq-patterns.mdreferences/polly-patterns.mdreferences/retry-strategies.mdThis skill provides guidance on implementing resilience patterns in .NET applications. It covers both synchronous resilience (HTTP clients, service calls) using Polly and asynchronous resilience (message handlers) using Brighter.
Key Principle: Design for failure. Systems should gracefully handle transient faults, prevent cascade failures, and provide meaningful fallback behavior.
Keywords: resilience, circuit breaker, retry, polly, brighter, fault tolerance, transient failure, DLQ, dead letter queue, timeout, bulkhead, fallback, http client resilience
Use this skill when:
For HTTP calls and synchronous service communication:
| Pattern | Purpose | When to Use |
|---|---|---|
| Retry | Retry failed operations | Transient failures (network, 503, timeouts) |
| Circuit Breaker | Stop calling failing services | Repeated failures indicate service is down |
| Timeout | Bound operation time | Prevent indefinite waits |
| Bulkhead | Isolate failures | Prevent one caller from exhausting resources |
| Fallback | Provide alternative | Graceful degradation |
For message-based and async operations:
| Pattern | Purpose | When to Use |
|---|---|---|
| Retry | Redeliver failed messages | Transient processing failures |
| Dead Letter Queue | Park unprocessable messages | Poison messages, business rule failures |
| Circuit Breaker | Stop processing temporarily | Downstream service unavailable |
| Timeout | Bound handler execution | Prevent handler blocking |
// Program.cs or Startup.cs
builder.Services.AddHttpClient<IOrderService, OrderService>()
.AddStandardResilienceHandler();
The AddStandardResilienceHandler() adds a preconfigured pipeline with:
builder.Services.AddHttpClient<IOrderService, OrderService>()
.AddResilienceHandler("custom-pipeline", builder =>
{
// Retry with exponential backoff
builder.AddRetry(new HttpRetryStrategyOptions
{
MaxRetryAttempts = 3,
Delay = TimeSpan.FromSeconds(1),
BackoffType = DelayBackoffType.Exponential,
UseJitter = true,
ShouldHandle = new PredicateBuilder<HttpResponseMessage>()
.Handle<HttpRequestException>()
.HandleResult(r => r.StatusCode == HttpStatusCode.ServiceUnavailable)
});
// Circuit breaker
builder.AddCircuitBreaker(new HttpCircuitBreakerStrategyOptions
{
FailureRatio = 0.5,
MinimumThroughput = 10,
SamplingDuration = TimeSpan.FromSeconds(30),
BreakDuration = TimeSpan.FromSeconds(30)
});
// Timeout per attempt
builder.AddTimeout(TimeSpan.FromSeconds(10));
});
Detailed Polly patterns: See references/polly-patterns.md
public class OrderCreatedHandler : RequestHandler<OrderCreated>
{
[UsePolicy("retry-policy", step: 1)]
public override OrderCreated Handle(OrderCreated command)
{
// Process order
return base.Handle(command);
}
}
var policyRegistry = new PolicyRegistry
{
{
"retry-policy",
Policy
.Handle<Exception>()
.WaitAndRetry(
retryCount: 3,
sleepDurationProvider: attempt =>
TimeSpan.FromSeconds(Math.Pow(2, attempt)))
}
};
services.AddBrighter()
.UseExternalBus(/* config */)
.UsePolicyRegistry(policyRegistry);
Detailed Brighter patterns: See references/brighter-resilience.md
Use retry when:
Don't use retry when:
Use circuit breaker when:
Configuration guidance: See references/circuit-breaker-config.md
Use DLQ when:
DLQ patterns: See references/dlq-patterns.md
For very transient failures:
.AddRetry(new RetryStrategyOptions
{
MaxRetryAttempts = 2,
Delay = TimeSpan.Zero // Immediate retry
});
For transient failures that need time:
.AddRetry(new RetryStrategyOptions
{
MaxRetryAttempts = 4,
Delay = TimeSpan.FromSeconds(1),
BackoffType = DelayBackoffType.Exponential,
UseJitter = true // Prevents thundering herd
});
Delays: 1s → 2s → 4s → 8s (with jitter)
For rate-limited services:
.AddRetry(new RetryStrategyOptions
{
MaxRetryAttempts = 3,
Delay = TimeSpan.FromSeconds(2),
BackoffType = DelayBackoffType.Linear
});
Delays: 2s → 4s → 6s
Full retry strategies: See references/retry-strategies.md
.AddCircuitBreaker(new CircuitBreakerStrategyOptions
{
FailureRatio = 0.25, // Open after 25% failures
MinimumThroughput = 5, // Need at least 5 calls to evaluate
SamplingDuration = TimeSpan.FromSeconds(10),
BreakDuration = TimeSpan.FromSeconds(60) // Stay open 60s
});
.AddCircuitBreaker(new CircuitBreakerStrategyOptions
{
FailureRatio = 0.5, // Open after 50% failures
MinimumThroughput = 20, // Need 20 calls before evaluation
SamplingDuration = TimeSpan.FromSeconds(30),
BreakDuration = TimeSpan.FromSeconds(15) // Quick recovery attempt
});
Detailed configuration: See references/circuit-breaker-config.md
1. Message received
2. Handler attempts processing
3. Failure occurs
4. Retry policy applied (1...N attempts)
5. All retries exhausted
6. Message moved to DLQ
7. Alert/monitoring triggered
8. Manual investigation
services.AddBrighter()
.UseExternalBus(config =>
{
config.Publication.RequeueDelayInMs = 500;
config.Publication.RequeueCount = 3;
// After 3 requeues, message goes to DLQ
});
Full DLQ patterns: See references/dlq-patterns.md
builder.Services.AddHttpClient<IPaymentGateway, PaymentGateway>()
.AddResilienceHandler("payment-gateway", builder =>
{
// Order matters: outer to inner
// 1. Total timeout (outer boundary)
builder.AddTimeout(TimeSpan.FromSeconds(30));
// 2. Retry (with circuit breaker inside)
builder.AddRetry(new HttpRetryStrategyOptions
{
MaxRetryAttempts = 3,
Delay = TimeSpan.FromMilliseconds(500),
BackoffType = DelayBackoffType.Exponential,
UseJitter = true
});
// 3. Circuit breaker
builder.AddCircuitBreaker(new HttpCircuitBreakerStrategyOptions
{
FailureRatio = 0.5,
MinimumThroughput = 10,
BreakDuration = TimeSpan.FromSeconds(30)
});
// 4. Per-attempt timeout (inner)
builder.AddTimeout(TimeSpan.FromSeconds(5));
});
public class ProcessPaymentHandler : RequestHandler<ProcessPayment>
{
[UsePolicy("circuit-breaker", step: 1)]
[UsePolicy("retry", step: 2)]
[UsePolicy("fallback", step: 3)]
public override ProcessPayment Handle(ProcessPayment command)
{
_paymentService.Process(command);
return base.Handle(command);
}
}
services.AddResiliencePipeline("my-pipeline", builder =>
{
builder.AddRetry(/* options */)
.ConfigureTelemetry(LoggerFactory.Create(b => b.AddConsole()));
});
| Metric | Purpose | Alert Threshold |
|---|---|---|
| Retry count | Track transient failures | > 3 per minute |
| Circuit state | Track service health | State = Open |
| DLQ depth | Track processing failures | > 0 |
| Timeout rate | Track slow services | > 5% |
Problem: Retrying too many times, too quickly.
// BAD: 10 immediate retries
.AddRetry(new RetryStrategyOptions { MaxRetryAttempts = 10 });
Fix: Use exponential backoff, limit retries:
// GOOD: 3 retries with backoff
.AddRetry(new RetryStrategyOptions
{
MaxRetryAttempts = 3,
Delay = TimeSpan.FromSeconds(1),
BackoffType = DelayBackoffType.Exponential
});
Problem: Retrying business logic failures.
// BAD: Retrying 400 Bad Request
ShouldHandle = new PredicateBuilder<HttpResponseMessage>()
.HandleResult(r => !r.IsSuccessStatusCode)
Fix: Only retry transient failures:
// GOOD: Only retry transient HTTP codes
ShouldHandle = new PredicateBuilder<HttpResponseMessage>()
.Handle<HttpRequestException>()
.HandleResult(r => r.StatusCode is
HttpStatusCode.ServiceUnavailable or
HttpStatusCode.GatewayTimeout or
HttpStatusCode.RequestTimeout)
Problem: Retrying endlessly when service is down.
Fix: Always pair retry with circuit breaker for external calls.
Problem: Messages go to DLQ and are never processed.
Fix:
references/polly-patterns.md - Comprehensive Polly v8 patternsreferences/circuit-breaker-config.md - Circuit breaker configuration guidereferences/retry-strategies.md - Retry strategy patternsreferences/brighter-resilience.md - Brighter message handler resiliencereferences/dlq-patterns.md - Dead letter queue patternsfitness-functions - Test resilience with performance fitness functionsmodular-architecture - Isolate resilience concerns by moduleadr-management - Document resilience decisionsLast Updated: 2025-12-22