Kafka plugins

Build production-ready data pipelines with Apache Airflow and dbt, manage scalable data warehouses, and implement vector search and RAG systems using embedding models and vector databases.

Delegate complex data engineering, ML, and AI workflows to specialized sub-agents that design scalable pipelines, build and optimize models, architect LLM systems, tune databases for performance, and deploy production infrastructure across clouds.

Implement Domain-Driven Design with tactical patterns (aggregates, entities, value objects), CQRS, event sourcing, and saga orchestration. Design bounded contexts, map integration contracts, and build event stores with read models for scalable, event-driven systems.

Create and repair Redpanda Connect pipeline configurations and Bloblang transformation scripts using natural language. Automates component search, config generation, and script debugging with optional sample data validation.

Develop, deploy, and manage Databricks-based data pipelines, ML models, AI agents, and dashboards using code, SQL, and declarative tools for multi-environment workflows.

Build, deploy, and operate applications on AWS with infrastructure-as-code (CDK, CloudFormation), core services (Lambda, API Gateway, Step Functions, ECS/Fargate, ECR, IAM, Bedrock with Knowledge Bases and Guardrails, AWS Blocks), observability (CloudWatch, X-Ray, CloudTrail, ADOT), messaging and streaming (SQS, SNS, EventBridge, Kinesis, MSK), AWS SDKs (boto3, JS v3, Swift), and cost optimization.

Interactively select architecture paradigms for software systems via scenario matching, compare trade-offs for team size and complexity, generate ADRs, and receive tailored implementation plans with steps, deliverables, risks, and mitigations for client-server, CQRS, event-driven, hexagonal, microservices, and more.

Connect to MongoDB databases, explore schemas, generate queries and aggregation pipelines from natural language, analyze query performance with index recommendations, implement Atlas Search and Vector Search, and manage stream processing workflows — all from within Claude.

Accelerate Spring Boot development with code generation for REST controllers, DTOs, MapStruct mappers, Spring Security configs, Kafka integration, and entity-to-DTO mapping. Includes structured implementation planning and Connekt-based API testing.

Delegate data engineering to AI agents that design ETL/ELT pipelines with orchestration via Airflow, Prefect, or Dagster; engineer data warehouses with star schemas, partitioning, and query optimization; and build streaming systems using Kafka, Flink, or Spark Streaming for real-time processing and monitoring.

Audit, review, and manage Apache Kafka clusters using the Lenses MCP server. Diagnose consumer lag, review connector configurations, schema compatibility, security settings, and dead letter queues. Scaffold Python Kafka producers/consumers and generate synthetic data tests with ShadowTraffic and TestContainers.

Build and deploy Kafka- and Flink-based streaming data pipelines on Confluent Cloud — from CDC ingestion with Debezius to stream processing with Kafka Streams or Flink UDFs, Schema Registry management, and Terraform-based schema migration.

Design and implement data foundations for financial systems: reference and market data management, integration architectures with REST APIs, WebSockets, gRPC, Kafka, FIX protocol, ISO 20022, batch feeds; plus data quality programs with validation, lineage, governance for regulatory compliance like BCBS 239 and MiFID II.

Delegate SDLC workflows to specialist AI agents that architect cloud-native systems, design databases, conduct deep web research, optimize performance and observability, distill repo knowledge, and build production agents via orchestrated pipelines.

Orchestrate disciplined, autonomous TDD/DDD development cycles with adversarial QA, code review, and persistent project memory that learns from execution traces to optimize future sessions.

Build Go microservices with the xgodev/boost framework using pre-built factory integrations for cloud databases, message queues, observability, and HTTP servers. Covers boot sequence, configuration, logging, caching, health checks, middleware, and dependency injection with uber-fx modules.

Process and transform data using jq, SQL, or pandas; design ETL/ELT pipelines for batch or streaming; perform time series forecasting, anomaly detection, and analytics; architect streaming systems with Kafka; generate insights and visualizations via natural language commands and specialist agents.

Design optimized message queue systems for Kafka, RabbitMQ, SQS using queue theory for throughput and latency. Implement async processing with workers, retries, backoff; manage backpressure via circuit breakers, rate limiting; architect distributed systems with consensus like Raft, CAP theorem, fault tolerance.

Develop and operate Camunda 8 automations end-to-end: author BPMN processes, DMN decisions, FEEL expressions, and Camunda Forms; build custom connectors and job workers; deploy resources to clusters, manage live process instances, and validate with Process Test coverage reports — all with CLI support for installation, configuration, and vector-search documentation lookup.

Review Spring Boot codebases using skill detection and parallel agents for DDD patterns, security, testing, observability, and Modulith modules. Verify upgrades to Spring Boot 4 with multi-phase analysis producing severity-categorized migration checklists and remediation steps. Implement data layers, REST APIs, and configurations via guided skills.

Run comprehensive multi-dimensional code reviews across architecture, security, performance, and best practices, generating structured reports and enhanced PR descriptions from git changes

Run first-principles study sessions on PDFs, web articles, GitHub repos, and YouTube videos, then auto-generate structured notes, spaced-repetition Slack quizzes, and technical blog posts with Notion integration