Carles Arnal
Principal Software Engineer at IBM, based in Barcelona. I work on distributed systems, schema governance, and ML pipelines. Core contributor to Apicurio Registry — an open-source schema and API registry for event-driven architectures.
What I Work On
Schema Governance
Core contributor to Apicurio Registry. Schema validation, compatibility rules, content canonicalization, and API design for multi-format registries (Avro, JSON Schema, Protobuf, OpenAPI).
Distributed Systems
Kafka-based event pipelines, Spark Structured Streaming, distributed tracing with OpenTelemetry, failure handling patterns (DLQ, backpressure, circuit breakers).
ML Pipelines
Real-time inference with dual-model architectures, model agreement tracking, Prometheus/Grafana observability, and schema-governed model metadata.
Cloud-Native Java
Quarkus, SmallRye Reactive Messaging, Kubernetes operators (Strimzi, Spark Operator), health probes, and production deployment patterns.
Deep Dives
Technical articles rooted in real engineering experience — not tutorials. Each one references actual code, architecture decisions, and production lessons.
Apicurio Registry as an Apache Iceberg REST Catalog
Apicurio Registry 3.2.0 now implements the Apache Iceberg REST Catalog API, turning your schema registry into a table catalog for Spark, Trino, DuckDB, and more. One service for both schema governance and lakehouse metadata.
Real-Time vs. Batch ML Inference: Engineering Tradeoffs You Won't Find in Tutorials
The inference spectrum isn't binary. Micro-batch, dual-model architectures, PySpark UDF tracing challenges, and what changes at 100x scale — lessons from building a real-time classification pipeline.
Model Agreement as a Proxy for Ground Truth in Streaming ML
When you deploy ML models in a streaming pipeline, you don't have labels. Dual-model agreement rate, disagreement taxonomy, confidence calibration, and drift detection without ground truth.
MCP Servers for Domain-Specific AI Tooling: Lessons from Apicurio Registry
Building an MCP server that wraps a schema registry for AI agent use. Designing the tool surface, security considerations, and patterns for wrapping any REST API as an MCP server.
Designing Schema Evolution for Event-Driven Systems
Schema evolution is the hardest coordination problem in event-driven architectures. Compatibility modes, common mistakes, and real-world patterns from building a Kafka-based ML pipeline with Apicurio Registry.
Building a RAG Chatbot with Schema Registry as the Knowledge Backend
Using Apicurio Registry to store versioned prompt templates as registry artifacts, then feeding them to a RAG chatbot. A unique intersection of schema governance and LLM tooling that nobody else has explored.
Why Schema References Break in Avro (And How to Design Around It)
Avro schema references promise reusable types across schemas. In practice, they break in subtle ways — circular dependencies, version pinning drift, namespace collisions. Here's what I learned contributing to Apicurio Registry.
Featured Projects
reddit-realtime-classification
End-to-end ML pipeline: Reddit API → Kafka (Strimzi) → Spark Structured Streaming (dual-model inference) → Quarkus consumer. Full observability with OpenTelemetry, Prometheus, and Grafana. Dead letter queues, backpressure, and 5 architecture decision records.
distributed-deep-dives
Technical deep dives into distributed systems, schema evolution, and ML inference tradeoffs. Each article includes runnable examples and references real engineering work.
apicurio-registry/support-chat
RAG-powered support chatbot for Apicurio Registry using LangChain4j, Ollama, and prompt templates stored as versioned registry artifacts.