Prompt templates are code. They have versions, they break when changed, they need rollback capability, and different versions produce different outputs. Yet most teams store them as hardcoded strings or, at best, in a config file. Schema registries already solve versioning, governance, and compatibility for schemas — why not extend the pattern to prompts?

This article is based on support-chat, a RAG-powered chatbot I built as part of Apicurio Registry that uses the registry itself as its prompt management backend.

Full article with examples: distributed-deep-dives/rag-with-schema-registry

The Architecture

The system has two services running in Docker Compose, plus a cloud LLM API:

  • Apicurio Registry 3.2.0 — Stores PROMPT_TEMPLATE artifacts (versioned prompts with variable substitution via the /render endpoint)
  • Quarkus app — LangChain4j integration with @RegisterAiService, in-process ONNX embeddings (bge-small-en-v15-q), RAG retrieval, and multi-turn conversation management
  • Google AI Gemini — Cloud LLM provider (gemini-2.0-flash) — no local GPU or Ollama required

The key insight: the registry isn’t just storing schemas for Kafka topics — it’s a general-purpose versioned artifact store. Prompt templates are artifacts that benefit from the same governance patterns: versioning, rollback, compatibility rules, and audit trails.

Prompt Templates as Registry Artifacts

Instead of hardcoding the system prompt, the chatbot renders it via the registry’s /render endpoint:

// Send variables to the registry, get back the rendered prompt
Map<String, Object> variables = Map.of(
    "supported_artifact_types", "AVRO, PROTOBUF, JSON, ...",
    "additional_context", ragContext
);
String rendered = renderPrompt("apicurio-support-system-prompt", version, variables);

The prompt template is stored as a YAML artifact following the PROMPT_TEMPLATE schema, with typed variables, defaults, and metadata. The registry handles variable substitution server-side — the application never implements a template engine.

This means I can update the system prompt — changing tone, adding context, adjusting behavior — without redeploying the application. I can A/B test prompt versions by passing different version parameters. And if a prompt change makes the chatbot worse, I roll back to the previous version in the registry.

RAG Pipeline

At startup, the DocumentIngestionService asynchronously fetches 12 pages of Apicurio Registry documentation, parses the HTML with JSoup, chunks the text (500 tokens, 50 token overlap), and embeds it with an in-process ONNX model. No external embedding service needed — the model runs inside the JVM as a Maven dependency.

ContentRetriever retriever = EmbeddingStoreContentRetriever.builder()
    .embeddingStore(embeddingStore)
    .embeddingModel(embeddingModel)
    .maxResults(5)
    .minScore(0.6)
    .build();

The minScore(0.6) threshold is critical — without it, the retriever returns marginally relevant chunks that confuse the LLM into hallucinating. Better to return fewer, higher-quality results.

What I Learned

Embedding model choice matters more than LLM choice for RAG quality. A better embedding model retrieves more relevant chunks, which gives the LLM better context. Switching from a generic embedding model to bge-small-en-v15 improved answer relevance more than switching between LLM models.

Chunk size is the most impactful hyperparameter. Too small (100 tokens) and you lose context. Too large (1000 tokens) and you dilute the relevant information. 500 tokens with 50 token overlap was the sweet spot for technical documentation.

Prompt versioning prevents “it worked yesterday” debugging. When the chatbot’s behavior changes, I can diff prompt versions in the registry. This is the same benefit schema versioning gives you for data contracts.

In-process embeddings simplify deployment dramatically. Moving from Ollama-hosted embeddings to ONNX-in-JVM eliminated an entire service from the Docker Compose stack. The embedding model is just a Maven dependency — no containers, no health checks, no model downloads.

This pattern generalizes beyond chatbots. Any system that uses prompts — summarizers, classifiers, code generators — benefits from versioned prompt management. The registry already has the infrastructure for versioning, compatibility rules, and access control.

Full article with Docker Compose setup and prompt template examples: distributed-deep-dives/rag-with-schema-registry