I built a Kubernetes-style control plane for AI agents

Why? Because I wanted to. And it feels so good to put it that way. Throughout my career, I feel like I’ve spent more effort justifying my decisions than actually bringing ideas to life. And I do resent that. But make no mistake: I understand that this is at the core of the job. As Gerald Weinberg put it, “No matter what the problem is, it’s always a people problem.” What makes or breaks a project is really its people and processes.

But ah, ah! Not this one. This one is the fruit of my pure indulgence, grown in my sweet solitude.

Over the past year, most of what I’ve done in my day job has been assembling different kinds of LLM-based workflows. I say “different”, and they are, in the sense that they solve different problems. But substantively? Not really. At least not by my standards.

I keep seeing the same three basic kinds of “AI project”: setups built around Claude SKILLS, collections of MCP servers, and backend components built with LangGraph, Semantic Kernel, Agent Framework, or some other framework. In all of them, the “AI” behaves like a supermassive object, pulling the rest of the system into its orbit. I find it especially problematic when a particular model provider’s capabilities become the system’s centre of gravity: every design conversation hinges on what the current “best model” can do.

I want to invert that dependency. The application backend tends to own the whole agent stack: the provider SDK, the model configuration, the tool loop, the memory strategy, and then all the plumbing. The application ends up being built around the agent. With this, I’m essentially trying to bring agents into the infrastructure layer and out of the application backend. The application should depend on a stable agent contract, while the platform owns how that agent is assembled, deployed, and run. The LLM should serve the application, not become the architecture around which the application is built.

That does not mean the application should know nothing about the agent. I mean, it should not have to care which model powers it, where its tools live, how its memory is stored, or how the agent is deployed. Swapping the model may still change the agent’s behaviour and require testing, of course. But it should not require rewriting the client application’s backend.

So I figured the best way to do that was to build a platform that treats agents as declarative resources. Applications invoke an agent through a stable interface; Rosen owns the plumbing behind it. Models, MCP servers, and memory backends can change behind that boundary without forcing changes into every application that uses the agent.

I have no serious expectation that this will ever grow into a major project. There are better-funded projects (or just funded, really; my funding for this is insomnia) that already have entire teams working in this space. I can’t compete with that alone. Hosting it on Codeberg and licensing it under AGPL-3.0 probably won’t do it any favours when it comes to wide adoption, either. For me, it’s a nice learning project (like I said, an indulgence). I won’t explain in this post why I made these choices. For now, let’s just say that if you’re a self-hosting tinkerer like me, maybe you’ll find this one interesting.

It’s called Rosen. I took the name from Do Androids Dream of Electric Sheep? and Blade Runner. I think it’s appropriate for an “AI project”.

Rosen is a control plane

…a Kubernetes control plane for agents, model endpoints, tools, and memory stores.

OK, OK, so in practice, this is what all that blabbing means: an AgentDeployment declares an agent and refers to a ModelEndpoint, plus whatever Tool and Memory resources it needs. A controller watches those declarations, creates or updates the runtime, and keeps the running system converged on the desired state. The application asks for an agent by name; Rosen owns how the pieces behind that name are wired together.

Something like this:

apiVersion: rosen.dev/v1
kind: ModelEndpoint
metadata:
  name: local-qwen
spec:
  engine: openai
  model: qwen2.5:3b
  baseURL: http://ollama.default.svc.cluster.local:11434/v1
---
apiVersion: rosen.dev/v1
kind: Memory
metadata:
  name: conversation
spec:
  backend: redis
  mode: conversational
  endpoint: redis://redis.default.svc.cluster.local:6379/0
---
apiVersion: rosen.dev/v1
kind: Tool
metadata:
  name: ticketing
spec:
  transport: mcp
  endpoint: http://ticket-tools.default.svc.cluster.local:8000/mcp
  permissions:
    - create_ticket
    - search_tickets
---
apiVersion: rosen.dev/v1
kind: AgentDeployment
metadata:
  name: support-agent
spec:
  runtime:
    image: support-agent:prod
  model:
    ref: local-qwen
  memory:
    - ref: conversation
  tools:
    - ref: ticketing
  replicas: 2

So it looks something like this:

Imagine Juicero wanted to take juice-making to the next next level: the agentic level! It would manage juice-extraction incidents on some kind of ticket board for a more AGILE incident-triage experience.

Juicero should not care which model serves the agent, where conversation history lives, or which MCP server exposes the ticketing tools. It should call Rosen and ask for support-agent. Whether Rosen represents that invocation as an AgentRun, dispatches it to a served agent over /invoke, polls a worker, or writes status back is control-plane business.

…And an SDK

A Python SDK (only Python, for now).

Rosen agents are deployed as containers running inside Kubernetes pods. That is the domain of the control plane, but it is not what I want the agent author to think about. The SDK has two jobs.

First, it lets developers define agents without tying the code to a specific model, tool server, or memory backend. Pretty much this: receive input, build a prompt, maybe save or recall some memory, and return an answer. Second, it lets that same agent connect back to the control plane without the agent code becoming control-plane code.

import rosen


def handle(ctx):
    ticket = ctx.input.strip()

    # Semantic memory: the context knows which store was bound in YAML.
    similar = []
    if ctx.memory.has_mode("semantic"):
        similar = ctx.memory.recall(ticket, k=3)

    prior = "\n".join(f"- {hit['text']}" for hit in similar) or "none"

    # The model remembers the conversation if a conversational memory store was set.
    answer = ctx.model.run(
        "Help Juicero answer this ticket.\n\n"
        f"Ticket:\n{ticket}\n\n"
        f"Similar past tickets:\n{prior}"
    )

    # The code still does not know if this is pgvector, faiss, or something else.
    if ctx.memory.has_mode("semantic"):
        ctx.memory.remember(ticket)

    return answer


if __name__ == "__main__":
    rosen.serve(handle)

That is the shape I wanted: the agent receives a context and does its thing. The code does not say which semantic memory store it is using. It just asks the context to recall similar tickets and remember the new one. When the agent is used in a conversation, the model remembers it because a conversational memory store was set for the agent in YAML, not because the handler learned how to manage chat history.

If the agent needs tools, the context has tools. If the model changes, the agent does not need to know. The YAML says which tools the agent may use; when the handler calls ctx.model.run(...), the SDK advertises those allowed operations to the model and routes any tool calls back through the right transport. serve() is what tells Rosen to run that function as a real service, with replicas, health checks, and the operating machinery hidden away.

It has its own Kubernetes CRDs

I said Rosen is built on K8s. I’ll try to elaborate without making it too eyelash-burning to follow. I shall get more technical in future entries of this series for those readers who may hold any hope of learning anything useful from me.

Something cool I learned while doing this project is that K8s lets you teach the API server new resource types through Custom Resource Definitions (CRDs). Once a CRD is registered, the API server treats your type the way it treats a Pod or a Service: it validates it, stores it, versions it, and serves it to kubectl and every other client. It effectively allows you to extend the control plane to your liking.

Using Kubebuilder makes it easy to generate CRDs. I just wrote plain Go structs and annotated the fields with Kubebuilder markers. controller-gen, the Kubebuilder code generator, reads those markers and produces:

• the CRD manifests the API server serves;
• the deepcopy boilerplate the runtime needs.

When I change a struct, I regenerate. The Go type stays the source of truth.

Rosen registers five kinds under the rosen.dev/v1 group: AgentDeployment, ModelEndpoint, Tool, Memory, and AgentRun. They are not independent. An AgentDeployment consists mostly of references: it binds one ModelEndpoint and, optionally, some Tool and Memory resources, each by name. A Memory in semantic mode points back at a ModelEndpoint to embed its entries. An AgentRun names the AgentDeployment it runs.

The controller resolves those references and reconciles. How? It’s pretty cool, I promise, but it doesn’t fit here. I’ll leave that teaser for this series.

Two of them carry a status field that the controller writes back: AgentDeployment and AgentRun. The other three are pure declarations. They describe things that live elsewhere, so there is nothing for a controller to observe about them.

One reference leaves the diagram. A ModelEndpoint names a built-in Kubernetes Secret for its API key, not a Rosen type, so credentials stay in the resource Kubernetes already has for them.

rosenctl writes that Secret for you:

rosenctl secret set model-creds --from-literal api-key=sk-...

The ModelEndpoint then points at it through spec.auth.apiKeySecretRef, so the key itself never lands in a manifest.

…And its own infrastructure adapters

A ModelEndpoint says engine: openai. A Memory says backend: redis. Those words are the interface. Each name resolves to an adapter, and the same pattern repeats three times:

engine: openai does not mean OpenAI the company. It means the OpenAI-compatible wire format, which vLLM, Ollama, and most self-hosted servers already speak. Moving a hosted model to a local one is a baseURL edit, not a code change:

spec:
  engine: openai
  model: gpt-4o-mini
  baseURL: https://api.openai.com/v1
---
# the same agent, now on a local model
spec:
  engine: openai
  model: qwen2.5:3b
  baseURL: http://ollama.default.svc.cluster.local:11434/v1

The list is short today, but nothing above the adapter cares which one you picked. Adding another is a new leaf: write a class that implements the interface, register its name, and you’re done. It is built to grow to as many backends as I care to write.

So… what’s next?

At the beginning, I explained why I wanted to do this, but not what my north-star vision looks like. And I won’t share it here. Frankly, I think it’s the single fully original idea in this project, and I’m not sure it will work or whether I’ll be able to bring it into reality. So I don’t want to make any promises.

I’m making this a series. There are a lot of juicy technical details about Rosen that I’d like to share, and they won’t fit in a single blog post.

I’m currently looking for contributors. If this post tickled your fancy, you can head over to the repo and check it out. I’ll be adding a few issues for features and bugs that I think would serve as nice introductions to the project. Just contact me if you’d like to be added as a contributor, and we can chat.