AIMux
Open-source agentic AI service mesh - infrastructure for observability, routing, and governance in multi-agent systems.
AIMux is an open-source agentic AI service mesh — infrastructure for the control plane that multi-agent systems need but don't yet have.
The Problem
As organisations deploy autonomous AI agents, the problems start to look remarkably like the problems microservices faced a decade ago: routing, observability, identity, and governance at the network layer. But the existing solutions — LangChain, CrewAI, AutoGen — are orchestration frameworks, not infrastructure. They own the agent lifecycle. AIMux doesn't. It operates at the infrastructure layer, securing and governing agents that run unmodified.
The Architecture
AIMux uses a three-layer architecture, each with a single responsibility.
Runs unmodified. It speaks whatever protocol it natively uses — MCP, A2A, plain HTTP, anything. It makes requests to localhost, completely unaware it's participating in a mesh.
A lightweight binary deployed alongside each agent. It proves the agent's identity via mTLS, sniffs outbound requests to detect the protocol and extract an intent hint, wraps the original payload untouched inside a normalised envelope with identity and intent metadata as headers, and enforces an outbound whitelist so the agent can only communicate through approved gateway addresses.
Receives normalised envelopes from sidecars, reads the metadata headers, performs semantic routing on the intent hint, enforces ACL policies against the cryptographically proven identity, collects observability metrics, and proxies the still-wrapped payload to the destination.
Key Design Decisions
The sidecar never translates, transforms, or interprets the payload. MCP adds a new message type? Doesn't matter — AIMux is wrapping it, not parsing it. A2A ships a v2 schema? Irrelevant. Protocol agnosticism comes from treating payloads as opaque.
Agents can't bypass the mesh because the sidecar controls outbound traffic. Identity is cryptographic (mTLS), not self-declared. The gateway enforces ACLs centrally. Revoking a certificate instantly disconnects an agent from the mesh.
Without sidecars, AIMux works as a single binary with YAML configuration and header-based identity — immediate value for experimentation. With sidecars, you get mTLS, zero-trust networking, protocol-agnostic routing, and cryptographic identity — hardened for production.
This architecture maps cleanly onto existing infrastructure patterns. Anyone who's deployed Istio with Envoy sidecars immediately understands the model.