LiteLLM, Kong, Cloudflare, Portkey, and Odock: An Honest AI Gateway Comparison

The short version

If you need the most mature open-source LLM gateway today, LiteLLM is hard to ignore. It has broad provider support, OpenAI-compatible access, virtual keys, spend tracking, budgets, rate limits, fallbacks, observability integrations, MCP gateway work, and a serious guardrail ecosystem.

If your company already runs Kong, Kong AI Gateway is a logical path. It extends a proven API gateway with AI proxying, prompt-aware plugins, semantic guards, rate limiting, logging, and enterprise deployment patterns. The biggest advantage is not only AI. It is Kong's existing API management muscle.

If you want quick edge-level visibility, caching, routing, DLP, and controls inside Cloudflare's network, Cloudflare AI Gateway is attractive. If you want a productized AI operations platform with dashboards, prompt management, guardrails, and hosted ergonomics, Portkey is strong. If your platform is deeply Kubernetes-native and you want open, Envoy-based routing primitives, Envoy AI Gateway is worth watching.

Odock should be judged differently. The goal is not to pretend that a newer project has more maturity than these platforms. The goal is to build an AI-native gateway where modular security and plugin-augmented workflows are not side features. They are the core operating model.

Where LiteLLM is strongest

LiteLLM's center of gravity is model access for platform teams. It is designed to give developers one OpenAI-compatible interface across many model providers while giving platform owners controls for cost, model access, keys, budgets, rate limits, fallbacks, observability, and guardrails.

That makes LiteLLM a strong fit when the main problem is provider sprawl. Teams can standardize access across OpenAI, Anthropic, Azure, Bedrock, Vertex, self-hosted models, and many others. It is also strong when finance and platform teams need usage attribution by key, user, team, or organization.

LiteLLM's guardrail approach is more mature than many people assume. It supports built-in and third-party guardrails, custom guardrail classes, and lifecycle hooks before the call, during the call, after the call, and across streaming responses. That is real extensibility.

The tradeoff is that LiteLLM's architecture still feels primarily like a model access gateway with guardrails and callbacks around it. That is not a weakness for many teams. It is exactly what they need. But if your core design goal is a modular security engine and workflow plugin graph that treats AI requests, MCP tool calls, policy decisions, and transformations as one programmable pipeline, Odock is aiming at a different shape.

Where Kong is strongest

Kong starts from a different world: API management. Kong AI Gateway is not just an LLM proxy. It is Kong Gateway plus AI-specific plugins. That matters because Kong already has a mature story for routing, auth, rate limiting, plugins, deployment modes, Konnect, Kubernetes, observability, governance, and enterprise operations.

For organizations already standardized on Kong, this is compelling. AI traffic becomes another class of API traffic with prompt-aware plugins layered in. Kong's AI Proxy handles provider translation. AI Proxy Advanced adds load balancing, retries, and fallback. Prompt guard plugins can scan user messages with regex or semantic similarity. Response guard and semantic cache plugins extend that model further.

Kong's strength is also its tradeoff. It is an API gateway first. The AI capabilities are powerful, but they live inside a broad API management framework. If you need enterprise API lifecycle management, that is excellent. If you only want a focused AI control plane for LLMs, MCP servers, security modules, and workflow plugins, Kong can feel larger than the problem.

Odock's plugin idea is narrower and more AI-specific: enrich or constrain the AI workflow itself, not only attach policies to an API route. That means plugins for prompt transformation, data masking, retrieval policy, tool-call approval, schema validation, output repair, routing hints, budget behavior, audit metadata, and tenant-specific workflow rules.

Where Cloudflare AI Gateway fits

Cloudflare AI Gateway is strongest when teams want a fast path to centralized AI observability and control at the edge. It emphasizes analytics, logs, caching, rate limiting, dynamic routing, model fallbacks, BYOK, guardrails, DLP, and cost tracking across supported providers.

The main advantage is operational convenience. If your application already sits behind Cloudflare, adding AI visibility and controls with minimal application changes is appealing. Caching and rate limiting also fit naturally into Cloudflare's infrastructure strengths.

The tradeoff is ecosystem gravity. Cloudflare AI Gateway is best when you are comfortable putting this control layer into Cloudflare's platform. For many teams that is a good decision. For teams that want a self-contained open-source gateway with custom workflow modules, deeper MCP control, and plugin logic that can run wherever the AI platform runs, Odock is aiming for more portability and custom extension.

Where Portkey fits

Portkey is closer to a productized AI operations platform. It combines a gateway with observability, logs, prompt management, guardrails, caching, retries, fallbacks, load balancing, budget limits, and agent framework integrations. It has a broad feature set and a polished operating model for teams that want dashboards and workflow around LLM usage.

Portkey is especially interesting for teams that want prompt lifecycle management and AI observability alongside gateway controls. Its guardrails can be deterministic, model-based, partner-provided, or custom, and guardrail outcomes can influence routing behavior.

The tradeoff is that Portkey is more of a full platform. That is useful if you want the platform. It is less ideal if you want to own a smaller gateway layer, keep workflow logic close to your infrastructure, and build custom security/plugin modules as code-first primitives.

Odock should not try to out-dashboard Portkey on day one. The sharper goal is to make the request lifecycle itself more programmable and governable for teams building their own AI platform.

Where Envoy AI Gateway fits

Envoy AI Gateway is important because it comes from the cloud-native infrastructure side. It builds on Envoy Gateway patterns and Kubernetes resources to route and manage LLM traffic. Its strengths are provider connectivity, OpenAI-compatible and Anthropic-compatible endpoints, model virtualization, fallback, token-aware rate limiting, observability, and Kubernetes-native configuration.

This is a strong direction for platform teams that already run Kubernetes and want AI traffic managed through familiar infrastructure primitives. It is also promising for high-performance, open, infrastructure-level deployments.

The tradeoff is product surface. Envoy AI Gateway is a lower-level infrastructure project compared with hosted AI ops platforms or app-facing gateways. If you want a rich modular security engine, workflow plugins, tenant-aware AI policies, and MCP governance as first-class user-facing concepts, you may need to assemble more of that yourself.

Where WSO2, Tyk, and Apigee fit

WSO2, Tyk, and Apigee come from enterprise API management. Their AI gateway work focuses on bringing LLM access, governance, traffic control, observability, cost tracking, model abstraction, safety policies, and developer access into established API management systems.

This is useful for larger organizations that already need portals, lifecycle workflows, enterprise governance, SSO, policy management, and compliance reporting. They are often a better fit for centralized platform teams than for small teams that want a lightweight AI-native gateway.

The tradeoff is complexity and focus. Enterprise API management platforms solve many problems beyond AI. Odock is intentionally narrower: one dock for LLM providers, MCP servers, security modules, budgets, quotas, and workflow plugins.

The real difference: approach

Most AI gateways now share the same checklist:

• Unified provider access
• OpenAI-compatible request paths
• Provider keys and credential management
• Logs, metrics, and usage tracking
• Budgets, quotas, and rate limits
• Caching, retries, routing, and fallbacks
• Prompt or response guardrails
• Some form of custom extension

That checklist does not explain the architectural difference. The better question is: what does the gateway consider its primary object?

• LiteLLM primarily treats the model request as the object to normalize, meter, route, and govern.
• Kong primarily treats AI traffic as API traffic enhanced by AI-aware plugins.
• Cloudflare primarily treats AI calls as traffic to observe, cache, route, and secure at the edge.
• Portkey primarily treats AI usage as an operations workflow with gateway, logs, prompts, guardrails, and dashboards.
• Envoy AI Gateway primarily treats AI traffic as cloud-native infrastructure routing with Kubernetes and Envoy primitives.
• Odock treats the AI workflow as the object: model calls, MCP tools, tenant policy, security checks, plugins, budgets, and observability running through one programmable control plane.

That is the positioning. Odock is not only a proxy. It is meant to be a workflow gateway.

Odock's modular security engine

Security in AI systems is not one filter. A real AI request has several moments where security decisions may be needed:

• Before the prompt reaches a provider
• After retrieval adds private context
• Before an MCP tool is exposed to an agent
• Before a tool call executes
• While routing to a provider with different data policies
• After the model returns output
• Before logs, traces, and spend records store metadata

Odock's security engine should be modular across those moments. A module can be deterministic, model-based, external, tenant-specific, or workflow-specific. One module might redact PII. Another might detect prompt injection. Another might enforce a customer data boundary. Another might approve or block an MCP tool call. Another might strip sensitive data before telemetry is written.

This is the important difference from a simple "guardrails" checkbox. The security engine is not just prompt moderation. It is a chain of policy modules that can inspect, transform, deny, route, annotate, or escalate at different stages of the AI workflow.

Odock's workflow plugins

Plugins are the other half of the approach. In many AI systems, application teams keep rewriting the same glue code: prompt formatting, schema checks, tool argument validation, output repair, fallback rules, customer-specific metadata, and audit hooks.

Odock's plugin layer is meant to move that logic into the gateway when it is shared, operational, or policy-sensitive.

• A validation plugin can reject malformed requests before they spend tokens.
• A transformation plugin can normalize provider-specific response shapes.
• A security plugin can mask secrets before a provider sees them.
• A routing plugin can add model hints based on tenant, workload, or budget state.
• A tool plugin can require approval before an MCP action executes.
• An observability plugin can attach audit metadata without exposing raw content.
• A response plugin can enforce JSON schema or citation requirements before returning to the app.

This is where Odock should differentiate: plugins should not be only generic HTTP middleware. They should understand the AI lifecycle and let teams augment the workflow around the model.

When you should choose each gateway

Choose LiteLLM if you want mature open-source model access, broad provider coverage, virtual keys, budgets, fallbacks, and a battle-tested AI gateway pattern today.

Choose Kong if your organization already uses Kong or needs enterprise API management with AI plugins layered into the same operational model.

Choose Cloudflare AI Gateway if you want fast edge-level analytics, caching, DLP, routing, and controls inside Cloudflare's platform.

Choose Portkey if you want a productized AI ops platform with gateway, prompt management, observability, guardrails, and dashboards.

Choose Envoy AI Gateway if your team wants open, Kubernetes-native, Envoy-based AI routing infrastructure.

Choose WSO2, Tyk, or Apigee if your organization already needs enterprise API lifecycle management and wants AI governance inside that broader platform.

Choose Odock if your priority is an AI-native gateway where modular security, MCP governance, and workflow plugins are central to the design, and where the gateway is not only routing requests but shaping the full AI workflow.

The honest Odock weakness

Odock should be clear about its current position. It is newer. It does not yet have the same production history, provider ecosystem, enterprise packaging, or community size as LiteLLM, Kong, Cloudflare, Portkey, Envoy, WSO2, Tyk, or Apigee.

That matters. A team with urgent production requirements should evaluate maturity, deployment model, support, security posture, and operational evidence before choosing any gateway.

The reason to bet on Odock is not that the market lacks gateways. The reason is that AI workflows are becoming more than model calls. They include tools, context, policies, approvals, transformations, budgets, and tenant-specific rules. Odock's opportunity is to make those workflow controls modular from the beginning.

Final take

The AI gateway market is splitting into several categories. LiteLLM is the practical open-source model gateway. Kong is the API management gateway extended for AI. Cloudflare is the edge control layer. Portkey is the AI operations platform. Envoy is the cloud-native routing foundation. Enterprise API platforms bring governance at organizational scale.

Odock's thesis is narrower and sharper: AI teams need a gateway where security modules and workflow plugins are first-class, because the hard production problems are no longer only "which provider do we call?" The hard problems are "what should happen before, during, and after this AI workflow, for this tenant, with this data, this model, and these tools?"

That is the difference in approach.