The 9 Best LLM Gateways in 2026, Grouped by the Job They Do
Navigating consolidation and tool sprawl to find the right infrastructure for your LLM stack.
Concentrate.ai
Navigating consolidation and tool sprawl to find the right infrastructure for your LLM stack.
Concentrate.ai
LLM Gateway
Change models, providers, add fallbacks, and more by running all AI usage through one platform.
Updated October 2026.
Most "best LLM gateway" lists still include tools that were acquired or frozen this year, and they put tools that do different jobs into one ranking. A self-hosted proxy, an inference marketplace and an enterprise API gateway all get called a gateway, but they solve different problems. This guide sorts them by job first, then ranks within each group.
Short answer:
Three of the best-known names in this category changed hands this year. Check whether a tool is still being built before you migrate to it.
| When | What happened | What it means for you |
|---|---|---|
| March 2026 | Mintlify acquired Helicone. The product is in maintenance mode. | Security fixes and new models only, no new features. Plan a move off the proxy. |
| May 2026 | Palo Alto Networks acquired Portkey, now sold as Prisma AIRS AI Gateway. | Upgrades route through Palo Alto sales. The roadmap follows Prisma's enterprise buyers. |
| August 2026 | Stripe acquired OpenRouter. | OpenRouter says its brand, product and neutrality stay the same. Your gateway is now owned by a payments company. |
An LLM gateway sits between your code and model providers. A good one gives you one API across providers, retries and fallbacks, spend tracking and limits, and a single place to enforce data policy. Some also sell you the tokens, so you can skip a contract with each provider.
| Gateway | Job | Pricing model | Open source | Biggest gap |
|---|---|---|---|---|
| Concentrate AI | Managed gateway + inference | Tokens only, $0 fees | No | No OTel export yet; US-only residency |
| OpenRouter | Managed marketplace | 5.5% credit fee; 5% BYOK above $25K/mo | No | Fees; built for model access more than company controls |
One OpenAI-compatible API (plus Anthropic-compatible /v1/messages) for 196 models across 20+ providers. It charges no platform, service or card fee, and BYOK is free.
The core reliability feature is same-model failover: Claude can fail over across Anthropic, Bedrock, Azure and Vertex with no code changes. It also has ZDR enforcement that fails closed (104 models), per-key PII redaction, real-time spend by team, project and key, budgets that count BYOK spend by default, free SSO, and SOC 2 Type II plus SOC 3 at trust.concentrate.ai.
Gaps: no OpenTelemetry export yet, US-only data residency, no batch API.
The largest model marketplace and the fastest way to try a new model. There's no inference markup, but a 5.5% (min $0.80) credit-purchase fee and 5% on BYOK above $25K a month. Stripe acquired OpenRouter in August 2026; It's now part of Stripe and says the product and its neutrality stay the same. Best for exploration and smaller teams. Larger teams usually add their own controls on top.
Zero markup and zero BYOK fee, with budgets at four scopes and OpenTelemetry trace drains. Team-wide ZDR and provider allowlisting cost $0.10 per 1,000 requests each. BYOK spend isn't counted in budgets. The best fit is teams already on Vercel.
The open-source standard, supporting 100+ providers, virtual keys and fallbacks. Free to run. SSO, SCIM, audit logs and RBAC are in Enterprise. You still need provider accounts or an inference layer behind it.
A Go gateway from Maxim, Apache 2.0, built for low overhead. It has failover, virtual keys and budgets. Clustering and guardrails are in the enterprise tier.
Acquired by Palo Alto Networks in May 2026 and now part of the Prisma AIRS line. It's strong on guardrails, prompt management and logs. Production is $49/month for 100K logs. SSO and granular budgets are Enterprise-only.
$25 per user per month for Pro, which includes pooled team budgets (up to 10 rules) and SSO. A good pick if you also want TrueFoundry's deployment platform.
Caching, analytics and rate limiting are free. Unified Billing adds 5% to credit purchases. Best if you already run on Workers.
AI plugins for Kong Gateway. Makes sense if your platform team already runs Kong for every other API.
A router picks which model or provider handles a request. A gateway is the whole control layer: auth, keys, budgets, logging, data policy and routing.
Managed gateways typically add tens of milliseconds. Concentrate passes streaming straight through.
Most don't in 2026. The costs that remain are card fees, BYOK fees, subscriptions and metered add-ons, so compare those.
Look for SOC 2 Type II, ZDR that fails closed instead of silently degrading, and PII redaction before prompts leave your network. Concentrate includes all three on pay-as-you-go.
Yes. A common setup is LiteLLM for in-house routing, with a managed gateway like Concentrate providing the inference.
| Vercel AI Gateway | Managed gateway + inference | List price; ZDR and allowlist metered | No | BYOK spend excluded from budgets |
| LiteLLM | Self-hosted proxy | Free OSS; Enterprise custom | Yes | You operate it; SSO and audit logs are in Enterprise |
| Bifrost | Self-hosted proxy | Free; enterprise tier | Yes (Apache 2.0) | Self-hosted only |
| Portkey / Prisma AIRS | Platform gateway | $49/mo Production + overages | Partly | SSO and granular budgets are Enterprise-only |
| TrueFoundry | Platform gateway | $25/user/mo Pro | No | Per-seat pricing |
| Cloudflare AI Gateway | Edge gateway | Free core; 5% Unified Billing | No | Attribution is DIY |
| Kong AI Gateway | API gateway plugin | OSS + Kong Enterprise | Partly | Heavy if you don't already run Kong |