BlogConcentrate adds governance and accountability that OpenRouter's routing alone cannot provide.
Concentrate.ai
LLM Gateway
Change models, providers, add fallbacks, and more by running all AI usage through one platform.
Every AI application that survives the prototype stage ends up depending on multiple models, providers, and billing systems. If a single-provider integration worked in development, it starts breaking production operations. The question this article answers is not whether you need a gateway, but which layer of gateway your team actually requires.
The failure modes arrive in a predictable sequence. One provider's rate limits bite first. Then an outage takes the application down. Then finance asks why a runaway job burned a month of budget in an afternoon. Then a cheaper or better model ships, and switching means touching every call site in the codebase.
Teams respond to each failure by bolting on fixes: retries here, a second provider there, a spend cap, a cache. Within a quarter they have built half a gateway badly, inside their own codebase, with no one owning it. The architectural answer has a name: an LLM gateway, one endpoint in front of many providers that centralizes auth, routing, fallback, spend tracking, and policy enforcement.
The question that follows is less comfortable. Not every gateway solves the same version of this problem. Some centralize model access and stop there. Others carry the governance and accountability layer that production teams in regulated or cost-sensitive environments cannot ship without. Concentrate, for one, handles automatic fallbacks across providers serving the same model, so failover logic stays out of application code.
OpenRouter's core value is breadth and speed of access. A single OpenAI-compatible endpoint reaches hundreds of models from dozens of providers, and it routes automatically to the cheapest or fastest provider. Opt-in model-level fallbacks let a single request degrade gracefully instead of failing. The service requires no infrastructure to operate, which makes it the fastest path from zero to multi-model access.
The routing surface is genuinely powerful. Provider sort options let callers prefer price, throughput, or latency. Fallback chains such as trying Claude, then GPT, then Llama mean one call can survive several provider failures in sequence. BYOK is supported, so teams with committed spend at a provider can route through it.
The data posture is honest and worth reading carefully. Zero data retention of prompt content by OpenRouter itself is the default, with prompt logging opt-in. But ZDR enforcement over downstream providers must be explicitly enabled and is not the default. That distinction matters for teams without a formal data governance posture. The safe default stops at the boundary of the platform itself.
Pricing is genuinely pass-through. Credit-based pricing charges against each model's native tokenizer, and independent reviews confirm catalogue rates match providers' published prices. The per-token economics are not where the cost hides. OpenRouter is hosted-only, which matters for teams with data-residency or infrastructure-control requirements.
The moment a team adds cost accountability across multiple stakeholders, handles sensitive data in any request path, or needs to prove to an auditor what ran and who authorized it, provider routing becomes necessary but not sufficient. What is required on top is a business layer: infrastructure that enforces policy in the request path rather than leaving enforcement to developer discipline. A business layer is operationally distinct from a model menu. The model menu translates between providers and removes integration friction. The business layer governs what the menu is allowed to serve. Five capabilities define it, and routing alone provides none of them.
First, real-time spend visibility broken down by team, project, key, model, and provider, not reconstructed from end-of-month invoices. Second, PII and PHI redaction running at one chokepoint, reviewable per request, enforced by policy. Third, RBAC gives you scoped keys with per-team and per-project budgets, so one bad loop or one junior developer cannot drain the account. Fourth, audit logs: a durable, reviewable record of who made which call, what data was sent, and what came back. Fifth, automatic fallbacks enforced by policy and logged, not hand-wired in application code. Concentrate bundles these five capabilities into a single managed API, with no infrastructure for teams to deploy or maintain.
The regulatory context is pushing in the same direction. All 50 US states have introduced AI bills, and the EU AI Act's high-risk obligations land on December 2, 2027. Governance infrastructure is becoming mandatory rather than optional, and the business layer is where that mandate gets enforced.
For engineering teams, the missing business layer means reliability, cost control, and security are each hand-wired separately. Each one breaks separately, and without warning. Fallback without policy means fallback without a log. Mid-stream failures, where tokens have already been sent, still require application-level handling. A good gateway is honest about this boundary. Without real-time granularity in spend tracking, the finance conversation happens after the damage. Knowing that Anthropic spend was high this month is not the same as knowing that the support bot, not the coding assistant, drove it, before the invoice arrives.
Security controls distributed across application code depend on developer discipline. Centralized PII redaction at the gateway layer is the only architectural guarantee that sensitive data does not reach a model provider unguarded. That guarantee cannot live in a thousand call sites and survive. It has to live in one chokepoint.
Finance and security teams feel the same absence even when engineering keeps the product running. Their version of the problem is visibility and accountability, not uptime. For finance, AI spend is now a budget line item. Without real-time granular tracking it cannot be managed. The question is not whether spend happened but which team, project, or model drove it, and whether it was authorized. Those anomalies are real: a successful product launch, a misconfigured workflow, a developer who left a loop running overnight.
For security, the exposure is wider than most engineering teams realize. A large share of enterprise AI interactions involve sensitive data, and much of it flows through unmanaged personal accounts that bypass corporate controls. The gateway is the chokepoint where that leak can be closed. Audit logs, RBAC, and per-request PII redaction review give security teams the ability to confirm what data was sent, confirm it was redacted, and see who made the call, without requiring developers to instrument each integration separately. SSO means identity management for AI access runs through the same system as the rest of the organization, not through a separate set of API keys that only engineering understands. OpenRouter's per-token economics are genuinely pass-through, but the fees attached to credit purchases and BYOK still affect total cost of ownership. For teams that need granular, real-time spend visibility broken down by team, project, key, and provider, or that require audit trails and policy-enforced fallbacks, that routing layer alone does not close the loop.
For teams handling PHI or operating in regulated industries, the governance requirements a business layer satisfies are not enhancements to developer experience. They are the legal floor below which the product cannot ship. HIPAA requires that PHI not reach model providers without a Business Associate Agreement covering that provider. A BAA must be in place before any patient data flows through a gateway. PHI redaction must run at one enforceable chokepoint, not distributed across application code where it can be missed or bypassed. Zero data retention is also a HIPAA-relevant control: prompt content must not be stored by the gateway or by upstream providers beyond what the BAA explicitly permits.
For AI-native healthcare startups, the conventional assumption has been that enterprise-grade compliance infrastructure requires enterprise procurement cycles. The gateway layer is where that assumption can be challenged. A managed gateway with a BAA available on the paid plan and redaction enforced at the request level changes what a small team can ship without a platform team behind it.
Concentrate and OpenRouter serve genuinely different primary use cases. OpenRouter is optimized for fast, broad model access with minimal setup. Concentrate is built for teams that need the full business layer, spend controls, PII and PHI redaction, RBAC, audit logs, and automatic fallbacks, on top of comparable model access, at no additional cost.
Concentrate provides one API for roughly 200 models from dozens of providers. The model breadth is comparable to OpenRouter's. The difference is what surrounds the routing. PII and PHI redaction runs in the request path, reviewable per request, enforced by policy rather than developer discipline. RBAC, SSO, and audit logs are available from day one, free, not gated behind an enterprise contract. Automatic fallbacks log every routing decision, so engineers can see exactly which request hit which provider and why.
The compliance posture is specific. Concentrate launched June 10, 2026, backed by True Ventures and RRE Ventures. The full governance stack is free and on by default for every customer. Zero data retention is the default. There is zero markup, no platform fee, card fee, or BYOK fee. Customers pay for tokens. A HIPAA-compliant gateway with a BAA available on the paid plan is designed for AI-native healthcare startups, not only hospital systems with enterprise procurement. The service is fully managed: no proxy to deploy or maintain, so engineers ship product rather than operating gateway infrastructure.
OpenRouter offers the broadest single-integration model catalog available as a hosted service, with hundreds of models from dozens of providers through one OpenAI-compatible endpoint. Provider routing sorts by price, throughput, or latency. Model-level fallback chains are configurable at the request level. BYOK is supported, though credit-purchase and BYOK fees apply despite the no-markup framing, and budgets should account for them. It is hosted-only. There is no per-team RBAC, audit logs are available per-key and per-workspace activity, PII redaction is available via Workspace Guardrails, and there is no HIPAA BAA. The governance layer is the team's responsibility to build elsewhere. That is the right trade-off for prototyping and experimentation. It is the wrong one when compliance and cost accountability are the requirement.
LiteLLM gives teams full infrastructure control and avoids per-token gateway fees. It is an open-source Python SDK and proxy server. Teams deploy it themselves, keep request data inside their own infrastructure, and own every routing and fallback configuration. The core SDK and proxy are free. The Enterprise commercial license adds SSO, RBAC, audit logs, and SLAs. When HIPAA coverage or policy enforcement inside your own infrastructure is the requirement, Portkey and Kong AI Gateway are the recommended alternatives. The hidden cost is engineering time. In March 2026, malicious LiteLLM versions 1.82.7 and 1.82.8 were published to PyPI, a supply-chain compromise that managed gateways avoid by design and a concrete illustration of the operational surface that self-hosting adds.
The right gateway choice follows from two questions most feature comparisons never ask directly. Who else in your organization needs visibility into AI usage? And what happens if sensitive data reaches a model provider unguarded? If the honest answer to both questions is that only engineers need visibility and the risk is acceptable for now, OpenRouter is the fastest path to multi-model access with no operational overhead. It is the right tool for that profile. If finance needs real-time cost attribution, or security needs to confirm PII never left the perimeter, or a compliance officer needs an audit trail, those requirements belong in the gateway layer, not in application code built on top of a routing-only service. If the team handles PHI and needs a BAA before any patient data flows through a model, that requirement sets the floor. Only gateways that offer a BAA and enforce redaction at the request level can meet it. If self-hosting is a hard procurement requirement, LiteLLM is the right starting point, with the understanding that SSO, RBAC, and audit logs require the Enterprise license and that operational overhead is real.
Provider complexity, routing, fallbacks, spend tracking, redaction, audit logging, belongs in the infrastructure layer, enforced by policy, not distributed across application code where it depends on every developer remembering to implement it correctly every time. The distinction matters more as teams grow. A two-engineer team can enforce policy through discipline. A twenty-engineer team cannot, and an audit does not accept discipline as a control. The market is moving toward requiring the business layer regardless of preference: the regulatory trajectory, the enterprise adoption rate, and the entry of finance and security teams into AI governance all point the same direction.
Choose the layer of the stack that matches where your team actually is today. But design the gateway decision for where your compliance and organizational requirements will be in twelve months. Switching gateways after a PHI incident or a runaway spend event is considerably harder than starting with the right layer.