Do You Need to Self-Host an LLM to Use It With HIPAA Data? Not With a HIPAA AI Gateway
A HIPAA AI gateway eliminates self-hosting by routing patient data only to BAA-eligible providers.
Concentrate.ai
On this page
No. You do not need to self-host an open-weight LLM to use it with protected health information (PHI). A HIPAA AI gateway like Concentrate gives you hosted inference on providers covered by a Business Associate Agreement (BAA), through one API and one signed BAA. With a BAA, Concentrate is $499/month, with no per-request fees and no markup on tokens.
Why have healthcare teams self-hosted models for HIPAA?
Healthcare teams self-hosted because it was the only way to be sure PHI never reached a vendor without a BAA. Under HHS guidance, a cloud or AI service that creates, receives, maintains or transmits ePHI for you is a business associate and must sign a BAA.
For open-weight models, that left three options:
Self-host: rent GPUs in your own HIPAA-ready cloud account and run the model yourself.
Negotiate BAAs one provider at a time: find each inference provider willing to sign, then repeat for every new model or provider.
Keep PHI out: skip the model for anything touching patient data.
Each option costs time. Self-hosting means owning GPU capacity, scaling, patching and uptime. Provider-by-provider BAAs mean a new legal review every time you want to try a new model.
A HIPAA AI gateway signs one BAA with you and routes your requests only to inference providers that are BAA-eligible. You call one API. The gateway handles which providers can see PHI.
The key difference from a bring-your-own-keys proxy is that the gateway is managed and provides the inference. A proxy that only forwards your own provider keys still leaves you negotiating a BAA with every model provider. Concentrate's BAA covers the gateway and the inference on BAA-eligible providers.
Can you use OpenRouter, Portkey or LiteLLM with PHI instead?
Not with one BAA that covers both the gateway and the inference. Each falls short in a different way:
OpenRouter: sells inference across many providers, but its enterprise page lists SOC 2 Type II and a DPA, not a BAA. Without a BAA, PHI should not go through it.
Portkey: a gateway that does not sell inference. Its virtual keys store your own provider API keys, so you still need a separate BAA with every model provider you route to.
LiteLLM: an open-source proxy you deploy and run yourself, with your own provider keys. You own the infrastructure and negotiate a BAA with every provider.
Concentrate is managed and provides the inference, so one BAA covers the gateway and the inference on BAA-eligible providers.
How does a HIPAA AI gateway work?
A request with PHI moves through four steps:
Your healthcare app sends a request to the Concentrate API, using a key from your HIPAA-enabled organization.
The Concentrate gateway checks the route, applies redaction rules, and allows only providers marked BAA-eligible.
A BAA-eligible inference provider runs the request.
The model returns its response back through the gateway to your app.
Self-hosting vs a HIPAA AI gateway: what are the tradeoffs?
Self-hosting
HIPAA AI gateway (Concentrate)
Infrastructure
You run GPUs, scaling and patching
Hosted inference, nothing to run
BAAs to sign
Your cloud provider
One BAA with Concentrate
Model choice
Models you deploy and maintain
Any model served on a BAA-eligible provider
Cost model
GPU capacity, paid whether you use it or not
$499/month with a BAA, plus provider token prices with no markup and no per-request fees
Operations
Your team owns uptime and upgrades
Concentrate routes across providers
Best fit
Teams that need full control of weights and hardware
Teams that want hosted models with PHI, fast
Self-hosting is still the right call if you need to fine-tune weights on your own hardware or keep everything inside your own network. For most teams that just want to call a model with patient data, a gateway removes the infrastructure work.
Example: how do you use GLM-5.3-Flash with HIPAA data?
You use GLM-5.3-Flash with PHI through Concentrate by routing only to its BAA-eligible providers. GLM-5.3-Flash is a z.ai model released August 26, 2026, with a 1M-token context window, image and PDF input, and function calling.
On Concentrate, GLM-5.3-Flash is BAA-eligible on 3 of its 13 providers today: Fireworks AI, Baseten and Azure Fireworks. z.ai's own API is not BAA-eligible on Concentrate, so PHI should not go there. These are examples as of November 2026. See Model Fortress for the live list.
Without a gateway, using GLM-5.3-Flash with PHI would mean self-hosting it or signing separate BAAs with providers that serve it. With Concentrate, it is one API call to a BAA-eligible route. Full details are in our GLM-5.3-Flash BAA guide.
How do I get started without self-hosting?
Contact Concentrate through concentrate.ai/compliance, start the $499/month BAA subscription and sign the BAA.
Enable a HIPAA organization.
Restrict PHI routing to providers marked BAA-eligible on each model page.
Use provider allowlisting, redaction before requests leave the gateway, and SSO with role-based access control.
You still own your own HIPAA obligations, such as access policies and what data your app sends. For more on the gateway itself, read HIPAA Compliant AI Gateway.
FAQ: self-hosting, HIPAA and BAAs
Do you need to self-host an LLM to use it with HIPAA data?
No. You can use a HIPAA AI gateway that signs a BAA and routes PHI only to BAA-eligible inference providers. Self-hosting is one option, not the only one.
Is an open-weight model HIPAA compliant?
No model is HIPAA compliant on its own. Compliance depends on the deployment: a signed BAA, a HIPAA-enabled organization, and routing only to BAA-eligible providers.
Can I use OpenRouter, Portkey or LiteLLM for PHI?
Not under one BAA that covers the inference. OpenRouter does not list a BAA, and Portkey and LiteLLM forward requests using your own provider keys, so you would need a BAA with every provider.
Does Concentrate sign a BAA?
Yes. BAA use is $499/month, with no per-request fees and no markup on tokens. Without a BAA, there is no subscription fee. See concentrate.ai/compliance.
Does one BAA cover other models?
Yes. One Concentrate BAA covers every model served on a BAA-eligible provider. Browse them in Model Fortress.
Can I send PHI to any provider through the gateway?
No. Send PHI only to providers marked BAA-eligible. Each model page shows which providers qualify.
Blog