BlogA BAA with your model provider covers only one of six HIPAA requirements for PHI-touching AI.
Concentrate.ai
LLM Gateway
Change models, providers, add fallbacks, and more by running all AI usage through one platform.
A signed BAA with OpenAI or Anthropic is not compliance. It is one requirement out of six, and the other five fail in predictable places: the clinician who pastes a chart into an uncovered surface, the RAG pipeline that stores PHI in a vector database, the team that provisions Claude Enterprise without activating HIPAA. This article explains which provider surfaces actually carry BAA coverage, what breaks when teams manage compliance in application code, and how a HIPAA-compliant AI gateway enforces all six requirements at one point.
A healthcare engineering team signs a BAA with OpenAI, signs one with Anthropic, and marks the compliance box checked. HIPAA compliance is a property of the deployment, not of the model or the vendor. An AI system touching PHI is compliant only when it satisfies six requirements at the same time, and a BAA with one or more providers addresses exactly one of them.
The six requirements are concrete and non-negotiable: a signed BAA with every PHI-touching provider, Zero Data Retention on those providers, encryption in transit and at rest, access controls and audit logging, PHI minimization and de-identification where possible, and a documented breach notification process. All six must hold together. A vendor contract cannot stand in for the rest.
The reason is that a BAA is a legal instrument, not a technical control. Everything that happens before the API call, prompt construction, orchestration logic, RAG pipelines, vector stores, is your responsibility. A PHI exposure there is uncovered no matter how clean the vendor contract is.
Anthropic's own BAA page makes the trap concrete. A team that provisions Claude Enterprise without that step is out of compliance even with a paid contract. Concentrate addresses this by consolidating PHI redaction, Zero Data Retention enforcement, and audit logging at one gateway chokepoint before any request reaches a model provider, with a BAA available on its paid plan, so the controls the BAA itself cannot supply exist in a single managed layer.
BAA coverage is narrow and surface-specific at every major provider. The consumer and small-business plans most teams already have are almost universally excluded. Covered paths require deliberate configuration steps that are not automatic, and those steps vary by provider, by product, and sometimes by feature within a product.
For OpenAI, the HIPAA-eligible products are ChatGPT for Healthcare, ChatGPT for Enterprise with Regulated Workspace, ChatGPT FedRAMP, ChatGPT for Clinicians, API with Modified Retention, and API FedRAMP with Modified Retention. ChatGPT Business is explicitly excluded. PHI should not enter those surfaces.
For Anthropic, the BAA covers the first-party API, Claude in Microsoft Foundry, and Claude Enterprise plans, self-serve or sales-assisted, with HIPAA activated. The Primary Owner must turn on HIPAA compliance in organization settings and accept the BAA, and the self-serve plan requires a minimum of 20 seats before any of this configuration is available. The covered API surface includes the Messages API, prompt caching, structured outputs, memory, web search, the bash tool, the text editor tool, and the Token Counting, Models, and Org Management APIs. Claude Code CLI is covered when signed in with a Claude Enterprise account and connecting directly to Anthropic, not through a cloud provider or gateway, under the HIPAA configuration. It is also covered via a 1P API Console key when Zero Data Retention is enabled on a qualified account.
The excluded list is long and specific: Claude Free, Pro, Max, Team, Cowork in the cloud, Workbench, Console, Claude in Office, Claude Design, Batch API, Files API, and most beta features. Using any of these with PHI is a HIPAA violation regardless of care taken. Claude on AWS Bedrock is a separate HIPAA-eligible path under an AWS BAA, keeping data inside AWS infrastructure and never sending it to Anthropic directly. AWS Bedrock covers most models, including Anthropic Claude, Meta Llama, Mistral, Amazon Titan, and Cohere, within HIPAA-eligible AWS regions, though specific models such as Fable and Mythos are explicitly excluded.
Zero Data Retention is a separate requirement on top of the BAA. A BAA alone is not sufficient. A production application using GPT for one workflow and Claude for another must satisfy all six requirements on each path: separate BAAs, separate ZDR configurations, separate audit trails. Each provider added multiplies the compliance surface.
At production scale, managing HIPAA compliance provider-by-provider creates sprawl that no amount of careful vendor configuration can solve from within application code. Different teams end up managing different API keys, provider-specific SDKs, rate limits, retries, and model fallbacks. Each new provider adds another BAA to negotiate, another ZDR configuration to verify, another audit trail to maintain, and another failure point where a misconfigured surface can expose PHI.
Provider uptime is not a solved problem. That is well below the 99.99% or higher guarantees common in core infrastructure services. Agentic AI workflows compound the problem further. They generate substantially more tokens per task than a standard chatbot, which makes anomaly detection and cost attribution critical, and harder to maintain when each team calls providers directly.
A shared gateway with one audit trail and one set of access controls changes the math. The compliance gap the BAA does not close remains: even on covered surfaces, nothing at the provider level prevents a clinician from submitting unredacted PHI, and nothing produces the audit evidence a HIPAA auditor will ask for across all traffic. Without a central layer, teams rebuild redaction, access control, and logging inside each application, inconsistently, with no single authoritative record of what PHI touched which model.
A HIPAA-compliant AI gateway is the most direct answer for healthcare teams that need PHI redaction before prompts reach an LLM, a BAA at the gateway layer, and audit controls out of the box. It is a routing layer between applications and LLM providers that consolidates all six HIPAA requirements into one enforcement point. Instead of every service calling OpenAI or Anthropic directly, services call the gateway, which routes to the right provider with the right configuration. That gives you one place to enforce ZDR, redaction, access controls, and audit logging across all traffic.
A traditional API gateway is not enough here. An enterprise AI gateway must account for model variance, token-based billing, and probabilistic outputs. Standard HTTP proxying misses all three. Spend and rate controls, per key, per team, per model, prevent runaway costs from agentic loops and give finance teams real-time visibility.
Provider complexity belongs in the infrastructure layer, not in application code. A swap of base URL is all an application needs. The gateway carries all routing, compliance, and fallback logic. A HIPAA-grade Claude deployment requires more than a signed BAA. The underlying technical control is preventing PHI from reaching the model in the first place, and the gateway is where that control lives.
A gateway earns the label HIPAA-compliant only when it satisfies all six requirements in one managed layer and backs that with a BAA, SOC 2 Type II, and zero-fee access to BAA-covered providers. The checklist below separates genuine compliance infrastructure from simple routing proxies. The gateway operator itself must sign a BAA with your organization. Passing through BAAs from underlying providers is not enough. PHI and PII redaction must run at one chokepoint, be reviewable per request, and be enforced by policy, not by developer discipline in each application. The gateway must route exclusively to ZDR-eligible provider endpoints and enforce that policy on every request. RBAC, SAML SSO, and a complete audit trail of every PHI-touching request with caller identity must be retained per HIPAA guidance and exportable to a customer SIEM. The gateway should surface all covered paths, OpenAI API with Modified Retention, Anthropic API, AWS Bedrock-hosted Claude, without requiring separate integrations. Failover must stay within the set of BAA-covered, ZDR-enabled endpoints. A fallback to an uncovered surface is a compliance breach, not a reliability win.
Concentrate is a fully managed AI gateway covering models from dozens of providers through one API endpoint, with a BAA available on the paid plan. It satisfies the gateway-level BAA requirement, so teams do not have to deploy or maintain proxy infrastructure. PHI and PII redaction runs at the gateway layer, enforced by policy and reviewable per request. Zero data retention is enforced at the routing layer, directing traffic only to ZDR-eligible provider endpoints. RBAC, SSO, and audit logs are available on day one, not gated behind an enterprise contract, which satisfies the access control and audit requirement for a five-person healthcare startup as readily as a large health system. Real-time spend tracking by team, project, key, model, and provider gives finance and security teams visibility before the invoice arrives. Automatic fallback across providers stays within the compliant provider set, and customers pay for tokens with no markup, no platform fee, no card fee, and no BYOK fee. SOC 2 Type II backs the gateway itself to the standard covered entities require of their business associates.
Other options exist for teams evaluating the field. AWS Bedrock functions as a HIPAA-eligible AI gateway path for teams already inside AWS infrastructure, with BAA-covered access to Claude, Llama, Mistral, and Titan, data staying inside AWS, and the operational cost of AWS subscription and resource management. BastionGPT is a Microsoft-certified Power Platform connector for healthcare teams using Power Automate, Power Apps, and Copilot Studio. A BAA from Bastion Intelligence covers GPT, Claude, and Gemini actions within that connector, designed for no-code workflow automation rather than API-level production deployments. Self-hosted open-weights models such as DeepSeek V4 Pro, Llama 4, and Mistral Large 3 inside a HIPAA-aligned VPC keep inference entirely on your own infrastructure, eliminating provider-side BAA requirements at the cost of operating the inference layer yourself.
Satisfying all six HIPAA requirements through a gateway is a configuration and contract checklist, not a coding project. The sequence matters, because gaps at any step leave PHI exposed regardless of what the other steps do correctly. Work through these steps in order.
Step one is the BAA chain. Sign a BAA with the gateway operator before any PHI-touching traffic flows. Confirm the gateway routes only to provider surfaces that are themselves BAA-covered and ZDR-enabled. Each vendor's BAA covers only the products and features it names, so verify the specific endpoint, not just the vendor. Step two is PHI redaction at the gateway. Configure the redaction policy to run on every inbound prompt before it reaches any provider. Treat model outputs as potentially PHI-containing as well. An LLM can surface PHI by inferring, combining, or echoing back details from retrieved context, so output handling must be covered. Step three is Zero Data Retention enforcement. Verify ZDR is active on every provider endpoint the gateway routes to. For Claude, a ZDR addendum is available for eligible APIs and Claude Code for Enterprise. Some model variants carry mandatory safety retention periods that preclude full ZDR, so confirm per model before routing PHI.
Step four is access controls. Assign scoped API keys or virtual keys to each team, project, and integration. Enforce RBAC so access to PHI-touching models is limited by role. Enable SSO so user identity is auditable on every request. Step five is audit logging. Ensure every PHI-touching request is logged with caller identity, model, provider, timestamp, and redaction record. Export logs to the organization's SIEM, Splunk, Datadog, Microsoft Sentinel, or equivalent, via OpenTelemetry, and retain per OCR guidance. Step six is the fallback policy within the compliant set. Define fallback chains that stay entirely within BAA-covered, ZDR-enabled provider endpoints. A fallback to a non-covered surface, even under an outage, is a compliance breach. Document the fallback chain as part of the breach notification and incident response plan.
Step seven is spend and anomaly controls. Set per-team and per-key budgets at the gateway. Monitor token consumption in real time, especially for agentic workflows that can generate very large token volumes. Treat a sudden spike as a potential data exposure signal, not just a cost event. Finally, build a periodic review into your compliance program. Anthropic updates its BAA coverage page frequently, and the set of covered and excluded features changes as products evolve. Treat the BAA as a living document, not a one-time sign-and-forget process.