Why HIPAA Compliance for Employee LLM Use Breaks Down Without a Centralized Redaction Chokepoint
Organizations need infrastructure to strip patient data before it reaches any LLM.
Concentrate.ai
Organizations need infrastructure to strip patient data before it reaches any LLM.
Concentrate.ai
LLM Gateway
Change models, providers, add fallbacks, and more by running all AI usage through one platform.
A signed Business Associate Agreement does not make an LLM deployment HIPAA compliant. It assigns the vendor's contractual obligations as a Business Associate, but every technical control still belongs to the deployer: redaction, audit logging, access governance, and the encryption and prompt-injection defenses that keep protected health information from leaking through system prompts, logs, or eval data. Stripping PII and PHI before a prompt ever reaches OpenAI, Anthropic, or any other provider requires a control that is in front of the API call itself, not a policy document that assumes employees will remember to redact by hand. That control is a centralized gateway, and understanding why requires walking through each layer where the BAA stops and the deployer's own architecture has to start.
The most dangerous assumption in healthcare AI right now is that a signed BAA equals a compliant deployment, and it does not. A BAA covers the vendor's role as a Business Associate: it prohibits the provider from training on customer data, it sets retention and deletion timelines, and it requires security controls consistent with the HIPAA Security Rule. None of that touches de-identification, audit logging, prompt-injection defenses, encryption, or access control, and none of it guarantees that PHI won't leak through a system prompt, a few-shot example, or a debug log. Those remain the deployer's burden, in full, regardless of how strong the contract is.
HIPAA's Privacy Rule and Security Rule were written before generative AI existed, but they apply anyway: an AI system processing PHI counts as a use or disclosure under the Privacy Rule, and as an access of electronic PHI under the Security Rule. Three properties of large language models make this harder than most teams expect. System prompts, few-shot examples, and retrieval contexts can all carry PHI, so every API call can be a disclosure path. Logs themselves count as PHI when they include identifiable patient context, and cloud providers retain prompts and outputs by default unless zero data retention is explicitly configured, a self-service setting on AWS Bedrock but something that requires an approved request or contractual amendment with OpenAI or Google Cloud. If a provider trains on customer prompts, that PHI enters the model's weights permanently. No-train clauses are not a nice-to-have.
As of May 2026, every major cloud-hosted LLM provider, OpenAI, Anthropic, Google, Microsoft, and AWS, offers a BAA-eligible enterprise tier. Meta's Llama carries no direct BAA and only becomes HIPAA-eligible through a third-party cloud's own BAA. The specifics differ in ways that matter operationally. OpenAI's ChatGPT for Healthcare tier, launched January 2026, comes with a BAA, no training on customer data, and healthcare-specific admin controls; its Enterprise and API tier also offers a BAA with no training by default, though the API version requires zero data retention to be configured; its Free and Plus consumer tiers carry no BAA at all and should never touch PHI. Anthropic's Claude Enterprise includes a BAA and no training, with a zero-retention tier available to qualified accounts, but that tier isn't automatic. You have to request it through sales, and Anthropic's Covered Models, including Fable 5 and Mythos 5, require 30-day retention even if a zero data retention agreement is in place. Claude's free, Pro, and Max consumer tiers have no BAA and opt-out training that's on by default, with multi-year retention. Microsoft's Azure OpenAI Enterprise offers a BAA, either click-through or negotiated, with no training, and a compliance posture that healthcare buyers generally treat as mature and well established. AWS Bedrock Enterprise offers a BAA with no training, but HIPAA eligibility there is set per model rather than per tier, so every individual model has to be verified before use. Google's Vertex and Gemini Enterprise tier offers a BAA with no training as well. Click-through BAAs are faster to put in place, but negotiated BAAs let a deployer set its own indemnity terms, breach notification timelines, and sub-processor approval rights, which larger health systems often require before signing on.
The enforcement backdrop makes the stakes concrete. Multiple seven-figure federal regulatory settlements landed in 2025 and 2026 because of inadequate security controls, and state Attorneys General now enforce just as aggressively alongside the federal regulator. Civil HIPAA penalties range from modest per-violation minimums to tens of thousands of dollars each, with substantial annual caps, and criminal penalties can reach a decade of imprisonment when a violation involves personal gain or malicious intent. Even a perfect stack of BAAs across every provider an organization uses leaves every one of those technical obligations sitting with the deployer. That gap is where employees walk straight through.
Organizations that have done the contractual work, enterprise BAAs signed with every provider they officially use, still run into a parallel problem: employees reach for AI tools the BAA never touched. Consumer-tier endpoints, free ChatGPT, Claude.ai's free tier, consumer Gemini, are explicitly excluded from BAA coverage. When a staff member pastes a patient note into one of those tools, that's a disclosure to a non-business-associate, and no enterprise agreement the organization holds anywhere else changes that fact.
This happens constantly. Inside healthcare organizations, a large share of AI interactions flow through unmanaged personal accounts that corporate controls never touch. The use cases are mundane and routine: drafting clinical notes, sketching a diagnostic plan, summarizing a billing note. A large share of enterprise AI interactions involve sensitive data, and much of that traffic moves through the same unmanaged channels. Staff aren't trying to break rules: they're trying to get work done, and when the approved tool is slow, unavailable, or doesn't support the task at hand, the nearest browser tab wins.
Training doesn't close this gap. HIPAA's Security Rule requires technical safeguards actually implemented in technology, and OCR has said directly in enforcement actions that policies and procedures alone don't satisfy that requirement. A memo telling employees not to use ChatGPT's free tier doesn't stop the request from leaving the building. Only something sitting between the employee and the model can do that, and that architecture is what the rest of this piece works through.
HIPAA's minimum-necessary standard and its audit-logging requirements ask for more than boxes you check on a policy document. They're architectural demands, and they both point to the same conclusion: PHI has to be intercepted and logged before a model ever sees it, not cleaned up afterward.
Minimum-necessary means each component or agent in an AI workflow should see only the PHI fields it actually needs to do its job, and that's a decision made in infrastructure, not in a cleverly worded prompt. Multi-agent clinical workflows create a new PHI boundary at every handoff: a triage agent calling a chart-summarization agent calling a billing-code agent, and each one needs the same controls applied to it. The exposure surface multiplies with every hop in the chain. Redaction has to happen before inference, full stop, because redaction that runs after a model has already processed the input means PHI has already reached a subprocessor. At that point the disclosure has already happened under the Privacy Rule, and no amount of after-the-fact cleanup undoes it.
What does an OCR-ready record actually need to contain? A HIPAA-compliant per-AI-decision log has to capture the clinician's identity and role, the model and endpoint used, the minimum-necessary prompt content, the system prompt version, the sub-processor route the request took, the response content, the specific redaction events (what was removed, when, and by whom), the policy version and decision outcome (allow, block, or redact), and the latency of the call. Without a log showing what was removed and when, an organization has no way to demonstrate compliance during an OCR audit, and no way to support breach-notification timelines if a leak is suspected later.
Logs fall under the same regulation they exist to prove you comply with. If application logs, eval datasets, or debugging traces carry identifiable patient context, the logging system itself becomes a covered component, and PHI inside it must be redacted or tokenized before it reaches any observability store that doesn't have its own BAA. AWS Bedrock's per-model HIPAA eligibility rule shows how this plays out in practice: a system that only checks BAA coverage at the tier level, not the model level, can route a technically compliant request to a model endpoint that isn't actually eligible, and the application calling it would have no way of knowing.
If each team builds redaction separately, for each application, against each model integration, no one can verify it. Every team ends up running a different classifier, applying a different policy, and keeping a different logging discipline, and there's no single control anyone can check to confirm all of it is working at once.
You see this often in fragmented multi-provider setups, where different teams manage their own API keys, their own provider SDKs, their own rate limits and usage reports. Redaction logic ends up embedded in application code instead of infrastructure, which makes it invisible to whoever is actually responsible for compliance review. HIPAA's 18 Safe-Harbor de-identification categories are precise and unforgiving: missing even one puts an entire dataset back in scope, and per-team implementations drift against that standard quietly, with no one noticing until an audit forces the question. Detection methods vary widely in how well they actually work. Regex works well for structured data like identification numbers or phone numbers, but it misses PHI mentioned in free text. NER models add context and catch more unstructured mentions, but accuracy swings widely. One benchmark found detection accuracy falling from near-perfect on data the model had seen patterns like before to very low on entity types outside that distribution. A team that validated its redaction against one kind of clinical note may have no real protection on a different kind.
A clinical-summary agent can surface a diagnosis inside a downstream message meant for an unauthenticated patient-portal context. A scheduling assistant can call a patient lookup tool that returns more fields than the task needs, and the model echoes one back. A model fine-tuned on supposedly de-identified notes can regenerate a rare diagnosis paired with a location detail specific enough that it re-identifies a real patient. The consumer-tier trap is really an application-level failure in disguise: application code holding a provider's API key does nothing to stop an employee from opening a browser tab to a consumer endpoint running the same underlying model. And when something does go wrong, per-application redaction leaves no consolidated audit trail behind it. Compliance officers can't stitch a complete picture of PHI exposure together from logs scattered across a dozen siloed systems, so the per-decision record the law requires doesn't exist in a form anyone can produce.
A centralized LLM gateway sits as a single enforcement layer between every application and every model provider, so it applies the same deterministic PHI classification and the same policy rules to every request, no matter which team or application sent it. The data path works the same way every time: a prompt enters the gateway, a deterministic classifier combining regex for structured fields and NER for unstructured mentions checks it before it ever leaves the organization's perimeter, policy rules decide whether to redact, block, or allow the request, and only the cleaned text crosses to the provider. For reversible redaction flows, a tokenization vault stays inside the organization's own environment, and re-identification happens on the way back so the end user sees correct output. For regulated industries, that redaction logic runs inside the organization's own VPC, so unredacted PHI never touches the public internet at any point in the exchange.
The gateway is also where minimum-necessary enforcement becomes real. It can strip fields at the route level, so a billing-code agent receives only what it needs to do coding work, not the full chart note. That's the actual mechanism the standard demands, not a prompt instruction a model may or may not follow. The same logic applies to BAA-eligibility: because AWS Bedrock sets HIPAA eligibility per model rather than per tier, routing has to be enforced at the model level for every request, and a gateway is the only piece of the stack positioned to do that consistently across every team using it.
Every decision the gateway makes writes a record: the request, the decision, the evaluator, the reason, the policy version in effect at the time. That record is generated at the point of interception, so it can't be missing entries because some team forgot to add logging to their own code. Access control lives at the same layer: centralized identity with SSO and MFA, role-based access aligned to job function, and break-glass flows that trigger immediate notification and enhanced audit logging when used. Provider and model routing follow policy automatically, which removes the risk of a team accidentally sending PHI to a Bedrock model without HIPAA eligibility or to a consumer-tier endpoint by mistake. And when a provider has an outage, the gateway can shift traffic to another BAA-eligible endpoint without touching application code, keeping the system up without trading away compliance to do it.
Even with BAAs in place across multiple providers, the deployer still carries the weight of the technical controls themselves: redaction, logging, access governance, and the audit trail an OCR investigator would actually ask to see. A gateway like Concentrate can centralize those controls across every provider integration an organization runs, so pre-inference redaction and compliance logging happen the same way every time, regardless of which provider or model ends up handling the request.
Concentrate offers a single unified API connecting teams to a wide range of LLMs, OpenAI, Anthropic, Google, and AWS among them, through one control plane, so PHI policy gets enforced once rather than rebuilt separately for every provider integration a team adds. PII and PHI redaction happens at the gateway boundary before any prompt reaches a provider, and redaction decisions get logged per request, producing the audit artifact an OCR investigation would expect to see. Model-level routing policy keeps requests directed only toward endpoints covered by the right agreements, including the kind of per-model HIPAA eligibility check AWS Bedrock requires. RBAC, SSO, MFA, and break-glass procedures are enforced at the gateway layer rather than left to each application team to implement on their own.
When employees can independently route requests to whatever LLM happens to be open in another tab, enterprise provider or consumer endpoint alike, redaction turns into a per-user, per-tool problem that no policy memo can solve on its own. A single, managed access point with redaction built into the request flow itself, enforced before PHI ever reaches a model, closes that gap between what the contracts promise and what actually happens on a given Tuesday afternoon. When employees try to solve a problem fast, they don't check BAA status or redaction settings. They reach for whatever tool is in front of them. The only pattern that holds up under that reality is one where sensitive data gets filtered automatically by the infrastructure itself, not by hoping everyone remembers the training, before any request ever leaves the organization's walls.