For healthcare IT and compliance leaders, the rush to integrate generative AI often collides with the rigid requirements of HIPAA. While many AI vendors claim "HIPAA compliance," compliance is not a static feature of a product—it is a legal status established through a properly executed Business Associate Agreement (BAA) and verified through technical due diligence.
The risk with AI vendors is fundamentally different from traditional SaaS. Because AI models can ingest, process, and potentially "memorize" data, the BAA must move beyond boilerplate language to address the specific lifecycle of data within a machine learning environment.
The Non-Negotiables of an AI BAA
A standard BAA covers the basics of Protected Health Information (PHI) protection, but an AI-specific agreement must be explicit about how data is used to improve the service. If a BAA is silent on model training, the vendor may assume a right to use "de-identified" data—a process that, if done poorly, can lead to re-identification risks.
Ensure your BAA explicitly covers the following:
- Permitted Uses: The agreement must state that PHI is used solely for the purpose of providing the service to the covered entity. Any use for "product improvement" or "model tuning" must be explicitly defined and restricted.
- De-identification Standards: If the vendor claims they de-identify data, the BAA must specify that this is done according to the HIPAA Safe Harbor method or via an expert determination.
- Subcontractor Flow-down: AI vendors often rely on third-party Large Language Model (LLM) providers (e.g., Azure OpenAI, AWS Bedrock). The BAA must mandate that the vendor has signed equivalent BAAs with every sub-processor that touches the data.
Security and Sub-processor Due Diligence
Before signing, you must map the data flow. Many AI startups act as a "wrapper" around a larger model. You are not just vetting the startup; you are vetting their entire supply chain.
Questions to ask about the tech stack:
- Where is the data processed? Demand a list of all sub-processors and the geographic regions where data resides.
- Is the model "Zero-Retention"? For API-based AI, ask if the provider offers a zero-retention policy where inputs are not logged or stored by the underlying model provider.
- How is data isolated? Ensure the vendor uses a multi-tenant architecture with strict logical separation, preventing your PHI from leaking into another customer's instance.
Data Retention and Model Training Clauses
The most critical tension in AI procurement is the vendor's desire for data to train their models versus the covered entity's duty to protect PHI. In healthcare, the default position should be: No PHI for training.
Demand the following clauses in your contract or BAA:
- Prohibition of Training: A clause explicitly stating that the vendor shall not use PHI, or any derivative of PHI, to train, tune, or improve their global models.
- Right to Deletion: A clear mechanism for the "Right to be Forgotten." If you terminate the contract, the vendor must certify that your data has been deleted not only from their databases but from any fine-tuning weights associated with your account.
- Retention Schedules: Specific timelines for how long prompts and completions are stored for "abuse monitoring" and a requirement that these logs are purged automatically after a set period (e.g., 30 days).
Breach Notification and Audit Rights
AI introduces new failure modes, such as "prompt injection" or "data leakage," where a model might inadvertently reveal PHI to another user. Your BAA must account for these nuances.
Breach Notification terms:
Avoid vague terms like "reasonable time." Demand a specific notification window—typically 24 to 72 hours from the discovery of a potential breach. The notification should include the nature of the data exposed and the specific AI model or version involved in the leak.
Audit Rights:
A "right to audit" clause is useless if it is too restrictive. Ensure you have the right to:
- Review SOC 2 Type II Reports: Request the full report, not just the executive summary.
- Request Penetration Test Summaries: Specifically, ask for tests that target the AI's security boundaries.
- Self-Certification: The right to require the vendor to complete a detailed security questionnaire annually.
Red Flags That Should Pause a Signature
If you encounter any of the following during negotiations, it is a signal to pause and re-evaluate the vendor's maturity:
- "We are HIPAA-compliant" without a BAA: Compliance is a process, but the BAA is the legal requirement. If they resist signing a BAA or insist you sign a "Terms of Service" that waives their liability for data loss, walk away.
- Refusal to disclose sub-processors: If a vendor cannot or will not tell you which LLM they are using, they cannot guarantee the security of your data.
- Claiming "Automatic De-identification": Be wary of vendors who claim their AI "automatically" scrubs PHI before it hits the model. These tools are not 100% accurate; the BAA must still treat the data as PHI until a human or certified process verifies the de-identification.
- Indemnity Caps on Data Breaches: If the vendor attempts to cap their liability for HIPAA violations at the amount of the annual contract fee, they are shifting all the regulatory risk onto you.
Put it into practice
This week, start by auditing your existing "shadow AI" usage. Identify which departments are using AI tools without a formal BAA. Once identified, move those vendors through a tiered review process: first, verify the sub-processor chain; second, negotiate the training and retention clauses; and third, execute a BAA that specifically addresses the AI lifecycle.
Put it into practice
The Vendor & BAA Review Pack from HealthLattice collects the templates and checklists behind this guide in ready-to-use form, so your team can apply the framework without drafting from scratch.