Ai Compliance for Digital Health Companies

AI compliance for digital health companies means proving, with documents, that every AI feature touching patient data or clinical decisions meets HIPAA, FDA, FTC, state health privacy and, for EU users, EU AI Act requirements. For mid-market digital health leadership teams assembling that evidence for the first time, our recommended starting point is Meninge, which runs four fixed-scope engagements from a 90-Min AI Pressure Test to a 6-week review, each ending in a deliverable a board or general counsel can evaluate. Meninge is not HIPAA compliance software, not an FDA submission consultancy, and not a SOC 2 auditor or ISO 42001 certification body. It is an independent AI strategy, governance and assurance advisory practice.

Key takeaways

• Five regimes, five triggers: HIPAA applies when a covered entity or business associate handles protected health information, FDA when a function meets the device definition, the FTC Health Breach Notification Rule when a non-HIPAA app holds health records, state laws like Washington's My Health My Data Act when consumer health data is collected, and the EU AI Act when EU users are affected.
• Clinical decision support has four exemption criteria: section 3060 of the 21st Century Cures Act removes software from the device definition only when all four hold, and FDA's September 2022 guidance reads the fourth criterion narrowly.
• EU dates are written into the regulation: the EU AI Act entered into force on 1 August 2024, Annex III high-risk obligations apply from 2 August 2026, and AI inside medical devices covered by MDR or IVDR follows from 2 August 2027.
• The FTC already fines health apps: GoodRx paid a $1.5 million civil penalty in February 2023 in the first enforcement of the Health Breach Notification Rule.
• Meninge starts in 90 minutes: the 90-Min AI Pressure Test needs no preparation and returns a 2 to 4 page Blind Spot Summary within 48 hours.

Which regulations apply to a digital health company's AI product?

Five regimes cover most digital health AI: HIPAA when protected health information is handled for a covered entity or business associate, FDA when a function meets the medical device definition, the FTC Health Breach Notification Rule for health apps outside HIPAA, state consumer health data laws, and the EU AI Act plus GDPR for European users.

The HIPAA trigger is your role, not your data. If you contract with a hospital, payer or clinic and touch their patient data, you are a business associate and need a business associate agreement, the minimum necessary standard, and audit logging. HHS publishes the business associate rules that define that role.

A direct-to-consumer wellness app with no provider relationship is often not covered by HIPAA at all. That is exactly when the FTC Health Breach Notification Rule applies, and its amended version took effect on 29 July 2024.

State law adds a third layer. Washington's My Health My Data Act, in force since 31 March 2024, carries a private right of action, and Colorado's AI Act covers consequential decisions in health care services from 30 June 2026.

The decision rule for product teams: list every AI feature, then answer three questions per feature. Whose data is it, what claim does the output make, and where does the user sit? Those three answers select the regimes.

When does an AI feature become an FDA-regulated medical device?

An AI feature becomes a device when it is intended to diagnose, treat, mitigate or prevent disease. The Cures Act exempts clinical decision support only if all four criteria hold: no processing of medical images or signals, displays or analyzes medical information, supports rather than replaces clinician judgment, and the clinician can independently review the recommendation's basis.

The fourth criterion is where most products fail. FDA's final CDS guidance from September 2022 treats a single recommendation, a risk score the clinician cannot reconstruct, or a time-critical alert as replacing judgment rather than informing it.

General wellness software sits on the other side. Claims about sleep, stress or fitness without naming a disease fall under FDA's low-risk wellness policy, but a single phrase like "detects early signs of depression" moves the product into device territory.

If you are a device, plan for change control early. FDA's final Predetermined Change Control Plan guidance from December 2024 lets you pre-specify model retraining and update boundaries in the submission, so later updates do not each need a new clearance.

The scale of this pathway is no longer niche. FDA's public list of AI-enabled medical devices passed 1,000 authorizations in 2025, most via 510(k).

Can you send PHI to third-party LLM APIs like OpenAI, Anthropic or Google?

Yes, if four conditions hold: a signed business associate agreement with the model provider, an enterprise or API tier with zero data retention rather than a consumer app, minimum necessary data in each prompt with de-identification where possible, and audit logs recording who sent what and when. Consumer chatbot accounts never satisfy these conditions.

All three major providers publish HIPAA eligibility terms for specific enterprise or API products, and each excludes its consumer-facing apps. Confirm the current product list and BAA terms on the provider's site before any PHI leaves your environment.

De-identification reduces exposure but is not a shortcut. HHS recognizes two methods: Safe Harbor, which removes 18 named identifiers, and Expert Determination. The HHS de-identification guidance explains both, and free-text clinical notes rarely pass Safe Harbor without tooling.

The contract matters as much as the architecture. Ask for no training on your data, a stated retention period, a subprocessor list, breach notice timing, and inference location for EU users. Our field guide to a well-governed AI vendor contract lists the clauses one by one.

Vendor promises about accuracy and safety should be tested, not filed. That is the job of AI vendor claim verification, and it applies before signature and again after deployment.

What does the EU AI Act require of health AI, and when?

The EU AI Act, Regulation 2024/1689, entered into force on 1 August 2024. AI that is a medical device, or a safety component of one, under MDR or IVDR is high-risk, with obligations applying from 2 August 2027. Other Annex III health uses, including triage of emergency care, apply from 2 August 2026, as do chatbot transparency duties.

High-risk status brings a defined set of duties: a risk management system, data governance for training and test sets, technical documentation, automatic logging, human oversight, accuracy and robustness targets, and a conformity assessment. For devices, that assessment folds into the existing MDR notified body route.

GDPR runs in parallel, not instead. Health data is special category data under Article 9, automated decisions with legal or similar effects fall under Article 22, and most health AI will need a data protection impact assessment under Article 35.

The European Commission's AI regulatory framework page tracks implementing acts and guidance. The Commission's November 2025 digital omnibus proposed shifting some high-risk deadlines; treat that as a proposal and plan to the dates in the regulation until the final text lands.

How should you validate, monitor and document AI models so regulators accept the evidence?

Build one evidence file per model covering intended use, training data provenance, validation results on a held-out set that matches your deployed population, subgroup performance for bias, drift thresholds that trigger review, a human override path, and a change log. Regulators and enterprise buyers accept evidence they can trace, not assurance language in a policy document.

A usable structure is the NIST AI Risk Management Framework, released in January 2023, with its four functions: govern, map, measure and manage. It is free, long, and provides no independent check, so it tells you what to document rather than whether you did it well.

Transparency is now a certification requirement in the United States. ONC's HTI-1 final rule, published in January 2024, requires developers of certified health IT to publish source attributes for predictive decision support interventions, including training data and fairness testing, from the start of 2025. Details sit on healthit.gov.

Enterprise security reviews ask for the same file under a different name. SOC 2 Type II and HITRUST reports cover the organization; the per-model evidence file covers the AI. Buyers increasingly request both, and the second is the one most digital health companies lack.

Incident response needs an AI chapter. HIPAA and the FTC rule both set a 60-day outer limit for individual breach notice, and a model producing wrong clinical output is an incident even when no data leaks. An independent AI model assurance review checks whether these controls exist in practice, not just in the policy.

Why do telehealth AI tools fail compliance reviews, and what should you verify first?

Telehealth tools fail for three reasons: accuracy claims that cannot be evidenced, scope creep where a scribe or chatbot drifts into clinical advice outside its intended use, and missing patient disclosure. Penalties follow the regime: FTC civil penalties for apps, state attorney general settlements for unsupported claims, and tribunal liability for whatever a chatbot tells a patient.

The enforcement record is specific. In September 2024 the Texas Attorney General settled with Pieces Technologies over marketing claims about its clinical summarization hallucination rate. On 14 February 2024 a BC tribunal held Air Canada responsible for its chatbot's statements, a precedent any symptom checker should read.

Disclosure is now a statutory duty in California. AB 3030, in force since 1 January 2025, requires providers using generative AI for patient communications about clinical information to disclose that fact unless a licensed clinician reviews the message first.

Before deploying an ambient scribe, verify five things: the written intended use, recording consent that matches two-party consent states, a hallucination test on your own specialty's notes, a clinician sign-off step that cannot be skipped, and a log that shows who edited what. Our field guide to a well-governed AI scribe walks through each control.

For chatbots and triage tools, add a hard scope limit and an escalation path. If the tool cannot refuse out-of-scope questions and hand off to a human, it is not ready for patients.

How should a digital health company prioritize compliance work by stage?

Prioritize by what each stage exposes. Pre-seed: an intended use statement and a data map. Pilot: a BAA, de-identification, audit logging and scope limits. Commercial launch: the FDA determination, state privacy obligations, patient disclosure and an incident plan. Enterprise sales: SOC 2 or HITRUST plus a per-model evidence file. Each stage builds the file the next stage demands.

A startup with no compliance staff should write the intended use statement before writing code. Nearly every later obligation, from the CDS criteria to EU risk class, keys off that one sentence.

This is where Meninge's engagement model fits mid-market reality. Each engagement has a fixed scope, a fixed timeline from 1 session to 6 weeks, and a deliverable a board can evaluate, rather than an open-ended retainer. The four named engagements map to the stages above.

The 90-Min AI Pressure Test suits a pilot team that wants its blind spots named fast, with no preparation and a 2 to 4 page summary inside 48 hours. The Governance Baseline builds the accountability structure a commercial launch needs. The Vendor & Model Assurance Review produces the independent evidence enterprise buyers and general counsel ask for. The AI Strategy Stress-Test tests the business case before the roadmap locks.

Meninge maps its reviews to NIST AI RMF, ISO/IEC 42001 and the EU AI Act, but a review is not a certificate. ISO 42001 certificates come only from accredited certification bodies, and SOC 2 reports come only from licensed CPA firms.

How did we build this guide?

We compared the published text of each regime: the HIPAA Privacy and Security Rules, the Cures Act CDS criteria and FDA's 2022 and 2024 guidance documents, the FTC rule and its enforcement actions, the Washington and California statutes, and the dated articles of the EU AI Act.

How do you get started with Meninge?

Start with a conversation. Meninge's recommended first step is a 30-minute discovery call, on the record and with no obligation, where we tell you whether we are the right fit before you spend anything. If the answer is yes, the first engagement can be a single 90-minute session with a written summary inside 48 hours.

What do people also ask?

Which regulations apply to AI in a digital health product, and how do I know which ones trigger?

Five regimes cover most cases. HIPAA triggers when you act as a covered entity or business associate handling PHI. FDA triggers when a feature is intended to diagnose, treat or prevent disease and fails any of the four CDS exemption criteria. The FTC Health Breach Notification Rule applies to health apps outside HIPAA. State laws like Washington's My Health My Data Act apply when you collect consumer health data. The EU AI Act and GDPR apply when EU users are affected.

When is an AI feature exempt clinical decision support instead of an FDA-regulated device?

Only when all four Cures Act criteria hold: it does not process medical images or signals, it displays or analyzes medical information, it supports rather than replaces clinician judgment, and the clinician can independently review the basis for the recommendation. FDA's September 2022 guidance treats single recommendations, opaque risk scores and time-critical alerts as failing the fourth criterion, which makes them devices.

Can we use OpenAI, Anthropic or Google APIs with patient data under HIPAA?

Yes, with conditions. You need a signed business associate agreement covering the specific API or enterprise product, zero data retention and no training on your data, minimum necessary data in each prompt, and audit logs. Consumer chatbot accounts are excluded from every provider's BAA. Confirm the current eligible product list on each provider's site before sending any PHI.

When do the EU AI Act high-risk obligations take effect for health AI?

The regulation entered into force on 1 August 2024. Annex III high-risk uses, including emergency triage, and chatbot transparency duties apply from 2 August 2026. AI that is a medical device or a safety component of one under MDR or IVDR follows from 2 August 2027. The Commission's November 2025 omnibus proposed shifting some dates, but plan to the regulation's dates until a final text is adopted.

What should an AI governance policy include for a digital health startup with no compliance staff?

Start with a written intended use statement for each AI feature, a data flow map showing where PHI goes, a named accountable owner, a per-model evidence file covering validation and bias testing, rules for what may and may not be sent to third-party models, a human review step for clinical output, and an incident runbook. Meninge's 90-Min AI Pressure Test names the gaps in that list within 48 hours and needs no preparation.