Regulatory Compliance for Enterprise AI

Executive Summary

Enterprise AI deployments in regulated industries must navigate a complex and rapidly evolving compliance landscape: HIPAA for healthcare data, SOC 2 Type II for operational security assurance, HITRUST CSF for healthcare-specific security certification, FDA regulations for AI as a medical device, and the EU AI Act for systems deployed in the European Union. Each framework makes distinct demands on how AI systems are designed, operated, documented, and audited. This chapter maps each framework to the specific AI implementation requirements it creates, with emphasis on the clinical AI context.

Learning Objectives

  • Identify which regulatory frameworks apply to a specific clinical AI deployment
  • Map SOC 2 trust service criteria to AI platform control requirements
  • Understand FDA Software as a Medical Device (SaMD) classification for clinical AI
  • Apply EU AI Act high-risk AI system requirements to clinical decision support deployments

Regulatory Framework Map

python
from dataclasses import dataclass
from enum import Enum
from typing import Optional

class RegulatoryScope(Enum):
    US_HEALTHCARE = "us_healthcare"
    US_FEDERAL = "us_federal"
    EU = "eu"
    GLOBAL_SECURITY = "global_security"


@dataclass
class RegulatoryFramework:
    name: str
    acronym: str
    scope: RegulatoryScope
    applies_when: str
    ai_specific_requirements: list[str]
    primary_governing_body: str
    enforcement_mechanism: str


REGULATORY_FRAMEWORKS = [
    RegulatoryFramework(
        name="Health Insurance Portability and Accountability Act",
        acronym="HIPAA",
        scope=RegulatoryScope.US_HEALTHCARE,
        applies_when="AI system creates, receives, maintains, or transmits PHI",
        ai_specific_requirements=[
            "Business Associate Agreement with all AI vendors processing PHI",
            "Access controls: role-based access to AI PHI features",
            "Audit controls: log all AI system PHI access with patient attribution",
            "Transmission security: TLS 1.3 for all PHI in transit",
            "Minimum necessary standard: AI context limited to required PHI",
        ],
        primary_governing_body="Department of Health and Human Services (HHS)",
        enforcement_mechanism="OCR enforcement; civil and criminal penalties",
    ),
    RegulatoryFramework(
        name="SOC 2 (Service Organization Control 2)",
        acronym="SOC 2",
        scope=RegulatoryScope.GLOBAL_SECURITY,
        applies_when="AI platform is a vendor to other organizations; or organization seeks security assurance for enterprise AI",
        ai_specific_requirements=[
            "CC6 (Logical and Physical Access): role-based access to AI platform components",
            "CC7 (System Operations): monitoring and alerting on AI system anomalies",
            "A1 (Availability): SLA commitments for AI platform uptime; incident response",
            "C1 (Confidentiality): PHI handling controls for confidential data classification",
            "PI1 (Processing Integrity): validation of AI output quality and accuracy",
        ],
        primary_governing_body="AICPA",
        enforcement_mechanism="Third-party audit; SOC 2 Type II report issued annually",
    ),
    RegulatoryFramework(
        name="HITRUST Common Security Framework",
        acronym="HITRUST CSF",
        scope=RegulatoryScope.US_HEALTHCARE,
        applies_when="Healthcare organization seeking comprehensive security certification",
        ai_specific_requirements=[
            "Control category 07 (Vulnerability Management): AI system patching; model update governance",
            "Control category 09 (Network Protection): segmentation of AI processing from PHI data zone",
            "Control category 01 (Information Protection): data classification applied to AI training data",
            "HIPAA controls + NIST CSF + PCI DSS controls mapped to AI systems",
        ],
        primary_governing_body="HITRUST Alliance",
        enforcement_mechanism="Third-party assessment; HITRUST r2 Certification",
    ),
    RegulatoryFramework(
        name="EU Artificial Intelligence Act",
        acronym="EU AI Act",
        scope=RegulatoryScope.EU,
        applies_when="AI system deployed in EU; any 'high-risk' AI use case",
        ai_specific_requirements=[
            "Clinical decision support classified as high-risk AI (Annex III)",
            "Conformity assessment required before deployment",
            "Risk management system documentation",
            "Data governance: training data documentation, bias assessment",
            "Transparency: disclosure to users that they are interacting with AI",
            "Human oversight: ability to override AI recommendations",
            "Accuracy, robustness, cybersecurity requirements",
            "Post-market monitoring plan",
        ],
        primary_governing_body="EU AI Office; national market surveillance authorities",
        enforcement_mechanism="Conformity assessment; CE marking for high-risk AI; fines up to €30M or 6% global revenue",
    ),
    RegulatoryFramework(
        name="FDA Software as a Medical Device",
        acronym="FDA SaMD",
        scope=RegulatoryScope.US_FEDERAL,
        applies_when="AI software intended to diagnose, treat, mitigate, cure, or prevent disease",
        ai_specific_requirements=[
            "IVD classification: diagnostic AI (risk class based on severity and indications)",
            "510(k) or De Novo clearance for moderate-risk clinical AI",
            "PMA for high-risk AI (life-sustaining, implantable device equivalent)",
            "Pre-determined Change Control Plan (PCCP) for AI model updates without resubmission",
            "Software Bill of Materials (SBOM) for ML components",
            "Predetermined performance metrics with clinical validation dataset",
            "Real-world performance monitoring post-clearance",
        ],
        primary_governing_body="FDA Center for Devices and Radiological Health (CDRH)",
        enforcement_mechanism="510(k) clearance; enforcement action for non-compliant devices",
    ),
]

SOC 2 Controls Mapped to AI Platform

python
# SOC 2 trust service criteria mapped to AI platform controls
# Educational example — not for clinical use

SOC2_AI_CONTROL_MAPPING = {
    "CC6.1 — Logical Access Controls": {
        "implementation": [
            "Role-based access control on AI gateway (clinical roles: physician, nurse, pharmacist, admin)",
            "mTLS for AI service-to-service authentication",
            "SMART on FHIR scope minimization for FHIR access",
            "LLM API key rotation on 90-day schedule",
        ],
        "evidence": "Access control policy document; IAM role audit reports; quarterly access reviews",
    },
    "CC6.6 — Logical Access Security": {
        "implementation": [
            "Multi-factor authentication required for clinician access to PHI-enabled AI features",
            "Device compliance verification (MDM enrollment) required for PHI AI access from mobile",
        ],
        "evidence": "MFA enforcement policy; MDM compliance reports",
    },
    "CC7.2 — System Monitoring": {
        "implementation": [
            "AI gateway metrics: latency, error rate, token usage per team",
            "SIEM rules for injection attempt spikes and unusual PHI access volume",
            "AI quality log monitoring: disclaimer presence rate, citation count, clinician feedback rate",
        ],
        "evidence": "SIEM alert configuration; monitoring dashboard screenshots; incident response records",
    },
    "CC7.3 — Security Incident Response": {
        "implementation": [
            "AI-specific incident response runbook: prompt injection response, PHI leakage response, model compromise response",
            "AI incident classification: P1 (PHI breach or patient safety risk), P2 (service degradation), P3 (quality regression)",
            "HIPAA breach notification triggered by P1 AI incidents",
        ],
        "evidence": "Incident response runbook; tabletop exercise records; incident tickets",
    },
    "PI1.1 — Processing Integrity": {
        "implementation": [
            "Clinical disclaimer present in AI outputs (disclaimer_present_rate metric)",
            "Source citations required for all clinical knowledge responses",
            "Output validation (PHI scanning, content policy compliance)",
            "Model version tracking: every AI output associated with model version",
        ],
        "evidence": "AI quality log reports; output validation configuration; model registry",
    },
}

EU AI Act High-Risk Requirements for Clinical AI

The EU AI Act classifies clinical decision support AI as a high-risk AI system under Annex III, imposing conformity assessment requirements before market deployment.

python
# EU AI Act Article 9 — Risk Management System requirements
# Educational example — not for clinical use

EU_AI_ACT_CLINICAL_REQUIREMENTS = {
    "risk_management_system": {
        "requirement": "Article 9 — Establish, implement, document, and maintain a risk management system",
        "clinical_ai_implementation": [
            "FMEA (Failure Mode and Effects Analysis) for clinical AI failure scenarios",
            "Clinical bias assessment across protected demographic groups",
            "Regular risk review as clinical AI capabilities evolve",
            "Risk management documentation updated with each model update",
        ],
    },
    "data_governance": {
        "requirement": "Article 10 — Training data governance",
        "clinical_ai_implementation": [
            "Document training data sources, composition, and preprocessing",
            "Demographic representation assessment for training datasets",
            "Data quality measures for training data (completeness, accuracy)",
            "Documentation of known gaps or biases in training data",
        ],
    },
    "transparency": {
        "requirement": "Article 13 — Transparency and provision of information to deployers",
        "clinical_ai_implementation": [
            "Disclosure to clinicians that AI-generated content is AI-generated",
            "Model card documenting: intended use, performance metrics, known limitations",
            "Clinical disclaimer on all AI outputs",
        ],
    },
    "human_oversight": {
        "requirement": "Article 14 — Human oversight",
        "clinical_ai_implementation": [
            "Physician review required before AI-generated clinical documents are finalized",
            "Easy mechanism for clinician to override AI recommendation",
            "Training for clinical staff on how to critically evaluate AI outputs",
            "Escalation path when AI output is uncertain or potentially incorrect",
        ],
    },
    "accuracy_robustness_cybersecurity": {
        "requirement": "Article 15 — Accuracy, robustness, cybersecurity",
        "clinical_ai_implementation": [
            "Predetermined accuracy metrics with clinical validation dataset",
            "Robustness testing against adversarial inputs (prompt injection evaluation)",
            "Cybersecurity controls per Section 06 of this repository",
            "Performance monitoring plan with defined metrics and alert thresholds",
        ],
    },
}

HITRUST and HIPAA Alignment

For healthcare organizations, HITRUST CSF r2 Certification provides a single framework that maps to HIPAA, NIST CSF, and other frameworks simultaneously. The AI platform controls in this repository align to HITRUST control categories:

HITRUST Control Category AI Platform Implementation
01 — Information Protection Program AI data classification policy; PHI access controls
07 — Vulnerability Management AI model update governance; dependency scanning
09 — Network Protection AI processing/PHI data zone segmentation
10 — Privacy Minimum necessary, de-identification, consent
12 — Audit Logging HIPAA audit trail; AI quality logs; SIEM

Enterprise Considerations

Regulatory overlap: HIPAA, SOC 2, HITRUST, and EU AI Act requirements frequently overlap. Map each requirement to a single implementation control rather than building separate compliance stacks. HITRUST CSF is specifically designed to harmonize HIPAA with other frameworks — use it as the single control framework if the organization is pursuing HITRUST certification.

Compliance as architecture: Regulatory requirements must be designed into the AI architecture from the beginning — not retrofitted. An AI system deployed without HIPAA audit logging cannot satisfy the HIPAA audit controls standard by adding logging after the fact to the same system that has already been used with PHI.

Evolving landscape: EU AI Act was published in 2024 with phased implementation timelines. FDA PCCP guidance evolves as clinical AI capabilities mature. HIPAA guidance on AI is developing. Establish a regulatory monitoring process to track changes that affect the AI platform.

Common Mistakes

1. Assuming SOC 2 covers HIPAA. SOC 2 is a security framework; HIPAA is a privacy and security law. An organization can be SOC 2 Type II certified while being out of HIPAA compliance. Both must be addressed independently.

2. Not classifying clinical AI as potentially SaMD. Organizations deploy clinical decision support AI without asking whether it meets FDA's definition of a medical device. If the AI is "intended to diagnose, treat, mitigate, cure, or prevent disease," it may require FDA clearance regardless of whether the organization considers it a "software tool."

3. Not including AI in the SOC 2 scope. SOC 2 audits cover the systems in scope. If the AI platform is not explicitly in scope, the audit does not cover AI controls. Ensure AI platform components are explicitly included in the SOC 2 scope definition.

Key Takeaways

  • Clinical AI is subject to multiple simultaneous regulatory frameworks: HIPAA (data privacy), SOC 2 (security assurance), HITRUST (healthcare security certification), FDA SaMD (medical device regulation), and EU AI Act (high-risk AI)
  • Each framework makes distinct implementation demands; map all to a single control implementation where possible
  • Clinical decision support AI is classified as high-risk under the EU AI Act, requiring conformity assessment and human oversight mechanisms
  • FDA SaMD classification must be evaluated before deploying AI for clinical decision support — "software tool" framing does not exempt clinical AI from medical device regulation
  • Compliance must be designed in from the start; it cannot be retrofitted to a deployed AI system

Further Reading