Architecture Review Facilitation#

@dataclass
class PreReviewArchitectureMap:
    """
    FDE's working model of the client's architecture before the review session.
    All items should be marked as CONFIRMED (verified) or ASSUMED (to confirm in review).
    """
    
    # EHR Layer
    ehr_system: str                    # "Epic FHIR R4, version 2023.1 — CONFIRMED"
    ehr_integration_standard: str     # "FHIR R4 + HL7 v2 ADT — CONFIRMED"
    smart_on_fhir_status: str         # "Available, App Orchard pending — CONFIRMED"
    cds_hooks_availability: str       # "Available in Epic — ASSUMED: confirm in review"
    
    # Cloud / Network
    cloud_provider: str               # "Azure — CONFIRMED"
    phi_in_cloud_policy: str          # "Approved for HIPAA BAA vendors — CONFIRMED"
    llm_outbound_access: str          # "Permitted — ASSUMED: test not yet run"
    ai_gateway_status: str            # "Not deployed — CONFIRMED"
    network_topology: str             # "Single datacenter + Azure region — ASSUMED"
    
    # Security
    tls_policy: str                   # "TLS 1.2+ required — CONFIRMED"
    audit_logging_capability: str     # "Splunk SIEM — CONFIRMED"
    baa_status: str                   # "Azure BAA signed; Anthropic BAA pending — CONFIRMED"
    security_review_process: str      # "IT Security review + Compliance sign-off — CONFIRMED"
    
    # Organizational
    ai_gateway_owner: str             # "IT Infrastructure team — CONFIRMED"
    model_governance: str             # "No formal process — ASSUMED: confirm in review"
    prompt_management: str            # "Ad hoc — ASSUMED"
    
    # Unknown / To Elicit
    open_questions: list[str]         # Items to resolve in the review session

[ ] Current state architecture map drafted with CONFIRMED / ASSUMED tags
[ ] Discovery Summary and Assessment Report reviewed
[ ] Integration risk patterns identified (from standard pattern library)
[ ] Open questions documented (to resolve in review session)
[ ] Review agenda prepared and shared with client 48h before session
[ ] Whiteboard or diagramming tool ready for live architecture drawing
[ ] Architecture Review Report template prepared (fill during session)

BLOCK 1 — Current State Architecture Walk (60–90 min)

Objective: Build an accurate current-state diagram collaboratively.
Method: FDE draws on whiteboard; client corrects and adds.

Opening: "I want to start by making sure I have an accurate picture of your
 current architecture. I've built a working map from our prior conversations —
 let's validate and correct it together."

Technique: Draw the FDE's pre-review map on the whiteboard. Ask the client
to correct it. Incorrect assumptions surface more information than open-ended
questions. "I have it that your integration engine sends ADT^A01 messages 
to Epic — is that right, or does it go the other direction?"

Cover:
  - Data flows for the target use case
  - Authentication and authorization paths
  - Network topology (on-prem / cloud / peering)
  - Security architecture (where PHI flows, what controls exist)
  - Existing integration points relevant to the AI use case
  - Current monitoring and observability

BREAK — 15 minutes

BLOCK 2 — Target State and Gap Identification (60 min)

Objective: Map where the AI system needs to connect and identify the gaps.

Technique: Add the target AI architecture to the current-state diagram.
Draw the connections that need to exist for the AI system to work.
Ask: "What has to be true for this connection to work? Does that condition
currently hold?"

Cover:
  - AI system component placement (SMART app, CDS Hooks service, AI gateway, LLM)
  - Data paths from EHR to AI to LLM and back
  - Authentication path (SMART token, AI gateway virtual key, LLM API key)
  - Security controls on each path
  - Failure modes for each connection

BREAK — 15 minutes

BLOCK 3 — Risk Assessment and Prioritization (45 min)

Objective: Identify, categorize, and prioritize the architectural risks.

Technique: Walk through each gap identified in Block 2. For each gap:
  1. Is it a risk? (Could it cause failure or security incident?)
  2. What is the likelihood? (High / Medium / Low given client's environment)
  3. What is the consequence? (Service unavailability / HIPAA incident / clinical harm)
  4. What is the mitigation?

Cover:
  - PHI data path risks
  - Availability risks (LLM downtime, latency, rate limits)
  - Security risks (authentication gaps, audit log gaps)
  - Governance risks (model updates without re-evaluation)
  - Clinical safety risks (AI failure affecting clinical workflow)

BLOCK 4 — Recommendations and Next Steps (30 min)

Objective: Agree on which risks to mitigate, in what order, and who owns each.

Output: Architecture Review Report action items with owners and dates.

ARCHITECTURE_RISK_PATTERNS = [
    {
        "pattern": "PHI in observability traces",
        "description": "LLM inference requests containing PHI are logged verbatim in observability systems",
        "detection_question": "Where do LLM inference request and response payloads go? Are they logged?",
        "consequence": "HIPAA Security Rule violation; PHI accessible to any engineer with logging access",
        "mitigation": "AI gateway scrubs PHI from all traces; only hashed patient IDs and metadata in logs"
    },
    {
        "pattern": "No AI gateway — direct LLM API calls",
        "description": "Application code calls LLM APIs directly without a centralized gateway",
        "detection_question": "Where do LLM API keys live? Are they per-application or centralized?",
        "consequence": "No cost attribution; no audit logging; no rate limiting; no prompt versioning",
        "mitigation": "Deploy AI gateway (LiteLLM, Azure AI Foundry) before production"
    },
    {
        "pattern": "CDS Hook with blocking LLM dependency",
        "description": "CDS Hook service makes synchronous LLM call without circuit breaker",
        "detection_question": "What happens in your CDS Hook service if the LLM API is unavailable or slow?",
        "consequence": "EHR workflow blocked when LLM API times out; patient care delay",
        "mitigation": "3-second timeout with circuit breaker; return empty card array on timeout"
    },
    {
        "pattern": "No model version pinning",
        "description": "Production system calls LLM API with 'latest' model or unpinned version",
        "detection_question": "How is the model version specified in production API calls?",
        "consequence": "Unexpected behavior change when vendor updates model; production incident",
        "mitigation": "Pin exact model version; define evaluation and approval process for updates"
    },
    {
        "pattern": "AI output without physician review gate",
        "description": "AI-generated clinical content is written to EHR without physician approval",
        "detection_question": "What is the workflow from AI generation to note appearing in the EHR?",
        "consequence": "AI error enters medical record; potential patient harm; liability",
        "mitigation": "Physician review and approval required before DocumentReference write; no auto-filing"
    },
    {
        "pattern": "No prompt version management",
        "description": "Prompts are modified directly in production without version control or evaluation",
        "detection_question": "How are prompts changed in production? Who approves changes?",
        "consequence": "Undetected quality regression; no rollback capability",
        "mitigation": "Prompt Registry with versioning; evaluation before production deployment"
    },
    {
        "pattern": "FHIR access with over-broad scopes",
        "description": "SMART application requests system-level FHIR scopes instead of patient-level",
        "detection_question": "What FHIR scopes does the application request? System/* or patient/*?",
        "consequence": "HIPAA Minimum Necessary violation; access to all patients instead of current patient",
        "mitigation": "Minimum Necessary scopes: patient/{resource}.read per use case"
    },
    {
        "pattern": "No fallback when AI service is unavailable",
        "description": "Clinical workflow has no fallback path when AI system is unavailable",
        "detection_question": "What do clinicians do if the AI tool is down? Is there a manual fallback?",
        "consequence": "Clinical workflow disruption; workarounds that bypass safety controls",
        "mitigation": "Design fallback workflow before launch; never make AI availability a dependency"
    },
]