Demo Engineering#

Executive Summary#

A technical demo is not a product walkthrough — it is an engineered artifact designed to produce a specific audience response under controlled conditions. Demo engineering is the discipline of designing, building, and operating live technical demonstrations that convey architectural credibility, surface product capability at enterprise scale, and survive the inevitable live failure mode. This chapter covers the full demo engineering lifecycle: demo architecture design, environment isolation, data strategy, failure handling, and the specific considerations for healthcare AI demos where regulatory sensitivity requires careful content design. FDEs who treat demos as ad hoc screen shares consistently underperform FDEs who engineer their demos with the same rigor they apply to production systems.

Learning Objectives#

• Design a demo architecture that is isolated, repeatable, and failure-resistant
• Select appropriate data strategies for different demo contexts (synthetic, de-identified, sandboxed live)
• Structure a demo script that leads with the problem, not the product
• Build failure handling into the demo design rather than hoping failure does not occur
• Apply healthcare-specific constraints to demo content including PHI avoidance and medical disclaimers

Business Problem#

Enterprise AI demos fail in three ways: they work technically but do not convince the audience; they work technically but the audience cannot see the relevance to their specific problem; or they fail live in ways that erode trust. The first two failures are design problems. The third is an engineering problem.

The business consequence of a failed demo in an enterprise AI engagement is disproportionate: a live failure in front of a client's executive team can set back an engagement by months, even if the failure was non-representative of the product's production reliability. Stakeholders who observe a failure during a demo form a strong and durable negative impression that factual correction rarely overcomes.

Demo engineering treats demo reliability as a first-class engineering constraint — not as an afterthought.

Conceptual Explanation#

A demo has three layers that must be engineered independently:

Layer 1 — The Environment: Where does the demo run? Is it isolated from production? Is it reproducible? What happens if the primary environment fails?

Layer 2 — The Data: What data does the demo process? Synthetic? De-identified client data? Sandbox data from the client's EHR? The data choice determines how convincing the demo is and what regulatory constraints apply.

Layer 3 — The Narrative: What story does the demo tell? Demos that start with features fail. Demos that start with the client's specific problem and show how the product solves it succeed.

All three layers must be designed together. A brilliant narrative with unreliable infrastructure produces a failed demo. A reliable infrastructure with generic data produces a forgettable demo.

Core Architecture: Demo Infrastructure#

Environment Design Principles#

Isolation: The demo environment must be completely isolated from any client data, production systems, or shared infrastructure that could introduce failures. A demo that depends on a production LLM endpoint that is experiencing latency will fail at exactly the moment the audience's attention is highest.

Reproducibility: Every component of the demo must be reproducible on demand. Scripts, not clicks. Infrastructure as code, not manual setup. A demo that requires 45 minutes of manual setup before each session is not demo-ready.

Fallback tiers: Every demo must have at least two fallback tiers:

• Tier 1: Live demo against the primary environment
• Tier 2: Live demo against a local backup environment (no internet dependency)
• Tier 3: Recorded walkthrough (never the first choice, but always available)

Implementation code omitted in the Playbook edition. For complete code examples, production patterns, and advanced implementation details, see the Enterprise AI Technical Reference.

Pinned versions: Every demo dependency — LLM model version, API version, library version — must be pinned. "Latest" in a demo is an undefined behavior contract. A model update that changes output style overnight can render a carefully scripted demo incoherent.

Data Strategy#

The data strategy for a demo determines its credibility and its regulatory risk:

For healthcare demos, the rule is absolute: no real PHI in demo environments. Synthetic patient data generators (Synthea, custom scripts) can produce realistic clinical encounters that demonstrate the AI's capability without creating HIPAA risk.

Implementation code omitted in the Playbook edition. For complete code examples, production patterns, and advanced implementation details, see the Enterprise AI Technical Reference.

Demo Script Engineering#

A demo script is not a spoken script — it is a structured guide that defines:

• The problem statement opening
• The transition from problem to product
• The specific actions performed in the product (keystrokes, clicks, inputs)
• The expected outputs and how to narrate them
• The audience questions anticipated at each stage
• The fallback response if the expected output does not appear

Demo script structure for HMS discharge summary demo:

Implementation code omitted in the Playbook edition. For complete code examples, production patterns, and advanced implementation details, see the Enterprise AI Technical Reference. mermaid graph TD subgraph "Demo Environment (Isolated)" FHIR["Synthetic FHIR Server\nHAPI FHIR or local"] DATA["Demo Data Store\nSynthetic patients (Synthea)"] APP["Demo Application\nSMART on FHIR app"] GW["Demo AI Gateway\nLiteLLM (demo config)"] LLM["LLM API\nPinned model version"] end

subgraph "Fallback Tiers" FB1["Tier 1: Cloud Primary\nFull capabilities"] FB2["Tier 2: Local Backup\nOllama + local HAPI FHIR"] FB3["Tier 3: Recording\nPre-captured walkthrough"] end

subgraph "Audience" CMIO["CMIO\nClinical value"] PHYS["Physician Champion\nWorkflow realism"] IT["IT Director\nIntegration architecture"] end

DATA --> FHIR FHIR --> APP APP --> GW GW --> LLM LLM --> APP

APP --> CMIO & PHYS & IT

FB1 -.->|"Network failure"| FB2 FB2 -.->|"Local failure"| FB3

## Common Mistakes

**1. Using "latest" as the model version.** A model update the night before the demo can change output style, introduce unexpected formatting, or produce unexpected content. Pin the exact model version used during rehearsal.

**2. Depending on live internet during the demo.** Hotel WiFi, conference center networks, and even enterprise office networks are unreliable during demos. The local backup environment must be ready and pre-tested.

**3. Starting with the product.** FDEs who open the demo application as their first action lose the audience who needed to understand the problem first. The problem statement is the opening.

**4. Not running rehearsal on the actual demo network.** Rehearsals on the FDE's home network do not reveal corporate proxy issues, DNS resolution failures, or port blocks that appear in the client's environment.

**5. Using obviously fake data that breaks clinical realism.** "Patient Name: Test User, Diagnosis: Test Diagnosis" destroys the demo's credibility. Synthetic data must look realistic.

**6. Not preparing for the "Can you show me X?" question.** Having only one pre-built scenario means that any question outside the scripted path produces an "I'll show you that later" deflection — which signals lack of depth.

## Best Practices

- Pin every version in the demo environment — model, library, API
- Always have a local backup environment ready and pre-tested on the day
- Open with the client's specific problem, not the product
- Synthetic patient data must be clinically coherent — have a clinician review it
- Prepare 3–5 pre-built scenarios for different clinical contexts
- Run a full rehearsal on the client's network (or representative network) the day before
- Have pre-generated fallback outputs for every generation step
- Never show real PHI in a demo — not even de-identified data that might be traceable
- End every demo with a specific next step — not "let us know if you have questions"

## Alternatives

| Demo Approach | When to Use | Trade-off |
|---------------|------------|-----------|
| Live interactive demo (primary) | When audience includes technical stakeholders; when client-specific data is available | Highest credibility; failure risk |
| Recorded walkthrough | When live demo is too risky (key executive meeting, spotty network) | No failure risk; lowest credibility |
| Client-data sandbox demo | When assessment is complete and sandbox access is available | Maximum relevance; setup complexity |
| Collaborative build session | When technical audience wants to see the engineering | Deepest credibility; time-intensive |

## Trade-offs

**Realism vs. risk:** The more realistic the demo (real client data structures, authentic clinical scenarios), the higher the audience engagement — but also the higher the setup complexity and the failure risk. FDEs must calibrate the realism level to the stakes of the meeting.

**Depth vs. breadth:** A 20-minute demo that goes deep on one use case is more credible than a 20-minute demo that skims five features. Healthcare FDEs should default to depth on the specific use case that matches the client's primary pain point.

## Interview Questions

### Q: How do you design a demo environment to be reliable for live client presentations?

**Category:** System Design
**Difficulty:** Senior
**Role:** FDE

**Answer Framework:**

Demo reliability is an engineering problem, not a hope. The design principles are isolation, reproducibility, and fallback tiers.

Isolation means the demo environment has no dependencies on production systems, shared infrastructure, or the client's network that the FDE does not control. All FHIR data is pre-fetched and cached. The LLM model version is pinned. All API credentials are demo-specific.

Reproducibility means everything that runs in the demo was created by a script, not manually. The environment can be torn down and rebuilt in under 30 minutes if something goes wrong.

Fallback tiers mean there are always at least two alternatives if the primary environment fails. A local container environment (Ollama for LLM inference, HAPI FHIR for data) can run the same demo without internet. Pre-recorded walkthroughs exist for catastrophic failure scenarios.

Additionally, all LLM generation steps have timeout handling that gracefully falls back to a pre-generated response. The narration for fallback activation is scripted in advance.

**Key Points to Hit:**
- Isolation from production/shared infrastructure
- Pinned versions for all dependencies
- Local backup environment always ready
- Pre-generated fallbacks for all generation steps
- Rehearsal on client-representative network

**Red Flags:**
- "We rely on the live API — it's usually fast enough"
- Not having a local backup environment

## Key Takeaways

- A demo is an engineered artifact, not a screen share — treat demo reliability as a first-class constraint
- Three layers must be engineered independently: environment, data, and narrative
- Pin every version — model, library, API — nothing "latest"
- Always have a local backup environment ready and pre-tested on demo day
- Open with the client's problem, not the product
- Synthetic data must be clinically coherent to retain healthcare audience credibility
- All healthcare demos require a medical disclaimer — verbal and visible
- Pre-generate fallback outputs for every LLM generation step

## Further Reading

- [FDE Role and Responsibilities](01-fde-role-and-responsibilities.md) — Where demo engineering fits in the FDE lifecycle
- [POC to Production](05-poc-to-production.md) — How demos evolve into POCs
- [EHR Integration](../07-Healthcare-AI/03-ehr-integration.md) — FHIR R4 and SMART on FHIR integration patterns for demo environments
- [Clinical Documentation AI](../07-Healthcare-AI/09-clinical-documentation-ai.md) — The HMS discharge summary use case demonstrated in this chapter