AI Foundations — Quick Reference#

Last Updated: 2026-06-30 Full Chapters: docs/01-AI-Foundations/

LLMs — Core Facts#

Embeddings — Core Facts#

RAG — Architecture at a Glance#

INDEXING (offline):
Documents → Parse → Chunk → Embed → Vector Store + Metadata DB

QUERY (real-time, target: <3s total):
Query
  → Preprocess (intent + synonym expansion)       ~50ms
  → Hybrid retrieval (dense + BM25, top-20)      ~100ms
  → Cross-encoder re-rank (top-8)                ~150ms
  → Context build (+ patient EHR context)         ~50ms
  → LLM generate (Claude Opus, streamed)          ~1–2s
  → Citation validate                             ~100ms
  → Response with citations

RAG Failure Modes — Quick Reference#

Prompt Engineering — Essential Patterns#

System Prompt Anatomy#

1. Role definition (who is the model + operational context)
2. Knowledge scope (only use provided context / retrieved content)
3. Hard constraints (never do X; always include Y)
4. Output format (JSON schema / section headers / exact structure)
5. Edge case handling (what to do when uncertain)

Temperature Selection#

Few-Shot: When to Use#

Add 2–5 examples in the system prompt when:

• Output format is complex and hard to describe in rules alone
• Task requires judgment (what counts as "significant"?)
• Zero-shot compliance rate is below ~85%

Fine-Tuning vs. RAG — Decision Table#

Evaluation — RAGAS Metrics#

Alert threshold: degrade alert if any metric drops > 10% from baseline.

Context Window Management#

Budget Template (200K Claude)#

Long Document Strategies#

• Fits in context (<150K tokens): Direct inclusion — include document then query
• Doesn't fit, need specific facts: RAG — retrieve relevant chunks
• Doesn't fit, need full understanding: Map-Reduce — process chunks in parallel, synthesize
• Sequential document with dependencies: Hierarchical summarization — level-by-level compression

Multimodal — Quick Reference#

Never use an LLM to diagnose from medical images without FDA 510(k) clearance.

Model Selection Guide#

Common Interview Questions#

1. "Why do LLMs hallucinate and how do you mitigate it in a clinical system?" → RAG + citation grounding + validation + human review
2. "When would you choose RAG over fine-tuning?" → Knowledge is dynamic, citation required, institution-specific → RAG
3. "Design a RAG system for a hospital's clinical guidelines" → See full system design in interview/architecture/01-design-enterprise-rag.md [PLANNED]
4. "What is RAGAS faithfulness and why does it matter for clinical AI?" → Measures whether claims are grounded in retrieved context; most safety-critical metric
5. "How does the context window affect RAG architecture?" → Creates retrieval budget; "lost in the middle" → order matters

See Also#

• LLM Fundamentals
• Embeddings
• RAG Architecture
• Agentic AI Cheat Sheet — [PLANNED]
• Healthcare AI Cheat Sheet — [PLANNED]