Event-Driven AI
Common Mistakes
1. Auto-commit of Kafka offsets. With enable<em>auto</em>commit=True, Kafka commits the offset as soon as the message is polled — before processing. If the AI worker crashes during processing, the event is marked as processed and never retried. Always use manual commit after successful processing.
2. Logging event payloads. Clinical event payloads contain PHI. Logging them to application logs (even at DEBUG level) violates HIPAA. Log event metadata only (eventid, eventtype, patient_id), never the full payload.
3. Not partitioning by patient_id. Round-robin partitioning distributes events across partitions but does not guarantee order for a given patient. If a discharge event for patient X is processed before the admission event (because they landed on different partitions), the AI risk assessment runs on incomplete context.
Key Takeaways
- Event-driven AI enables autonomous response to clinical events at scale without blocking clinical workflows
- Kafka partition by patient_id to guarantee per-patient event ordering
- Always use manual offset commit: commit only after successful processing to ensure exactly-once semantics
- PHI in Kafka topics requires encryption at rest, in transit, and per-consumer-group access control
- Consumer lag is the key operational metric; alert when lag exceeds 1000 messages sustained