Model Security

Key Takeaways

  • Models fine-tuned on clinical data can memorize and reproduce training data — conduct memorization audits before deployment
  • Membership inference allows adversaries to determine if specific patients' data was used in training, even without extracting content
  • Differential privacy (DP-SGD) provides the strongest mathematical protection against both memorization and membership inference — at a model quality cost that must be evaluated
  • Encrypt model weights at rest using envelope encryption with KMS customer-managed keys
  • Models fine-tuned on real (even de-identified) clinical data should be classified as "restricted" with API-only access and full audit logging