Training ML Models with Differential Privacy (DP-SGD)
DP-SGD (Differentially Private Stochastic Gradient Descent): A modified training algorithm that adds noise to gradients during each optimization step. The trained model satisfies differential privacy, meaning an attacker cannot determine if any specific example was in the training data by examining the model.
Why Standard Training Leaks Privacy
Neural networks memorize training data. Language models reproduce verbatim text from training corpora. Image classifiers reveal whether specific photos were used. Membership inference attacks can determine with high accuracy whether a particular record was in training data. Even without explicit memorization, model parameters encode information about individual examples. DP-SGD prevents this by ensuring the model would look nearly identical whether or not any single training example was included.
How DP-SGD Works
Three modifications to standard SGD: Per-example gradients: Compute gradient for each example separately (instead of batch average). Gradient clipping: Bound each gradient to maximum norm C. This limits how much any single example can influence the update. Noise addition: Add Gaussian noise calibrated to C and epsilon to the clipped gradient sum. The clipping ensures bounded sensitivity; the noise provides privacy. The model trains slowly and less accurately, but the final parameters satisfy differential privacy.
Privacy-Utility Trade-off
DP training significantly degrades model accuracy. On standard benchmarks like CIFAR-10, non-private models achieve 95%+ accuracy; DP models with epsilon=1 achieve 60-70%. The gap narrows with more data and compute but never closes. Practical epsilon values for production (epsilon between 1 and 10) typically reduce accuracy by 5-30% compared to non-private training. The acceptable accuracy loss depends on your privacy requirements and legal constraints.
Composition Warning: Each training epoch consumes privacy budget. Training for 100 epochs costs 100x the single-epoch privacy cost. Use privacy accounting (moments accountant) to track cumulative epsilon across all epochs.