Auditing machine learning while preserving privacy requires evaluation protocols that combine formal privacy guarantees, adversarial testing, and cryptographic safeguards. These protocols are relevant because models trained on sensitive data can leak personal information, causing legal, cultural, and individual harms; understanding causes and consequences helps design audits that protect vulnerable groups and respect territorial regulations such as GDPR enforced by the European Data Protection Board.
Core protocols
At the foundation is differential privacy, a mathematically rigorous framework developed by Cynthia Dwork Harvard University and Aaron Roth University of Pennsylvania that quantifies privacy loss through a parameter often called epsilon. Differential privacy is used both during training and when releasing evaluation metrics so that audit outputs do not reveal individual records. Complementing this are cryptographic techniques: secure multi-party computation and homomorphic encryption allow distributed evaluations where raw data never leaves its owner, following principles pioneered by researchers such as Craig Gentry IBM Research for homomorphic encryption. For iterative or cross-silo workflows, federated evaluation promoted by Brendan McMahan Google enables auditors to compute model statistics across custodians without centralizing sensitive data.
Evaluation protocols and metrics
Privacy-preserving audits combine formal guarantees with empirical tests. Auditors run controlled membership inference and attribute inference probes to measure leakage; foundational work from Reza Shokri Cornell Tech demonstrated how such attacks reveal overfitting and memorization risks. Protocols should report both formal parameters like epsilon and empirical attack success rates, contextualized by model utility so stakeholders see the trade-off between privacy and accuracy. Cryptographic attestation and reproducible audit logs using secure enclaves or zero-knowledge proofs enable third parties to verify that an audit followed agreed protocols without exposing raw inputs. Standards guidance from the National Institute of Standards and Technology helps structure these processes and align them with organizational risk assessments.
Careful protocol design must consider causes and consequences: models trained on underrepresented communities may disproportionately leak sensitive attributes, creating cultural and territorial harms if data crosses borders. Environmental costs arise because repeated, privacy-safe audits often require more computation; auditors should balance sustainability by sampling audits and using cost-aware DP mechanisms. Combining formal privacy, adversarial testing, and cryptographic transparency yields audits that preserve individual privacy while producing verifiable evidence for regulators, affected communities, and system designers — a necessary practice to maintain trust and reduce harm in real-world deployments.