EDPS publishes Guidance on AI Risk Management

The European Data Protection Supervisor (EDPS) has issued guidance to help EU institutions and agencies manage risks when developing, procuring, and deploying AI systems that process personal data. Anchored in the EUDPR’s accountability principle and aligned with ISO 31000:2018, the document focuses on technical measures tied to core data protection principles—fairness, accuracy, data minimisation, security—and on practical enablers such as interpretability and explainability. It also maps risks across the AI lifecycle and procurement phases, making clear that controllers must identify, treat, and document risks throughout, including when using external providers, without relying on this guidance as a compliance checklist.

On fairness, the EDPS highlights multiple bias sources—poor data quality, sampling and historical bias, algorithmic bias, overfitting, and interpretation bias—and prescribes countermeasures such as quality policies for training data, representative datasets, feature engineering, fairness-aware objectives, regular bias audits, and explainability techniques (e.g., LIME, SHAP). For accuracy, the guidance distinguishes legal accuracy of personal data from statistical accuracy of models and warns about hallucinations and data drift. Mitigations include robust validation with diverse and edge cases, hyperparameter optimisation, ongoing monitoring for drift, regular retraining, and human oversight. Procurement demands transparency from providers on model design, training data governance, validation methods, and fairness/accuracy metrics.

Security risks specific to AI include disclosure of training personal data through inversion or membership inference, storage breaches, and API leakage. Recommended controls cover data minimisation, perturbation (generalisation, aggregation, differential privacy), synthetic data with caution, encryption, secure development, MFA, RBAC, throttling, TLS, SIEM-based monitoring, audits, and patching. Finally, the guidance addresses data subjects’ rights: controllers must be able to identify personal data in training sets and models (via metadata and retrieval tools), and support rectification or erasure (including machine unlearning where feasible, or output filtering). The EDPS emphasises continuous risk management, lifecycle governance, and documentation to uphold fundamental rights and public trust.

Key takeaways

• Interpretability and explainability are prerequisites for compliance and auditability, distinct from transparency to data subjects.
• Fairness risks stem from data quality, sampling and historical bias, algorithmic design, overfitting, and interpretation bias; apply representative data, fairness-aware objectives, and regular audits.
• Legal accuracy (of personal data) differs from statistical accuracy (model performance); mitigate hallucinations, data drift, and deteriorating input quality via monitoring and retraining.
• Data minimisation must be enforced despite AI’s data hunger; prefer sampling, anonymisation or pseudonymisation, and only necessary features.
• AI-specific security threats include model inversion, membership inference, regurgitation, data/model poisoning, and insecure APIs; implement strong technical protections.
• Procurement requires demanding provider transparency on training data governance, validation, metrics, and model integrity.
• Controllers must support data subject rights with metadata, retrieval tools, machine unlearning where possible, and output filtering if needed.
• Risk management per ISO 31000 should be continuous across development and procurement lifecycles, with documented measures and residual risk assessments.