A View on Predictive Policing in the AI Act

The recently approved Artificial Intelligence Act by the EU Council marks a significant step in regulating AI, particularly by prohibiting certain high-risk applications like predictive policing. This regulation aims to protect fundamental rights and promote safe AI development through a risk-based approach. However, the effectiveness of the ban on predictive policing is questioned due to potential loopholes such as the “human in the loop” defense and national security exemptions, which may undermine its intended impact.

Predictive policing, which involves using AI to forecast criminal activity, has faced criticism for potential human rights violations, including privacy breaches and discriminatory biases. Despite being initially classified as high-risk, predictive identification is now prohibited under Article 5(1)(d) of the Act. Though, the ambiguity in terms like “profiling” and “meaningful human intervention” may allow law enforcement to circumvent the ban by demonstrating human involvement in decision-making processes.

Further complicating the issue is the Act’s exemption for AI systems used for national security purposes, which could be exploited by Member States to justify the use of predictive identification. This broad exemption risks undermining the ban’s effectiveness, raising concerns about the potential for continued discriminatory practices under the guise of national security. To ensure the Act’s goals of safeguarding fundamental rights and developing human-centric AI, clearer definitions, stricter guidelines, and a more nuanced approach to national security exceptions are needed.

European Law Blog has dived into the issue in their article Predictive Policing in the AI Act: meaningful ban or paper tiger?

KEY TAKEAWAYS

1. The AI Act aims to regulate high-risk AI applications to protect fundamental rights.
2. Predictive policing is now prohibited under Article 5(1)(d) of the Act.
3. Ambiguities in terms like “profiling” and “meaningful human intervention” may weaken the ban.
4. The national security exemption poses a significant risk to the ban’s effectiveness.
5. Clearer definitions and stricter guidelines are needed to ensure the Act’s goals are met.