Could users contest or challenge the decisions made by the AI system?

This page is a fallback for search engines and cases when javascript fails or is disabled.
Please view this card in the library, where you can also find the rest of the plot4ai cards.

Accountability & Human Oversight CategoryEthics & Human Rights Category
Design PhaseModel PhaseOutput PhaseMonitor Phase
Could users contest or challenge the decisions made by the AI system?

Some AI systems make or support decisions that significantly affect individuals, such as in hiring, lending, or criminal justice. If users cannot challenge these decisions or request human review, the system may violate oversight obligations and erode trust. Lack of contestability undermines accountability and may breach Article 22(3) of the GDPR or Article 14 of the EU AI Act, both of which require mechanisms for human intervention and review.

If you answered No then you are at risk

If you are not sure, then you might be at risk too

Recommendations

  • Ensure the AI system includes mechanisms for contestability, allowing users to challenge or seek review of decisions that negatively impact them. Wrong decisions could also have an impact on people that have not been the target of the data collection (data spillovers).
  • Provide clear instructions on how users can initiate such challenges and ensure that this process is transparent, accessible, and user-friendly.
  • Incorporate features that enable human oversight in decision-making processes, ensuring users have the option to escalate issues to human operators.
  • Establish a redressal process that includes timelines for resolution, a clear escalation hierarchy, and mechanisms for feedback integration to improve the system’s decision-making over time.
  • Regularly audit and evaluate the decision-making outcomes of the AI system, focusing on areas where users frequently raise disputes. Use these audits to improve system accuracy and reduce the need for contestation.
  • Provide detailed and comprehensible explanations of the system’s outputs to users, ensuring they understand how decisions are made and what data was used.
  • Engage relevant stakeholders, including legal experts, ethicists, and representatives from affected user groups to design and evaluate the contestability mechanisms and ensure they meet ethical and regulatory standards.
  • Train system operators and customer support staff to handle disputes arising from the AI system effectively, ensuring they are equipped to assist users in navigating the contestation process.

Charter of Fundamental Rights of the European Union

Interesting resources/references