Could our AI system affect access to services such as healthcare, housing, insurance, benefits or education?

Ethics & Human Rights Category
Design PhaseInput PhaseModel PhaseOutput PhaseMonitor Phase
Could our AI system affect access to services such as healthcare, housing, insurance, benefits or education?
  • The output of your model could be used to deny access to certain fundamental rights.
  • How can you be sure that the decisions of your AI system are always fair and correct?
  • How can you prevent causing harm to individuals?
  • AI systems intended to be used to determine access or admission, evaluate learning outcomes, or monitor students’ behaviors are classified as “high risk” by the AI Act (Annex III).
  • Could the AI system create barriers to healthcare access for some groups or individuals?

If you answered Yes then you are at risk

If you are not sure, then you might be at risk too

Recommendations

  • Adhere to EU Trustworthy AI guidelines to ensure fairness and accountability
  • Use diverse, representative training data to reduce biases that could disproportionately impact certain groups.
  • Regularly audit the system for unintended discriminatory effects and address identified issues.
  • Provide clear explanations for decisions made by the AI system, including the data and logic used. Allow users to challenge decisions and request human reviews.
  • Establish post-market monitoring processes to detect and address issues that arise after deployment.
  • Update the system regularly to account for changes in legal requirements, societal norms, and data quality. For high-risk applications, such as determining healthcare access or evaluating job candidates, establish stringent safeguards to minimize the risk of harm. Implement thresholds and fail-safes to ensure critical decisions are accurate, fair, and reliable.
  • Work with regulatory bodies, civil society organizations, and industry peers to establish best practices and promote fairness and equity in AI systems. Ensure the AI system is designed to accommodate a wide range of users, including those with varying needs and abilities. Regularly test the system in diverse real-world settings to validate its accessibility and fairness. Use stakeholder consultations to understand the specific needs and vulnerabilities of affected groups.

Interesting resources/references