Does the AI system use behavioral data in ways that may raise ethical, privacy, or human rights concerns?

Ethics & Human Rights CategoryPrivacy & Data Protection CategoryBias, Fairness & Discrimination CategoryData & Data Governance Category
Design PhaseInput PhaseModel PhaseOutput PhaseMonitor Phase
Does the AI system use behavioral data in ways that may raise ethical, privacy, or human rights concerns?
  • Behavioral data includes individuals' actions, habits, preferences, or biometric responses, such as keystrokes, browsing history, device usage, or emotional expressions.
  • AI systems that track and learn from behavior can create serious risks, such as:
    • Privacy violations through covert or unconsented surveillance.
    • Profiling and discrimination, as behavioral traits may act as proxies for protected characteristics (e.g., gender, ethnicity, age).
    • Manipulation and behavioral exploitation, especially if labeling or feedback loops reinforce conformity or nudge users toward certain actions.
    • Chilling effects on autonomy and expression, particularly in politically sensitive or authoritarian contexts.
  • These risks implicate key rights under the EU Charter of Fundamental Rights, the ECHR, and the EU AI Act, which designates behavior-influencing AI as high-risk.

If you answered Yes then you are at risk

If you are not sure, then you might be at risk too

Recommendations

  • Define and document how behavioral data is collected, labeled, and used, including value judgments behind 'positive' or 'negative' classifications.
  • Obtain explicit, informed consent for behavior tracking and provide opt-out mechanisms.
  • Implement privacy-preserving techniques (e.g., differential privacy, federated learning) to reduce data exposure.
  • Regularly audit for bias in behavior-based profiling and assess the representativeness and fairness of training data.
  • Conduct Human/Fundamental Rights Impact Assessments or DPIAs where applicable.
  • Apply safeguards to prevent misuse in sensitive domains (e.g., employment, finance, public services), and assess whether the system qualifies as high-risk under the EU AI Act.

Interesting resources/references