This page is a fallback for search engines and cases when javascript fails or is disabled.
Please view this card in the library, where you can also find the rest of the plot4ai cards.
Could an agent be misused to manipulate or deceive users?
Could an agent be misused to manipulate or deceive users?
- Agentic systems capable of persuasive language or personalized interaction can be manipulated to influence human decisions, emotions, or behaviors.
- This creates risks of social engineering, phishing, misinformation, or undue influence, especially if the agent mimics authority figures or trusted personas.
- The risk is amplified when agents use persistent memory or learn user preferences over time.
If you answered Yes then you are at risk
If you are not sure, then you might be at risk too
Recommendations
- Impose ethical use constraints and define red lines (e.g., no impersonation, no medical/legal advice without oversight).
- Use transparency mechanisms to disclose when users are interacting with agents.
- Enable user control and opt-out of persuasive or adaptive behaviors.
- Monitor for behavior that resembles coercion, manipulation, or impersonation.