Synonyms: algorithmic complacency, over-reliance on automation, system trust bias, AI over-reliance, machine trust bias
Do not index
Definition
Automation bias is the tendency for people to trust recommendations from automated systems too quickly, even when those recommendations are wrong or incomplete.
This usually happens when software appears more authoritative or reliable than human judgment.
Use cases
Unchecked automation bias can lead to serious mistakes because users stop questioning the system’s output.
Clinical decision support: a doctor accepts an AI triage recommendation without re-reading the chart, and misses a contraindication.
AI coding assistants: developers merge auto-suggested code that compiles but introduces a subtle SQL injection.
Navigation apps: drivers follow GPS into a closed road or river because the map "felt" authoritative.
How it's used in practice
Add review steps: Require users to confirm important decisions before automated actions are finalized.
Explain recommendations clearly: Show the signals or data points behind a suggestion when possible.
Support easy rejection: Rejecting or editing a recommendation should feel as easy as accepting it.
Label system-generated content: Clearly separate AI-generated outputs from human input or verified information.
🪄
Pro-tip: Match the UI confidence to the system confidence.
If the AI is uncertain, the interface should communicate uncertainty too. Avoid presenting low-confidence suggestions with overly authoritative language or visual styling.
Challenges & limitations
The efficiency paradox: Friction reduces bias but also slows down the user, which can frustrate power users.
The trust gap: If you make users double-check everything, they may eventually view the automation as useless.
