Synonyms: AI alignment, value-aligned AI, human value alignment, aligned AI, intent alignment
Do not index
Definition
Value alignment is the work of making an AI system's goals and behavior match what people actually want and value, not a proxy that drifts from their intent. A model tuned for the wrong target (clicks, watch time, a narrow reward) can hit its metric while failing the user. In products, alignment spans the model (how it's trained) and the design (what you reward, and how users can correct it).
Use cases
Get the objective wrong, and the system works against the user while looking like it's succeeding. The cost is engagement traps, sycophantic AI, and eroded trust.
The engagement trap. A feed tuned for time-on-app learns to push outrage, winning the metric while making users feel worse.
The yes-man assistant. An AI trained to be "preferred" learns to flatter and agree, even when the user is wrong. That feels nice and quietly misleads.
The literal genie. A user tells a bot to "close my account fast," and it skips the data export they'd have wanted.
How it's used in practice
Pick a proxy that tracks the real goal: before you chase a metric, ask where it diverges from what users value, and add a guardrail metric to catch the gap.
Surface and confirm intent: for ambiguous or high-stakes requests, restate what you think the user wants before acting.
Give users correction levers: let people rate, edit, or override AI behavior, and route that feedback into how the system improves (the practical core of RLHF).
Make the system's values legible: state what the AI will and won't do, so its behavior is predictable.
🪄
Pro-tip: Watch for reward hacking in your own product, not just the model. Any metric you chase gets gamed, sometimes by the system and sometimes by your team.
If thumbs-up rate is the target, you'll ship an AI that's agreeable over accurate. Pair every target metric with a counter-metric that would move the wrong way if the system started cheating.
Challenges & limitations
Whose values? "Human values" aren't one thing. They differ across cultures and users, and someone has to decide which to encode, which is a power most teams don't acknowledge.
Alignment has a tax. Making a model safer or more honest can make it less capable or more cautious, and teams feel that tradeoff in product quality.
You can measure the proxy, not the value. Thumbs-up and retention are trackable, but "did this actually serve the user" resists clean measurement, so misalignment hides behind good-looking numbers.
Free resources
IBM: What Is AI Alignment? — a clear primer on alignment, RLHF, and where models diverge from human values.
