Benchmarking is measuring a product's user experience with consistent metrics so you can compare it over time, against competitors, or against a target. Instead of "the redesign feels better," you get numbers, like task success, time on task, and satisfaction, that show whether it actually improved.
Use cases
Skip the baseline and you're left arguing from gut feel, which loses to whoever's most senior in the room.
No proof of progress: A team rebuilds onboarding and everyone agrees it's better. Without a "before" number, they can't show it improved, or notice if it quietly got worse.
Knowing where you stand: Competitor benchmarking on the same tasks shows whether your checkout is actually faster than a rival's, instead of assuming.
How it's used in practice
Pick a few real metrics: Track task success rate, time on task, and a satisfaction score like SUS, tied to actual tasks.
Capture a baseline first: Measure before you change anything. You can't show improvement without a starting point.
Hold the method steady: Use the same tasks, metrics, and setup each round, or the comparison is meaningless.
Re-measure on a cadence: Run it after each major redesign or quarterly, and watch the trend rather than a single score.
🪄
Pro-tip: The value is in the consistency, not the first number. One benchmark tells you almost nothing; the trend over time is the insight.
The most common mistake is changing the tasks or method between rounds, which turns the comparison into garbage. Lock the protocol, even if it's imperfect, and change it only on purpose. And benchmark against your own past self before you obsess over competitors, since beating last quarter is more actionable than beating a rival you can't see inside.
Challenges & limitations
Numbers don't explain themselves: A metric shows what changed, not why. You still need qualitative research to find the cause.
Metric choice is a trap: Pick a vanity or easy-to-game number and the benchmark misleads everyone who trusts it. Competitor benchmarking is also limited to their public flows.
Commonly Used Metrics & Tools:
System Usability Scale (SUS) — a 10-question survey that gives a comparable satisfaction score. Best for a quick, standardized attitudinal benchmark.
Task Success Rate — the percentage of users who complete a task. Best for the core "does it work" measure.
Maze — a platform for unmoderated testing and benchmarking. Best for collecting task and survey metrics at scale.
