Back to Glossary
Prompt Evaluation
Synonyms: prompt evals, prompt testing, LLM evals, AI evals, prompt regression testing
Do not index
Definition
Prompt evaluation is testing a prompt against a fixed set of real inputs to check whether its outputs are good enough to ship. Code, a person, or another model grades each output, and every prompt version runs on the same set.
Use cases
Without evals, a fix for one complaint can break answers that used to work, and users find out first.
- The 4-question ship: A PM tries a new support prompt on 4 questions and ships it. A week later it refuses refund requests the old prompt handled.
- The judge that likes long answers: In the MT-Bench study, GPT-4 grading other models favored longer answers and whichever answer it read first.
How it's used in practice
- Start from real failures: Read real conversations and group what went wrong into failure types. Each type becomes a test.
- Begin with 20 to 50 cases: Add edge cases: empty input, very long input, off-topic questions, and other languages.
- Grade with code first: Use exact-match or format checks where you can. Save model graders for tone, with a pass/fail rubric.
- Run old and new prompts on the same set: Ship only if no case that passed before now fails.
Test the judge before you trust its scores. Have a person grade 30 outputs, then compare with the model grader. If they disagree often, fix the rubric first.
Challenges & limitations
- No single right answer: Summaries and emails are hard to score with code, so teams rely on model graders and their biases.
- Stale test sets: Users keep asking new things, and a set built at launch misses the new failures.
- Cost: Each run calls the model once per case, and again if a model grades it.
Free resources
- Define Success Criteria and Build Evaluations (Anthropic) — writing measurable criteria and grading outputs with code, people, or models
- Evaluate Prompts in the Developer Console (Anthropic) — generating test cases and comparing prompt outputs next to each other
- Evaluation Best Practices (OpenAI) — structuring evals for outputs that change from run to run
