9 prompts to test, score and stress-test your AI prompts like a QA engineer
Stop guessing whether your AI prompt works. Use these 9 prompts to find out.
Most people are practising the wrong skill.
They’re getting better at writing prompts. Tighter instructions, cleverer personas, more specific examples. And the outputs do get better, for a while, on the tasks they thought to test. The problem is that “it looked good when I tried it” is not a quality standard. It’s a hope.
Tools like DSPy are already optimising prompt text automatically. The writing part is being automated. What can’t be automated, at least not yet, is knowing whether the output is actually good. That requires a human to define what good looks like, write down the test cases, and run them consistently.
That’s the skill shift. From writing prompts to testing them.
These 9 prompts take you through the full process: building test cases, scoring outputs, finding failure modes, running comparison tests and locking in the version that actually works.
Why this matters right now
Automated prompt optimisation is moving fast. DSPy, PromptFoo, and similar tools can iterate through hundreds of prompt variants in the time it takes you to write one manually. The bottleneck they all share is the same: someone has to tell the system what a good output looks like.
If you can’t answer that question in writing, with specific criteria, the tools are useless. The people who’ll get the most from the next wave of AI tools aren’t the best prompt writers. They’re the best prompt evaluators.
Prompt 1: Build your first test case
What it does: Produces a structured test case document for any prompt you’re currently using, including input variations, success criteria and failure flags.
When to use it: Before you rely on any prompt for real work, or when an existing prompt is producing inconsistent results.
The Prompt:
I want to build a proper test case for an AI prompt I’m using. Here is the prompt I want to test:
[PASTE YOUR PROMPT HERE]
My use case is: [DESCRIBE WHAT YOU USE THIS PROMPT FOR, e.g. “writing LinkedIn posts for a B2B software company”]
My target output quality is: [DESCRIBE WHAT GOOD LOOKS LIKE, e.g. “under 200 words, no jargon, ends with a question, sounds like a person not a brand”]
Please produce a test case document with: 1. 5 input variations that cover the range of real scenarios I’d use this prompt for (from easy/typical to difficult/edge case) 2. For each input, a written description of what a passing output would look like 3. A list of 5 specific failure flags (things an output could do that would make it a fail regardless of other quality) 4. A scoring rubric with 4 criteria, each scored 1-5, that I can apply to any output from this prompt
Format the result so I can paste it into a document and reuse it.
How to use it:
Paste your existing prompt into the first placeholder exactly as you currently use it
Write the use case description in plain language, not aspirationally, describe what you actually use it for
For target output quality, be specific: length, tone, structure, what it must and must not contain
Example input: Prompt = a prompt that writes cold email subject lines. Use case = “outreach emails for a freelance UX designer targeting e-commerce startups.” Target quality = “under 8 words, no question marks, specific to the recipient’s industry, doesn’t use the word ‘quick.’”
What you’ll get: A reusable test document with 5 scenario inputs, pass/fail descriptions for each, a list of automatic failure conditions and a 4-criteria scoring rubric. Takes about 3 minutes to fill in and saves hours of guesswork later.
Advanced note: Run the same test case document through two different models (Claude and GPT-4o, for instance) using identical inputs. The differences in where each model fails tell you more about prompt weaknesses than any amount of manual tweaking will.
ooops, a paywall…
That test case gives you something most people never have: a written definition of what good actually means for your specific prompt.
Behind the upgrade, there are 8 more prompts that take this further:
Prompt 2: Score an existing output: Apply your scoring rubric to a real output and get a structured evaluation with a numerical score and specific improvement notes, not just “this could be better.”
Prompt 3: Find failure modes: Run your prompt against 6 adversarial inputs designed to break it, so you know where it falls apart before your actual work does.
Prompt 4: Write a comparison test: Set up an A/B test between two prompt variants using identical inputs and a blind scoring method, so you can make version decisions on evidence rather than preference.
Prompt 5: Extract implicit criteria: Feed Claude a set of outputs you rated highly and ask it to reverse-engineer your unstated preferences into explicit criteria you can add to your rubric.
Prompt 6: Build a regression log: Create a living document that records prompt versions, test scores and change notes, so you can roll back to a previous version if a tweak makes things worse.
Prompt 7: Stress-test for edge cases: Generate 10 edge-case inputs specific to your use case, including unusual formats, missing information and off-topic requests, to find where your prompt gets confused.
Prompt 8: Write a minimum viable spec: Turn your test case document into a one-page prompt specification that another person (or an automated tool like DSPy) could use to optimise or rebuild your prompt from scratch.
Prompt 9: Audit a prompt you didn’t write: Apply the full testing process to a prompt you found online or inherited from someone else, so you can decide whether it’s actually fit for purpose before using it in production.
Plus: The Eval Scorecard, a reusable plain-text template you can copy into any document and fill in for any prompt in under 10 minutes.
If you only do one thing differently this week, building that first test case is a solid start. The full testing workflow is behind the upgrade.