Millan Kaul

less than 1 minute read

Written by - Millan Kaul

I assumed the longer prompt would win. It didn’t.

Image comparing responses from short and long prompts
More prompt words do not always mean more quality.

What I did today:

I tested a short prompt against a more detailed version on five realistic examples. I compared whether the output followed the task, stayed concise, and gave useful answers.

What surprised me:

The longer prompt added instructions, but also made some responses more rigid and less useful.

My QA takeaway:

  1. Prompt changes are product changes. Test them like you would any other change.
  2. More prompt words do not always mean more quality.

Do you test prompt edits before shipping them?

#PromptEngineering #AIEvals #QualityAssurance

You may also enjoy

My first AI eval was boring - day 02

less than 1 minute read

Setting up a basic LLM evaluation only to find the initial results predictable, but discovering why simple tests are the best place to start.