I turned a defect into an eval - day 15
I found one AI bug today—and turned it into a permanent test in 10 minutes.
Recent posts
I put a price tag on quality - day 10
I tested two models today. One answer was only slightly better and much more expensive.
I tested the ‘boring’ requirement - day 05
Putting predictable constraints and standard expectations to the test, evaluating how the LLM judge grades standard formatting and basic rules.
My first AI eval was boring - day 02
Setting up a basic LLM evaluation only to find the initial results predictable, but discovering why simple tests are the best place to start.
The same prompt, 10 different answers - day 01
Running the exact same prompt multiple times reveals the inherent variability in LLM responses, a core challenge in AI testing.