Adding a CI pass fail threshold - day 30
I brought my eval journey to the finish line today by integrating our small test dataset directly into our CI/CD pipeline with a pass/fail threshold.
Recent posts
I turned a defect into an eval - day 15
I found one AI bug today—and turned it into a permanent test in 10 minutes.
I put a price tag on quality - day 10
I tested two models today. One answer was only slightly better and much more expensive.
I tested the ‘boring’ requirement - day 05
Putting predictable constraints and standard expectations to the test, evaluating how the LLM judge grades standard formatting and basic rules.
My first AI eval was boring - day 02
Setting up a basic LLM evaluation only to find the initial results predictable, but discovering why simple tests are the best place to start.