I made AI Judge - Day 06
Evaluating the logic behind having an LLM judge another LLMās output. Establishing a helpfulness rubric and confronting definitions.
Recent posts
I tested the āboringā requirement - day 05
Putting predictable constraints and standard expectations to the test, evaluating how the LLM judge grades standard formatting and basic rules.
Five examples are enough to start - day 04
You donāt need thousands of test cases to begin. Five carefully chosen examples are more than enough to boot up your AI evaluation pipeline.
I tested a longer prompt - day 03
Increasing the complexity of prompt inputs to see how the LLM judge responds and analyzing its limits when prompt length grows.
My first AI eval was boring - day 02
Setting up a basic LLM evaluation only to find the initial results predictable, but discovering why simple tests are the best place to start.