How to test the ai llm judge - day 07
Evaluating the evaluator: why your LLM-as-a-judge is a system under test that needs its own validation.
Recent posts
I made AI Judge - Day 06
Evaluating the logic behind having an LLM judge another LLMās output. Establishing a helpfulness rubric and confronting definitions.
I tested the āboringā requirement - day 05
Putting predictable constraints and standard expectations to the test, evaluating how the LLM judge grades standard formatting and basic rules.
Five examples are enough to start - day 04
You donāt need thousands of test cases to begin. Five carefully chosen examples are more than enough to boot up your AI evaluation pipeline.
I tested a longer prompt - day 03
Increasing the complexity of prompt inputs to see how the LLM judge responds and analyzing its limits when prompt length grows.