I put a price tag on quality - day 10
I tested two models today. One answer was only slightly better and much more expensive.
Recent posts
Better answers took longer - day 09
The better answer was also the slower answer. That is a product decision, not just a model decision.
The ‘best’ model lost - day 08
The model everyone calls ‘best’ lost my tiny test today. Learn why a cheaper, faster model might outperform state-of-the-art models for specific tasks.
How to test the ai llm judge - day 07
Evaluating the evaluator: why your LLM-as-a-judge is a system under test that needs its own validation.
I made AI Judge - Day 06
Evaluating the logic behind having an LLM judge another LLM’s output. Establishing a helpfulness rubric and confronting definitions.