Filed under Evaluation

A saved prompt is not a complete experiment record. The model, relevant settings and test inputs can all affect the result. When comparing an OpenAI workflow, record the exact model identifier used, the prompt version…

Prompt tuning is hard to assess when the definition of “better” changes after every answer. Freeze a small acceptance set first. For a document-routing task, define the allowed destinations and label a collection of…

A model can handle ordinary support tickets well and still fail on the exceptions that consume most of your team's time. Build a small evaluation set from representative, appropriately sanitized cases. Include a missing…