Opik fine-tunes 1.7B judge using Ollie conversations
Researchers fine-tuned a 1.7B-parameter open model into a decision judge using 600 labelled conversational turns from an AI assistant. On a held-out test set, the fine-tuned judge outperformed off-the-shelf alternatives like Jev and gpt-4o-mini at recognizing interaction shapes such as declines and scope, but trailed Jev on factual answer soundness where domain knowledge is required. Open article →