While a single AI agent’s performance may seem flawless based on isolated interactions, it can still reveal underlying product flaws. This discrepancy has prompted businesses to shift from individual trace evaluations to comparing user cohorts against established baselines.
At VB Transform 2026, industry leaders including Harrison Chase, CEO of Langchain, Hui Zhang, CTO of Conviva, and Emmanuel Turlay, Director of Engineering at CoreWeave, discussed this transformation along with the movement toward more cost-effective, specialized Judge models.
The concept of Agent-as-Judge—where one AI agent assesses the output of another—does not replace LLM-as-Judge, which remains the primary method according to Chase. Zhang notes a significant challenge lies in balancing automated evaluations against human reviews.
“Automated reviews are scalable but often lack grounding. Whether judged by an agent or an LLM, scoring and grading remain difficult to ground in reality, leading to reliance on human evaluations, which aren’t scalable,” Zhang explained. “The industry faces a critical decision: which path to take?”
Evaluation Criteria as Product Specifications
This gap—where high evaluation scores may indicate a flawed product—is what the team aims to address by implementing a comprehensive evaluation suite before product launch. However, Chase believes this approach may not be effective.
“Some teams become trapped in evaluation paralysis,” Chase said, emphasizing the need for iteration. “Effective teams prioritize launching and refining.” Assess metrics as dynamic specifications rather than static test suites. It resembles a product requirements document, setting the criteria for what an application should accomplish. “Eval is akin to the new PRD,” he stated. “It outlines the capabilities expected of agents.”
Turlay viewed the same failure from an alternate perspective. “Achieving 100% coverage in testing did not prevent production bugs,” he noted, indicating that even a complete test suite may miss critical elements. This aligns with the gap Chase identified in evaluations.
He suggested that continuous monitoring can uncover more real-world failures than exhaustive pre-launch tests. Teams should initiate extensive online checks to identify common failure types, then develop targeted offline assessments rooted in these identified issues.
Why Individual Trace Scoring Can Lead to Misjudgments
A well-structured evaluation can still produce inaccurate results. Zhang contests the common practice of evaluating samples one-by-one, arguing that it overlooks trends only evident when analyzing user cohorts against a baseline, a method he terms “contrastive analysis.”
For instance, in retail, if a shopper asks an agent for running shoes ahead of a half marathon, and the interaction seems satisfactory, an individual score may not reflect underlying issues. The agent’s follow-up question rate for this category was three times higher than the baseline, while the conversion rate of purchases made outside of conversations was five times higher than average.
Such metrics, invisible from a single interaction, can reveal category-specific troubleshooting opportunities. Zhang argues the industry lacks complementary data—insights on user interactions before, during, and after conversations.
Sizing Evaluation According to Judge Responsibilities
Once contrastive analysis highlights broken categories, the next challenge is ongoing monitoring and associated costs. Turlay’s approach focuses on employing the most capable model initially to validate task solvability, followed by optimization. If the leading model fails to perform, smaller models are unlikely to succeed. Once an effective pattern is established, teams can sample a portion of user traffic, enabling simpler tasks like binary classification to be handled by smaller open-source models.
LangChain has enhanced this strategy by fine-tuning its model to identify perceptual errors—instances where users perceive agent mistakes. “We modified the Qwen model from Alibaba’s open-source family,” said Chase. Achieving good results through manual labeling and distillation can yield significant efficiency, “as seen with Claude Sonet, allowing savings of 10 to 100 times based on delivery methods,” added Chase.
Not all safeguards require complex models. Chase pointed to Claude Code’s guardrails, asserting they mainly employed regular expressions—standard programming practices used for pattern validation in code. “Their guardrails were essentially regex patterns, not sophisticated LLMs,” he highlighted.
The Role of Humans Even in LLM Judging
A fundamental question arises: does utilizing LLMs as judges diminish the necessity for human oversight?
Turlay drew from his experience at a self-driving car company, stating that despite accelerated data processing and model retraining cycles, human approval remained essential.
“I could confidently support the deployment of a model in our vehicles,” he remarked, stressing the importance of human involvement in sectors like law, finance, and healthcare. “It will take time before AI can independently manage responsibilities that require legal accountability.”
Even if machines can analyze patterns on a scale that outstrips human capability, Zhang concurs: human guardianship is crucial in specialized scenarios, as machines might excel at pattern recognition but not at oversight.
Chase emphasized that human checks extend beyond safety nets. “Human engagement is vital for establishing trust in these agent systems and is essential for their continuous learning and memory,” he concluded. “Active interaction is necessary for these systems to evolve.”
Source: venturebeat.com