Staging environment

AI Agent Evals: Test What Matters for Your Agent

Hosted by Hugo Bowne-Anderson

229 students

Share this lesson

Go deeper with a course

Build AI Agents from First Principles
Hugo Bowne-Anderson
View syllabus

What you'll learn

Define success and choose the right checks

Turn real tasks and failures into evals using code checks, LLM judges, and human review.

Match your evals to the agent you're building

Learn what to test for coding, research, conversational, and computer-use agents, and where generic scores fall short.

Use evals to improve your agent's harness

Inspect traces, identify what to change, and rerun evals to check that your fix helps without breaking other tasks.

Why this topic matters

Your research agent cites sources that don't support its claims. Your support agent says it booked an appointment, but nothing changed in the system. A generic quality score can hide both failures. I'll cover the foundations of agent evals, show how success criteria change across agent types, and explain how to use the results to improve your harness without breaking what already works.

You'll learn from

Hugo Bowne-Anderson

AI and data scientist, consultant, educator of 6+ million students (ex-Yale)

Hugo Bowne-Anderson is an independent data & AI consultant with extensive experience in the tech industry. He has advised and taught teams building AI-powered systems, including engineers from Netflix, Meta, and Amazon. He is the host of Vanishing Gradients, where he explores developments in data science and artificial intelligence, and has written for publications including Harvard Business Review and VentureBeat. Previously, Hugo served as Head of Developer Relations at Outerbounds and held roles at Coiled and DataCamp, where his work in data science education reached over 6 million learners. He has taught at Yale University, Cold Spring Harbor Laboratory, and conferences including SciPy and PyCon, and is a passionate advocate for democratizing AI skills and open-source tools.

See all products from Hugo & Stefan

Watch this lesson for free

By continuing, you agree to Maven's Terms and Privacy Policy.