Staging environment
Free Lesson

Start AI Evals With Datadog, Splunk, New Relic

Hosted by Stella Liu

88 students

In this video

What you'll learn

Use current observability tools for AI evals

See how platforms such as Datadog, Splunk, and New Relic can already support your AI evals needs

Turn observability data into evaluation signals

Learn which telemetry to capture and use to evaluate real-world AI agent interactions.

Build evaluation strategy on top of observability

Observability tools make evaluation easier, but success starts with knowing what to measure.

See Datadog LLM Observability in action

Watch a live Datadog walkthrough of tracing an AI agent, inspecting LLM and tool calls, and reading evals results

Why this topic matters

You may already have what you need to start evaluating AI systems. Tools like Datadog, Splunk, and New Relic can capture the traces, logs, and telemetry behind agent behavior. Join this session to see how that familiar observability can support AI evals, and why defining success, failure modes, and meaningful metrics matters more than finding the “perfect” platform.

You'll learn from

Stella Liu

Head of Applied Scientist, AI Evals

Stella Liu is an AI Evaluation practitioner and researcher, specializing in frameworks for large language models and AI-powered products.


Since 2023, she has led real-world AI evaluation projects in EdTech, where she established the first AI product evaluation framework for Higher Education and continues to advance research on the safe and responsible use of AI. Her work combines academic rigor with hands-on product experience, bringing proven evaluation methods into both enterprise and educational contexts.


Earlier in her career, Stella worked at Shopify and Carvana, where she built large-scale data-driven automation systems that powered product innovation and operational efficiency.


Stella's AI Evals newsletter on Substack: https://datasciencexai.substack.com/

Follow Stella on LinkedIn

See all products from AI Evals and Analytics

Go deeper with a course

AI Evals Certification: Master Building Reliable AI
Stella Liu and Amy Chen
View syllabus