Test Driven Agent Development

Unlock Premium Features with a Subscription

Live Tarining:

Full access to all live workshops and training sessions.
20+ Expert-Led Workshops:

Dive deep into AI Agents, RAG, and the latest LLMs
ODSC Conference Discounts:

Receive extra discounts to attend ODSC conferences.

Get Premium Now

Course Overview:

Duration: 1 Hour

Building reliable AI agents often feels more alchemy than engineering. We will try to showcase how it is possible to evolve the classic red-green-refactor cycle for the non-deterministic nature of language models. The key lies in extending exact-match assertions to write tests that validate an agent's intermediate steps: Did it choose the right tool? Were the arguments correct?

The session will cover practical techniques like evaluating traces with a combination of assertions and other patterns like ""LLM-as-a-judge"".

Instructor:

John Dickerson, PhD

CEO | Co-founder and Chief Scientist at Mozilla.ai | Arthur

John Dickerson is the Mozilla.ai's CEO, Arthur's co-founder and Chief Scientist as well as an Associate Professor of Computer Science at the University of Maryland. He works at the intersection of machine learning and economics, with a focus on designing incentives that promote "good" participation in complex systems. His research centers on solving practical economic problems using techniques from computer science, stochastic optimization, and ML. He received his PhD in computer science from Carnegie Mellon University (SCS CSD PhD '16).