Aisha Thornton
Staff Engineer
Stripe
Aisha Thornton is a staff engineer at Stripe, where she leads the team responsible for reliability and evaluation across Stripe's AI-powered product features. She has spent the last three years solving the problems that appear after the prototype works — when LLMs meet real users at scale and the edge cases multiply faster than the team can catch them.
Her work at Stripe spans model evaluation frameworks, automated regression testing for AI outputs, and the monitoring infrastructure that catches failures before users do. She was one of the first engineers at Stripe to push for a formalized LLM evaluation process, a framework that has since been adopted across multiple product teams inside the company.
At BuildWithAI Summit, Aisha delivers a Day 1 talk on reliable LLM outputs at scale, then goes hands-on in a Day 2 workshop where attendees leave with a working evaluation framework for their own AI features. Both sessions are built for engineers who have already shipped something and are now trying to keep it working.
Session
Reliable LLM Outputs at Scale
Day 1
Engineering
What Stripe's engineering team learned after 18 months of running LLMs in production — where outputs break, how to catch failures before users do, and the evaluation framework that changed how the team ships. A practical, no-hype session for engineers who have moved past the prototype stage.
Session
Workshop — Evaluating LLM Outputs — A Practical Framework
Day 2
Engineering
How to know if your LLM is actually working — a hands-on session on building an evaluation framework for any AI feature, from simple output checks to automated regression testing. You'll leave with a template your team can start using the same week.
Pick your seat.
VIP Access
$799
$599
✓
VIP Access to speakers
✓
Mentorship with speakers
✓
Access to all session recordings (released within 2 weeks)
✓
Digital speaker resources & slide decks
Available until August 1, 2026 or offer Sold Out
Sponsors & Partners
These are the tools, platforms, and funds that make BuildWithAI Summit possible.
Want to put your brand in front of 600 builders?
