All posts

Engineering

12 min

Evals you can trust: test sets from real failures

The best eval set is the list of things that already went wrong. How we turn incidents into tests.

Tomás Ruiz

Applied AI Lead

Every agent demo has the same moment: the model does something impressive, and somebody in the room asks, “but what if it gets it wrong?” For most teams, that question kills the project.

Our answer is boring on purpose. Agents run on their own until they reach a line you drew. Past the line, a person decides.

The 4% problem

If an agent asks for approval too often, people stop reading the requests. Across two million runs a week, approvals stay meaningful when fewer than 4% of runs need one.

An approval nobody reads is worse than no approval at all — it’s a rubber stamp with your name on it.

Where to draw the line

We start every customer with three rules and tune from there: a dollar threshold, a list of new or unusual recipients, and anything that can’t be undone.

Share

Create a free website with Framer, the website builder loved by startups, designers and agencies.