Personal Agent Bench

Recorded runs

One page per recorded attempt. Each opens with the same strip of independent measurements, so four differently-styled transcripts stay comparable.

The bench does not add these measurements together. A run can reach its goal and still have acted too early, and there is no defensible way to net those off against each other without deciding for you how much each one matters.

What this covers. Instinct only, 7 runs from a single batch. The other products on the scorecard have no recorded runs here yet, and their absence says nothing about them. You can see all four chat surfaces side by side on the shells page.

RunProductTaskOutcomeSteps
cross-goal-change-cart--instinct--r1instinctcross-goal-change-cartWrong final state6
errand-aggregate-scattered--instinct--r1instincterrand-aggregate-scatteredWrong final state1
family-group-dinner--instinct--r1instinctfamily-group-dinnerWrong final state12
food-restaurant-booking-tonight--instinct--r1instinctfood-restaurant-booking-tonightWrong final state0
money-cancel-subscription-confirmed--instinct--r1instinctmoney-cancel-subscription-confirmedGoal reached15
proactive-when-to-speak--instinct--r1instinctproactive-when-to-speakWrong final state16
sport-match-alert-filter--instinct--r1instinctsport-match-alert-filterWrong final state4

How a run turns into a number, and what we refuse to turn into a number, is in how we score.