WebArena benchmark: leaderboard, tasks and versions

The WebArena benchmark is a web agent test built on self-hosted websites, published in July 2023 by researchers at Carnegie Mellon, and it solves a problem most web agent evaluations pretend doesn't exist. If you score an agent against a real website, you're scoring it against a thing that changed while you were measuring, and next month's run isn't comparable to this month's. So the authors built their own web instead.

• 812 tasks across four kinds of site the paper names, e-commerce, a social forum, collaborative software development and content management, with a map, a wiki, a calculator and a scratchpad alongside them as tools.
• Success is checked programmatically against the end state of the site, not against text the agent wrote about what it did.
• The original paper's best GPT-4 agent scored 14.41% against a human baseline of 78.24%. On 5 November 2026 the leaderboard's top entry, dated September 2026, scores 86.1%; the next best is 74.3%.
• The human baseline was measured on one sampled task per template by five graduate students, not on the 812 tasks the agents run. An agent number above it is not the same as beating people at WebArena.

What the WebArena benchmark actually is

A set of working websites you host yourself, plus 812 tasks phrased the way a person would phrase them.

The sites are functional rather than mocked. There's an e-commerce shop with a product catalogue, orders and customer reviews; the seller-side admin interface behind it; a GitLab instance with repositories, issues and commit history; a forum modelled on Reddit; and a map. Around those sit a wiki, a calculator and a scratchpad, so a task can require looking something up or holding a number while doing something else.

The tasks read like requests rather than instructions. Cancel order 307. Tell me the number of reviews mentioning the word "best". Find the closest national park to the largest city in Maine. Add a new colour option to one size of one product. A few need two sites at once, which is why the environment supports multiple tabs, and seventeen of them pair the wiki with the map.

Counted from the repository's own task file on 22 September 2026, the distribution runs 187 tasks on the shop, 182 on its admin side, 180 on GitLab, 109 on the map and 106 on the forum, with the remainder spanning two sites.

Why self-hosting is the whole WebArena design

Everything WebArena is good at follows from the decision to ship the websites rather than point at somebody else's.

A live site changes its layout, its stock, its prices and its data, so two agents evaluated a month apart were not given the same test. Worse, a live site can be modified by the agent in ways that affect the next agent's run. Self-hosting and resets make repeated runs possible, but comparing results across years still needs the same task set, evaluator, environment and step budget. Original and Verified scores do not meet that condition.

The second consequence is how grading works. Because the site is yours, you can inspect it afterwards. WebArena checks functional correctness: did the order actually get cancelled, does the repository actually exist, is the returned string the right one. An agent that narrates a successful cancellation without cancelling anything scores zero, which is exactly the failure a text-similarity grader would miss.

Where the WebArena leaderboard stands

The shape of that table matters more than the top line. When the paper was published, the best GPT-4 agent reached 14.41% and the authors' point was the size of the gap. The second row, at 74.3%, was the top row when we read the sheet on 12 October 2026, and the climb to it took about two and a half years on an environment that has not moved.

The new top row needs reading with care. The sheet lists Narada AI Code at 86.1%, dated September 2026, with its own team as the result source and its "Open?" column marked no. Narada described it in a blog post dated 16 October 2026: the score is its Operator agent running workflows built by a second product, Narada Code, and the post names no underlying model, run count or evaluation protocol. We found no paper or released trajectories as of 5 November 2026; our Narada AI page sets out what is public. That is not evidence of anything wrong with it. It does mean the one score above the human figure is also the one a reader can do least to check, and a Berkeley team has shown that the original grader can be gamed (see the next section). Just below the top five, two more rows, DeepSky Agent at 66.9% and an earlier Narada AI entry at 64.2%, are marked self-reported.

The rest of the top rows are not frontier models run as-is either. They're scaffolds, memory schemes and fine-tunes built for this environment. A row here tells you what a configuration achieved, not what a product does.

The sheet is still being added to. On 5 November 2026 it held 48 rows, the newest dated September 2026, which is unusual for a 2023 benchmark.

WebArena-Verified, and which WebArena a score refers to

WebArena-Verified is a re-release from a ServiceNow team, presented at a NeurIPS 2025 workshop with a paper and made public in December 2025; its latest release on GitHub is v1.2.3, from 7 February 2026. It keeps the same 812 tasks and the same kinds of site, but every task, reference answer and evaluator was reviewed by hand and corrected. Scoring is deterministic: it drops the model-as-judge checks and the loose substring matching of the original, and it can score a run offline from recorded network traffic. It also offers a 258-task "Hard" subset for cheaper runs.

That matters because the original grader was loose enough to be exploited. A Berkeley team reported in April 2026 that an agent could read the reference answers straight out of the task configuration and reach roughly 100% on all 812 tasks without doing any of them. The leaderboard above is the original sheet. We found no official leaderboard for the Verified release, and its scores, wherever they appear, are not comparable with this table.

WebArena-Pro is a third, separate variant. ServiceNow's July 2026 workshop publication describes 300 tasks across 20 self-hosted applications, including audio and video observations. That count is the July snapshot; we could not confirm a later total. So a WebArena score can mean the original 812 tasks, the corrected Verified release, its Hard subset, or Pro, and it is only meaningful next to the name of the set.

WebArena Infinity is something else again, and not a version of the 812 tasks at all. Shuyan Zhou of Duke University, first author of the original WebArena, published it in March 2026 as a blog post rather than a paper, under the same GitHub organisation and an MIT licence. It is a pipeline that builds imitation web apps and their tasks automatically. The first release has ten apps, among them two imitations of Gmail, a PayPal wallet, GitLab and Xero, with 1,260 tasks between them. Grading reads each app's stored state through an API instead of inspecting the rendered page. There is no leaderboard: the post reports three agents its author ran, led by Gemini-3-Flash with Browser Use at 69.3% (873 of 1,260), and the repository has not changed since 24 March. WebArena Infinity post, repository.

Our own exam imported the 60 tasks of its Email Management app, one of the Gmail imitations, on 14 October, as the exam changelog records. They are not in the fixed draw and we have no product results on them, so there is no score to report. Two things limit what a score would mean. The tasks are mailbox housekeeping, 20 each at easy, medium and hard: starring and archiving, labels, filters and settings, up to multi-step setups such as a filter that labels and marks invoices as read. None asks the agent to compose, reply to or send a message; the nearest is a filter that forwards matching mail. And the app is a web page that looks like Gmail, not Gmail. An assistant that reaches your mail through Google's own connection cannot reach it, so only products that drive a browser can attempt these tasks, and a result would measure operating a Gmail-like page rather than handling your email.

WebArena's human baseline is not measured on the same tasks

That 78.24% gets quoted constantly, including by us, and almost nobody quoting it says how it was produced.

Five computer science graduate students were each given one sampled task from each of the 170 templates the tasks were generated from. They completed 78.24% of them, taking an average of 110 seconds per task, and did better on tasks that required changing something (81.32%) than on tasks that required finding something out (74.68%). The failures were mostly misreadings: answering with travel distance when asked for travel time, giving a name when asked for a name and an email, filling in half of a product's details.

So the human figure and the agent figures are not the same measurement. One is five people on roughly 170 sampled tasks; the other is a system on all 812. The sheet's own note on the human row says "Selected tasks by templates", and that note survives almost no citation. Treat the gap as a signal about difficulty, not a like-for-like scoreline, and be sceptical of anyone announcing that agents have passed humans on WebArena without mentioning any of the above. With a row now above 78.24%, that announcement is easy to make.

Unachievable tasks, and the trick they catch

Some WebArena tasks cannot be completed, on purpose. The information isn't there, the permission isn't there, or the site simply doesn't do that.

This is the most quietly clever thing in the benchmark, because it tests the behaviour that makes an agent safe to leave alone: knowing when to stop. An agent that confidently reports success on an impossible task is worse than useless, since you'd have no reason to check.

The paper's numbers on this are striking. Told explicitly to stop when a task looks impossible, GPT-4 correctly gave up on 77.78% of the unachievable set but its overall score dropped, because the same caution made it abandon achievable tasks too. Without the hint it scored higher overall and identified only 44.44% of the impossible ones. The humans got 100% of them. The sheet still carries both GPT-4 rows, 14.9% without the hint and 11.7% with it.

That trade-off was measured on one model in one experiment. We have not run an equivalent comparison across the products on our scorecard, so we cannot say how each of them handles it.

What WebArena can't tell you about a personal agent

The sites aren't yours, and they hold nothing about you.

That's the structural limit, and a future version couldn't patch it without becoming a different benchmark. The shop has no saved card of yours, no delivery address you've used twice, no history of the three things you always reorder. Each task starts from a reset environment with seeded accounts, so nothing an agent learns in task 200 is available in task 201, and a product that remembers you scores exactly what a product that forgets you scores.

Consider what that removes. A standing preference is worth nothing. A correction you made last week is worth nothing. A routine that fires every Monday without being asked cannot be expressed as a WebArena task at all.

There's a second gap. Every WebArena task starts with a person typing an instruction. Nothing in the benchmark starts because a clock reached a time or an email arrived, while at least one product we've taken apart, Town, has routines that a clock, an email or a calendar entry can start.

And a third, which is about consequence. WebArena's shop takes no real money. An agent that cancels the wrong order in a Docker container has caused no harm, so nothing in the score reflects the asymmetry between a wrong answer and a wrong action. That asymmetry is most of what makes a personal agent worth being careful about.

Where these WebArena numbers came from

We didn't read any of the figures above off a screenshot or a summary.

The 812 task count is a count: the repository's own task configuration file, fetched and counted on 22 September 2026, which also gives the per-site distribution. The leaderboard table is the sheet the project page links to, exported as CSV and sorted locally on 22 September, 12 October and 5 November 2026, with the open and self-reported flags taken from the sheet's own columns. The paper figures, including the human baseline and the unachievable-task results, come from the paper itself.

One thing to flag if you go looking. The project site has become WebArena-x, a suite covering several benchmarks, and the original WebArena project page now lives one level down at /og/. That page still carries the leaderboard link, which is the only place the current numbers exist.

WebArena answers "can this system operate a website it has never seen, reliably enough to be graded on the result". That's a real question, answered rigorously, and it isn't the question you're asking when you decide whether to give something your email and a payment method. Those questions are on our scorecard: where the model runs and what it can reach, how a tool definition gets in front of it, where your memory is kept and whether you can correct it, what you get to approve, and how you reach it. Read how we score, start with a teardown, or see what the other agent benchmarks measure.

FAQ

What is the WebArena benchmark? A self-hosted web environment with 812 tasks across a shop, an admin back office, a GitLab instance, a forum and a map, released by Carnegie Mellon researchers in July 2023. Agents are graded on whether the site ended up in the right state, not on what they said they did.

What is the highest WebArena score? On 5 November 2026 the top row of the official sheet is Narada AI Code at 86.1%, dated September 2026 and not marked as open. The highest entry marked open is WebTactix's DeepSeek v3.2 setup at 74.3%. The original paper's best GPT-4 agent managed 14.41%.

Have AI agents beaten the WebArena human baseline? One sheet entry now scores above the 78.24% human figure, but that figure comes from five students on about 170 sampled tasks, not the 812 agents run, so the two are not a like-for-like comparison.

Is WebArena still maintained? The original sheet still receives submissions, the newest dated September 2026. A separate WebArena-Verified release fixed the task evaluators, and its scores aren't comparable with this sheet.

What is WebArena-Verified? A re-release of the same 812 tasks by a ServiceNow team, with every task, answer and evaluator audited by hand, deterministic scoring and a 258-task Hard subset. It was made public in December 2025.

How is WebArena different from GAIA? GAIA asks multi-step research questions and grades the answer string. WebArena asks an agent to change something on a website and grades the website afterwards. Both now have a leaderboard entry above their published human figure.

Can I run WebArena myself? Yes, which is unusual and is the reason its results can be reproduced. The sites ship as Docker images and the task set is a configuration file in the public repository, so a published score can be checked rather than believed.

The other benchmark pages

• GAIAEvery question arrives cold. Nothing carries over from the last one, so a system that remembers you scores exactly the same as one that doesn't.
• BrowseCompReading, not doing. Nothing is booked, cancelled, paid for or sent.
• OSWorldSkill, not memory. It asks whether an agent can operate the machine, never whether it knows why you'd want it to.
• Tau-benchThe user is simulated and the domains are narrow, but this is the closest thing on the list to what a personal agent actually does.
• AgentBenchNone of the environments is your life. No calendar, no inbox, no bill to dispute.
• Terminal-BenchAlmost nothing a personal agent does happens at a terminal.
• SWE-benchA closed problem with a written specification. An errand has neither, and it cannot be rolled back.
• AssistantBenchStill one-shot. No standing preferences, no routine that fires next Monday.