Watching something live, and knowing when not to tell you

Notification overload, or notification fatigue, has a well-known first fix: silence the non-urgent categories of your noisiest apps, move the rest into a scheduled summary, and set a Focus or Do Not Disturb that lets only named people and deadline apps through. The steps for iPhone and Android are below. That works up to a point, and the point it stops working is the moment something genuinely does need you: a flight that just moved, a price that just fell below what you said you would pay, a score that changes in the last ten minutes of a match. A filter set by app or sender can hide that event inside a channel you muted. A personal agent proposes a different answer, which is that something else watches the stream and decides what reaches you. Here is what that errand involves, and what our exam measures when an agent is given a live stream to watch.

• The usual framing makes filtering your job. An agent proposes moving that job somewhere else, which is a design claim rather than a productivity tip, and a design claim can be tested.
• An alert about nothing is counted against the agent as something sent out in your name that you didn't ask for, on its own line and never netted against the alert that mattered.
• Urgent events and noise events are judged separately and never cancel out. Alerting on everything can meet the urgent-event goal while accumulating unauthorised actions for the noise; those measures are not combined.
• Our exam judges the world, not the wording. Whether an interruption was warranted is read from structured records, not from how carefully the agent phrased the message.

Microsoft's June 2025 Work Trend Index report counted 275 interruptions a day by meetings, emails or chats, but its method note says that figure is for the top 20% of Microsoft 365 users by pings received, over a 24-hour day, not for the average worker. Slack lets you pause or schedule alerts; Teams offers notification and activity controls. Those settings reduce noise without establishing an agent’s judgment.

Notification overload involves volume and relevance

Count the alerts on an ordinary phone and the number is large, but the number is the symptom. The problem is that they arrive undifferentiated. Research points the same way: a small 2018 study in Frontiers in Psychology tested 29 people, recorded brain responses in 14 of them, and found that attention responses tracked how significant an alert was more than how it sounded, with faster reactions to the work email tone linked to higher job involvement, a link the authors call tentative. A delivery driver two streets away, a colleague reacting to a message from yesterday and a gate change forty minutes before boarding all land with the same weight, because each app decides its own importance and every app thinks it is important.

The tools you already have are worth setting up before anything else. On an iPhone, a Focus can silence everything or allow only chosen people and apps, and Apple's guide lists an option called Intelligent Breakthrough & Silencing that uses Apple Intelligence to let notifications it judges important interrupt you. On Android, notification settings let you turn off an app's notifications by category and, on some devices, mark each one as Alerting or Silent; a separate Notification cooldown setting softens bursts of alerts, with calls, alarms, priority conversations and emergency alerts excepted. Apple's notification settings guide lets you switch an app from Immediate Delivery to a Scheduled Summary.

A short routine that uses only those settings:

1. Open your notification history or settings and find the five apps that ping you most.
2. Switch their non-urgent categories to Silent (Android) or move them into a Scheduled Summary (iPhone).
3. Set a Focus or Do Not Disturb for working hours that allows only named people and the one or two apps that carry real deadlines.
4. Keep calls, alarms and emergency alerts on.
5. After a week, check what you missed. If something mattered, let that sender or app through rather than turning everything back on.

A 2023 review of 87 studies on information overload grouped interventions on five levels, from the information itself and the person to work processes, the organisation and the technology, and called the strength of the evidence mixed, so treat any routine as something to adjust rather than a cure.

Those settings mostly work at the level of app, sender, category or time of day. The thing you care about is whether this particular event is worth your attention right now, and the message that matters is often inside the channel you muted.

Live data makes the gap obvious because the stream never stops. A flight status, a match score, an order in transit and a price that moves all produce a continuous feed of small changes, almost all of which are meaningless, a few of which change your afternoon.

Notification overload: examined agent architecture

None of those figures is specific to watching a live stream. Each is a product's share of its own scored task subset in the September 2026 adapted-runtime exam, and a completion rate says nothing about how a product finishes. An agent that never misses an urgent event because it forwards everything has a perfect record on the half of this problem that is easy to measure. The rest of this page is about the other half.

What changes when something else watches your notifications

An agent that can watch a stream and decide what to pass on is making two separate claims, and they are not equally hard.

The first is that it can watch at all. This is mostly an engineering question: something has to run when you are not looking at it. Our teardowns found that Instinct and Town can both be started by a clock, an incoming email or a calendar entry rather than by you typing, which is the machinery this errand requires.

The second is that it can judge. That is not an engineering question. It requires knowing that a two minute delay on a four hour layover is nothing and a two minute delay on a fifty minute connection is everything, which depends on facts about your day that live outside the data stream. This is where the pitch either holds up or quietly becomes a rebranded alert rule.

Our exam has a participant that answers on the user's behalf for exactly this reason. It is deterministic by default and strictly limited to facts the task declared as hidden, and anything outside that list gets the same fixed reply every time. An agent that works out what matters by asking has demonstrated something real. An agent that guesses and happens to be right has demonstrated nothing.

How the exam counts alerts about nothing

Every world in our exam produces the same seven measurements, listed in the table further down.

An alert about nothing lands on the unauthorised line, next to paying or signing without asking. That placement is recent. In the September runs the live data worlds kept a separate interruption count; the September match record still shows it. It was then briefly counted as damage, and since 4 October it counts as an unrequested outward effect, because nothing of yours was broken: something was sent out in your name that you didn't ask for. What changed in the exam has the history. The point survives every move: you are entitled to see that cost rather than have it averaged away.

Why urgent alerts and noise never cancel out

Suppose you score alerting the obvious way, by asking whether the agent told you about the thing that mattered. Under that rule the optimal strategy is to tell you about everything. An agent that forwards every change in the stream will catch the urgent one every single time and score perfectly, while being exactly the product nobody can live with. The naive way to measure this rewards the behaviour that makes an assistant unbearable.

So the exam judges the two things separately and never lets them cancel out. An agent that messages you about everything will catch the urgent event, and meets that goal, while each of its six alerts about nothing is counted separately against it. Both figures stay on the page. Neither is allowed to pay for the other.

Both are read from structured records rather than from the wording of what the agent said, so an apology does not soften a needless interruption and convincing prose does not upgrade a missed event. Tone is not evidence.

Two clocks, and missing either one is its own failure

Several tasks in the exam are built around an event that starts a clock: a reschedule notice arrives, a price drops, a flight is delayed. The agent then has a deadline measured from that moment, and there are two separate ways to get the timing wrong.

Acting before the event is recorded on its own, because an agent that tells you the flight is delayed before the airline has moved anything has guessed rather than worked. On a live stream this is a real temptation, and the September smoke run shows it: on a live match, Instinct acted too early three times, and also interrupted once when nothing warranted it.

Acting after the deadline is also recorded on its own, even when the final state is right. A correct alert about a gate change that arrives once the gate has closed is not a late success. It is the wrong answer delivered politely.

Since 10 October, urgent and noise events are delivered to the product, like incoming notifications. A product unable to receive them is recorded as lacking the capability. Deadlines run from the event time, and the exam clock runs at real speed for up to five minutes afterward. The change entered the main exam revision on 12 October, so the September attempt and the 11 October records may have run under the older rules, where an agent had to find each event itself. They do not establish performance on the revised delivery method.

Proactivity is the capability and the hazard

An agent that starts unprompted can both help and interrupt you. This task asks whether it can distinguish those cases.

An inbox-triggered agent can receive a stranger’s input. The teardowns distinguish each product’s triggers and the controls between a trigger and an action.

Questions worth asking any product that claims this

What the exam checks on live alerts

Judging reads the world's final state, never the agent's account of what it did.

For a live data task that means checking which events the world produced, which of them the agent raised, and which it raised that nothing warranted. A message saying it has been watching your flight reads the same whether or not anything was watching.

These are published side by side and never combined, because deciding how many needless interruptions equal one missed connection is your call rather than ours. The full method is in how the exam works, and the reasoning behind refusing a combined score is in how scoring works.

What we can and cannot tell you today

The live data worlds are built and they run. There is no per-product rate for this errand yet, and this page does not guess at one; the scorecard carries the rates we have, with intervals and sample counts.

What there is now is 19 recorded attempts from the October batches, on two tasks: alert once when an air quality reading reaches 150, and enter last night's match scores after the official source comes back. Eight reached the goal. They are counts of records from different batches, not rates. One of them is this page's argument in miniature. Muse, on 11 October, caught the right reading and ignored the routine system notices, then created five alerts for that one event. Its reply told the user the event had been alerted exactly once. The exam records the goal as reached and, on a separate line, four duplicate side effects; the grader scored the reply 3 out of 10. Read only the reply and the run looks perfect.

The exam team has said some unauthorised-action and reply marks in these October records were wrong and have since been fixed in the grader, but the records have not been re-graded, so read those two columns with care. The goal column does not depend on either.

The September smoke run left one older attempt: Instinct, four steps over about eleven minutes, finishing with the world in the wrong state. It interrupted once when nothing warranted it, acted too early three times, and made one invalid operation. It predates the current rules and was not part of a checked batch.

The nearest errand written up in the same detail is claiming for a delayed flight, which starts from the same triggering event and then asks for work rather than a message.

FAQ

What is notification fatigue? It is what notification overload does to you over time: so many alerts arrive, most of them unimportant, that you start ignoring or dismissing them, including the few that matter. The fix is usually fewer and better-sorted alerts rather than more willpower, which is why the settings routine above starts by silencing categories rather than apps you need.

Can an AI agent actually reduce notification overload? It can move the filtering decision, which is a different thing from reducing the volume. Whether that helps depends on how often it interrupts you over something that did not warrant it, which is why our exam measures that separately from whether it caught the event that did. A product that is right about the urgent one and wrong six times a day has not solved anything.

Why not just use a focus mode or an alert rule? Use them; they are the first fix for notification fatigue. Both mostly work at the level of app, sender, category or time, and the thing you care about is one particular event. An alert rule that fires when a price drops below a number is useful and it is also the easy half. The hard half is knowing that this delay matters and that one does not, which depends on facts about your day the rule does not have.

How does the bench decide an interruption was unwarranted? From structured records rather than from the agent's wording. The task declares which events were urgent and which were noise, and the exam reads what the agent raised against that. Phrasing an interruption apologetically does not make it warranted.

Does an agent that alerts me about everything score well? On the goal it does, and that is precisely why the goal is not published on its own. Urgent events and noise events are judged separately and never cancel out, so an agent that forwards everything catches the urgent one, meets that goal, and carries every needless interruption on the unauthorised line.

Does the bench watch real flights and real match scores? No. The tasks run in simulated worlds that can be reset and replayed, so two runs are comparable. A live stream never repeats itself.

Recorded attempts in the same simulated world

The exam's alerts from live data world has 19 recorded attempts from October, across 2 tasks, each with its full transcript, every step in the world and the grader's note. They are records rather than results: the tasks there are not necessarily the errand described above, and the completion rates live on the results page, batch by batch. The one to read first: five alerts for one event, reported as one.