--- title: "AI agent teardown: how we take a personal agent apart" description: "Our AI agent teardown method: real accounts used properly, behaviour observed from outside, findings published and never the material behind them." url: https://personalagentbench.com/how-we-take-them-apart/ last_updated: 2026-10-16 --- # How we take a personal agent apart An AI agent teardown, as we use the term, maps a shipping agent's parts from the outside: where it runs, which tools it can call, what it remembers, what it asks before acting, and how you reach it. That sounds like it should involve something clever. Mostly it involves getting a real account (paid where the product charges), using the product properly for long enough that it holds state about you, and writing down what the interface is willing to tell you. - Every teardown finding on this site was found from the outside, using the product the way an ordinary customer uses it. No interception, no account sharing, no getting anything out of anyone. - We publish findings and never the material behind them: no instruction text, no internal documents, no schemas. - Anything we could not establish goes in the last row of the scorecard, in the same typeface as the rows that flattered us. - Repeated observations support broader findings. Single-call discrepancies, such as Pine’s event and credit counts, remain single-case evidence. - Automated gates stand between a draft and this site. The two that guard what we publish have both blocked real material rather than sitting there decoratively. ## A personal agent's surface is the product The useful realisation behind our account-based approach is that a personal agent has to tell you a great deal in order to be usable at all. It has to show you what it is doing, or you would not trust it with a phone call. It has to name the tools it is using, or its progress messages would be gibberish. It has to explain why it stopped, or you would not know whether to intervene. Every one of those is a design requirement, and every one of them leaks structure. So we use the product. We give it work. We watch what it says while it does that work, what it refuses, what it asks permission for, how it describes its own failures, and what it leaves behind afterwards. Then we do it again with the variable changed. None of that requires access to anything that is not ours. ## The layers an AI agent teardown separates The [scorecard](https://personalagentbench.com/) separates six layers, and [how we score](https://personalagentbench.com/how-scoring-works/) explains why they matter. - **Execution:** where work runs and what it can touch. - **Tools:** which actions are available and how their definitions reach the model. - **Memory:** what persists, where it lives and whether the user can inspect it. - **Approvals:** which actions require permission and how that permission is enforced. - **Channel:** where you reach the agent, including message, clock and event triggers. - **Unknowns:** what we could not establish. ## What a teardown actually involves, in order **Get a real account and use it properly.** Paid where the product charges, invited where it is invite-only. Not a demo account, and not for an afternoon, because memory takes time to show up. When Dhravya Shah probed Instinct's memory, one preference he gave it took about 23 hours to show up as a change in its memory files, by [his own account](https://supermemory.ai/blog/reverse-engineering-instinct-memory/). An hour-long trial would have recorded that the product forgot. **Give it work that has edges.** A task that succeeds tells you less than a task that nearly succeeds. Ask for something at the boundary of what it can do and it will tell you where the boundary is, usually in the process of apologising for it. **Change one thing.** The difference between a single observation and a finding is a controlled second run. If a product behaves differently when a tool is unavailable, that difference is the finding, and it needs both halves. **Count what is countable.** Tool names, action counts, section counts, time between a request and a reconciliation. Numbers are the part of a teardown that other people can check, which is exactly why we prefer them and why we publish how we counted. **Write down what refused to resolve.** Every teardown here ends with a list of things we tried to establish and could not. That list is not an apology; it is the part that makes the rest credible. ### One AI agent teardown from start to finish: Pine Here is how those steps played out on one product, using only what the [Pine teardown](https://personalagentbench.com/agents/pine/) already publishes. The account was a real one, on a product that charges for its time; [Pine's pricing page](https://www.19pine.ai/pricing) sells personal plans by the week with a monthly credit allowance. Pine sells more than one thing under its name, so the first decision was scope: we examined the bill-dispute product and said so, rather than letting one quarter of the brand stand in for the rest. The work with edges was the job Pine exists for, a billing call to a company. One of those calls turned into a three-way identity check, and that is where the useful evidence came from. We watched 62 events go by live; the saved history afterwards held 30 rows. The same call reported 70 credits in its final state and took 74 off the balance. Neither number is dramatic. Both are counts, anyone with an account can make the same comparison on their own calls, and both say something about what the record you keep is worth. Counting gave the rest. The client shows 26 tool names and none of their schemas, which is the kind of thing a product states outright simply by displaying it. One tempting reading had to be thrown out: the task cards carry a field called `prompt`, and it turned out to be the user's own message, not Pine's instructions. Then the list of what refused to resolve. The production instruction text came back empty across roughly 126 probes. The calling tool's schema, the carrier that places the calls and whether work is split between a fast and a slow model all went into the last row of the [scorecard](https://personalagentbench.com/), where Pine's column says exactly that. ## What we do not do in a teardown This matters more than the previous section, so it is stated in the plainest way we can manage. We do not intercept traffic that is not ours to read. We do not use anyone else's account, and we do not ask anyone inside these companies for anything. We do not obtain or use credentials. Checking whether a door is closed, from inside our own account, is part of the work; when it is, we record that as a finding and do not break it open. Pine's instruction text is the clearest case: roughly 126 attempts came back empty, and the [Pine teardown](https://personalagentbench.com/agents/pine/) reports that as a fact about the product rather than hiding it as a gap in our work. We also do not publish the material itself. Instruction text, internal documents, schemas and prompts stay where they are. What gets published is what we concluded, restated in our own words. That rule costs us the most quotable parts of every teardown and we keep it anyway, because a comparison site that redistributes other companies' material has swapped one kind of credibility for another. ## Why findings and not artefacts The tempting version of this site is the one that prints the artefacts. It would be more dramatic, it would travel further on social media, and it would be worth less. A leaked prompt is a snapshot. It is accurate for as long as nobody edits the file, which for a product shipping weekly is about a week. A finding about structure survives the edit: knowing that a product loads tool definitions one namespace at a time is still true after the namespaces change, and knowing that another keeps no verbatim record of what was said on your behalf is still true after the summary format changes. There is also the question of what we would be asking you to trust. An artefact invites you to verify it against a copy you cannot obtain. A finding invites you to get the product and watch it behave the way we said it does, which is a check anyone reading this can actually run. ## The gates, and what they have caught Several automated checks run before anything is pushed to this site. Some enforce writing and search rules; two guard what we publish, and neither is decorative. The first compares every draft against our own private research notes and refuses anything that overlaps them beyond a short phrase, along with anything that looks like a credential, an internal address or a trace of how material was obtained. Translated pages go through the same check, because a translation of a private note is still the private note. The second governs anything we restate from other people's published work. It requires a source you can open, checks that the platform we claim matches the address, caps quotes at twenty-five words, rejects a restatement that merely contains its own quote, refuses second-hand hedging with no named source, and refetches every cited source to confirm it still resolves. Both have regression suites whose job is to prove they still block. A gate nobody has watched refuse something is worth about as much as the assumption it replaced. ## Three evidence labels in an AI agent teardown, one excluded from publication Our research notes mark every claim with one of three labels, and only two of them are allowed onto this site. **Stated outright** means the product says it, in words, somewhere a customer can read. A tool name in an interface, a refusal message, a count in a settings page. These are the cheapest claims to make and the easiest for you to check, which is why the pages lean on them hardest. **Reconstructed** means several separate observations agree and no single one of them says it. That a request is assembled on your own machine rather than fetched ready-made is a reconstruction: no screen announces it, and several behaviours only make sense if it is true. Reconstructions carry more risk than stated facts, so they are written in language that shows they are inferences rather than quotations. **Inferred** means it is our best guess and nothing has confirmed it. This label never reaches the site. Not in a hedged sentence, not in a footnote, not as "appears to". If the only thing supporting a claim is that it would make sense, the claim does not go up, and if it matters it goes in the list of what we could not establish instead. That third rule is the one that costs most. Every teardown here has interesting guesses attached to it that you will never read, and the pages are duller and more reliable for it. ## When somebody else gets to an AI agent first We are not the only people doing this, and pretending otherwise would be silly. On 20 September Dhravya Shah, the founder of supermemory, [published a teardown](https://supermemory.ai/blog/reverse-engineering-instinct-memory/) of [Instinct's](https://personalagentbench.com/agents/instinct/) memory layer, reached entirely by probing it over iMessage, and [posted it on X](https://x.com/DhravyaShah/status/2101745550752428340). He arrived at the same structure we had: memory kept as git-tracked markdown, looked up by grep-style keyword queries rather than vector search, and ingested by a pass he infers runs once every 24 hours. He labels the result a reconstruction that may contain errors, and it describes the categories of context the model receives without reproducing any instruction text. It runs the other way too. Every's [side-by-side comparison of personal agents](https://every.to/personal-agents-comparison) lists our Instinct teardown among its sources, which is the use we hoped findings would get: somebody else's evidence, checked and built on, with the link left in. Two teams working from different angles and agreeing is worth more than either result alone, so when it happens we say so and link to the other one. Our [radar](https://personalagentbench.com/radar/) picks up published work like that, restates it with the author's name on it, and points at the original. ## What a teardown cannot tell you about an AI agent Behaviour observed from outside is still behaviour observed from outside. We can tell you a product exposes twenty-six tool names and no schemas. We cannot tell you what those schemas contain. We can tell you a request is assembled on a machine rather than fetched, because the assembly is observable. We cannot tell you what was in the request that left, because we never saw it leave. Structure is inferable. Contents usually are not. Every teardown here separates those two things, and where a conclusion rests on several observations agreeing rather than on one thing stated outright, it is written to make that visible rather than to sound more certain than it is. The [scorecard](https://personalagentbench.com/) separates evidence types, and the exam is a different kind from a teardown. The September completion rates came from the exam team as totals with no public batch record, and the older 114-task comparison came with transcripts we have not republished. The October batches are the first with every attempt behind them: 245 attempts are on the [recorded runs](https://personalagentbench.com/runs/) pages, and we checked each published October rate against them. All of those results come from replicas of each product run on a substituted model, so they say how the runtime copes with the errands, not what we saw in the consumer app. Public-source product pages identify their sources, and teardown findings describe our own observations of the real product. ## FAQ **Is reverse engineering an AI product legal?** What we do here is use products as customers and record how they behave, which describes our access boundary, not a legal conclusion. Contract terms, jurisdiction and the particular activity still matter. We do not intercept traffic, use other people's accounts, or obtain credentials, and we do not republish material that belongs to the companies we examine. **Why not publish the prompts you found?** Because a prompt is accurate until somebody edits the file, and because redistributing another company's material would trade one kind of credibility for another. Findings restated in our own words survive product changes and can be checked by anyone with an account. **How do you distinguish a repeated finding from a single case?** We seek repeat observations or explicit product statements. Some published findings, including Pine's call-history and credit discrepancies, come from one recorded case and should be read at that scope. They do not establish a frequency or universal behaviour; failures to reproduce remain part of the uncertainty. **Can I check any of this myself?** Yes, and that is the point of working this way. Our observed-behaviour claims are bounded by the account and surface examined. Account flags, unavailable integrations and inaccessible model traffic can limit reproduction; the pages identify those gaps. Counts are reproducible only with a matching surface and counting rule. **What happens when a product changes?** The teardown goes stale, which is why each page carries the date it was last verified. We would rather mark a page than quietly let it age into fiction. **Do you accept corrections?** Yes. Email [official@zentor.ai](mailto:official@zentor.ai) with the page and the claim. If something here is wrong we will fix it with the date on it, and say what changed rather than editing quietly. ### Reading a teardown once you know how it was made Knowing the method changes how the pages read. A number is countable and was counted. A structural claim was reproduced. A sentence about what we could not establish is a real boundary rather than modesty. Start with the [scorecard](https://personalagentbench.com/) for all of them side by side, the [teardowns](https://personalagentbench.com/agents/) for one product at a time, or [how we score](https://personalagentbench.com/how-scoring-works/) for what the six axes are and why those six. --- Published by Personal Agent Bench (https://personalagentbench.com), run by the Zentor team. Zentor builds agent software of its own, so its products are not on the scorecard and will not be added.