How we take a personal agent apart
Reverse engineering an AI agent sounds like it should involve something clever. Mostly it involves buying an account, using the product properly for a week, and writing down what the interface is willing to tell you.
- Everything on this site was found from the outside, using the product the way a paying customer uses it. No interception, no account sharing, no getting anything out of anyone.
- We publish findings and never the material behind them: no instruction text, no internal documents, no schemas.
- Anything we could not establish goes in the last row of the scorecard, in the same typeface as the rows that flattered us.
- A single observation is not a finding. Where we could not reproduce something, we say so or leave it out.
- Two automated gates stand between a draft and this site, and both of them have blocked real material rather than sitting there decoratively.
The surface is the product
The useful realisation, and the one that makes all of this legal and most of it easy, is that a personal agent has to tell you a great deal in order to be usable at all.
It has to show you what it is doing, or you would not trust it with a phone call. It has to name the tools it is using, or its progress messages would be gibberish. It has to explain why it stopped, or you would not know whether to intervene. Every one of those is a design requirement, and every one of them leaks structure.
So we use the product. We give it work. We watch what it says while it does that work, what it refuses, what it asks permission for, how it describes its own failures, and what it leaves behind afterwards. Then we do it again with the variable changed.
None of that requires access to anything that is not ours.
What we actually do, in order
Buy the thing and use it properly. Not a trial, not a demo account, and not for an afternoon. A product behaves differently in week two, when it has state about you, than it does in the first hour. Several of the findings on this site only appear once there is history.
Give it work that has edges. A task that succeeds tells you less than a task that nearly succeeds. Ask for something at the boundary of what it can do and it will tell you where the boundary is, usually in the process of apologising for it.
Change one thing. The difference between a single observation and a finding is a controlled second run. If a product behaves differently when a tool is unavailable, that difference is the finding, and it needs both halves.
Count what is countable. Tool names, action counts, section counts, time between a request and a reconciliation. Numbers are the part of a teardown that other people can check, which is exactly why we prefer them and why we publish how we counted.
Write down what refused to resolve. Every teardown here ends with a list of things we tried to establish and could not. That list is not an apology; it is the part that makes the rest credible.
What we do not do
This matters more than the previous section, so it is stated in the plainest way we can manage.
We do not intercept traffic that is not ours to read. We do not use anyone else's account, and we do not ask anyone inside these companies for anything. We do not attempt to obtain credentials, and where a product deliberately closed a door, we record that it is closed rather than looking for a window. Two of the four products on this bench refused us something specific, and in both cases the refusal is written into the teardown as a finding about the product rather than a gap in our work.
We also do not publish the material itself. Instruction text, internal documents, schemas and prompts stay where they are. What gets published is what we concluded, restated in our own words. That rule costs us the most quotable parts of every teardown and we keep it anyway, because a comparison site that redistributes other companies' material has swapped one kind of credibility for another.
Why findings and not artefacts
The tempting version of this site is the one that prints the artefacts. It would be more dramatic, it would travel further on social media, and it would be worth less.
A leaked prompt is a snapshot. It is accurate for as long as nobody edits the file, which for a product shipping weekly is about a week. A finding about structure survives the edit: knowing that a product loads tool definitions one namespace at a time is still true after the namespaces change, and knowing that another keeps no verbatim record of what was said on your behalf is still true after the summary format changes.
There is also the question of what we would be asking you to trust. An artefact invites you to verify it against a copy you cannot obtain. A finding invites you to buy the product and watch it behave the way we said it does, which is a check anyone reading this can actually run.
The gates, and what they have caught
Two automated checks sit between a draft and this site, and neither is decorative.
The first compares every draft against our own private research notes and refuses anything that overlaps them beyond a short phrase, in either English or Chinese, along with anything that looks like a credential, an internal address or a trace of how material was obtained. A translation does not get a pass; the check works on both languages for exactly that reason.
The second governs anything we restate from other people's published work. It requires a source you can open, checks that the platform we claim matches the address, caps quotes at twenty-five words, rejects a restatement that merely contains its own quote, refuses second-hand hedging with no named source, and refetches every cited source to confirm it still resolves.
Both have regression suites whose job is to prove they still block. A gate nobody has watched refuse something is worth about as much as the assumption it replaced.
The three labels we use internally, and why two of them never reach you
Our research notes mark every claim with one of three labels, and only two of them are allowed onto this site.
Stated outright means the product says it, in words, somewhere a customer can read. A tool name in an interface, a refusal message, a count in a settings page. These are the cheapest claims to make and the easiest for you to check, which is why the pages lean on them hardest.
Reconstructed means several separate observations agree and no single one of them says it. That a request is assembled on your own machine rather than fetched ready-made is a reconstruction: no screen announces it, and several behaviours only make sense if it is true. Reconstructions carry more risk than stated facts, so they are written in language that shows they are inferences rather than quotations.
Inferred means it is our best guess and nothing has confirmed it. This label never reaches the site. Not in a hedged sentence, not in a footnote, not as "appears to". If the only thing supporting a claim is that it would make sense, the claim does not go up, and if it matters it goes in the list of what we could not establish instead.
That third rule is the one that costs most. Every teardown here has interesting guesses attached to it that you will never read, and the pages are duller and more reliable for it.
When somebody else gets there first
We are not the only people doing this, and pretending otherwise would be silly.
In September the founder of supermemory published a teardown of one product's memory layer, reached entirely by probing its messaging interface, and arrived at the same structure we had: version-controlled markdown, keyword lookup rather than vector search, reconciliation running about once a day. That post passed 280,000 views without reproducing a single line of the product's own material.
Two teams working from different angles and agreeing is worth more than either result alone, so when it happens we say so and link to the other one. Our radar picks up published work like that, restates it with the author's name on it, and points at the original.
What this method cannot tell you
Behaviour observed from outside is still behaviour observed from outside.
We can tell you a product exposes twenty-six tool names and no schemas. We cannot tell you what those schemas contain. We can tell you a request is assembled on a machine rather than fetched, because the assembly is observable. We cannot tell you what was in the request that left, because we never saw it leave.
Structure is inferable. Contents usually are not. Every teardown here separates those two things, and where a conclusion rests on several observations agreeing rather than on one thing stated outright, it is written to make that visible rather than to sound more certain than it is.
FAQ
Is reverse engineering an AI product legal? What we do here is use products as customers and record how they behave, which is ordinary evaluation. We do not intercept traffic, use other people's accounts, or obtain credentials, and we do not republish material that belongs to the companies we examine.
Why not publish the prompts you found? Because a prompt is accurate until somebody edits the file, and because redistributing another company's material would trade one kind of credibility for another. Findings restated in our own words survive product changes and can be checked by anyone with an account.
How do you know a finding is real and not a one-off? A single observation is not a finding here. Anything that made it onto these pages either reproduced across runs or was stated outright by the product. Where neither applies, it appears in the list of things we could not establish.
Can I check any of this myself? Yes, and that is the point of working this way. Every claim is about behaviour you can observe by using the product. The numbers are the easiest place to start, because counting is reproducible.
What happens when a product changes? The teardown goes stale, which is why each page carries the date it was last verified. We would rather mark a page than quietly let it age into fiction.
Do you accept corrections? Yes. If something here is wrong we will fix it with the date on it, and say what changed rather than editing quietly.
Reading a teardown once you know how it was made
Knowing the method changes how the pages read. A number is countable and was counted. A structural claim was reproduced. A sentence about what we could not establish is a real boundary rather than modesty.
Start with the scorecard for all of them side by side, the teardowns for one product at a time, or how we score for what the five axes are and why those five.