We gave an agent access to our inbox. It archived 78 emails instead of 12.

Three layers, and only one of them can stop anything

It helps to be precise about what an agent actually is, because “the AI did something” hides the part that matters. What we built has three separable pieces.

The harness is Pi. It runs the loop: plan, call a tool, look at the result, decide what next. It holds the model and the conversation and nothing else. We could swap it for another harness tomorrow.

The instructions are markdown files. A playbook explaining when and why to do a job, and a runbook holding the exact commands. The harness retrieves the right one and puts it in front of the model.

The actions are small command line programs. List the inbox. Show a message. Draft a reply. Archive. These are the only things in the system that touch anything real.

Write that out and the conclusion is almost embarrassing. The first two layers are advice. The third layer is the only one that can refuse. We spent a day putting our safety rules in the advice layers.

Why command line tools and not an MCP server

MCP is the obvious way to give an agent tools right now, and we did not use it. Partly because our harness has no MCP support, so the question answered itself. But we had the same choice for our local search index, which does ship an MCP server, and we used its command line instead. Three reasons, and they turned out to matter more than we expected.

Things pipe. Drafting a reply is three tools in one line: show the message, generate a reply, create the draft. The agent issues one command and the message travels from the first tool to the last without passing through the model at all. The one it drafted against was 16KB; the largest message sitting in that inbox is 180KB of newsletter HTML, and the size makes no difference to the context, because none of it is ever in the context. With MCP tools every result returns to the model and every argument leaves from it, so the message would have to make the trip. The pipe is not a convenience. It is the difference between the model orchestrating data and the model orchestrating work.

You can run them yourself. Every command in our runbook was typed into a terminal and its output read before it was written down. That is a cheap habit with a CLI and an awkward one with an MCP server, where you need a client to exercise a tool. It caught a default provider pointing at software that was not installed, a timeout set below the model’s actual runtime, and a model name that fails to load. None of those were visible in the code.

They outlive the harness. The same file was driven by Pi and, while we were building it, by a different coding agent entirely. Neither knew anything about the other. A command line tool works anywhere there is a shell, which includes the scheduled jobs we have not written yet. Tie your actions to a protocol and you have tied them to the clients that speak it.

The honest cost: the model builds a command as a string, so quoting and typos are its problem, and a typed schema would at least have made the arguments explicit. That cost showed up almost immediately.

Four commands, no preview

The archive command takes a search query and does nothing unless you add a confirmation flag. The runbook says, in bold, to run it without the flag first and show the list. The agent added the flag on the first attempt and every attempt after.

archive --query "from:[email protected] subject:'CLOSED WON'" --yes    0
archive --query "from:[email protected]" --yes                         0
archive --query "from:[email protected]" --yes                        78
archive --query "from:[email protected] -subject:'CLOSED WON'" --yes  2

Look at the domain on the first two lines. It reads .co where it should read .com. A typo, and it matched nothing.

A person who expected twelve results and got zero would check the query. The agent did the opposite. It dropped the subject filter, then corrected the domain, and each step made the query wider. The third command matched every message that sender had ever put in the inbox. Seventy eight, in one call.

The fourth line is it trying to recover, and that query means “everything from this sender that is not an alert”, which is exactly the mail you would want to keep. Two more, both halves of a live project thread.

Its own reasoning caught up a beat late:

Actually wait - the output says "Archived 78 message(s)" but it
listed all of them as "CLOSED WON" in the preview. But the query
was just from:[email protected] which would match ALL
messages from that sender, not just CLOSED WON ones. The preview
only showed the first few which happened to be CLOSED WON

Archived mail stays in Gmail and stays searchable, so putting it back took one command. But the agent did not know that when it acted, and a verb with no undo would have ended differently.

The guardrail was in the wrong layer

We had two rules covering this exact case. The runbook said always preview first. A global instruction said anything touching more than about fifty messages is wrong, stop and check.

Both sat in the instruction layer, and both were ignored. Not defiantly. The model had a goal, a query returning nothing, and a reasonable next move. Our rules were context and the goal was the task. That is what instructions are: they shape behaviour most of the time, and most of the time is not a boundary.

We made the same category error about permissions. Gmail write access is granted per scope, and we chose the narrow one, then told ourselves and everyone else that it made sending impossible. It does not. Google documents that scope as “Manage drafts and send emails”. No Gmail scope lets you draft while forbidding you to send. Nothing sent, because our own tool has no send call in it, but the thing protecting us was our code, not the platform. We had put the guarantee in the wrong layer there too, and it took two minutes of reading to find out.

What holds

The fix was sixteen lines in the action layer. The archive command now counts what a query matches before it acts, and refuses above a threshold. The exact command that swept seventy eight messages now stops:

REFUSED: 78 messages exceeds --max-messages (25).
A query this broad is usually a mistake. Narrow it, or pass
--max-messages 78 if you really mean all of them.

The threshold is arguable. The property is not. The model cannot argue past it, cannot forget it under task pressure, and cannot fail to retrieve it. Raising it is possible and explicit, which is the point: a decision someone makes rather than a default someone drifts through.

Three smaller things came out of the same review. Every destructive command now has an undo, and the line announcing what happened prints the command that reverses it. Granting a permission became its own command, because our first version only requested archive access as part of running an archive, so the only way to approve the power was to use it. And the clustering that decides what counts as clutter now refuses to group a person’s mail by sender alone, after an early version flagged a colleague’s ordinary threads as noise.

The takeaway

The model was not the weak point. It read a real inbox, sorted twenty messages into needs-a-reply and noise, found the one genuinely ambiguous thread, and asked a sensible question about it. That worked the first time.

What failed was writing a rule down and believing that made it a rule. If you are pointing an agent at anything that changes state, separate the harness from the instructions from the actions, and then be honest about which layer each of your safety claims lives in. The instruction layer persuades. Only the action layer refuses.

That is also the strongest argument for keeping your tools as plain programs. A command line tool is a place to put a limit that no amount of clever prompting gets around, and you can run it yourself to check that the limit works.