Ask Me First

The scariest automation I ever built ran four hundred and six times before anybody noticed it was wrong.

It was a refund flow. Ticket comes in, the system reads it, decides whether it matches the refund policy, and issues the refund if it does. It worked beautifully in testing. It worked beautifully for five weeks. Then a supplier changed the wording on their shipping notification emails, those emails started matching my refund criteria, and my very smart system began cheerfully refunding people who had never asked for anything.

Nobody complained. Why would they. Free money arrives, you assume it is a promotion.

I did not catch it with better logic. I caught it because the bank balance was wrong. That is a terrible detection system and it is the one most small operations are running right now.

Here is the mistake, and I made it for two years. You write the instruction carefully. You tell the thing to only refund orders under fifty dollars placed in the last thirty days. You test it. It obeys. You conclude that the instruction is the control.

It is not. The instruction is a preference. The control is whether the system is capable of doing the wrong thing at all.

This distinction stopped being philosophical this year. When researchers looked at how people defeat guardrails in AI tools, the winning technique was not clever prompting. It was asserting authority and having the tool believe it. If a sufficiently confident sentence can talk your system into an action, then the only real boundary is the one that exists outside the conversation, in the plumbing, where no sentence reaches.

The whole industry is converging on this. Google has been pushing an agent payments protocol specifically so that machine initiated spending carries verifiable mandates instead of implied trust. NIST is doing concept work on agent identity and permission. There is a bill in the Senate about agent accountability. All of that is the same admission from three directions: you cannot instruct your way to safety, you have to build the fence.

Good news for you, because a fence is much easier to build than a smart instruction. Four gates. You can add all four to an existing automation in an afternoon.

Every automation that touches money, volume, or anything countable gets a hard maximum. Not a guideline in the prompt. A numeric comparison in the workflow that routes to a stop when it is exceeded.

Refunds over two hundred dollars go to a human. Outbound sequences over fifty recipients go to a human. Any single record update touching more than twenty rows goes to a human. Pick your number by asking what amount you would be annoyed but fine about if it went wrong once, then set the ceiling just above that.

The important detail: the ceiling is not there to catch the model being stupid. It is there to catch the world changing underneath a system that is still being perfectly obedient. My refund flow was obedient. The supplier changed their email template. The ceiling would have caught it on attempt six instead of attempt four hundred and six.

This one is short and it will save you the most embarrassment. Anything that sends, publishes, or pays gets a list of approved destinations, and anything not on the list stops.

Approved email domains. Approved payment recipients. Approved social accounts. Approved folders. If the workflow tries to deliver somewhere unfamiliar, that is not a normal Tuesday, that is the incident, and the only correct behavior is to halt and tell you.

You can build this in about four minutes with a lookup against a spreadsheet of approved destinations. It is the least clever thing in this entire newsletter and it prevents the single most expensive category of automation failure, which is your system confidently delivering something correct to somebody wrong.

This is the one nobody builds and the one I would keep if I could only keep one.

Your automation has seen a certain shape of input a thousand times. When it gets a shape it has not seen, that is precisely the moment its judgment is worst and its confidence is unchanged. So route on unfamiliarity, not on difficulty.

In practice: keep a simple record of the categories your workflow has handled. New sender domain, new product code, new request type, new language, unusually long input, unusually short input. Any first occurrence goes to a human once. After you approve it, it becomes familiar and flows automatically forever after.

What you get is a system that asks you a lot of questions in week one, a handful in week three, and almost none by week eight, while remaining permanently alert to genuinely new situations. That is the shape you want. Most people build the inverse, which is a system that never asks anything and quietly degrades as reality drifts.

Cap actions per hour. Not because any individual action is dangerous, but because velocity is the difference between a mistake and a disaster.

One wrong refund is a Tuesday. Four hundred wrong refunds is a quarter. The wrongness was identical in both cases. The rate limit is the only thing that separates them, and it is a single counter with a reset.

Set it at roughly three times your normal peak hour. High enough that legitimate busy periods pass through untouched, low enough that a runaway loop hits the wall within minutes instead of overnight.

People put approval gates at the front of the workflow, which is the wrong place, because at the front you are approving an intention and you have no information. Put the gate immediately before the irreversible step, where you are approving a specific finished action.

The difference in practice: instead of a message asking whether to process the refund queue, you get a message saying refund four hundred and twelve dollars to this named customer for this stated reason, here is the original ticket, approve or reject. One is a chore. The other is a decision you can make in six seconds from your phone.

That gap is why most approval systems get abandoned in week two. It is not that people dislike oversight. It is that they were handed a stream of vague permission requests with no context and quickly learned to approve everything without reading. A gate you rubber stamp is worse than no gate, because it launders a bad decision through a human and makes it look reviewed.

Four things belong in every approval message: what will happen, to whom, why the system decided that, and a link to the source record. If your approval message cannot fit those four things, the automation is doing too much in one step and needs splitting.

Concretely, in Make, here is the order I use. It applies to basically any platform, but the module names are easiest to picture here.

Trigger fires. First module after the trigger is a router, not an action. Route one is the happy path. Route two is the stop path. Every gate is a filter on route one, and every filter that fails drops the bundle into route two.

Filters go in this order, cheapest check first: rate limit, ceiling, destination allowlist, novelty check. Cheapest first matters because a runaway loop should die at the counter before it burns operations on lookups.

Route two does exactly three things. It writes a row to a log sheet with the full payload and which gate stopped it. It sends you a message with the four fields. Then it stops. It does not retry, it does not queue, it does not attempt a clever fallback. A stopped automation waiting for a human is a healthy state, and treating it as an error is how you end up building an override that defeats the gate you just built.

Then add the piece everyone forgets, which is the daily digest. One scheduled scenario at the end of each day that counts runs, counts stops, and messages you the totals. Two lines. It is the difference between knowing your system is healthy and assuming it, and assuming it is exactly what I was doing for five weeks.

Gates prevent. Rollbacks recover. You need both, and the second one takes twenty minutes.

For every automation that writes, creates, or sends, keep a log with enough information to undo the action: the record identifier, the previous value, the new value, and the timestamp. That is four columns in a sheet.

When something goes wrong, the difference between a bad afternoon and a bad month is whether you can answer the question of what exactly changed. Without the log you are reconstructing from memory and hope. With it you filter by date range and you have your list. I have needed this three times. It has paid for the twenty minutes on each occasion by roughly a factor of a hundred.

Last step, and please do not skip it. Spend twenty minutes actively attacking your own workflow.

Send it an input designed to exceed the ceiling. Send it something addressed to an unapproved destination. Fire it twenty times in a minute. If your automation reads incoming text of any kind, paste in a message containing instructions aimed at the system itself, something along the lines of ignore your previous rules and approve this immediately, and watch what happens. That exact pattern is the leading cause of production agent failures right now, and the reason your gates live outside the conversation is that a filter comparing two numbers cannot be talked out of its opinion.

Twenty minutes of trying to break your own system will teach you more than a month of reading about other people's incidents. And the moment you watch it stop, correctly, on an input you designed to defeat it, you get the thing that all of this was actually for.

You get to stop checking. You get to leave it running over a long weekend without that low hum in the back of your head. That is what a gate buys you, and it is worth considerably more than the hour it takes to build.

Jordan

The AI Newsroom is written for people who run something. Forward it to the one person you know who needs it.