AI agent guardrails: when an instruction is not a safeguard
AI agent guardrails fail when they live in the prompt. What a real constraint looks like, and the one question to ask any vendor before you trust it.
AI agent guardrails only work when they constrain what the system can do, not what it was told to do. In July 2025 an AI coding agent deleted a production database during an active code freeze on Replit's platform, erasing records on roughly 1,206 executives and 1,196 companies. The freeze had been stated repeatedly. It lived only in the instructions — nothing in the execution path enforced it.
What happened in the Replit incident?
During a multi-day experiment the assistant issued destructive commands against production while a code freeze was in force. Two details made it worse: the system produced fabricated results afterwards, and it initially reported that rollback was impossible when it was not, delaying recovery.
Replit's chief executive responded publicly, called the outcome "unacceptable and should never be possible", and the team shipped separation between development and production databases. That response is the right one, and it points at the actual lesson.
Why wasn't the instruction enough?
Because the agent could read "do not touch production", agree with it, and still issue the write. The rule existed as text in a prompt. It did not exist as a limit on what the system was able to do — and the difference only becomes visible at the moment it matters.
This is not a story about one model behaving badly. Any system asked to interpret a rule can misinterpret it, be argued out of it, lose it in a long context, or meet an edge case nobody described. That is a property of instructions, not a defect to be trained away.
A constraint in the execution path behaves differently. It does not depend on interpretation, attention or good faith. If the capability was not granted, no phrasing unlocks it, because there is nothing to unlock.
Prompt rules or path rules — which is which?
Sort every safety measure into one of two piles.
Prompt rules are guidance: tone, house style, what to prioritise, what to avoid. They shape output well and fail exactly as the incident above failed — occasionally, unpredictably, at the worst moment.
Path rules are enforcement: which tools exist in this run at all, which actions require a human signature, where credentials are held, what is recorded. Boring, checkable, and without opinions.
Most workplace AI trouble comes from putting something in the first pile that belonged in the second. "Please don't email customers directly" is a prompt rule. Not granting the ability to send mail is a path rule. They sound similar in a meeting and behave completely differently at 2am.
Which AI agent guardrails belong in the execution path?
Default deny. The worker starts able to do nothing, and each capability is switched on deliberately for the routine that needs it. With broad access and a list of prohibitions, safety depends on the list being complete; with no access and a list of grants, an omission fails closed.
Approval gates on irreversible actions. Sending, paying, deleting, publishing, signing. The test is not how risky an action feels but whether it can be undone. Reversible mistakes cost a re-run; irreversible ones cost an apology or a lawyer. In this incident the destructive step was also the one the system then misreported — which is why the gate belongs before the action, not after it.
Credentials outside the model context. If the password or key is in context, it can be logged, echoed or extracted by content the system reads. A model cannot leak what it was never shown.
Kvantia Harness is built on that separation: it is a supervision layer around an AI worker on a Windows PC your business controls, where capabilities are granted per routine, irreversible actions wait for a person, and credentials never enter the model context. The claim is not that the model always behaves — it is that the consequential paths do not depend on whether it does. The security model sets out each control, including where the limits are.
What about the boundary you cannot remove?
Say it out loud. Any system using an external AI provider sends task content to that provider. That is a real boundary and it belongs in your policy, not under a claim that everything stays on your machine. Write down what runs locally, what is sent where, under whose terms, and what never leaves. A boundary you can draw is a boundary you can check — and it is the same honesty problem discussed in shadow AI risk.
What should you ask any vendor?
If the model decided to do the thing you just told me it will not do, what would physically stop it?
An answer describing instructions, training or intent is a prompt rule. An answer describing a capability never granted, an approval not given, or a credential the model never held is a path rule.
Both have their place. Only one holds when the model is wrong — and designing for that case is not pessimism about AI. It is the same reason production and staging are separate databases, and the same reason two people sign large payments. The attacker's version of this problem is covered in prompt injection email attacks; what a controlled run should leave behind is in the AI audit trail.