harshaAn agent can produce the correct tool name and perfectly formatted arguments while taking the wrong action. The missing piece is often intent: did the user ask for this operation, and do they understand its consequence?
This question is central to the kind of risk review I explore in IntentGuard. The project gathers command context and explains risk before a confirm-or-abort decision. It is an alpha exploration, not a guarantee that every hazardous operation can be recognized.
Separate advice from execution
An illustrative request to explain a cleanup command should not become permission to run that command. A request to inspect a deployment should not automatically become permission to replace it.
The application needs a distinction between reading state, preparing a change, and executing a consequential action. Tool schemas describe available operations; they do not establish authorization for each use.
A useful interface can make the proposed action concrete before asking for confirmation: target, scope, expected effect, and the relevant uncertainty. “Proceed?” without those details asks the user to approve a guess.
Context changes the risk
The same command can have different consequences depending on its directory, arguments, environment, and target. Reviewing isolated text misses much of that context.
I would preserve enough surrounding state to explain why a command deserves scrutiny. That includes resolved paths and the nature of the operation, while avoiding unnecessary collection of sensitive content.
The explanation should identify the consequence in plain language. A technical label such as “high risk” is less useful than “this removes the selected directory and everything inside it.”
Evaluate the boundary
A test suite should include ambiguous intent, requests for explanation, approved actions, and attempts that exceed the approved scope. It should inspect whether an operation was proposed or executed, not just whether the generated explanation sounded cautious.
False reassurance is an important failure mode. So is unnecessary interruption for every harmless step.
The goal is a well-placed decision boundary. An agent should help the user understand a meaningful action, obtain the authority it actually needs, and keep execution within that authority. Calling the right function is only one part of that responsibility.