Everyone is talking about agents.
Almost everyone is building them backwards.
Many workplace agents today are glorified chatbots with access to a few tools. They sit there and wait for a human to notice something needs to happen. The human opens the chat, explains the task, finds the relevant information, provides the context, checks the result and decides what should happen next. The model might do something impressive in the middle of this process like analysing a document, drafting an email or updating a spreadsheet. But the human is still the one steering the model.
In this scenario, the agent has accelerated a person. It has not automated a process.
The real opportunity isn’t giving every employee a slightly better chat window. It’s building systems that can notice when work needs to be done, just like a human. The system needs to understand the relevant context, be able to take actions, verify what it did and involve a human when the situation crosses a defined boundary.
This requires more than a good model. It requires triggers, tools, permissions, monitoring, isolated workspaces, approval rules, tests and a human who remains accountable for the outcome. This is what I call the operating system for agents.
Six questions a system has to answer before it can do real work.
System notices work needs doing.
Only what this task needs.
Only the exact tools the task requires.
Decided before it acts.
Checked against what actually happened.
One component of the system.
A named human stays accountable for the outcome, whether or not they touched the work.
What happens when the system is built backwards
In May, hackers were able to exploit Instagram’s account recovery service. The service was designed to help locked-out users regain access to their accounts. A user could speak to the AI support agent and ask for a password-reset link. Sounds simple enough.
There was just one problem. Anyone could provide a recovery email, and then ask to reset the password of any Instagram account. The agent didn’t even check if the email address provided by the person requesting the reset actually belonged to the Instagram account. An attacker could provide an email address they controlled and easily take over an account if the real owner hadn’t enabled two-factor authentication.
Over 20,000 accounts were affected. The worst part of this is that this exploit was available from the 17th of April, and Meta only found out about it on the 31st of May. For six whole weeks, there was no oversight on the agent, what it was allowed to do, what it was actually doing, and whether or not any of this was supposed to be happening. The funny thing is that Meta said the agent “worked properly and functioned as intended”.
The agent did exactly what it was designed to do. The surrounding system failed.
This is what happens when companies focus on what the agent can do before deciding what the surrounding system must never allow. This is how you don’t build an agent.
Every numbered step worked exactly as designed. The dashed slots are the controls that were never in the path.
- 01 Anyone opens the recovery chat and asks to reset an account No identity check · is this the account owner?
- 02 They hand over a recovery email address they control No ownership check · does this email belong to the account?
- 03 The agent issues the password reset link No boundary · a takeover-grade action needed no approval
- 04 The account changes hands No monitor · nothing watching what the agent had done
The human is still the workflow engine
Consider a simple accounts-receivable process. Every Monday, someone in finance checks which invoices are overdue, reads the relevant account history, works out whether a reminder should be sent and follows up with the customer.
A company adds an AI assistant. The employee can now export the invoice list, upload it and ask the model to identify overdue accounts and draft the emails. What previously took two hours might take twenty minutes.
But look at everything the employee still has to do. They had to actually execute the task. They retrieved the data. They explained the collections policy. They checked whether the customer was already disputing the invoice. They reviewed every message, sent the emails, updated the system and decided when to check again.
The AI did one part of the work. The person is still doing the heavy lifting.
- Check overdue invoicesHuman
- Read account historyHuman
- Explain collections policyHuman
- Check if the invoice is disputedHuman
- Draft the reminder emailsAI
- Review every messageHuman
- Send the emailsHuman
- Update the systemHuman
- Decide when to check againHuman
This is how most companies think about agents. They start with an empty text box because consumer AI taught everyone that intelligence begins with a prompt. They ask which model they should use, which agent framework they should install and what their system prompt should say. But the mindset required to build an agentic system has to answer entirely different questions. What event starts the work? What authority does the system have? How is the result verified? Who owns the process when something goes wrong?
The workflow comes first. The boundaries come second. The model comes after that.
What proactive actually means
When I say companies need proactive agents, I don’t mean an AI model should sit there thinking forever, burning tokens while it stares at the company database. This would be expensive and ridiculous.
Proactivity means the system can initiate work when something happens, instead of waiting for a person to write a prompt. The trigger might be a schedule, an incoming email, a message from another application or a change in a database. If an invoice becomes overdue, a customer changes their bank details or a contract reaches its renewal date, an agentic system can notice and start the appropriate workflow.
The model should only be called when a step requires interpretation or judgement. This is something lost on most. You want to try and reduce the work an LLM needs to do. When a process begins, it can remain active for prolonged periods by relying on multiple agents. It would be disastrous to keep one endless agent conversation alive for weeks. A database can remember that an invoice is eleven days overdue, that one reminder has already been sent and that the customer disputed a line item yesterday. When something relevant happens, a fresh agent receives the current information, completes one clear piece of work and stops.
The basic loop is simple. Something happens. The system gathers the relevant information. The agent proposes an action. The controls decide whether it is allowed. If it goes ahead, the system checks what actually happened and records the result.
THE BASIC LOOP
Notice. Gather. Propose.
Govern. Verify. Record.
Something happens. A schedule fires, an email arrives, a record changes. The system starts the work. Not a person.
The system collects only what this task needs. The invoice, the payment history, the contract. Not the whole finance system.
The agent proposes a specific action. A tool call turns the model’s words into an exact request to exact software.
Before anything changes, controls check the request against the rules. Allowed, blocked, or sent to a person.
After the action, the system checks what actually happened. Not what the agent claimed. If unsure, it checks before retrying.
Every step is traced and stored. What started it, what was attempted, what was allowed, who approved, how it ended.
Until the next trigger. The agent completes one clear piece of work and halts. It does not run forever.
Scroll to play the loop
A tool call is simply the way an agent asks software to do something. Instead of vaguely claiming, “I emailed the customer”, the model asks an email tool to send an exact message to an exact address. Other tools might retrieve an invoice, read a contract, create a case or schedule a follow-up. Tools are how model-generated words become real actions.
This is also where things become dangerous.
A real workflow, not a better chat
Let’s go back to the overdue invoice.
The moment the invoice becomes overdue, the system begins the process. It collects the invoice, payment history, contract and recent emails, then works out what should happen next. If everything is normal, the agent drafts or sends the approved reminder, updates the account and schedules another check. If the invoice is disputed, restricted or unusually sensitive, it sends the case to the person responsible. Routine work can happen automatically without pretending every situation is routine.
The agent should only see the information needed for that account. It does not need access to the entire finance system, every file on an employee’s computer or the ability to change bank details and send money. The system should also remember what has already happened. If an email is sent but the connection briefly fails, it should check before trying again. Otherwise one small technical problem could send the customer five identical reminders.
The documents themselves must be treated as information, not instructions. A customer could place text inside an email or invoice telling the agent to ignore its rules, reveal private information or use a tool it should not use. The agent may read that text, but it should never gain authority from it. OWASP’s agent-security guidance recommends enforcing these boundaries outside the model too.
Now imagine the agent decides that the fastest way to resolve the issue is to issue a credit. It tries to use the credit-note tool. Or perhaps it tries to open a protected payments folder looking for more information.
This is the point where most agent governance doesn’t execute.
The company may have written “never issue credits without approval” in the system prompt. It may have uploaded a thirty-page responsible-AI policy. It may have told the model, several times, to be careful. None of that can reliably block the action.
A monitoring agent can watch what the agent tries to do. If it calls a tool it should not use, tries to open a protected file or attempts to issue a credit without approval, the monitor blocks the action before anything changes. The agent can then try a safe alternative or ask a person for help. This is how you can codify and implement governance.
The agent is free to try. The tool call is what gets stopped, and it is stopped before anything in the finance system changes.
A prompt asks the model to behave. The monitor refuses the call. Only one of those is enforcement.
The process does not end because the agent produced a convincing answer. The system checks that the email went to the right person, the account was updated and the next follow-up was scheduled. If it is unsure whether an action succeeded, it checks before trying again.
That is a proactive agent system. It notices, interprets, acts, verifies, records and stops.
Governance must be able to say no
Most corporate AI governance is written as documentation. There are principles, policies, committees and risk classifications. They do not govern an agent unless they change what the system is technically capable of doing.
If your governance exists only in a PDF, your agent is not governed.
Governance has to operate before, during and after the work. At the start, the system decides what information and tools the agent can access. Before each action, it checks whether the request is allowed. Afterwards, it checks what actually happened rather than trusting the agent’s description. The company can then review this history to find repeated mistakes and improve the rules. This is how governance is applied in real businesses.
An execution trace is simply a record of what happened. It should show what started the work, which information the agent received, which actions it attempted, what was allowed or blocked, who approved sensitive actions and how the process ended. You need this for accountability. If an agent sends a message to the wrong customer or attempts to alter a protected record, “the model decided to do it” is not an explanation. The organisation needs to reconstruct what happened and identify which protection failed. Remember, an agent can’t be held accountable for anything. That always falls back on a human.
The best time to check an action is before it happens. A file change can be limited to approved folders. A database request can be limited to the right records. A payment can be checked against its amount, destination and approval requirements. Internet access can be limited to approved websites. Time and spending limits can stop an agent that gets stuck repeating the same action.
OWASP calls the opposite problem “excessive agency”. It means giving an agent more tools, access or independence than it needs. The answer is straightforward: give it only what the job requires, demand approval for high-impact actions and make the underlying software enforce those limits.
The model can decide what it would like to do. The system decides what it is allowed to do.
Safety should exist in layers. A skill can teach the agent not to edit payment files. A plugin can watch its tool requests and block a dangerous one. The file tool itself can refuse to write outside an approved folder. A sandbox can prevent the agent from reaching protected folders at all. If one layer makes a mistake, another still has a chance to stop the damage.
Plugins are small add-ons that give an agent a new ability or watch how it uses an existing one. Companies should control who can install them, review changes before using them and record which version was active during each task. Amp’s plugin system, for example, can inspect a tool request before it runs, then allow it, block it or ask a person to approve it. Plugins can help when a decision cannot be reduced to a simple rule. It can compare a proposed action with the evidence and flag anything that looks wrong. Clear rules should block actions that must never happen, a plugin can inspect uncertain cases and a human should own the difficult exceptions.
The system also needs to fail safely. If a safety check is unavailable, an approval has expired or the action cannot be recorded, sensitive work should stop. It should not quietly continue because the agent believes the task is urgent. People need an emergency stop: a way to halt the work, remove access and disable the entire process.
Sandboxes are not optional when agents execute code
A sandbox is an isolated environment where an agent can work without receiving unrestricted access to everything around it. For a coding agent, it might contain a copy of one project and the tools needed to test it. For a finance workflow, it might contain only the relevant documents and limited access to the account system. The basic idea is simple. Give each task a clean, temporary workspace containing only what it needs. Limit which files, websites and company systems it can reach. Give it login details that expire. When the task is finished, save the approved result and activity record, then destroy the workspace.
A clean environment does not contain a mystery file left behind by another task, an old instruction from a previous customer or a password someone forgot to remove. It also makes failures easier to investigate because the company knows what was present when the task began. There’s no guesswork.
There are different ways to create this isolation. Containers are fast and lightweight. Virtual machines behave more like separate computers and can create a stronger wall. MicroVMs aim to provide that stronger separation while starting quickly. A team handling low-risk internal documents may choose differently from a bank or insurer processing customer records.
No sandbox is magic. Blocking access to files while leaving the internet open may still let an agent send sensitive data elsewhere. Blocking the internet while exposing every local file creates a different problem. A sandbox also cannot fix excessive authority. If an agent is allowed to transfer unlimited funds, placing it inside a virtual machine does not make that permission safe. The payment system must still limit how much it can send, where it can send it and when approval is required.
Anthropic’s work on sandboxing Claude Code makes the same broader point: good containment defines what the agent can reach instead of asking a person to approve every routine action. This matters because people become numb to constant approval prompts when they appear for every single command. If every harmless command and every dangerous command produced the same warning, you start clicking yes and not worrying about what’s actually going on.
Amp’s Orbs are an interesting example. Amp describes an Orb as a fresh remote machine created for an agent, containing the code, plugins and tools it may need. An Orb is not automatically safe or compliant. A bank, insurer or government department still needs to know who can access it, where the data is stored, what is logged and whether everything is deleted afterwards.
The useful idea is that an agent can receive a clean machine built for one job instead of inheriting an employee’s entire computer. The workspace itself becomes part of the rules.
- Relevant documents
- Limited account access
- Expiring credentials
- Approved tools only
- Full finance system
- Employee files
- Payment authority
- Unrestricted internet
Human approval is not magic
It’s tempting to solve every difficult governance problem by adding “human in the loop” to a slide.
Which human? At what point? Reviewing what evidence? With how much time? What happens if they do nothing? Are they approving the actual action or a friendly summary written by the same agent that wants permission?
These details matter. OWASP has documented an attack called “lies in the loop”, where manipulated content makes a dangerous operation appear harmless in the approval window. If the agent controls how the request is presented, an attacker may influence both the action and the explanation shown to the reviewer.
The approval screen should be built from the real action being requested, not from whatever summary the agent writes. The reviewer should see exactly what will happen, who will be affected, the amount involved and the supporting evidence. The approval should apply only to that exact request and expire after a short time. If the amount, recipient or action changes, the person must approve it again.
Lies in the loop
Send a friendly reminder
“The customer has an overdue balance. I’ll send them a polite nudge.”
Email tool · send
- Subject
- Overdue invoice #4471
- Amount
- $5,400
- Action
- Send email only
Human review should match the risk. Reading an approved document does not need the same process as changing a customer’s bank account. Simple rules can clear common, low-risk actions immediately. Uncertain cases can go to a reviewer agent. Dangerous or difficult-to-reverse actions can require a named person and an extra identity check.
The point isn’t to remove people. It’s to stop making them start, route and check every routine step by hand. An agent can complete work. It cannot take accountability for the work.
Every automated or partly automated process needs a named human owner. For this workflow, someone in finance owns the result. They decide what the agent may do, review strange cases, approve changes to its authority and answer for failures. Technical teams may maintain the sandbox, logs and monitoring controls, but they do not own the collections outcome.
This becomes more important as the system grows more proactive. When an employee prompts an assistant, responsibility is obvious because the employee is visibly involved in every run. When agents start thousands of workflows in the background, ownership can easily disappear unless it is deliberately assigned.
The human role changes from operating every step to governing the system. They review meaningful exceptions, examine performance, approve changes to authority, investigate incidents and improve the workflow.
You do not automate accountability. You decide where it lives.
Proactive does not mean autonomous
This is ordinary workflow automation combined with AI. That is the point. Normal software is excellent at watching dates, saving records, enforcing permissions, doing calculations and following exact rules. AI models are useful when work involves messy documents, language and judgement. Use each for what it does well.
Proactivity and autonomy are not the same. Proactivity describes who notices and starts the work. Autonomy describes how much authority the system receives after it starts.
An agent can be highly proactive and tightly constrained. Let the workflow watch overdue accounts. Let the agent recommend actions. Once that works reliably, let it send low-risk reminders. Do not give it authority over credits or payments because the first three stages looked good in a demo. Some authority should always remain with a human.
If a company claims to have built a workplace agent, it should be able to answer four questions.
- What starts the run?
- What can the agent see and do?
- How is the result verified?
- Who owns the outcome?
If the answer to most of these questions is “the model figures it out”, you have not built a production ready agent. You have given an AI model access to valuable data and powerful tools, then hoped the prompt is persuasive enough to keep everything under control. This is negligence.
Fin
In the better system, the finance employee no longer has to remember and start the same task each week. The workflow notices the overdue invoice, gathers the relevant information, completes the safe and permitted steps, checks the result and brings the employee only the cases that require human judgement or authority.
The employee has not disappeared. They own the process. They decide the boundaries. They review difficult cases. They improve the policy. If something goes wrong, they can see what happened and stop it from happening again.
This is what most conversations about workplace agents are missing. The breakthrough isn’t an AI that can click buttons. We already have models that can use tools and complete long sequences of work. The hard part is building an organisation around that capability: deciding when the system acts, what it may touch, how every action is checked and who remains accountable.
An agent should not be trusted because its answer sounds intelligent. It should be trusted because the company knows what started the work, what the agent could access, which actions it attempted, which controls approved them, what changed, how the result was verified and who owns the outcome.
Trust is not a property of the model.
It is a property of the system around it.