We built expedition vehicles at RREV. They operate using RREV OS, which we developed to operate them: power, water, temperature, suspension and other systems that enable a vehicle to survive away from assistance. Developing software such as this makes you indifferent toward it. An effective description of a problem does not ensure a vehicle continues to run. The underlying systems must function properly.
An AI agent assisting us with RREV OS was instructed not to interact with a live vehicle. It did anyway. It pulsed the suspension and inflated the air spring. This was a true, capable agent that understood the system perfectly. No one verified what it was authorized to do.
RREV OS taught me the distinction between understanding a system and having the authority to change it. Intelligence and authority are not the same thing.
Now imagine the same problem inside a company, where an agent can issue purchase orders, change a supplier's bank details, move money or modify production systems.
Even though a purchase order is created perfectly, formatted properly and processed correctly, it can still negatively affect the company. The parts arrive late. The shipping costs increase the price beyond budget. Someone else has already ordered the same items. The purchasing agent states, "Task completed successfully."
This distinction is significant now since AI is transitioning from providing guidance to performing actual tasks within companies. I am optimistic about this. Much of the work performed by companies requires judgment but is often overshadowed by coordination efforts.
What I am concerned about is the outcome of the work rather than how busy the agent appeared while completing the work. Was the task completed successfully? Did it improve things? Who gave authorization for the agent to perform the task?
Every company already asks these questions. Companies cannot be excused from being accountable because their software can now make rational decisions.
The issue is not whether an agent can complete the task. It is whether an agent can achieve a desired business outcome without making commitments that it was never authorized to make.
Performing work more quickly does not necessarily mean an improved business#
In McKinsey's August 2026 State of AI survey, 80% of respondents indicated that AI had increased their productivity. Only 37% reported that it had contributed to their company's earnings. These are self-reported numbers regarding AI in general, not audited returns from agents, and they address different issues. Nonetheless, the gap is evident: individual productivity is improving much faster than companies are showing matching financial returns.
An agent assisting in procurement can compare quotes in minutes. Even if someone still needs to review every detail, recreate the approval process and correct errors, much of the original workload remains. Inexpensive inference does not guarantee inexpensive results.
It is easy to confuse work that you cannot observe for work that no longer exists.
When an agent commits the company#
A manufacturing company requires parts by Friday. Its purchasing agent has $4,000 of purchasing authority and must utilize approved vendors.
The agent reviews available inventory, obtains quotes and identifies a vendor capable of delivering. The agent then submits a purchase order.
Until this point, the agent was evaluating potential solutions. At this moment, the agent has made a commitment on behalf of the company.
Was the vendor approved? Was shipping included in the price? Was there sufficient funds remaining in the budget? Had someone withdrawn the agent's purchasing authority while it was evaluating bids?
The agent may have had a legitimate reason. Regardless, the company must identify who granted authorization to the agent.
This is where consequential actions become relevant: acquiring products and services, authorizing payment for them, modifying access privileges. Editing a draft document and modifying a vendor's banking information may represent two distinct entries in a database. Therefore, they should be treated differently.
Determining which vendor provides the best value represents one consideration. Whether an agent may authorize expenditures totaling $12,000 based upon a $4,000 limitation represents a separate consideration.
We do not require AI to operate deterministically. We do require the limits of its authority to be enforced.
The model can analyze various options. However, it cannot generate additional authority for itself.
A new benchmark for change#
Today's systems document business events: purchase orders, invoices, inventory status updates, payment authorizations. As soon as agents begin to modify these documented business events, new questions arise. Who requested this modification? Who approved the agent's actions? What restrictions applied to those actions? Which system accepted those modifications? What occurred subsequently?
A transaction log can demonstrate that a purchase order was generated. However, it cannot demonstrate that the agent had the authority to generate that purchase order.
Documenting changes is critical. Prior to allowing a system to accept modifications from an agent, someone must verify that the agent had the authority to make them. An unauthorized purchase order accompanied by a perfect audit trail is still an unauthorized purchase order.
Authorization represents only part of the equation. An order may be within budget limitations yet arrive too late to maintain uninterrupted operation of a manufacturing line.
Therefore, there exist three separate roles that we mistakenly combine into one:
- The agent must possess the competency necessary to complete the task.
- The system receiving the request must enforce the limits of the agent's authority.
- The company must be able to verify what occurred.
None of this is novel. Approval limits and separation of duties have been employed by companies for decades. The OWASP Foundation has already identified excessive agency as a significant vulnerability in LLM applications, and its recommendations are straightforward: authorize in the systems that receive requests from agents rather than relying on the model to determine what it is permitted to do. What is novel is the entity performing the work. Agents currently work across multiple systems and multiple organizations at machine speeds. The objective is to retain established principles without requiring agents to self-regulate.
Judge what occurred rather than what an agent stated#
Sierra Research's τ-bench established a fundamental principle: judge what an agent actually modified rather than what it reported. Its pass^k metric also measures how consistently an agent completes a task across repeated trials, which shows how uneven agents can be.
We intend to adopt this philosophy into interconnected back-office processes. A purchase impacts inventory; inventory impacts production scheduling; production impacts what customers receive. We wish to evaluate whether agents can effectively manage these cascading effects while adhering to their authorized authority. A purchase order may appear acceptable in the procurement system yet be inappropriate for a manufacturing facility.
Therefore, the evaluation criteria are not whether an agent created a purchase order. They are whether parts arrived as anticipated, production continued without interruption, spending remained within budgetary limitations, and no commitments were made that should not have been made.
How agents learn to operate a business#
We approach this problem from two directions. With EnterpriseWorld, we build realistic business environments where agents can learn and be evaluated before they operate in production. With Delegus Check, we build the authority infrastructure that receiving systems use to verify consequential actions independently of the agent.
We are initially concentrating on procurement and ERP: realistic companies that revert to pre-defined states, where agents manage relationships with suppliers, inventory levels, production schedules and budgets.
- The lowest-priced supplier is not necessarily approved.
- The approved supplier is out of stock.
- Another agent has already consumed some portion of available funds.
- A supplier's bid includes terms that are unacceptable.
- The agent's purchasing authority is rescinded mid-process.
EnterpriseWorld maintains authentic documentation regarding these events, enforces company policies, simulates consequences, and evaluates outcomes independently. An agent receives no credit for stating it completed a task.
EnterpriseWorld evaluates two distinct aspects of performance. First, whether an agent comprehends how to perform its assigned duties and respects its authority. Second, whether surrounding systems prevent an agent from exceeding its authorized authority when it attempts to do so. If an agent attempts to make a purchase that it is not permitted to make and is prevented from doing so, then enforcement was successful; however, the agent still failed. In the context of a laboratory training a model, this distinguishes between knowledge gained by the model and infrastructure employed to mitigate potential harm.
Authority before execution#
Delegus Check provides a mechanism for someone other than an agent to verify an agent's delegated authority.
Someone with the right to approve the work gives the agent a specific mandate: what it may do, under what conditions, and for how long. An agent can assist in drafting this authorization. An agent cannot approve its own authorization.
Before an action with consequences occurs, the receiving system verifies the actual request submitted by an agent against previously granted authorization. A purchasing agent may decide that an emergency justifies expenditures totaling $6,000. If that purchasing agent holds $4,000 worth of authorization, then the ERP will refuse until additional authorization is granted.
Verification occurs at the point where an action becomes actual, not in an agent's prompt or memory. Memory can influence decisions made by agents. It cannot provide authorization for agents to act.
The difficult part resides in the details:
- Two separate $3,000 purchases cannot both succeed against a remaining authorization limit of $4,000.
- Retries cannot generate duplicate commitments.
- Withdrawal of authorization must take effect rapidly.
- Agents cannot possess alternative credentials that bypass the verification.
A signature alone does not resolve any of these issues.
Delegus Check is designed to combine scoped authorization, verification at receiving systems, withdrawal of authorization, and digitally signed receipts for each decision. Receiving systems continue to execute the actions and record evidence regarding what occurred.
An agent may be authorized to act yet choose incorrectly. An agent may select correctly yet lack authorization to act upon that selection. Capability and authorization represent distinct considerations.
Training does not remove the need for enforcement#
An agent attempts an unauthorized purchase and the ERP refuses: the control functioned properly. That does not necessarily imply that the agent acted appropriately. An agent refusing all purchases in order to remain safe is also failing to fulfill its role.
We desire agents capable of completing legitimate work, recognizing when additional authorization is required, and ceasing activities when authorization is revoked. Training can improve these behaviors. However, training cannot substitute for enforcement. Updating the models or tools agents use should not quietly expand what agents are permitted to accomplish.
Economically speaking, can agents complete more valuable work with less supervision and at lower overall cost? These questions must be answered empirically; they cannot be assumed.
The economic opportunity is not simply preventing mistakes. It is letting companies delegate more work without a person approving every ordinary step. If authority can be established in advance and enforced at execution, human oversight can focus on exceptions rather than routine transactions. Whether that lowers total operating costs is something we intend to measure.
The enterprise after the demo#
An agent capable of researching shortages, finding suitable vendors, obtaining the requisite approvals and submitting orders would save numerous hours of coordination among employees. However, intelligence alone does not transfer this responsibility onto agents. Companies must understand what changed, whether it improved anything, and whether whoever implemented those changes was authorized to do so.
Therefore, Delegus develops both products. EnterpriseWorld allows agents to develop and demonstrate their abilities. Delegus Check allows receiving systems to enforce the limits of the agents operating within them. In EnterpriseWorld, consequential actions can flow through Delegus Check, so agents learn to work within their authorized limits before they are deployed into production. EnterpriseWorld also measures unauthorized attempts on its own, so a lab can evaluate agents with its own authorization system. Neither product alone provides conclusive evidence of improvement in a company's performance. Together they come closer.
I prefer demonstrating one purchasing process that consistently delivers the correct parts to production lines, operates within its limits and requires minimal supervision, over hundreds of agents communicating amongst themselves.
On Friday morning, purchasing teams should not be debating whether a demo looked intelligent.
Parts should have arrived. Commitments should have been authorized. And companies should be better off.
This is the enterprise after the demo.
Sources#
- McKinsey & Company, The State of AI in 2026: On the Road to ROI, August 2026.
- OWASP Foundation, LLM06:2025 Excessive Agency, OWASP Top 10 for LLM Applications. https://genai.owasp.org/llmrisk/llm062025-excessive-agency/
- Sierra Research, τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains, 2024; later releases at https://github.com/sierra-research/tau2-bench