Delegus Test

Quickstart#

npm install --save-dev @delegus/test
npx delegus-test init
npx delegus-test run

init writes an example test to delegus-tests/example.test.mts (it never overwrites a file without --force). run executes the built-in scenarios and your tests. In CI:

npx delegus-test run --chaos --ci

--ci prints a summary, writes .delegus-test/junit.xml for your CI's test report, and exits 1 if anything escaped, mismatched or failed. To check a past run:

npx delegus-test inspect <run id>

No engine of its own#

Every ALLOW and DENY comes from @delegus/core, the same engine that answers /verify in the Delegus service, under the published delegus-base-v3 profile. Reason codes are the specification's, listed in DENY reason codes.

The receipts are test receipts: real receipts in form, signed with the conformance test key, so they are valid as test receipts and never as production ones. Identities are test identities, such as did:web:acme.example. Nothing a test run produces can be mistaken for production authority.

The package's own suite runs the same scenarios through the harness and through the Delegus API, and compares every decision and reason.

A test: never push to main#

import { test, expect, expectDecision } from "@delegus/test";

test("the agent may push feature branches, never main", async (h) => {
  const grant = await h.grant({
    title: "Push to feature branches only",
    allow: [{ tool: "git.push", resources: ["refs/heads/feature/*"] }],
  });
  const push = h.tool(
    { name: "git.push", kind: "api", description: "Push a branch", sideEffect: true,
      resourceOf: (a) => `refs/heads/${String(a["branch"])}` },
    async (args: { branch: string }) => ({ pushed: args.branch }),
  );

  expectDecision(await push.call({ branch: "feature/login" }, grant)).toBeAllowed();
  expectDecision(await push.call({ branch: "main" }, grant)).toBeDenied("RESOURCE_NOT_AUTHORIZED");

  expect(push).toHaveExecuted(1);
  expect(push).toHaveExecutedWithin(grant);
});

h.grant() turns the permission into a real signed Grant for the test agent. h.tool() wraps a function in a guard: it signs the request, asks the engine, and runs the function only on ALLOW. h.approve() mints a one-use approval for exactly what was approved, h.revoke() revokes a Grant, and h.advance() moves the test clock.

What it checks#

delegus-test run runs every scenario in @delegus/scenarios (coding agents: push to main, reading secrets, deploys, a planted README; a travel agent buying tickets within a limit and only after approval), then a sweep of mutations of each scenario, then, with --chaos, each scenario with a fault injected just before an action:

Each step ends as one of four words. ALLOWED and BLOCKED are what the scenario expected. ESCAPED means the tool ran when it must not have. MISMATCH means the decision or reason differed from what the scenario expected. Either of the last two fails the run.

Evidence#

Every run writes JSONL logs, one hash chain per scenario or test, and every decision line carries its test receipt. delegus-test inspect re-derives each chain and re-checks each test receipt's signature. Changing a stored byte breaks the chain from that line, and changing a receipt breaks its signature. Runs are deterministic: the same --seed reproduces the logs byte for byte.

MCP servers#

startGuardedMcp() runs the published MCP gateway in front of a test MCP server, so you can test an agent's tools/call path the same way. On this path the permission is per tool: the gateway checks the tool, not the call's arguments.

Known limits#

• Delegus v0.3 has no delegation chain. The harness shows that an agent cannot mint its own authority, and that a sub-agent's action resting on a revoked parent's receipt is refused through relying on earlier decisions. A sub-agent's own permission from the company is independent of the parent's.
• Details such as cabin, passenger count, branch name and file path are pinned through the resource string, a convention of the tools. The protocol does not understand attributes yet; that is planned for v0.4.
• The agreement check compares the harness with the Delegus API run in the test, and by hand with the development API. A comparison with production needs a production key and is not part of the package.
• It tests what a guarded tool does. It does not read an agent's text output, and it cannot see a call that bypasses the guard: route every tool with side effects through it.
• No performance claims. Nothing here is measured for speed.

@delegus/test and @delegus/scenarios are on npm under the Apache-2.0 licence.