This page is what that suite is, what it proves, and how you run it against an implementation that is not ours.
What exists today#
Thirty-six vectors, one per outcome that matters. Three ALLOW paths (allow-, allow-api-call, allow-) and one DENY vector for every reason code in the specification's §5.2 and §5.3 — AUTHORITY_, PROOF_REPLAYED, AMOUNT_, AUDIENCE_, GRANT_EXPIRED, ISSUER_, three kinds of SERVICE_ (issuer lookup, status list, replay store), and the rest — plus extras that pin readings an implementer might otherwise get subtly wrong: a grant signed with alg: none, a proof whose request URI is non-canonical, an unknown constraint the engine must refuse rather than ignore. Every reason code in the frozen tables has a vector; the engine's own test asserts that coverage so a new code cannot land without one.
Three normalization sets. Separate from the case vectors: the htu set (request-URI normalization per §4.3, inputs → canonical form, plus malformed inputs that must be rejected), the JCS set (RFC 8785 canonicalization of Action objects and the resulting action_hash), and the jti format set. These are the places where two correct-looking implementations disagree by one byte and produce different hashes. The sets exist so they can't.
Reproducible byte for byte, on any platform. Fixed Ed25519 test keys (seeds 0x01–0x07), a fixed clock (2026-), and non-production identifiers only (did:web:acme., did:web:test.). Ed25519 signing is deterministic. The one place platform drift usually creeps in — gzip output, which differs between zlib versions — is closed by encoding the status-list bitstrings with a fixed pure-JS encoder using stored blocks. The result is that every artifact in the suite, including every signed receipt, is the same bytes on every machine.
A portable format. Each case is a JSON file with three parts:
input— the grant, proof, action, relying party, evaluation time, receipt id, proof window, the service's DIDs and public keys, the profile document, the trust configuration, andstate: what the three service seams observed before the request (issuer lookup after P1; status list before P6; replay store after P1–P14).statecan also be"unavailable", which is how the fail-closed vectors are expressed.expected— decision, reason, the protocol and trust results, and the fullchecksmaps.receipt— the signed receipt this engine produces for that input, with the test receipt key.
A third-party implementation reproduces expected from input. An implementation of the receipt format additionally reproduces receipt exactly when it signs with the test key.
Four things the conformance test does every run. It executes every vector definition live; checks the committed JSON has not drifted from the definitions; re-runs each committed JSON through the engine using only its own contents; and re-verifies each committed receipt offline. That last step is the same procedure anyone uses on a production receipt — see the receipts page — so the suite exercises offline re-verification on every commit, not only on the day it was written.
The suite runs on every commit of the reference engine, and it is published as @delegus/ (Apache-2.0), so anyone can run it — see "Run it" below.
What it does and does not prove#
It proves that an implementation of the open protocol layer — parsing, canonicalization, the twenty checks in their fixed order, receipt assembly — produces the specified decision and reason for every specified situation, and that its receipts are byte-identical to the reference. That is exactly the layer the specification's constitution puts on the open side.
It does not, and cannot, prove the service side: that an issuer is really who they say, that a proof has never been seen before anywhere, that the evidence will still be there in seven years. Those are the trust checks (T1–T5) and the evidence obligation (§8), and the receipt's trust block is explicitly Delegus's attestation, not reproducible by design. Conformance tells you a verifier is a correct Delegus verifier. It does not make it Delegus.
Beyond the vectors#
Three pieces have landed on top of the vectors: the standalone package, a hosted runner, and a Python port.
- Vectors as a standalone npm package with a CLI — available now:
npx @delegus/runs the full set against any implementation exposing the engine interface and prints which vectors pass.conformance - A hosted conformance runner — live on
api.delegus.ai: put your verifier behind one HTTPS endpoint, in any language, and Delegus runs every published test case against it and reports pass or fail for each one. See "Test your verifier over HTTPS" below. Reports are not signed and are not a certification. - A Python port of the protocol layer passes all thirty-six vectors — every receipt byte-identical, all three sets, every committed receipt re-verified offline. It is Delegus's own port, written to prove the vectors are implementable outside TypeScript. No independent implementation has passed the set yet.
When an implementation that is not ours passes the full set, that is the moment the open layer becomes a standard rather than a format, and Delegus becomes its reference verifier rather than its only one. That is the point of publishing the vectors.
Run it#
The vectors are published as a standalone package. Against the reference engine:
npx @delegus/conformanceIt prints each vector's result and finishes with the summary line — 36 of 36 vectors, all three sets, every committed receipt re-verified offline. To run the same vectors against your own implementation of the protocol layer, point the runner at it:
npx @delegus/conformance --impl ./my-verifier.jsThat is how you find out whether what you built is a correct Delegus verifier — the same question, answered by the same bytes, that the reference engine answers on every commit.
If your verifier sits behind an HTTPS endpoint (the contract is in the next section), you can run the same checks the hosted runner does from your own machine, before you start a hosted run:
npx @delegus/conformance --endpoint https://verifier.example/conformanceTest your verifier over HTTPS#
If your verifier is not JavaScript, or you would rather not run anything locally, the hosted runner does the same job over the network. You expose one HTTPS endpoint that speaks the small contract below; Delegus sends it every published test case and tells you, case by case, whether your answers match.
Start a run with any Delegus API key:
POST https://api.delegus.ai/conformance/runs
{"endpoint": "https://verifier.example/conformance"}The answer is 202 with a run_id. Then read the result:
GET https://api.delegus.ai/conformance/runs/{run_id}status is running, passed, failed or error, with the report for each test case.
The contract your endpoint speaks. Every request is a JSON POST carrying "v": 1 and "vectors": "<fingerprint of the test set>", plus one operation:
| Operation | Request adds | Your endpoint answers |
|---|---|---|
describe | — | {"ops": [...]} listing the operations you support, or {"unsupported": true} |
evaluate | "input" (one test case's input) | the evaluation: decision, reason, protocol_result, trust_result, protocol_checks, trust_checks, and optionally receipt |
normalizeHtu | "inputs": [...] | {"results": [...]} |
canonicalize | "inputs": [...] | {"results": [...]} |
sha256 | "inputs": [...] | {"results": [...]} |
isJti | "inputs": [...] | {"results": [...]} |
Every operation answers HTTP 200. An operation you do not implement answers {"unsupported": true}, and only then is its set of checks skipped. A 404 or any other error counts as a failure.
Limits. Public https endpoints only. At most 50 requests and two minutes per run, one run at a time per account, and ten new runs per hour per API key. Each report records the fingerprint of the exact test set it ran, so a result always says which cases it covers.
What a report is, and is not. It is Delegus's unsigned account of which cases your endpoint answered correctly. It is not a certificate or a badge, and no implementation outside Delegus has passed the full set yet.
Related: Specification §15 · Receipts and offline re-verification · Open format, closed service