Open beta — free until the beta window closes: checkout is switched off, no card, nothing to pay. Real-engine allowance: 3 welcome + 30 beta-pack runs per account; simulation stand 25 runs / 24 h. Every engine run starts with the canonical payment gate: fail it and the run is refunded (net 0) — you learn for free whether your payment layer stalls even against a well-behaved counterparty.

Dashboard

adversarial testing lab — AI agents that move money, and the rails they settle on

We break AI agents that move money.
We sign our findings, not our opinions.

Argus runs pre-registered, black-box batteries against buyer and merchant agents — and the payment path they rely on: x402 payment flows, credit and settlement handling, idempotency guarantees. An agent is only as solvent as the code that signs for it. Every verdict is a sealed record in a public hash chain that anyone can verify in one command.

sealed records, hash-chained
22
sealed records, hash-chained
systems under test
6
systems under test
pre-registered probes
60+
pre-registered probes
findings retracted
0
findings retracted

the registry

A chain of seals, not a list of claims

Each node is an Ed25519-signed record chained to the hash of its predecessor: finding or battery, verdicts, findings, hash-pinned evidence. Select a seal to inspect it.

latest seal

registry intact · 22 records · verify-registry.mjs ↗

argus-seal-00212026-11-12agent-passport-system/agent-passport-gateway @ 99888e8f143eb0c281ccd62759750038c1a1aa9e

Nano cross-process idempotency battery @ agent-passport-gateway 99888e8 — pre-registered R1/R2a/R2b/R3/R4 + invariant (PR #14)

verdicts

r1-1: PASSr1-2: PASSr1-3: PASSr2a-1: PASSr2a-2: PASSr2b-1: PASSr2b-2: PASSr2b-3: PASSr3-1: PASSr3-2: PASSr3-3: PASSr4-1: PASSinvariant: PASS

chain 07a5412b → 47acb4ef ✓ · Ed25519 · prev_record_hash → chain head

latest correction argus-correction-0017 · CORRECTION-0012-01 — seal #0012 fixture label erratum: probe_start.control_under_test stale template default; build/install receipt added ↗

hash-pinned evidence (7)

$ git clone github.com/igor-vii/Argus-Agent-Test-Lab && node tools/registry/verify-registry.mjs → registry intact: 22 records, chain head 47acb4ef…

attestations/records — hash chain explorer
argus-seal-00212026-11-12agent-passport-system/agent-passport-gateway @ 99888e8f143eb0c281ccd62759750038c1a1aa9e

Nano cross-process idempotency battery @ agent-passport-gateway 99888e8 — pre-registered R1/R2a/R2b/R3/R4 + invariant (PR #14)

Battery result: 12 trials + invariant = 13/13 rows PASS, 106/106 checks, zero divergences (verdicts.md). R1 (both orderings + simultaneous): exactly one reservation wins in every ordering; the loser observes the original operation (replay: true, same op id) without reserving or signing again; delegation budget decremented exactly once (1000 raw on both hops of the two-hop chain in every trial); the winner alone prepares (one account_info, one work_generate), commits and publishes once — exactly one process call, and the published block is byte-identical to the stored block_json. R2a (pre-existing key, both processes race with changed amount/recipient): both receive idempotency_mismatch (= 409 under the pinned route mapping); zero new reservations (the pre-op's holds only); the stored operation row is byte-identical after the race; budget untouched; zero node calls. R2b (fresh key, different amounts 1000 vs 2000, both orderings + simultaneous): exactly one reservation — the first writer under BEGIN IMMEDIATE commits, the loser receives idempotency_mismatch (409) against the winner's stored fingerprint; budget decremented exactly once, by the winner only (1000 on both hops when A wins, 2000 on both hops when B wins). R3 (different keys, same wallet, first payment unfinished, both orderings + simultaneous): the second operation is refused with wallet_busy (409); exactly one wallet_ops row, status reserved (unfinished); neither process signs — zero node calls at all; budget decremented once. R4 (retry after publication, across processes): process A reserves, prepares the one block, commits and publishes (accepted); process B retries the same key and payment, receives the original operation (replay: true, same op id, status admitted), never calls prepareOp — B's node log contains exactly one action: process; B takes over the expired lease, commits one more attempt and publishes the SAME stored block — the two process calls are byte-identical and both match the stored block_json; attempt history [1: accepted, 2: accepted]; attempt_count = 2; one operation, one reservation, budget decremented once. Invariant: reservation count == 1 in every R1 trial (distinct reserved op ids, one set of holds); no budget underflow (spend_used_minor <= spend_limit_minor on every delegation in every trial); no double-reserve anywhere.

verdicts

r1-1: PASSr1-2: PASSr1-3: PASSr2a-1: PASSr2a-2: PASSr2b-1: PASSr2b-2: PASSr2b-3: PASSr3-1: PASSr3-2: PASSr3-3: PASSr4-1: PASSinvariant: PASS

7 hash-pinned evidence files

findings

  • nano-pin-acquisition (info) Pin acquisition: commit 99888e8 dated 2026-11-11 22:18:15 -0800 (= 2026-11-12T06:18:15Z), an ancestor of current main (PR #14 merged as 4784153) — plain clone+checkout, no fetch-by-SHA. All validation and the battery ran on the pinned checkout only, per the owner's explicit request.
  • nano-gate-line-by-line (info) Layer-1 gate passed before the first artifact: all thread quotes verified verbatim at the pin (S1-S8, layer1-gate.md) — reserveSend single .immediate() transaction with key+budgets+inserts, replayOrMismatch fingerprint semantics, refuseIfAccountBusy wallet_busy, 409 route mapping, commitAttempt/publishAttempt one-token stored-block semantics (vendor's own C3 pins retry-identical-block), takeOver claim fencing, WAL/busy_timeout substrate. Ten pinned files hash-pinned. Vendor's own wallet-ops suite: 22/22 PASS at the pin in this environment before the battery was built.
  • nano-harness-iterations (info) Harness iterations disclosed: an infrastructure smoke trial (distinct key, not a registered row) and one full-battery iteration exposed three harness defects (missing dbFile param; fixture delegation chain not linked so the budget decremented one hop only; R4 role-A control-flow fall-through into the retry branch). All three are battery-harness bugs, not SUT divergences; no registered expectation was altered to accommodate them. The sealed verdicts are from the corrected runner; observed SUT behavior was identical across iterations.
  • nano-scope-boundary (info) Scope facts: the battery drives the vendor's real code cross-process; sim trials observe a true simultaneous race (winner recorded, assertions order-agnostic), sequenced trials fix the interleave. The fake Nano node records every call (account_info/work_generate/process) — signature count is observable through the node log since prepareOp is the only signing path. No HTTP routes booted: 409 expectations verified at the WalletOpError layer with the pinned route mapping cited. No defect claim — all registered rows PASS on the pinned implementation.

$ node tools/registry/verify-registry.mjs → registry intact: 22 records, chain head 47acb4ef32bab1df… records ↗

method

Four rules that make a verdict worth something

The lab's credibility is structural: it comes from what a probe must go through before anyone is allowed to call it a failure.

01

Black-box by construction

The SUT is always the published artifact — npm dist, exact pin, unmodified. No source access, no vendor cooperation required. If it can't be shown against the artifact users actually install, it isn't shown.

02

Pre-registered before the run

Required behaviors and verdict rules are fixed in committed matrix files before a single probe fires. No post-hoc reshaping of what a failure means — the rig can't drift toward the conclusion.

03

Offline by default

All traffic on 127.0.0.1 against adversarial stands and honest mock backends modeled from the vendor's own contracts, each decision citing a vendor source line. No vendor-operated service is touched without written authorization.

04

Sealed and hash-chained

Every record is signed with Ed25519 and chained to the previous record's hash. Anyone can verify the whole chain from the public repo in one command. Seals attest run validity and findings completeness — not bug-freeness.

verdict taxonomy

FAIL
Vendor-side failure under the probe — the SDK or platform loses money or integrity under the tested fault.
HOLD
The SDK rejected or limited the fault — a genuine guard fired where the battery expected one.
PASS
The platform held beyond the documented scenarios.
OBSERVED
Observed behavior matches the vendor's published disclosure — an honest-posture result, not a defect.
DX
Developer-experience finding: no direct money-loss path, but the tooling hides a hazard the runtime itself flags.

cases

Every verdict, public and linked

Six fronts so far — from a live merchant agent to the SDKs and rails that sign and settle on their behalf. Each card links back to sealed records and public threads; nothing on this page exists outside the registry.

x402-foundation

x402 reference client + PR #3920 core · @x402/core 2.28.0

A11 closedA10c/d PASSA10a/b open

Issue #3866 rounds 2–4 + PR #3920 cycles 1–3: A11 verified closed, conflicting-offer invariant holds, normalization delta verified.

  • Origin finding (#0000): the reference client signs any server-declared 402 amount.
  • Round 2–4 engagement on issue #3866 with control batteries and delta probes.
  • Cycle-2 at PR head b1a4d8ce: non-atomic amount rejection verified closed; binding race (signer ×2) reproduces.
  • Cycle-3 at e993bef (17/19 PASS): conflicting-offer races A10c/A10d PASS — one commitment + one pre-signer rejection; case-variant recipient normalization A12/A13 PASS.
  • Author accepted the cycle-3 results; remaining gate: upstream maintainer integration review.
seals#0000#0003#0004#0008#0012

Upstream engagement · cycles 1–3

BlockRunAI

@blockrun/nano-client · 0.8.0 (npm exact pin)

3 FAIL3 HOLD

A signature over the attacker's payTo, ×50 with no cap, and a README claim its own advice breaks.

  • F1: shape-only payTo check signs authorization over attacker-controlled recipient.
  • F2: no default spend cap — ×50 of quoted price signed on default config.
  • F3: lost-response dead-end; the README's own recovery advice routes into a double charge.
  • Earlier report #15 closed as not planned; the sealed battery answers the closure premise.

Public · awaiting vendor

Nevermined

@nevermined-io/payments · 1.13.0 (npm exact pin)

7 FAIL1 DX3 HOLD

Seven failures in credit and settlement handling — including a double charge on retry.

  • F1: lost-response retry double-charges on the default token; no idempotency key, no settlement query.
  • F2: single-use guard is per-process and reactive — concurrent race delivers a free service.
  • F3: any settle failure except one special case delivers the resource unpaid — with a receipt.
  • The class was already flagged in the vendor's own tracker (#374/#367, open since June); the battery makes it sealed and reproducible.

Sealed · publication ready

Inngest

inngest + @inngest/test · 4.26.0 / 1.0.0 (npm dist)

0 FAIL2 DX

Thirteen probes, zero failures: behavior matches the vendor's own disclosure on every probe.

  • Commit-then-crash window verified live against the documented A3 model, including supervisor-restart variant.
  • Event-ID dedup holds under a 12-way concurrent race; prescribed mitigations verified effective.
  • Two developer-experience findings in the vendor's test engine; triaged upstream (Linear EXE-2315).
  • Lab close-out with seal #0005 published to the issue thread on closure.
seals#0005

Closed · close-out published

tripl3tr3s

mcp-tool-idempotency · 4d665ae → v0.2.0

3 fixed10/10 verified

Reported privately, fixed publicly, verified independently — credited in SECURITY.md.

  • Three durability windows: lost response after commit, crash after commit, cross-instance race.
  • Coordinated disclosure before publication; maintainer shipped owner-gated claim leases.
  • Ten-window battery on v0.2.0: 10/10 PASS, suite 91/91 at tag.

Fixed · verified · credited

AgentMiddleware

amw-paypal-agent-demo · live demo (Block D)

REPRODUCED1 high finding

The lab's first live round — and the clean-machine reproduction that set the standard.

  • S10/S11 executed live against a real merchant agent demo with raw evidence captured.
  • Independent clean-machine reproduction from committed sources: verdicts identical.
  • The two-seal discipline (run + reproduction) applied ever since.
seals#0001#0002

Live round · reproduced

engage

Run your code before your users do

Two working tracks, both ending in the same artifact: a sealed verdict anyone can verify. Engagements begin with written scope; nothing about an engagement changes a verdict.

A

Coordinated production round

Point us at a live agent that moves real money; we run the battery under written scope authorization: funded test wallet, agreed time window on 48 hours' notice, a no-debit/refund arrangement for probe transactions, full request capture shared with you. Deliverable: a sealed production-round report — either documenting that the production endpoint held against the full battery, or documenting the exact boundary where defects become live money movement.

from $990 / round

B

Fix verification

Our batteries become your acceptance rig. You ship a patched artifact; we re-run the sealed suite against the published release and seal PASS/FAIL per finding. A clean seal is an independently checkable artifact you can cite to integrators and in your security narrative.

Per-finding sealed verdict

terms of engagement

  • Fixed per-round fee or a short retainer, agreed before any run.
  • Reports and seals remain the lab's to publish; vendor responses get equal space.
  • Production rounds require written scope, a funded test wallet, and a no-debit arrangement.
  • Private resolution is welcome — the seal is not a weapon, it is a receipt.

github.com/igor-vii/Argus-Agent-Test-Lab · issues open

  • §1Praise in public where it is earned — as loudly as failures get stated.
  • §2Break in public where it is broken — with line numbers, not adjectives.
  • §3Vendor responses get equal space alongside every published finding.
  • §4No vendor operates our stands; no engagement buys a verdict.