Benchmarks and Evidence

NoKV benchmark claims must exercise the same path-native metadata model, revision-owned object layout, root routing, and durability profile as the product. A key codec or Holt engine result is useful diagnostic evidence; it is not automatically a CLI, SDK, recovery, failover, or end-to-end result.

Evidence Levels

Level	Boundary	What it can establish
Key/codec	Canonical key and durable record functions	Encoding cost, size, ordering, allocation
Holt engine	Named trees, point/range reads, atomic batches	Engine throughput, conflicts, WAL and checkpoint cost
Metadata domain	Workspace marker, paths, commands, operations	Namespace amplification, visibility, replay and lifecycle cost
Service	Protocol through shard owner and object provider	Routing, serialization, fencing, provider latency and recovery
Product	Native CLI, direct SDKs, optional MCP sidecar, or transport-free Workbench facade	User-visible latency, errors, retries and end-to-end throughput

Every report names its level. Results from different levels are not interchangeable.

Executable Metadata Diagnostic

nokv-bench metadata exercises the production protocol DTO executor over one MetaShard backed by a real HoltStore. Setup creates one root, one owner, one visible Workbench, and a deterministic PathCurrent tree with available ArtifactRevision and strong path references. The timed interval measures existing/missing exact reads and paginated recursive/non-recursive prefix reads; it does not measure object upload or publication.

Use a new, nonexistent metadata directory for each retained file-backed run:

cargo run --release --locked -p nokv-bench --features metadata-read-stats \
  --bin nokv-bench -- \
  metadata \
  --metadata-dir /absolute/path/to/new-metadata-dir \
  --iterations 1000 \
  --warmup 100 \
  --direct-children 96 \
  --leaves-per-child 64 \
  --page-limit 32 \
  --seed 42 \
  --revision <commit-or-patch-label> \
  --harness-revision <benchmark-tree-digest> \
  --dirty-worktree

Omit --metadata-dir only for an in-memory diagnostic. Label the actual dirty state accurately. For retained comparisons, set reviewed machine and physical metadata-device labels; missing details remain unknown rather than being inferred.

The v3 JSON report retains dataset and source labels, durability, warmup, latency distribution, throughput, result checksums, correctness assertions, logical-result digest, logical metadata-read amplification, and Holt cursor and store-counter evidence. visited is a Holt cursor work unit, not a physical row or device read. File-backed store-wide deltas are exactly attributable only for the runner’s dedicated-store, concurrency-one profile.

This command is a metadata-domain diagnostic. It does not frame network bytes, access object storage, invoke the SDK, or exercise the product CLI. Its own report marks the release-performance acceptance gate NOT QUALIFIED because it omits the required product, cold-cache, concurrency, host-utilization, and failure/recovery matrices.

Required Profile

Every retained row records source commit and dirty state, toolchain, machine and OS, metadata device, object-provider class, durability profile, topology, roots and workspaces, path and payload distribution, concurrency, duration, seed, and cache state.

Keep the durability boundaries separate:

• local_wal: acknowledge at the configured shard-local Holt WAL boundary;
• shared_log: acknowledge at the configured shared logical-log boundary.

Never merge or average rows across those profiles.

Workload Coverage

A release evidence set separates at least:

• exact read and ordered list, including missing paths and cursor pages;
• conditional create, replace, append, edit, conflict, and exact replay;
• indexed search and aggregate at one declared read version;
• snapshot, commit, tag, restore, abort, revision GC, and reconciliation;
• root distribution, owner replacement, restart, replay, and stale-owner rejection.

Object and metadata time must be reported separately for publication. Root placement never varies by filename; a workload that hashes paths across shards measures a different system.

Comparison And Qualification

Compare only matching operation semantics, payloads, concurrency, durability, cache state, topology, object-provider class, and machine class. State semantic differences next to any external-system result instead of collapsing unlike operations into one percentage.

A result is qualified only when correctness checks pass before and after the timed interval, the exact command and raw output are retained, skipped environment checks are marked NOT QUALIFIED, benchmark-only behavior does not change the product path, and the report links its acceptance gate. Retain p50, p95, p99, maximum, error counts, resource utilization, and recovery work—not only an average.

Historical Agent-Interface Evidence

The separate interactive benchmark preserves archived June 2026 agent-interface results pinned by the 2026-06-11 source revision 0b7cfa757fafe37c71db43117abdf8bc04efa7cf. Its boundary is a fixed, stateless raw-SQLite-versus-seven-verb namespace harness over committed telemetry. It is not a current nokv-bench workload, a storage/FUSE comparison, or a release qualification result.

No current FUSE, AI-training filesystem, or single-node roadmap result is presented on this page.