MEASURE WHAT MATTERS

A faster number needs
a fair comparison.

Understand the benchmark conditions before turning a result into a product decision.

Three different kinds of evidenceAnthropic’s operational results are company-reported. OpenScion’s timings below are upstream-reported. Our own published check is functional and contains no performance measurements.

Upstream-reported synthetic timings

The OpenScion project reports the following medians for 10,000 synthetic 4 KiB files on an Apple Silicon laptop, release build, five runs, dated October 14, 2026. We have not reproduced these timings.

ScenarioGitOpenScion
Import3,966 ms245 ms
Commit one file21.8 ms22.3 ms
Full clone2,343 ms544 ms
Sparse clone158 ms25.3 ms

Source: OpenScion benchmark README at the reviewed revision. The 10,000-file single-file commit is not faster on the OpenScion side.

Why these are not production speedups

  • OpenScion keeps all server data in memory and does not fsync or compress it. Git writes durable, compressed objects.
  • The benchmark invokes OpenScion’s client in-process; Git pays command-process startup costs.
  • OpenScion transfers raw objects; Git’s packing and compression can change the result on a real network.
  • A synthetic small-history fixture does not reproduce a long-lived, heavily concurrent monorepo.

Our local functional evidence

Generated files survived complete and sparse clone round-trips. A commit could be cloned back, and a stale ref update was refused. The same check observed unauthenticated ref reads/writes and loss of repository state on restart. These are behavior checks, not throughput claims.

Download the local test record →

Design your own comparison

  1. Freeze one source revision and the exact paths required for the real task.
  2. Compare against partial/sparse Git clones and an existing mirror or shared-cache baseline.
  3. Separate cold from warm caches and record cache topology, hardware, network and concurrency.
  4. Verify the resulting files and run the actual build or Agent task.
  5. Measure p50/p95 readiness time, bytes transferred, compute, storage, failures and recovery.

Read the full evaluation guide or download the blank benchmark worksheet. The worksheet has no pre-filled results.

What would change our view?

A durable implementation, a reproducible workload and a measured advantage over an optimized Git baseline. Any adoption decision should include the new service’s own operating and integration costs.