Benchmark

Varc benchmarks are named, deterministic workloads stored in varc.toml. A benchmark can use a built-in preset, a weighted command mix, or a captured workload artifact exported by the desktop app.

Configuration and artifact validation are free. Running a benchmark requires a Pro or Team CLI Token with the load-test capability.

Create and validate a benchmark

Use the interactive initializer to create or edit a named benchmark:

varc benchmark init baseline
varc benchmark validate baseline

Add --dry-run to preview the proposed TOML without changing the file. The initializer preserves unrelated sections and comments, never stores a CLI Token, and refuses to prompt when input is non-interactive.

A minimal read-only benchmark looks like this:

schema_version = 1
default_connection = "local"

[licensing]
token_env = "VARC_CLI_TOKEN"

[connections.local]
url_env = "REDIS_URL"

[benchmarks.baseline]
requests = 100000
clients_per_runner = 8

[benchmarks.baseline.workload]
type = "preset"
preset = "get_only"

Set exactly one termination condition: requests or duration. See the varc.toml reference for every benchmark field, limit, and default.

Run locally

Resolve and test the target before generating load:

export REDIS_URL='redis://127.0.0.1:6379'
export VARC_CLI_TOKEN='varc_cli_...'

varc connection test
varc topology inspect
varc benchmark validate baseline
varc benchmark run baseline

On an interactive local terminal, run automatically opens the live terminal dashboard. Press q, Escape, or Ctrl-C to cancel. Use --no-tui for plain progress or --tui to require the dashboard. JSON output and distributed runs do not use the local dashboard.

Choose a workload

Presets

Preset names are get_only, set_only, mixed_50_50, read_heavy, write_heavy, realistic_cache, realistic_session, pipeline, high_concurrency, and large_payload.

Only get_only is classified as read-only. Presets never seed keys implicitly.

Weighted commands

Use weighted operations for an application-shaped mix:

[benchmarks.session_mix]
connection = "local"
requests = 250000
clients_per_runner = 16
key_prefix = "varc-session:"

[benchmarks.session_mix.safety]
allow_writes = true
environment = "development"

[benchmarks.session_mix.workload]
type = "weighted"
key_space = 100000
key_distribution = { type = "zipfian", exponent = 0.99 }
value_distribution = { type = "compressible", bytes = 512 }

[[benchmarks.session_mix.workload.operations]]
label = "read-session"
weight = 9
command = "HGET"
args = [{ type = "key" }, { type = "literal", text = "payload" }]

[[benchmarks.session_mix.workload.operations]]
label = "write-session"
weight = 1
command = "HSET"
args = [
  { type = "key" },
  { type = "literal", text = "payload" },
  { type = "value" },
]

Literal and fixed values support text, hex, or Base64 encodings, so command templates remain binary-safe. Every command also passes Varc’s conservative command policy before execution.

Captured workload artifacts

The desktop app can export bounded .varc-workload.jsonl profiles and exact traces. Inspect and validate an artifact before using it:

varc benchmark artifact inspect captured.varc-workload.jsonl
varc benchmark artifact validate captured.varc-workload.jsonl

[benchmarks.captured]
connection = "local"
requests = 10000

[benchmarks.captured.safety]
allow_writes = true
environment = "development"

[benchmarks.captured.workload]
type = "artifact"
path = "captured.varc-workload.jsonl"
speed = 1.0
iterations = 1
exact_data_acknowledged = true
incomplete_trace_acknowledged = true

An artifact supplies workload data; it does not replace varc.toml. Exact data and incomplete traces each require their own explicit acknowledgement.

Make writes deliberate

Every workload that may write requires safety.allow_writes = true. This includes all presets except get_only, captured artifacts, explicit seeding, and weighted commands outside the conservative read-only set.

Use a benchmark-only key_prefix. When environment is prod or production, acknowledgement must exactly match that value:

[benchmarks.production_probe.safety]
allow_writes = true
environment = "production"
acknowledgement = "production"

This is an operator gate, not a recommendation to run synthetic load against a production service.

Enforce results in CI

Assertions evaluate the final local or merged report:

[benchmarks.ci_read.assertions]
max_error_rate = 0.01
max_p95_ms = 20.0
min_requests = 100000

[benchmarks.ci_read.outputs]
json = "artifacts/ci-read.json"
jsonl = "artifacts/ci-read.jsonl"
junit = "artifacts/ci-read.xml"
replace = true

varc --non-interactive benchmark validate ci_read
varc --non-interactive --format json benchmark run ci_read

Progress stays on stderr and one stable terminal result is written to stdout. An assertion failure exits with code 6; an incomplete or indeterminate result exits with code 5.

Render a saved aggregate without connecting or loading configuration:

varc benchmark report artifacts/ci-read.json
varc --format json benchmark report artifacts/ci-read.json

Run across multiple machines

Add a distributed table to the same benchmark:

[benchmarks.ci_read.distributed]
runners = 4
run_id_env = "VARC_LOAD_RUN_ID"
join_timeout = "10m"
prepare_timeout = "10m"
start_delay = "10s"
report_timeout = "10m"

Start the identical command on exactly four machines with the same non-secret run ID, target, configuration, and artifact inputs:

export VARC_LOAD_RUN_ID="build-${CI_BUILD_NUMBER}"
varc --non-interactive --format json benchmark run ci_read

Do not assign a controller or leader. Runners coordinate through the measured server, reject mismatched inputs before execution, freeze membership, partition global work, and merge one report. If a runner disappears during measurement, the others finish their bounded work and the result is incomplete; Varc does not rebalance or replay indeterminate writes.

Use --distributed COUNT to enable or override a runner count and --single to execute a configured distributed benchmark in one process.

Next steps

• Set up the CLI
• Read the complete TOML reference
• Plan and run a migration
• Validate and run scripts