perfshield

perfshield is a build-system-agnostic benchmarking tool for comparing two
versions of a JavaScript library across JS runtimes. It borrows statistical
ideas from Tachometer (confidence intervals, adaptive sampling) and focuses on
runtime JS engines (Node/V8 today).

What it does

- Builds your benchmark bundle using your own build command.
- Saves a baseline bundle (prepare).
- Builds the current bundle and compares it to the baseline (compare).
- Reports results in console and/or JSON.
- Exits with code 1 when a regression is detected (both relative and absolute CIs exclude 0 in the slower direction).

Requirements

- Node.js (used for the CLI and the Node engine).
- A build command that emits a single ESM benchmark bundle.

Install

``
pnpm add -D perfshield
`

Quick start

1. Create a benchmark module that exports benchmarks:

`js
export const benchmarks = [
{
name: "sum",
iterations: 1,
async fn() {
let total = 0;
for (let i = 0; i < 10_000; i += 1) {
total += i;
}
return total;
},
},
];
`

2. Create a perfshield.config.json:

`json
{
"artifactsDir": ".perfshield",
"build": {
"command": "pnpm run build:bench",
"output": "dist/bench.js"
},
"engines": [
{
"name": "node",
"command": "node"
}
],
"sampling": {
"minSamples": 30,
"timeoutMs": 10000,
"conditions": [0]
},
"report": {
"formats": ["console", "json"]
}
}
`

3. Run the workflow:

`
perfshield prepare --config perfshield.config.json
perfshield compare --config perfshield.config.json
`

4. (Optional) Calibrate sampling defaults based on the prepared baseline:

`
perfshield calibrate --config perfshield.config.json
`

This prints a JSON snippet with recommended sampling values you can paste into
your config.

Benchmark bundle contract

The build output must be a single ESM file that exports:

`js
export const benchmarks = [
{
name: "example",
fn: async () => {},
setup: async () => {},
teardown: async () => {},
unit: "ms",
iterations: 1,
metadata: { tag: "optional" },
},
];
`

Rules:

- benchmarks is required.
- fn is required.
- setup/teardown run once per sample (per benchmark, per version).
- If setup throws, the sample fails and the run is aborted.
- iterations repeats fn inside a single sample to reduce noise.

Artifacts

- prepare stores a baseline bundle in artifactsDir (default .perfshield).
- compare uses the stored baseline and the newly built bundle.

Reports

Supported formats:

- console: human‑readable summary.
- json: machine‑readable report.

If any benchmark shows a regression (both relative and absolute CIs exclude 0 in the slower direction),
the process exits with code 1.

Examples

See examples/simple` for a minimal setup you can run locally.