modestbench automatically tracks benchmark results over time in a local .modestbench/ directory. This history enables you to:

- Track performance trends - See how your code's performance changes across commits
- Detect regressions - Compare current results against previous runs to catch slowdowns
- Analyze improvements - Validate that optimizations actually improve performance
- Document progress - Export historical data for reports and analysis

`bash


List recent runs

Define budgets in your modestbench.config.json:

`json
{
"budgetMode": "fail",
"budgets": {
"benchmarks/critical.bench.js": {
"default": {
"parseConfig": {
"absolute": {
"maxTime": "10ms",
"minOpsPerSec": 100000
}
}
}
}
}
}
`

Budget Types:

- Absolute Budgets: Fixed thresholds
- maxTime - Maximum mean execution time (e.g., "10ms", "500us")
- minOpsPerSec - Minimum operations per second
- maxP99 - Maximum 99th percentile latency

- Relative Budgets: Comparison against baseline
- maxRegression - Maximum performance degradation (e.g., "10%", 0.1)

#### Wildcard Patterns

Apply budgets broadly using wildcards:

`json
{
"budgets": {
"*/.bench.js": {
"*": {
"*": {
"relative": { "maxRegression": "15%" }
}
}
},
"benchmarks/critical.bench.js": {
"*": {
"*": {
"relative": { "maxRegression": "5%" }
}
}
}
}
}
`

- Files: Use glob patterns (*/.bench.js, benchmarks/*.bench.js)
- Suites/Tasks: Use * to match any name
- Precedence: Most specific pattern wins (exact matches override wildcards)

Budget Modes:

- fail (default) - Exit with error code if budgets fail
- warn - Show warnings but don't fail
- report - Include in output without failing

#### Managing Baselines

`bash


Save current run as a baseline

`bash


Profile a command

- [command] - Command to profile (e.g., npm test, node script.js)
- --input, -i - Path to existing V8 profile log file
- --filter-file - Filter functions by file glob pattern
- --min-percent - Minimum execution percentage to show (default: 1.0)
- --top, -n - Number of top functions to show (default: 25)
- --group-by-file - Group results by source file
- --color - Enable/disable color output

How It Works:

The profile command uses Node.js's built-in V8 profiler to identify functions that consume the most execution time. It automatically filters out Node.js internals and node_modules to focus on your code.

Functions that appear at the top of the profile report are good candidates for benchmarking, as optimizing them will have the most impact on overall performance.

$3

Already have test files? Run them as benchmarks without writing any new code:

`bash


Run Jest tests as benchmarks

- jest - Jest test files with describe/it/test syntax
- mocha - Mocha test files with describe/it syntax
- node-test - Node.js built-in test runner (node:test)
- ava - AVA test files

How It Works:

The test command captures test definitions from your test files and runs each test as a benchmark task. Test suites map to benchmark suites, and individual tests become benchmark tasks. Setup/teardown hooks (beforeEach/afterEach) are preserved and run with each iteration.

This is useful for:

- Quick performance checks - See how fast your tests actually run
- Regression detection - Identify tests that have become slower
- Optimization targets - Find slow tests that might benefit from optimization

Test Command Options:

- --iterations, -i - Number of iterations per test (default: 100)
- --warmup, -w - Number of warmup iterations (default: 5)
- --bail, -b - Stop on first failure
- --json - Output results as JSON
- --quiet, -q - Minimal output

Configuration

$3

Create modestbench.config.json:

`jsonc
{
"bail": false, // Stop execution on first failure
"exclude": ["node_modules/**"], // Patterns to exclude from discovery
"excludeTags": ["slow", "experimental"], // Tags to exclude from execution
"iterations": 1000, // Number of samples per benchmark
"limitBy": "iterations", // Limit mode: 'iterations', 'time', 'any', 'all'
"outputDir": "./benchmark-results", // Directory for results and reports
"pattern": "bench/*/.bench.{js,ts}", // Glob pattern to discover benchmark files
"quiet": false, // Minimal output mode
"reporters": ["human", "json"], // Output reporters to use
"tags": ["fast", "critical"], // Tags to include (if empty, all benchmarks run)
"time": 5000, // Time budget in ms per benchmark
"timeout": 30000, // Task timeout in ms
"verbose": false, // Detailed output with debugging info
"warmup": 50, // Warmup iterations before measurement
}
`

Configuration Options:

- pattern - Glob pattern(s) to discover benchmark files (can be string or array)
- exclude - Glob patterns for files/directories to exclude from discovery
- excludeTags - Array of tags to exclude from execution; benchmarks with ANY of these tags will be skipped (default: [])
- iterations - Number of samples to collect per benchmark task (default: 100)
- time - Time budget in milliseconds per benchmark task (default: 1000)
- limitBy - How to limit benchmarks: "iterations" (sample count), "time" (time budget), "any" (whichever comes first), or "all" (both thresholds required)
- warmup - Number of warmup iterations before measurement begins (default: 0)
- timeout - Maximum time in milliseconds for a single task before timing out (default: 30000)
- bail - Stop execution on first benchmark failure (default: false)
- reporters - Array of reporter names to use for output (available: "human", "json", "csv")
- outputDir - Directory path for saving benchmark results and reports
- quiet - Minimal output mode, suppresses non-essential messages (default: false)
- tags - Array of tags to include; if non-empty, only benchmarks with ANY of these tags will run (default: [])
- verbose - Detailed output mode with additional debugging information (default: false)

> Note: Smart defaults apply for limitBy based on which options you provide. See Controlling Benchmark Limits for details.

$3

modestbench supports multiple configuration file formats, powered by cosmiconfig:

- JSON: modestbench.config.json, .modestbenchrc.json, .modestbenchrc
- YAML: modestbench.config.yaml, modestbench.config.yml, .modestbenchrc.yaml, .modestbenchrc.yml
- JavaScript: modestbench.config.js, modestbench.config.mjs, .modestbenchrc.js, .modestbenchrc.mjs
- TypeScript: modestbench.config.ts
- package.json: Use a "modestbench" field

Generate a configuration file using:

`bash
modestbench init --config-type json # JSON format
modestbench init --config-type yaml # YAML format
modestbench init --config-type js # JavaScript format
modestbench init --config-type ts # TypeScript format
`

Configuration Discovery: modestbench automatically searches for configuration files in the current directory and parent directories, following standard conventions.

Output Formats

$3

Real-time progress bars with color-coded results and performance summaries.

$3

Structured data perfect for programmatic analysis and integration:

`json
{
"results": [
{
"file": "example.bench.js",
"suite": "Array Operations",
"task": "Array.push()",
"hz": 1234567.89,
"stats": {
"mean": 0.00081,
"stdDev": 0.00002,
"marginOfError": 2.45
}
}
],
"run": {
"id": "run-2025-10-07-001",
"timestamp": "2025-10-07T10:30:00.000Z",
"duration": 15420,
"status": "completed"
}
}
`

$3

Tabular data for spreadsheet analysis and historical tracking.

Advanced Features

$3

`javascript
const state = {
data: [],
sortedData: [],
};

export default {
suites: {
Sorting: {
setup() {
state.data = generateTestData(1000);
},
benchmarks: {
// Shorthand syntax for simple benchmarks
'Quick Sort': () => quickSort(state.data),
'Merge Sort': () => mergeSort(state.data),
},
},

Searching: {
setup() {
state.sortedData = generateSortedData(10000);
},
benchmarks: {
'Binary Search': () => binarySearch(state.sortedData, 5000),
'Linear Search': () => linearSearch(state.sortedData, 5000),
},
},
},
};
`

$3

`javascript
export default {
suites: {
'Async Performance': {
benchmarks: {
// Shorthand syntax works with async functions too
'Promise.resolve()': async () => {
return await Promise.resolve('test');
},

// Full syntax when you need config, tags, or metadata
'Fetch Simulation': {
async fn() {
const response = await simulateApiCall();
return response.json();
},
config: {
iterations: 100, // Fewer iterations for slow operations
},
},
},
},
},
};
`

$3

modestbench supports a powerful tagging system that lets you organize and selectively run benchmarks. Tags can be applied at three levels: file, suite, and task. Tags automatically cascade from parent to child, so tasks inherit tags from their suite and file.

#### Adding Tags

Tags can be added at any level:

`javascript
export default {
// File-level tags (inherited by all suites and tasks)
tags: ['performance', 'core'],

suites: {
'String Operations': {
// Suite-level tags (inherited by all tasks in this suite)
tags: ['string', 'fast'],

benchmarks: {
// Task inherits: ['performance', 'core', 'string', 'fast', 'regex']
'RegExp Test': {
fn: () => /pattern/.test(str),
tags: ['regex'], // Task-specific tags
},

// Task inherits: ['performance', 'core', 'string', 'fast']
'String Includes': () => str.includes('pattern'),
},
},
},
};
`

#### Filtering Benchmarks

Use --tag (or -t) to include only benchmarks with specific tags (OR logic - matches ANY tag):

`bash


Run fast algorithms

Use --exclude-tag (or -T) to skip benchmarks with specific tags:

`bash


Exclude slow benchmarks

`bash


Run fast benchmarks, but exclude experimental ones

`javascript
export default {
tags: ['file-level'], // All tasks get this tag

suites: {
'Fast Suite': {
tags: ['fast'], // Tasks get: ['file-level', 'fast']
benchmarks: {
'Task A': {
fn: () => {},
tags: ['math'], // This task has: ['file-level', 'fast', 'math']
},
'Task B': () => {}, // This task has: ['file-level', 'fast']
},
},
},
};
`

Filtering Behavior:

- --tag math → Runs only Task A (matches 'math')
- --tag fast → Runs both Task A and Task B (both have 'fast')
- --tag file-level → Runs both tasks (both inherit 'file-level')
- --exclude-tag math → Runs only Task B (Task A excluded)

#### Suite Lifecycle with Filtering

Suite setup() and teardown() only run if at least one task in the suite matches the filter. This prevents unnecessary setup work for filtered-out suites.

Integration Examples

$3

`yaml
name: Performance Tests
on: [push, pull_request]

jobs:
benchmark:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/setup-node@v3
with:
node-version: 18

- run: npm ci
- run: npm run build

- name: Run Benchmarks
run: |
modestbench \
--reporter json \
--reporter csv \
--output ./results

- name: Upload Results
uses: actions/upload-artifact@v3
with:
name: benchmark-results
path: ./results/
`

$3

`javascript
// scripts/check-regression.js
import { execSync } from 'child_process';
import { readFileSync } from 'fs';

// Run current benchmarks
execSync('modestbench --reporter json --output ./current');
const current = JSON.parse(readFileSync('./current/results.json'));

// Load baseline results
const baseline = JSON.parse(readFileSync('./baseline/results.json'));

// Check for significant regressions
for (const result of current.results) {
const baselineResult = baseline.results.find(
(r) => r.file === result.file && r.task === result.task,
);

if (baselineResult) {
const regression = (baselineResult.hz - result.hz) / baselineResult.hz;
if (regression > 0.1) {
// 10% regression threshold
console.error(
Performance regression detected in ${result.task}: ${(regression * 100).toFixed(1)}%,
);
process.exit(1);
}
}
}

console.log('No performance regressions detected ✅');
`

Programmatic API

`typescript
import { modestbench, HumanReporter } from 'modestbench';

// initialize the engine
const engine = modestbench();

engine.registerReporter('human', new HumanReporter());

// Execute benchmarks
const result = await engine.execute({
pattern: '*/.bench.js',
iterations: 1000,
});
`

Contributing

We welcome contributions! Please see our [Contributing Guide][contributing] for details.

$3

`bash


Clone the repository

- Built on top of the small-but-mighty benchmarking library, [tinybench][]
- Interface inspired by good ol' [Benchmark.js][]
- Built with [zshy][] for dual ESM/CJS modules
- AccurateEngine` statistical analysis inspired by the excellent work of [bench-node][]

Resources

- [Issue Tracker][bugs]
- [Discussions][]

License

Copyright © 2025 Christopher Hiller. Licensed under the [Blue Oak Model License 1.0.0][license].

[license]: https://blueoakcouncil.org/license/1.0.0
[tinybench]: https://github.com/tinylibs/tinybench
[benchmark.js]: https://benchmarkjs.com/
[bugs]: https://github.com/boneskull/modestbench/issues
[discussions]: https://github.com/boneskull/modestbench/discussions
[zshy]: https://github.com/colinhacks/zshy
[contributing]: CONTRIBUTING.md
[bench-node]: https://github.com/bench-node/bench-node