numopt-js

A flexible numerical optimization library for JavaScript/TypeScript that works smoothly in browsers. This library addresses the lack of flexible continuous optimization libraries for JavaScript that work well in browser environments.

Documentation

- API Reference (GitHub Pages): https://takuto-na.github.io/numopt-js/
- Source Repository: https://github.com/takuto-NA/numopt-js

Features

- Gradient Descent: Simple, robust optimization algorithm with line search support
- Line Search: Backtracking line search with Armijo condition for optimal step sizes (following Nocedal & Wright, Numerical Optimization (2nd ed.), Algorithm 3.1)
- Strong Wolfe Line Search: Robust line search satisfying Strong Wolfe conditions (recommended for quasi-Newton methods)
- BFGS: Quasi-Newton method that updates a dense inverse Hessian approximation (unconstrained smooth optimization)
- L-BFGS: Memory-efficient quasi-Newton method (two-loop recursion) for larger parameter counts
- CMA-ES: Derivative-free black-box optimization using a covariance-adapting search distribution (unconstrained)
- Gauss-Newton Method: Efficient method for nonlinear least squares problems
- Levenberg-Marquardt Algorithm: Robust algorithm combining Gauss-Newton with damping
- Constrained Gauss-Newton: Efficient constrained nonlinear least squares using effective Jacobian
- Constrained Levenberg-Marquardt: Robust constrained nonlinear least squares with damping
- Adjoint Method: Efficient constrained optimization using adjoint variables (solves only one linear system per iteration instead of parameterCount systems)
- Numerical Differentiation: Automatic gradient and Jacobian computation via finite differences
- Browser-Compatible: Works seamlessly in modern browsers
- TypeScript-First: Full TypeScript support with comprehensive type definitions
- Debug-Friendly: Progress callbacks, verbose logging, and detailed diagnostics

Requirements

- Node.js >= 18.0.0
- Modern browsers with ES2020 support (for browser builds)

Installation

``bash
npm install numopt-js
`

Start Here (Pick Your Path)

- Minimize a scalar cost (smooth unconstrained optimization): use Gradient Descent (cost: (p) => number, grad: (p) => Float64Array). Start at Gradient Descent.
- Minimize a scalar cost (faster than GD for many problems): use BFGS or L-BFGS (cost: (p) => number, grad: (p) => Float64Array). Start at BFGS / L-BFGS.
- Minimize a scalar cost (black-box, no gradient): use CMA-ES (cost: (p) => number). Start at CMA-ES.
- Fit a model with residuals (nonlinear least squares): use Levenberg–Marquardt or Gauss–Newton (residual: (p) => Float64Array, optional jacobian: (p) => Matrix). Start at Levenberg-Marquardt.
- Equality-constrained problems \(c(p, x) = 0\): use Adjoint / Constrained GN/LM (constraint: (p, x) => Float64Array). Start at Adjoint Method.
- Browser usage: start at Browser Usage.

Why Float64Array? This library uses Float64Array for predictable numeric performance. You can always convert from normal arrays with new Float64Array([1, 2, 3]) (more details below).

Cost Function vs Residual Function (Important)

- Cost function: cost(p) -> number (used by gradientDescent)
- Residual function: residual(p) -> Float64Array (used by gaussNewton / levenbergMarquardt), where the library minimizes \(f(p) = 1/2 \|r(p)\|^2\)

Result Object (What to Look At)

Most algorithms return a result with these fields:

- Common (all algorithms): finalParameters, converged, iterations, finalCost
- Gradient Descent: finalGradientNorm, usedLineSearch
- BFGS / L-BFGS: finalGradientNorm
- CMA-ES: functionEvaluations, finalStepSize, finalMaxStdDev
- Gauss-Newton / Levenberg–Marquardt: finalResidualNorm (and LM also has finalLambda)
- Constrained algorithms / Adjoint: finalStates, finalConstraintNorm

Note: result.parameters is a deprecated alias of result.finalParameters and will be removed in a future release.

60-second Quick Start (Node)

Pick one of the following and run it.

$3

Create quick.mjs:

`js
import { gradientDescent } from 'numopt-js';

const cost = (params) => params[0] params[0] + params[1] params[1];
const grad = (params) => new Float64Array([2 params[0], 2 params[1]]);

const result = gradientDescent(new Float64Array([5, -3]), cost, grad, {
maxIterations: 200,
tolerance: 1e-6,
useLineSearch: true,
});

console.log(result.finalParameters);
`

BFGS / L-BFGS

Use these for smooth unconstrained problems when you can provide a gradient.

`js
import { bfgs, lbfgs } from 'numopt-js';

const cost = (params) => (params[0] - 1) 2 + (params[1] + 2) 2;
const grad = (params) => new Float64Array([2 (params[0] - 1), 2 (params[1] + 2)]);

const bfgsResult = bfgs(new Float64Array([10, 10]), cost, grad, {
maxIterations: 200,
tolerance: 1e-8
});

const lbfgsResult = lbfgs(new Float64Array([10, 10]), cost, grad, {
maxIterations: 200,
tolerance: 1e-8,
historySize: 10
});

console.log(bfgsResult.finalParameters, lbfgsResult.finalParameters);
`

Run:

`bash
node quick.mjs
`

$3

Create quick.cjs:

`js
const { gradientDescent } = require('numopt-js');

const cost = (params) => params[0] params[0] + params[1] params[1];
const grad = (params) => new Float64Array([2 params[0], 2 params[1]]);

const result = gradientDescent(new Float64Array([5, -3]), cost, grad, {
maxIterations: 200,
tolerance: 1e-6,
useLineSearch: true,
});

console.log(result.finalParameters);
`

Run:

`bash
node quick.cjs
`

CMA-ES (Black-box Optimization)

Use CMA-ES when your cost function is a black box and you don’t have a gradient.

Two important inputs:
- initialStepSize (sigma0): your initial error guess / search radius
- randomSeed: set this for reproducible runs (recommended for testing and debugging)
- restartStrategy: use "ipop" for multi-modal problems (λ doubles per restart)
- profiling: enable lightweight timing breakdown in the result

`js
import { cmaEs } from 'numopt-js';

const sphere = (params) => params.reduce((sum, v) => sum + v * v, 0);

const result = cmaEs(new Float64Array([10, -7, 3, 5]), sphere, {
maxIterations: 200,
populationSize: 20,
initialStepSize: 2.0,
randomSeed: 123456,
targetCost: 1e-10,
restartStrategy: "none",
profiling: true,
});

console.log(result.finalParameters, result.finalCost, result.stopReason, result.profiling);
`

$3

For multi-modal problems (e.g., Rastrigin), use IPOP restarts:

`js
const result = cmaEs(new Float64Array([5, 5, 5, 5, 5, 5, 5, 5, 5, 5]), sphere, {
maxIterations: 1200,
populationSize: 20,
initialStepSize: 2.5,
randomSeed: 123456,
restartStrategy: "ipop",
});
`

Node Usage (ESM + CommonJS)

numopt-js supports both ESM (import) and CommonJS (require) in Node.js.

$3

Note:
- If your project is CommonJS (default), use a .mjs file to run ESM.
- If you want ESM by default, set package.json to "type": "module" (then .js is treated as ESM).

`typescript
import { gradientDescent } from 'numopt-js';
`

$3

Note:
- If your project is ESM ("type": "module"), use a .cjs file to run CommonJS.

`js
const { gradientDescent } = require('numopt-js');
`

Browser Usage

numopt-js is designed to work seamlessly in browser environments. The library automatically provides a browser-optimized bundle that includes all dependencies.

Important:
- Don’t use file:// for the import-maps / direct-import examples. Serve your files via a local static server (for example: npx serve, python -m http.server, or Vite) so ES modules load correctly.
- SSR frameworks (Next.js, etc.): run numopt-js on the client side. If you hit SSR errors, move the code into a client component ("use client") or dynamically import it with SSR disabled.

$3

If you're using a bundler (Vite/Webpack/Rollup), just import from the package and the bundler will resolve the browser build via package.json exports.

`typescript
import { gradientDescent } from 'numopt-js';
`

$3

If you're using import maps (no bundler), map numopt-js to the browser bundle:

`html


`

$3

Import the browser bundle by path. (In this mode, you cannot use the bare specifier numopt-js.)

`html

`

$3

Problem: ReferenceError: exports is not defined when using in browser

Solution: Make sure you're using dist/index.browser.js instead of dist/index.js. The browser bundle includes all dependencies and is pre-configured for browser environments.

Problem: Module not found errors

Solution:
- Ensure you're using a modern bundler that supports package.json exports
- For direct browser usage, use import maps or explicitly import from dist/index.browser.js
- Check that your build tool supports ES modules

Examples

After installing dependencies with npm install, you can run the example scripts with npm run