Load detection and shedding capabilities for http, express, restify and koa
npm install overload-protectionLoad detection and shedding capabilities for http, express, restify, and koa
overload-protection provides integration for your framework of choice.
If a threshold is crossed for a given metric, overload-protection
will send an HTTP 503 Service Unavailable response, with (by default)
a Retry-After header, instructing the client (e.g. a browser or load balancer) to
retry after a given amount of seconds.
Current supported metrics are:
* event loop delay (is the JavaScript thread blocking too long)
* Used Heap Memory
* Total Resident Set Size
For a great explanation of Used Heap Memory vs Resident Set Size see
Daniel Khans article at
Create a config object for your thresholds (and other overload-protection)
options.
``js`
const protectCfg = {
production: process.env.NODE_ENV === 'production', // if production is false, detailed error messages are exposed to the client
clientRetrySecs: 1, // Retry-After header, in seconds (0 to disable) [default 1]
sampleInterval: 5, // sample rate, milliseconds [default 5]
maxEventLoopDelay: 42, // maximum detected delay between event loop ticks [default 42]
maxHeapUsedBytes: 0, // maximum heap used threshold (0 to disable) [default 0]
maxRssBytes: 0, // maximum rss size threshold (0 to disable) [default 0]
errorPropagationMode: false // dictate behavior: take over the response
// or propagate an error to the framework [default false]
logging: false, // set to string for log level or function to pass data to
logStatsOnReq: false // set to true to log stats on every requests
}
Then pass the framework we're integrating with along with the configuration object.
For instance with Express we would do:
`js`
const app = require('express')()
const protect = require('overload-protection')('express', protectCfg)
app.use(protect)
With middleware based frameworks, always put the overload-protection middlewareoverload-protection
first. In default mode this means will take over the response
and prevent any other middleware from executing (thus taking further potential pressure off
of the process).
Restify, and Koa all work in much the same way, call the overload-protection
module with the name of the framework, a config object and pass the resulting
instance to – e.g. Koa would be:
`js`
const Koa = require('koa')
const protect = require('overload-protection')('koa', protectCfg)
const app = new Koa()
app.use(protect)
For pure core HTTP the overload-protection instance can be calledreq
at the top of the request handler function. With two arguments (just and res)true
the function will return if protection/shedding has been provided, or false
if not. If has taken over (the true case), then we should
exit the function and do no further work:
`js
const http = require('http')
const protect = require('overload-protection')('http', protectCfg)
http.createServer(function (req, res) {
if (protect(req, res) === true) return
res.end('content')
})
`
With three arguments (the third argument being a callback), the rest of the
work should be done within the supplied callback.
`js
const http = require('http')
const protect = require('overload-protection')('http', protectCfg)
http.createServer(function (req, res) {
protect(req, res, function () {
// when errorPropagationMode mode is false, will only
// be called if load shedding didn't occur
// (if it was true we'd need to check for an Error object as first arg)
res.end('content')
})
})
`
`sh`
npm install overload-protection --save
`sh`
npm install
npm test
The overhead of using overload-protection is minimal, run the benchmarks to conduct overload-protection
comparative profiling of using versus not using it for each supported framework.
`sh`
npm run benchmarks
The framework argument is non-optional. It's a string and may be one of:
* express
* koa
* restify
* http
The opts argument is optional, as are all properties.
Options (particularly thresholds) are quite sensitive and highly relevant on
a case by case basis. Possible options are as follows:
#### production: process.env.NODE_ENV === 'production'
The production option determines whether the client receives an error message
detailing the surpassed threshold(s). (It may also be used in future for other such
good practices or performance trade-offs).
#### clientRetrySecs: 1
By default, overload-protection will add a header to the 503 responseRetry-After
called . It's up to the client to honour this header, which
instructs the client on how many seconds to wait between retries.
Defaults to 1 seconds.
#### sampleInterval: 5
In order to establish whether a threshold has been crossed, the metrics
are sampled at a regular interval. The interval defaults to 5 milliseconds.
#### maxEventLoopDelay: 42
Synchronous work causes the event loop to freeze, when this happens
an interval timer (which is our sampler) will be delayed by the amount
of time the event loop was stalled for while the thread processed synchronous
work. We can measure this with timestamp comparison. This option sets a threshold
for the maximum amount of stalling between intervals we'll accept before our
service begins responding with 503 codes to requests. Defaults to 42 milliseconds.
When set to 0 this threshold will be disabled.
#### maxHeapUsedBytes: 0
Disabled by default (set to 0), this defines maximum V8 (Node's JavaScript engine) used heap size.
If the Used Heap size exceeds the threshold the server will begin return 503 error codes
until it crosses back under the threshold.
See
for more info on Used Heap from a V8 context.
#### maxRssBytes: 0
Disabled by default (set to 0) maximum process Resident Set Size. If
the RSS exceeds the threshold the server will begin return 503 error codes
until it crosses back under the threshold.
#### errorPropagationMode: false
This is relevant to middleware integration only
By default, overload-protection will handle and end the response,
without calling any subsequent configured middleware. The point here
is to avoid any further processing for an already (by definition)
over loaded process.
However, it could be argued, from a puritanical perspective, that middleware
should defer to the framework and that any HTTP code of 500 or above should
be generated by propagating an error through the framework.
This option prevents overload-protection from manually ended the response andError
instead generates an object (with additional properties as per http-errors as used by Express and Koa) next
and propagates it through the framework (either by throwing it in Koa, or passing through the callback).
#### logging: false
The logging option can be set to a string or a function.
If logging is set to a string, the string should indicate the desired log logging
level for notifying that a 503 response was given. When is a stringreq
a request bound Log4j-style logger is assumed. This means the object (or the ctx object in the case of Koa) log
should have a object which contains methods corresponding to log levels. So if logging
was set to (logging: 'warn') then req.log.warn is expected to be presentbunyan-express
and be a function. A number of logging libraries follow this pattern, such as and all of the pino express-pino-logger
middleware loggers (, koa-pino-logger, restify-pino-logger, pino-http).
If the application isn't using a request bound Log4j-style logger, the logging logging: console.warn
option can be set to a function which receives a log message. This function is
then responsible for writing the log. We could also simply set it to one of
the console methods, e.g. .
This is primarily for usage when errorPropagationMode is false. If errorPropagationMode true
is set to , we may want to instead log once the error has propagated to a handler.
#### logStatsOnReq: false
Set logStatsOnReq to true log the profiled stats on every request. In order to use this option, the logging option must not be false. Bear in mind that using this option will
add extra pressure on the event loop in itself, so use with caution.
The returned instance (which in many cases is passed as middleware to app.use), overload
has an property. This begins as false. If any of the thresholds have true
been passed this will be set to . Once all metrics are below their thresholdsfalse
this would become again.
This allows for any heavy load detection required outside of a framework.
The returned instance (which in many cases is passed as middleware to app.use), eventLoopOverload
has an property. This begins as false. If the maxEventLoopDelay
threshold is passed this will be set to . Once it's below the configured thresholdfalse
this would become again.
This allows for any event loop delay detection necessary outside of a framework.
The returned instance (which in many cases is passed as middleware to app.use), heapUsedOverload
has a property. This begins as false. If the maxHeapUsedBytes
threshold is passed this will be set to . Once it's below the configured thresholdfalse
this would become again.
This allows for any heap used threshold detection necessary outside of a framework.
The returned instance (which in many cases is passed as middleware to app.use), rssOverload
has a property. This begins as false. If the maxRssBytes
threshold is passed this will be set to . Once it's below the configured thresholdfalse
this would become again.
This allows for any heap used threshold detection necessary outside of a framework.
The delay in milliseconds (with additional decimal precision) since the last sample.
If maxEventLoopDelay is 0, the event loop is not measured, so eventLoopDelay will always
be 0 in that case.
Corresponds to the opts.maxEventLoopDelay option.
Corresponds to the opts.maxHeapUsedBytes option.
Corresponds to the opts.maxRssBytes` option.
- loopbench: Benchmark your event loop
- autocannon: Fast HTTP benchmarking tool written in Node.js
- express: Fast, unopinionated, minimalist web framework
- koa: Koa web app framework
- koa-router: Router middleware for koa. Provides RESTful resource routing.
- pre-commit: Automatically install pre-commit hooks for your npm modules.
- restify: REST framework
- standard: JavaScript Standard Style
- tap: A Test-Anything-Protocol library
MIT
Kindly sponsored by nearForm