Transform Pino HTTP log messages with a format string
npm install pino-toke[![stability][0]][1]
[![npm version][2]][3]
[![dependencies freshness][14]][15]
[![js-standard-style][10]][11]
Transform Pino HTTP log messages with a format string
This CLI tool and module is a transform transport for the pino, specifically for processing HTTP
log messages (see the Supports section). Output is configured using the tokenized
format string as used in the morgan logger.
* express-pino-logger
* restify-pino-logger
* koa-pino-logger
* pino-http
* hapi-pino
You can use this module as a pino transport like so:
``js`
const pino = require('pino')
const transport = pino.transport({
target: 'pino-toke',
level: 'info',
options: {
destination: 1, // optional (default stdout)
ancellary: 2, // optional
format: ':method :url :status :res[content-length] - :response-time ms', // required
keep: false // optional
}
})
pino(transport)
The options object's properties are described below.
Pino supports a legacy transport interface
that is still supported by this module.
`sh`
$ npm install -g pino-toke
`sh`
$ pino-toke -h
`sh
pino-toke [-d] [-k] [-a] [tokens]
-d | --dest | stderr | stdout (default) or Number. Specify output fd
--destination
-a | --ancillary stderr | stdout or Number. Specify JSON logs fd
-k | --keep true | false (default) Retain transformed logs in ancillary output
tokens :id :pid :level :hostname :url :date[format] :time[format]
:method :response-time :status :referrer :remote-addr
:remote-user :http-version :user-agent :req[header] :res[header]
`
Spin up a server that uses a pino http logger (see the Supports section),
pipe it to pino-toke and desribe the format in tokenized form
`sh`
$ node server | pino-toke :method :url :status :res[content-length] - :response-time ms
By default, logs are output to STDOUT, however we can set the -d (alias, --dest, --destination), flag to a a stderr, or a number (1 for stdout, 2 for stderr, 3 or more for custom file descriptor):
`sh`
$ node server | pino-toke -d stderr :status :get :url - :response-time ms
The above is equivalent to:
`sh`
$ node server | pino-toke -d 2 :status :get :url - :response-time ms
We can also direct formatted log output to custom file descriptors, but we must
use bash redirection (in some form) from that file descriptor, otherwise the process
will most likely immediately crash (this is to do with how unix works).
`sh`
$ node server | pino-toke -d 8 :status :get :url - :response-time ms 8> ./http-logs
By default, any logs which aren't an HTTP log (meaning, they don't have req and res
properties and the isn't "request complete") are filtered out.
However, we can specify an ancillary (secondary) output for other log messages, using
the -a (alias --ancillary) flag.
The following will write reformatted HTTP logs to STDOUT and original JSON logs
which are not HTTP logs to STDERR.
`sh`
$ node server | pino-toke -a 2 :status :get :url - :response-time ms
The following achieves the reverse effect:
`sh`
$ node server | pino-toke -d 2 -a 1 :status :get :url - :response-time ms
The next example creates an custom file descriptor, and redirects output from that
descriptor to a ./logs file, whilst outputting formatted HTTP logs to STDOUT
`sh`
$ node server | pino-toke -a 4 :status :get :url - :response-time ms 4> ./logs
This will sends formatted HTTP logs to the ./http-logs file and pipe all other log messages to pino-elasticsearch
`sh`
$ node server | pino-toke -a 1 -d 4 :status :get :url - :response-time ms 4> ./http-logs | pino-elasticsearch
The -a (--ancillary) flag can be coupled with the -k (--keep) flag so that
raw HTTP JSON logs are also piped to the ancillary output stream, along with any
filtered output.
The following will pipe all formatted logs to the 4 file descriptor which is redirected to a file,
while all original JSON logs (instead of non-HTTP logs) are written to STDOUT.
`
$ node server | pino-toke -k -a 1 -d 4 :status :get :url - :response-time ms 4> ./http-logs
pino-toke supports all the same tokens found in the morgan
logger, with additional tokens based on the information we have in the pino HTTP log format.
Tokens supported in addition to morgan's token set are :id, :pid, :level,:hostname, and :time.
#### :id
Logs from pino-http add a unique identifer to each request, this gives the unique id
#### :pid
Process Id (as in all pino logs)
#### :level
Log level (as in all pino logs)
#### :hostname
Server hostname (as in all pino logs)
#### :url
URL of the request (as per req.url, so does not include domain name)
#### :date[format]
Date and time based on the time key in the pino log.
[format] can be:
* web - RFC 1123 format (default) - Fri, 28 Apr 2017 14:12:42 GMT
* iso - ISO 8601 format -
* clf - date format used in Common Log Format
When format is omitted, or not recognized, defaults to web
#### :time[format]
The time based on the time key in the pino log
[format] can be:
* ms - milliseconds (default)
* iso - the time format after T and before the decimal (.) in ISO 8601 string 14:12:42
When format is omitted, or not recognized, defaults to ms
#### :method
Request method
#### :response-time
Response time as recorded by pino-http in the res object.
#### :status
HTTP Status
#### :referrer
Referrer header (normalizes alternative "referer" heading)
#### :remote-addr
Remote address (IP)
#### :remote-user
If Basic auth is being used, this will contain the user name provided via basic auth
#### :user-agent
Contents of the User-Agent header
#### :req[header]
Given header in the request
#### :res[header]
Given header in the response
#### toke(format, destination, ancillary)
Returns a stream that we write Pino JSON logs to.
The format parameter is required and can be a format string, a function or
an object.
A format string represents how a log line should be written out, using a "tokens",
which are essentially placholder elements in the :token. The string
can also contain normal text, which will appear between the tokens.
Unrecognized tokens will appear in the output as normal text.
`js`
const { toke } = require('pino-toke')
toke(':method :url :status')
If format is a function it should have the signature (o, tokens). tokens
The parameter is an object of mapping functions, which pick o
properties from the object (see ./tokens.js file. o
The parameter will be an object representation each HTTP JSON log line.
It's return value will be the contents of the transformed log line.
`
toke((o, tokens) => 'some log line')
If format is an object, should have a property called format that contains theformat string or a function.
`js`
toke({format: ':method :url :status'})
`js`
toke({format: (o, tokens) => 'some log line'})
When format is an object, additional options can be provided. keep
There's only one valid option, :
`js`
toke({format: ':method :url :status', keep: true})
When keep is true all JSON log messages will be written verbatim ancillary
to the write stream. It keep is false, only messagesancillary
which we not processed as HTTP related messages will be written to
the write stream. This only applies if an ancillary stream
is provided.
The optional destination parameter is a writeable stream that receivesprocess.stdout
the reformatted log messages. It defaults to .
The optional ancillary parameter is a writable stream that
#### toke.compile(format)
Compile a format string (format) in a function. The returned function tokens
takes a and o parameters. The tokens parameter is an objecto
of mapping functions, which pick properties from the object. Theo parameter is an object representation of a pino JSON log line.
This can be useful for pre- and post-processing when providing a a function to format.
MIT
* Inspired by the morgan` logger
* Sponsored by nearForm
[0]: https://img.shields.io/badge/stability-stable-green.svg?style=flat-square
[1]: https://nodejs.org/api/documentation.html#documentation_stability_index
[2]: https://img.shields.io/npm/v/pino-toke.svg?style=flat-square
[3]: https://npmjs.org/package/pino-toke
[8]: http://img.shields.io/npm/dm/pino-toke.svg?style=flat-square
[9]: https://npmjs.org/package/pino-toke
[10]: https://img.shields.io/badge/code%20style-standard-brightgreen.svg?style=flat-square
[11]: https://github.com/feross/standard
[12]: https://npm.im/debug
[13]: https://npm.im/github/pinojs/pino
[14]: https://david-dm.org/pinojs/pino-toke/status.svg
[15]: https://david-dm.org/pinojs/pino-toke