Multi-threaded generate command for Nuxt.js


> Use multiple workers to generate the static files for your Nuxt.js project

Setup

Install the package
``
yarn add nuxt-generate-cluster
`

Add a generate script to your package.json
`js
"scripts": {
"generate": "nuxt-generate -w 4"
}
`

Nuxt config options

Configure the generate options in nuxt.config.js
`js
generate: {
workers: 4,
workerConcurrency: 500,
concurrency: 500,
routes (callback, params) {
return axios.get('https://api.example.com/routes?since=' + params.lastFinished)
.then((res) => {
return res.data
})
},
done ({ duration, errors, workerInfo }) {
if (errors.length) {
axios.post('https://api.example.com/routes', { generate_errors: errors })
}
}
}
`

$3

- Default: number of processors

The number of workers that should be started. It probably has no use to start more workers then number of processors in your system.

$3

- Default: 500

To even the load between workers they are sent batches of routes to generate, otherwise a worker with 'easy' routes might finish long before others. Workers will also still use the concurrency option from Nuxt.

$3

The default Nuxt.js routes method has been extended so you can pass additional parameters to it, see params parameter in example config under Setup. By default
it will list 3 timestamps:

- lastStarted
The unix timestamp when the nuxt-generate command was last executed, should be just now

- lastBuilt
The unix timestamp when the nuxt project was last built by nuxt-generate

- lastFinished
The unix timestamp when nuxt-generate last finished succesfully (eg not interrupted by ctrl+c)

> Timestamps are locally stored in ~/.data-store/nuxt-generate-cluster.json, see data-store

$3

This method is called on the master just before the workers are started/forked. It receives the Nuxt options as first argument.

Use this method if you experience serialization issues or when your Nuxt config is too big. The Nuxt options are stringified and then passed as environment variable to the workers, on Windows there seems to be a maximum size of 32KB for env variables.

Properties which should be safe to remove are (not a complete list):
- buildModules (and their options)
- serverMiddleware

$3

This method will be called when all workers are finished, it receives two arguments:

- The first argument is an object with statistics:
- duration
The total time in seconds that the command ran

- errors
An array of all the errors that were encountered. Errors can have type handled or unhandled, for the latter the error message will contain the stacktrace

`js
[ { type: 'handled',
route: '/non-existing-link',
error:
{ statusCode: 404,
message: 'The message from your 404 page' } }
]
`

- workerInfo
An object with detailed information about each worker. Data passed is from the watchdog object that we use internally to monitor the worker status.

`js
{{ '6707': // the process id of the worker
{ start: [ 1929158, 859524606 ], // process.hrtime the worker was started
duration: 109567, // how long the worker was active
signal: 0, // the signal by which the worker was killed (if any)
code: 0, // the exit status of the worker
routes: 73, // the number of routes generated by this worker
errors: [] }, // the errors this worker encountered, errors of all workers
// combined is the error argument above
}
`

- The second argument is the Nuxt instance

Command-line options

> Please note that you need to explicitly indicate with -b that you want to (re-)build your project

`
$ ./node_modules/.bin/nuxt-generate -h

Multi-threaded generate for nuxt using cluster

Description
Generate a static web application (server-rendered)
Usage
$ nuxt-generate

Options
-b, --build Whether to (re-)build the nuxt project
-c, --config-file Path to Nuxt.js config file (default: nuxt.config.js)
-h, --help Displays this message
-p, --params Extra parameters which should be passed to routes method
(should be a JSON string or queryString)
-q, --quiet Decrease verbosity (repeat to decrease more)
-v, --verbose Increase verbosity (repeat to increase more)
--fail-on-page-error Immediately exit when a page throws an unhandled error
-w, --workers [NUM] How many workers should be started
(default: # cpus)
-wc [NUM], How many routes should be sent to
--worker-concurrency [NUM] a worker per iteration

`

If you need to have more control which routes should be generated, use the -p option to pass additional parameters to your routes method.

`


nuxt.config.js

generate: {
routes (callback, params) {
console.log(params)
}
}

$ nuxt-generate -w 2 -p id=1&id=2
// will print =>
{ id: [ '1', '2' ],
lastStarted: 1508609323,
lastBuilt: 0,
lastFinished: 0 }
`

If you are using a npm script under bash use -- to pass the parameters to nuxt-generate instead of npm:

`
$ npm run generate -- -p '{ "id": [1,2,3] }'
// will print =>
{ id: [ 1, 2, 3 ],
lastStarted: 1508786574,
lastBuilt: 0,
lastFinished: 0 }
`

Logging

You can use multiple -v or -q on the command-line to increase or decrease verbosity.
We use consola for logging and set a default log level of 3 unless DEBUG is set, then its 5.
If you want to log debug messages without setting DEBUG you can use -vv on the command-line

The difference between log levels 2, 3 and 4 are:

- Level 2 (-q)
Only print master messages like worker started / exited. No worker messages.

- Level 3
Also print which routes the workers generated

- Level 4` (-v)
Also print how much time the route generation took and messages between master and workers