node-sequelize-stream

node-sequelize-stream is a library allowing to stream data with sequelize.

npm install node-sequelize-stream

To have "findAllWithStream" method in your models you need to pass your sequelize instance to sequelizeStream function:

``
const sequelizeStream = require('node-sequelize-stream');
const sequelize = new Sequelize(...);
// models initialization
sequelizeStream(sequelize, defaultBatchSize, isObjectMode);

`

defaultBatchSize - is an optional parameter that means default amount of batches to be fetched with each chunk for all models. Default value is 100.

To get stream object you need to do:
`
const stream = db.models.User.findAllWithStream({batchSize: 50, isObjectMode: true, ...otherSequelizeParams});
stream.pipe(res);
`

`
const stream = db.models.User.bulkCreateWithStream(
[{id: 1, name: 'SomeUser'}, ...],
{batchSize: 50, isObjectMode: true, ...otherSequelizeParams}
);
stream.pipe(res);
`

`
const stream = db.models.User.updateWithStream(
{name: 'UpdatedName'},
{batchSize: 50, isObjectMode: true, where: {...}, ...otherSequelizeParams}
);
stream.pipe(res);
`

`
const stream = db.models.User.destroyWithStream({
batchSize: 50,
isObjectMode: true,
where: {...},
...otherSequelizeParams
});
stream.pipe(res);
`

`
app.get('/getCsv', (req, res) => {
res.set({
'Content-Type': 'text/csv; charset=utf-8',
'Content-Disposition': 'attachment; filename="testFile.csv"',
})

const stream = User.getCsvfindAllWithStream();
stream.on('data', chunk => {
console.log('\n\nchunk', typeof chunk, chunk.toString());
});
stream.on('error', error => {
console.log('\n\nERROR!!!!!', error);
res.end(error)
});
stream.on('end', () => {
console.log('\n\nEND!!!!!');
res.end()
});
stream.pipe(res);
})
`

batchSize - is an optional parameter that means default batch size for target action (model batch size or default batch size will be taken if parameter is not defined).

isObjectMode - is an optional parameter that switches object mode for a stream. If enabled then stream chunks will be an object. If disabled then stream chunk is a buffer as usual.


Also batch size can be set for each model separately like this:
`
const User = instance.define('User', {...});
User.BATCH_SIZE = 10;
User.IS_OBJECT_MODE = true;
``