Tiny and fast async generator to read streams line by line
npm install readlines-ng
Node 10+ is required. This package provides very simple but feature-reach functionality to consume readable streams or files line by line. It's also quite fast (up to 30% faster than readline) and has no dependencies.
readlinesLineStream\n)encoding option is nullindex.js, it's really tiny. Or check the usage example below.javascript
const { readlines, LineStream } = require('readlines-ng');// As a generator
for await (const line of readlines(filename, { encoding: 'utf8' })) {
console.log(line);
}
// As a readable stream
const reader = new LineStream(filename, {
chunkSize: 256*1024,
encoding: null,
});
for await (const line of reader) {
console.log(line);
}
// Custom separator
const { PassThrough } = require('stream');
const datasource = new PassThrough();
datasource.write('qux42');
datasource.write('foobar42');
datasource.end();
const reader = new LineStream(filename, { sep: '42' });
for await (const line of reader) {
console.log(line);
}
`Tests
bash
npm run test
`Benchmark
Trivial impl using this package exposes the performance better than the standard readline module.
`bash
npm run bench # Node 11+ is requiredFile size ~2.8 GB
> Benchmark: lines counter (readline module)
> 5974719 lines found
> Done in 5.757290349 sec> Benchmark: lines counter (async generator)
> 5974719 lines found
> Done in 4.707053160 sec
> Benchmark: lines counter (async generator UTF-8)
> 5974719 lines found
> Done in 3.937784544 sec
> Benchmark: lines counter (readable stream)
> 5974719 lines found
> Done in 4.984460949 sec
> Benchmark: lines counter (readable stream UTF-8)
> 5974719 lines found
> Done in 4.354594750 sec
``