Synchronous shared context compression with node's native zlib
npm install fast-zlibzlib wrapper for Node.js that enables synchronous shared context compression.
_processChunk() method, which this package makes use of to provide an easy and fast way to synchronously process chunks of data without losing the zlib context.
js
const zlib = require("fast-zlib");
let inflate = new zlib.Inflate();
let deflate = new zlib.Deflate();
let inflateRaw = new zlib.InflateRaw();
let deflateRaw = new zlib.DeflateRaw();;
let gzip = new zlib.Gzip();
let unzip = new zlib.Unzip();
let gunzip = new zlib.Gunzip();
let brotli = new zlib.BrotliCompress();
let debrotli = new zlib.BrotliDecompress();
`
ES6 imports and typescript users can also use the following:
`ts
import zlib from "fast-zlib";
let deflate = new zlib.Deflate();
`
`ts
import { Deflate } from "fast-zlib";
let deflate = new Deflate();
`
Each fast-zlib class is a wrapper around the original zlib class and looks like this:
$3
Create a new instance of a fast-zlib class.
* options - An optional object of zlib or brotli options as per node's zlib documentation.
$3
Process a chunk of data.
* data - A Buffer of data to be compressed or decompressed. Non-Buffers will be internally converted to Buffer.
* flag - An optional flush flag to override the default flag.
* => Buffer - A Buffer of processed data.
$3
Close the zlib handler and shut down the instance.
* => void
$3
Access the underlying zlib instance for advanced usage.
Examples
Usage is very simple, compress a chunk of data and decompress it elsewhere. The instance keeps track of its compression state and sliding window contexts.
`js
let zlib = require("fast-zlib");
let deflate = new zlib.Deflate(); // create a deflator
let inflate = new zlib.Inflate(); // create an inflator
let data = "123456789";
let chunk1 = deflate.process(data);
// Buffer(17) [120, 156, 50, 52, 50, 54, 49, 53, 51, 183, 176, 4, 0, 0, 0, 255, 255]
// the first chunk of data is fully processed
let chunk2 = deflate.process(data);
// Buffer(9) [50, 132, 49, 0, 0, 0, 0, 255, 255]
// reusable patterns from previous compressions are referenced
let chunk3 = deflate.process(data);
// Buffer(8) [130, 51, 0, 0, 0, 0, 255, 255]
// and the context continues to adapt to all subsequent chunks
inflate.process(chunk1).toString(); // "123456789"
inflate.process(chunk2).toString(); // "123456789"
inflate.process(chunk3).toString(); // "123456789"
`
Decompression must be done in exactly the same order as compression because chunks sequentially complement each other. Attempting to decode a chunk out of order may throw an error and reset the decompressor so it has to either restart from the beginning or you will have to destroy both and create a new pair.
`js
let chunk1 = deflate.process(data);
let chunk2 = deflate.process(data);
inflate.process(chunk2); // error
inflate.process(chunk1); // works
inflate.process(chunk2); // works
`
Each zlib class can be passed an options object as per zlib's documentation.
`js
let deflateRaw = new zlib.DeflateRaw({
chunkSize: 128 * 1024,
level: 8
});
let inflateRaw = new zlib.InflateRaw({
chunkSize: 64 * 1024
});
`
This library uses Z_SYNC_FLUSH as the default flush flag in order to return data immediately. For more control over the compression process, flush flags can be set as an option and also passed directly to the process function.
`js
let deflate = new zlib.Deflate({
flush: zlib.constants.Z_NO_FLUSH // set default flag to Z_NO_FLUSH
});
let inflate = new zlib.Inflate();
deflate.process("123"); // add data
deflate.process("456");
deflate.process("789");
// process all data added so far by passing Z_SYNC_FLUSH
let data = deflate.process("hij", zlib.constants.Z_SYNC_FLUSH);
inflate.process(data).toString(); // 123456789hij
`
Other flush flags are also available and can be used to achieve fine control over the process. Not all classes support the same flags and there might be differences in behavior between them. For example brotli uses BROTLI_OPERATION_PROCESS and BROTLI_OPERATION_FLUSH instead of Z_NO_FLUSH and Z_SYNC_FLUSH. Check zlib's documentation for more details about how each class works.
`js
// default flag is zlib.BROTLI_OPERATION_FLUSH
let compress = new zlib.BrotliCompress();
let decompress = new zlib.BrotliDecompress();
let compressed = compress.process("abc");
decompress.process(compressed).toString(); // abc
compress.process("123", zlib.constants.BROTLI_OPERATION_PROCESS);
compress.process("456", zlib.constants.BROTLI_OPERATION_PROCESS);
let data = compress.process("789", zlib.constants.BROTLI_OPERATION_FLUSH);
decompress.process(data).toString(); // 123456789
``