A typescript first implementation of a parallel transform stream
npm install @dvd-88/typescript-parallel-transformA typescript first implementation of a Node.js
Transform stream
with concurrency control.
To install middy you can use NPM:
``bash`
npm install --save @dvd-88/typescript-parallel-transform
There are two main classes that extends the Node.js Transform stream and allows
you to control the concurrency of the stream in this package:
- ParallelTransform: the order of the data is not preserved. Data is pushed toOrderedParallelTransform
the next step as soon as it is available.
- : the order of the data is preserved. Data is pushed
to the next step in the same order as the chunk it originates from was
received.
Usage example: Parallel Transform
`bash`
npx tsx src/examples/parallel-transform.ts
Running this example will output something similar to this:
`
Elapsed time: 1004.164841ms
Result: [
10, 9, 8, 7, 6,
5, 4, 3, 2, 1
]
The elapsed time is around 1 second because it processed 10 items concurrently
(the default max concurrency is 16) and the slowest took 1 second to be
processed. Moreover the order of the items in the result array is reversed since
the computational time of each item is inversely proportional to its value and
the input array is sorted in ascending order.
Usage example:
Ordered parallel Transform
`bash`
npx tsx src/examples/ordered-parallel-transform.ts
Running this example will output something similar to this:
`
Elapsed time: 1004.970725ms
Result: [
1, 2, 3, 4, 5,
6, 7, 8, 9, 10
]
The elapsed time is around 1 second because it processed 10 items concurrently
and the slowest took 1 second to be processed. Moreover the order of the items
in the result array is equal to the input one.
Usage example:
Parallel Transform Factory
`bash`
npx tsx src/examples/parallel-transform-factory.ts --ordered
Since we selected the ordered parallel transform implementation by passing the
---ordered parameter, the output will be similar to the ordered example:
`
Elapsed time: 1007.248033ms
Result: [
1, 2, 3, 4, 5,
6, 7, 8, 9, 10
]
Usage example:
Parallel Transform Factory (promisified)
`bash`
npx tsx src/examples/promisified-parallel-transform-factory.ts --ordered
Since we selected the ordered parallel transform implementation by passing the
---ordered parameter, the output will be similar to the ordered example:
`
Elapsed time: 1006.521075ms
Result: [
1, 2, 3, 4, 5,
6, 7, 8, 9, 10
]
Licensed under MIT License.