A concurrent version of Array.map()
npm install @silyze/concurrent-mapA concurrent version of Array.map that limits the number of concurrent asynchronous operations.
This package allows you to process large arrays of items with an asynchronous function (asyncFn) while controlling how many promises run in parallel. This is useful to prevent resource exhaustion (e.g., API rate limits, database connections) when dealing with many asynchronous tasks.
---
Install via npm:
``bash`
npm install @silyze/concurrent-map
---
In CommonJS:
`js`
const concurrentMap = require("@silyze/concurrent-map");
In ES Modules / TypeScript:
`ts`
import concurrentMap from "@silyze/concurrent-map";
---
`ts`
export default async function concurrentMap
items: T[],
asyncFn: (item: T) => Promise
maxConcurrency: number = 8
): Promise
- items: T[]
The array of input items to be processed.
- asyncFn: (item: T) => Promise
An asynchronous function that takes an item of type and returns a Promise resolving to a result of type R. This function will be invoked for each element in items.
- maxConcurrency: number (optional, default: 8)asyncFn
Maximum number of calls to run in parallel. Once this limit is reached, concurrentMap waits for the first pending promise to settle before scheduling a new one.
- Promise
A promise that resolves when all invocations have settled (either fulfilled or rejected). The resolved value is an array of results of type R. Note that the order of results in the array corresponds to completion order, not input order. If you need to preserve original order, see the "Ordering Results" section.
---
`ts
import concurrentMap from "@silyze/concurrent-map";
(async () => {
const items = ["a", "b", "c"];
const results = await concurrentMap(
items,
async (item) => {
// Simulate an async task
await new Promise((resolve) => setTimeout(resolve, 1000));
console.log(Processed: ${item});
return item.toUpperCase();
},
2
);
console.log(results); // e.g. ['A', 'B', 'C'] (order may vary)
})();
`
In this example:
1. Two tasks run concurrently (maxConcurrency = 2).results
2. Each item is processed with a 1-second delay.
3. The final array contains the uppercase letters, but may not be in the original input order.
If you need the output array to maintain the same order as the input array, wrap each result with its index and sort after processing:
`ts
import concurrentMap from "@silyze/concurrent-map";
(async () => {
const items = ["a", "b", "c"];
const indexedResults = await concurrentMap(
items.map((value, index) => ({ index, value })),
async ({ index, value }) => {
const result = await someAsyncOperation(value);
return { index, result };
},
3
);
// Sort by original index
indexedResults.sort((a, b) => a.index - b.index);
// Extract values in order
const orderedResults = indexedResults.map((item) => item.result);
console.log(orderedResults);
})();
`
If any invocation of asyncFn rejects, concurrentMap still waits for all pending tasks to settle before resolving the returned promise. Rejected tasks will be omitted from the results array; if you need to capture errors, you can catch them inside asyncFn and return an object with either value or error:
`ts
import concurrentMap from "@silyze/concurrent-map";
type Outcome
| { status: "fulfilled"; value: R }
| { status: "rejected"; error: any };
(async () => {
const urls = ["url1", "url2", "url3"];
const outcomes = await concurrentMap(
urls,
async (url) => {
try {
const response = await fetch(url);
const data = await response.json();
return { status: "fulfilled", value: data } as Outcome
} catch (error) {
return { status: "rejected", error } as Outcome
}
},
5
);
outcomes.forEach((outcome) => {
if (outcome.status === "fulfilled") {
console.log("Data:", outcome.value);
} else {
console.error("Error:", outcome.error);
}
});
})();
`
---
- Batch Size: Choose maxConcurrency` based on your environment (CPU, memory, network). Too high can exhaust resources; too low underutilizes.
- Task Duration: If tasks vary significantly in duration, the order of completion (and thus result positions) will reflect that.