fibersync - Callbacks and promises simplification for Node.js with synchronized+asyncawait (by fibers, bluebird and lodash)
npm install fibersync
Do you need this library?
This library has enabled async/await coding style in Node.js since 2014. But JavaScript now has native async/await.
JS async/await was standardized as part of ES2017, and has been enabled by default in Node.js since v7.6.
So, do you still need this library? If you are just starting to use async/await, the answer is probably no. Use native
async/await. If you're maintaining a codebase that uses this library, or that needs to run on a old version of Node.js,
then you may want to keep using it, but consider migrating to native async/await eventually. If you need deep coroutines
for an advanced scenario, there may still be a case for using this library, since native async/await only supports
shallow coroutine semantics.
fibersync ( asyncawait + synchronize ) v1.0async in Depth: Suspendable Functionsthis Contextasync.mod Functionawait in Depth: Awaitable Expressionsawait?awaitasyncawait addresses the problem of callback hell in Node.js JavaScript code. Inspired by C#'s async/await feature, asyncawait enables you to write functions that appear to block at each asynchronous operation, waiting for the results before continuing with the following statement. For example, you can write the following in plain JavaScript:``javascript`
var foo = async (function() {
var resultA = await (firstAsyncCall());
var resultB = await (secondAsyncCallUsing(resultA));
var resultC = await (thirdAsyncCallUsing(resultB));
return doSomethingWith(resultC);
});
which, with one proviso, is semantically equivalent to:
`javascript
function foo2(callback) {
firstAsyncCall(function (err, resultA) {
if (err) { callback(err); return; }
secondAsyncCallUsing(resultA, function (err, resultB) {
if (err) { callback(err); return; }
thirdAsyncCallUsing(resultB, function (err, resultC) {
if (err) {
callback(err);
} else {
callback(null, doSomethingWith(resultC));
}
});
});
});
}
`
The function foo does not block Node's event loop, despite its synchronous appearance. Execution within foo is suspended during each of its three asynchronous operations, but Node's event loop can execute other code whilst those operations are pending. You can write code like the above example in a HTTP request handler, and achieve high throughput with many simultaneous connections, just like with callback-based asynchronous handlers.
In short, asyncawait marries the high concurrency of asynchronous code with the visual clarity and conciseness of synchronous code. Rather than passing callbacks and error-backs, you can return values and use try/catch blocks. Rather than requireing specialised asynchronous control-flow constructs like each and whilst, you can use plain JavaScript constructs like for and while loops.
can suspend a running function without blocking Node's event loop. Both libraries are built on coroutines, but use different technologies. co uses ES6 generators, which work in Node >= v0.11.2 (with the --harmony flag), and will hopefully be supported someday by all popular JavaScript environments and toolchains.asyncawait uses node-fibers. It works with plain ES3/ES5 JavaScript, which is great if your tools do not yet support ES6 generators. This may be an important consideration when using compile-to-JavaScript languages, such as TypeScript or CoffeeScript.A similar outcome may be achieved by transforming JavaScript source code in a preprocessing step. streamline.js is an example of this method. Code using
asyncawait is executed normally without any code tranformation or preprocessing.4. Compared to...
asyncawait represents one of several viable approaches to writing complex asynchronous code in Node.js, with its own particular trade-offs. Notable alternatives include async, bluebird and co, each with their own trade-offs. The following table summarises some of the alternatives and their pros and cons. For more information about how the alternatives compare, take a look in the comparison folder.asyncawait may be a good choice if (a) you need highly concurrent throughput, (b) your asynchronous code must be clear and concise, (c) your code targets Node.js, and (d) you are limited to ES3/ES5 syntax (e.g. you write in TypeScript or CoffeeScript).| | Max. throughput (full event loop utilisation) | Concise, clear code (control-flow, data-flow and error-flow) | Max. support for Node.js dev/build tools | Max. support for JS envs (eg Node + browsers)
|---|---|---|---|---|
| Plain synchronous code | :heavy_exclamation_mark:[1] | :white_check_mark: | :white_check_mark: | :white_check_mark: |
| Plain callbacks | :white_check_mark: | :heavy_exclamation_mark:[2] | :white_check_mark: | :white_check_mark: |
| Callbacks + control-flow (e.g.
async) | :white_check_mark: | :heavy_exclamation_mark:[3] | :white_check_mark: | :white_check_mark: |
| Promises + control-flow (e.g. bluebird) | :white_check_mark: | :heavy_exclamation_mark:[3] | :white_check_mark: | :white_check_mark: |
| Coroutines with co | :white_check_mark: | :white_check_mark: | :heavy_exclamation_mark:[4] | :heavy_exclamation_mark:[5] |
| Coroutines with asyncawait | :white_check_mark: | :white_check_mark: | :white_check_mark: | :heavy_exclamation_mark:[6] |Footnotes:
[1] Each synchronous call blocks Node's event loop. All concurrent tasks are blocked, and the event loop sits idle, until the call completes.
[2] Plain callbacks rapidly become unwieldy for complex asynchronous tasks. See comparison.
[3] Whilst better than plain callbacks, these styles still produce longer and more complex code than synchronous or coroutine-based code. See comparison.
[4] Some tools do not (yet) support ES6 generators, including compile-to-JavaScript languages such as TypeScript and CoffeeScript.
[5] ES6 still has patchy browser support.
[6] Strictly limited to Node.js environments (i.e. no browsers) due to the use of
node-fibers.5. Performance
How well does asyncawait perform? The answer depends on what kinds of performance you care about. As a rough guide, compared with bare callbacks, expect your code to be 70% shorter with 66% less indents and run at 79% of the speed of bare callbacks. OK, so don't trust those numbers (which actually are real) but do check out the code in the comparison folder, and do run your own benchmarks.6. Quick Start
$3
npm install fibersync$3
provides just two functions: async() and await(). You can reference these functions with the code:
`javascript
var async = require('asyncawait/async');
var await = require('asyncawait/await');
`
Use to declare a suspendable function. Inside a suspendable function, use await to suspend execution until an awaitable expression produces its result. Awaitable expressions typically involve performing asynchronous operations.Note the spacing after
async and await in the examples. They are just plain functions, but the space makes them look more like keywords. Alternatively if you really want them to stand out, you could use names like __await__ or AWAIT, or whatever works for you.$3
`javascript
const async = require('./fibersync/').async;
const await = require('./fibersync/').await;
const dwait = require('./fibersync/').dwait;
const defer = require('./fibersync/').defer;var Promise = require('bluebird');
var fs = Promise.promisifyAll(require('fs')); // adds Async() versions that return promises
var path = require('path');
var _ = require('lodash');
/* Returns the number of files in the given directory. /
var countFiles = async (function (dir) {
console.log('current dir content: ', dwait (fs.readdir('.', defer()))); // additional separate example of additional dwait-defer pair
var files = await (fs.readdirAsync(dir)); // old exmaple of regular async-await pair
var paths = _.map(files, function (file) { return path.join(dir, file); });
var stats = await (_.map(paths, function (path) { return fs.statAsync(path); })); // parallel!
return _.filter(stats, function (stat) { return stat.isFile(); }).length;
});
// Give it a spin
countFiles(__dirname)
.then (function (num) { console.log('There are ' + num + ' files in ' + __dirname); })
.catch(function (err) { console.log('Something went wrong: ' + err); });
`The function
returns the number of files in a given directory. To find this number, it must perform multiple asynchronous operations (using fs.readdir and fs.stat). countFiles is declared as a suspendable function by wrapping its definition inside async(...). When countFiles is called with a dir string, it begins executing asynchronously and immediately returns a promise of a result. Internally, countFiles appears to have synchronous control flow. Each await call suspends execution until its argument produces a result, which then becomes the return value of the await call.$3
The examples folder contains more examples. The comparison folder also contains several examples, each coded in six different styles (using plain callbacks, using synchronous-only code, using the async library, using the bluebird library, using the co library, and using this asyncawait library). 7.
async in Depth: Suspendable Functions
The subsections below refer to the following code:
`javascript
var suspendable = async (function defn(a, b) {
assert(...) // may throw
var result = await (...)
return result;
});
var suspendable2 = async.cps (function defn(a, b) {...});
var suspendable3 = async.thunk (function defn(a, b) {...});
var suspendable4 = async.result (function defn(a, b) {...});
`$3
Suspendable functions may accept arguments. Calling will in turn call defn(1, 2). Suspendable functions may be variadic. They report the same arity as their definition (i.e. suspendable.length and defn.length both return 2).A suspendable function's definition may return with or without a value, or it may throw. Returning without a value is equivalent to returning
undefined. The return value of the definition function becomes the result of the suspendable function (see Obtaining Results from Suspendable Functions). $3
A suspendable function's definition may throw exceptions directly or indirectly. If any of the await calls in defn asynchronously produces an error result, that error will be raised as an exception inside defn.Within the definition of a suspendable function, exceptions may be handled using ordinary
try/catch blocks. Any unhandled exception thrown from within defn will become the error result of suspendable.$3
A suspendable function executes asynchronously, so it cannot generally return its result (or throw an error) directly. By default, async produces suspendable functions that return promises. suspendable returns a promise that is fulfilled with defn's return value, or rejected with defn's exception. Other ways of communicating results/errors are also supported:- Returning a promise:
suspendable(1, 2).then(function (val) {...}, function (err) {...});
- Acceptng a node-style callback:
- returning a lazily-executed thunk:
- returning the value directly: Note that
can only be called from inside another suspendable function. Also, it is possible to create suspendable functions that comminucate results in multiple ways, such as both accepting a callback and returning a promise. You can use the async.mod function to achieve this.$3
When a suspendable function is called, its this context is passed through to the call to its definition. For example, when suspendable.call(myObj, 1, 2) is executed, defn will be called with arguments 1 and 2 and a this value of myObj.$3
The async function can be used to create asynchronous iterators. These are analogous to ES6 iterators, except that the next() function is a suspendable function obeying all the rules described in this section. async.iterable creates an iterable which returns an asynchronous iterator whose next() function returns a promise of a {value, done} result.Asynchronous iterators have a
forEach() method for iterating over their values. For more information, take a look at the descendentFilePaths.js and iteration.js examples.$3
Calling a suspendable function such as suspendable starts its asynchronous execution immediately, as per the normal semantics of promises. In contrast, thunk-returning suspendable functions do not begin executing until a callback is passed to the thunk. Suspendable functions such as suspendable3 thus have lazy semantics.$3
Suspendable functions may be called in await expressions, since they return promises (or thunks or values) and are therefore awaitable. It follows that calls to suspendable functions may be arbitrarily nested and composed, and may be recursive.$3
Every variant of the async function (i.e. async, async.cps, async.iterable, etc) has a mod method that accepts an options object and returns another async function variant. The options object may contain any combination of the following four properties:`javascript
{
returnValue: ; // Recognised values: 'none', 'promise', 'thunk', 'result'
acceptsCallback: ;
isIterable: ;
maxConcurrency: ; // Recognised values: falsy values and positive numbers
}
`
Omitted properties will inherit their value from the variant being modded. For example, the calls async.mod({acceptsCallback:true}) and async.cps.mod({returnValue:'promise'}) are equivalent. Both calls return an async function that may be used to create suspendable functions that both accept a callback and return a promise.8.
await in Depth: Awaitable Expressions
The subsections below refer to the following code:
`javascript
var suspendable = async (function () {
var promise1 = new Promise(.../ eventually produces the value 'p1' /);
var promise2 = new Promise(.../ eventually produces the value 'p2' /);
var thunk1 = function(callback) {.../ eventually produces the value 't1' /});
var thunk2 = function(callback) {.../ eventually produces the value 't2' /});
var thunk3 = ..., thunk4 = ...;
var r1 = await (promise1);
var r2 = await (thunk1);
var r3 = await (3.14);
var r4 = await ([promise2, 2, ['222', thunk2]]);
var r5 = await ({ t3: thunk3, t4: thunk4 });
return [r1, r2, r3, r4, r5];
});
`$3
takes a single argument, which must be an awaitable expression. An awaitable expression may be any of the following:1. A promise (or any
thenable object), as in var r1 = await (promise1). The function suspendable will be suspended until promise1 is settled. The promise's resolution value ('p1') will become the await call's return value, and assigned to r1. If promise1 is rejected, the rejection value will be thrown as an exception inside suspendable.
2. A thunk, as in var r2 = await (thunk1). The thunk thunk1 will be called immediately, and suspendable will be suspended until control is returned to the thunk's callback. The thunk's result ('t1') will become the await call's return value, and assigned to r2. If thunk1 returns an error, the error value will be thrown as an exception inside suspendable.
3. A primitive value, such as a number, string or null, as in var r3 = await (3.14). The await call will return immediately with the primitive value, in this case assigning the value 3.14 to r3.
4. An array or plain object, whose elements are all awaitables, as in var r4 = await ([promise2, 2, ['222', thunk2]]). Note this definition is recursive and allows nested object graphs. The function suspendable will be suspended until all contained awaitables (promise2, 2, '222' and thunk2) have produced their value, at which time the await call will return a clone of the object graph with all awaitable expressions replaced by their results (['p2', 2 ['222', 't2']]). If any of the contained awaitables produces an error, the error value will be thrown as an exception in suspendable.Note that calling
await with more than one argument (or with zero arguments) is equivalent to calling await with a single array containing all the arguments.$3
In conventional Node.js code, asynchronous functions take a callback as their last parameter and don't return any value. Therefore, calls to these functions are not awaitable. However, awaitable versions may be obtained with relative ease using something like bluebird's promisifyAll(), or thunkify.$3
A series of await calls are executed serially. For example, execution of var r1 = await (promise1) is completed before execution of var r2 = await (thunk1) begins.In contrast, a single
await call on an array or plain object processes all of the contained awaitables concurrently. For example, when the statement var r5 = await ({ t3: thunk3, t4: thunk4 }) both thunk3 and thunk4 are called immediately, and their asynchronous tasks are executed concurrently. Libraries such as lodash and underscore interoperate smoothly with
asyncawait, for both producing arrays of concurrently executing tasks, and for consuming arrays of results.$3
There are several variations of the await function, with alternative behaviour when the awaitable expression is an array or plain object. Take a look at awaitTop.js for a usage example.The
await.top(n) variant accepts a number n, and resumes the suspendable function when the first n awaitable expressions contained in the awaitable array or plain object produce their value. The return value of the await.top(n) call is an array containing the fastest n results in the order they were resolved.The
await.in variant is like await, but does not clone the awaitable expression it recieves as an argument. The results of the contained awaitables are substituted in place into the original awaitable array or plain object, which becomes the return value of the await call.9. Recipes
$3
Coming soon...$3
`js
const assert = require('assert');
const fs = require('fs');const dwait = require('./fibersync/').dwait;
const defer = require('./fibersync/').defer;
const asyncBdd = require('./fibersync/').asyncBdd;
const {it, before, after} = asyncBdd.wrap(global); // main call in the async tests example
describe('thing', function() {
let data;
before(function() {
// We can load things in the before hook.
data = dwait (fs.readFile('./data', 'utf-8', defer()));
});
it('should have data', function() {
// We can arbitrarily yield in it blocks.
dwait(setImmediate(defer()));
assert(typeof data === 'string' && data.length > 0);
});
});
`10. API Reference
$3
Creates a function that can be suspended at each asynchronous operation. contains the body of the suspendable function. async returns a function of the form (...args) --> Promise. Any arguments passed to this function are passed through to fn. The returned promise is resolved when fn returns, or rejected if fn throws.$3
Variant of async that produces a suspendable function that accepts a node-style callback and returns nothing. See Obtaining Results from Suspendable Functions.$3
Variant of async that produces a suspendable function that returns a thunk. See Obtaining Results from Suspendable Functions.$3
Variant of async that produces a suspendable function that returns its result directly, but can only be called from inside another suspendable function. See Obtaining Results from Suspendable Functions.$3
Variant of async that produces a function which returns an asynchronous iterator, whose next() method is a suspendable function that returns a promise. See Creating and Using Asynchronous Iterators.$3
Enables the creation of arbitrary variants of the async function. Accepts an options object and returns an async function variant. See The async.mod Function.$3
Suspends a suspendable function until the awaitable expression expr produces a result. The result becomes the return value of the await call. If expr produces an error, then an exception is raised in the suspendable function.$3
Variant of await whose result consists of the n fastest-resolving awaitables contained in its argument. See Variations of await.$3
Variant of await that returns the original array/object, rather than a cloned array/object, substituting the results of contained awaitables in-place. See Variations of await.11. Acknowledgements
asyncawait uses the following technologies:- node-fibers: This implementation of coroutines is unfortunately limited to Node.js. ES6 generators may be simpler, but fibers are more flexible and support a far broader space of design possibilities. It would be great if ES6 generators were this open and flexible.
- bluebird: this promise library is both a core component of
asyncawait and a great source of inspiration for writing high-performance JavaScript code. Bluebird is optional.
- TypeScript: asyncawait` is written in TypeScript (look in the src folder), and includes a type declaration file. TypeScript makes JavaScript development faster, less error-prone, more scaleable, and generally more pleasant. TypeScript is optional.