flow-engine

The flow-engine is a node.js module specializing in processing of a sequence of
tasks. The tasks execution sequence is defined in JSON object's assembly
property and passed to flow-engine for execution.
In The following example, flow-engine executes three tasks - if, if, and
response.

``
{
"assembly": {
"execute": [
{
"if": {
"condition": "'$(request.verb)' === 'POST'",
"execute": [
{
"message-mediation": {
"target": "context.response",
"value": "{ status: 200, message: 'This is a POST response' }"
}
}
]
}
},
{
"if": {
"condition": "'$(request.verb)' === 'GET'",
"execute": [
{
"message-mediation": {
"target": "context.response",
"value": "{ status: 200, message: 'This is a GET response' }"
}
}
]
}
},
{
"response": {
"payload": "context.response"
}
}
]
}
}
`

Each task is a JavaScript function that can
- execute any code
- make change to the context object
- instruct flow-engine to continue or stop the remaining tasks execution

The flow-engine provides some prebuilt tasks at lib/task.
You can use these prebuilt tasks to compose your flow or use them as references
to build customized ones. Putting the customized tasks to the lib/task
directory, and flow-engine can pick up the new tasks without additional
configuration.

flow-engine and IBM microgateway

The flow-engine is designed to be the heart of a gateway or reverse proxy.
It is included and used as part of the microgateway module that
serves as API policies enforcement point.

Each API policy on the microgateway is implemented as a flow-engine
task. And what and when API policies need to be enforced is describe as the
assembly flow to be executed by the flow-engine.

#Installation
npm install flow-engine

#APIs
To use the FlowEngine, one must require('flow-engine').

- setup function require('flow-engine')(configObj):
configObj supports the follow options:
- flow : the flow definition file and it shall be a YAML file
- paramResolver: a module's location that is used to resolve the
placeholder inside the flow assembly
- tasks: an object that contains all custom task modules' name and
location
- baseDir: the base directory that will be used to load modules

and the return value is a middleware handler

- Flow(assembly, optionObj):
Create a flow instance with the flow assembly
- assembly: the flow's assembly and it's JSON object
- optionObj: supports the following options:
- paramResolver: a function that is used to resolve the placeholder inside
the flow assembly
- tasks: an object that contains all custom task modules' name and their
handler functions. Use this option to control the creation/lifecycle of
task handlers by yourself.
- tid: transaction id. if it doesn't present, flow-engine uses tid() to get
tid that is maintained in flow-engine's scope.
- logger: a bunyan logger obj that flow-engine uses it for logging. if it
doesn't present, flow-engine creates its own logger.
- logLevel: flow-engine uses it to create its own logger if logger is not
there.

- Flow.prototype.prepare(context, next):
Pass the context object and next function to the flow instance that you
create by the Flow ctor

- Flow.prototype.run():
Start to execute the flow assembly

- Flow.prototype.subscribe(event, handler):
Subscribe an event with an event handler.
- event: 'START', 'FINISH', 'ERROR' or 'pre|post:task-name'
- handler: a handler should be:
function(event, next)
A handler function would get the event name and a done callback(next).
The handler must call next() when it finishes.

- Flow.prototype.unsubscribe(event, handler):
Remove the handler from the event's subscriber list

- tid():
get a flow-engine scope transaction Id. Basically, it's a global counter
that start with 1 and increase 1 by each call.

To execute a flow described in YAML:

`
var flow = require('flow-engine')({ config: "flow.yaml" });
//the flow is a middleware handler and ready to be used

`

Or directly create a Flow instance with a specific JSON config
`
const Flow = require('flow-engine').Flow;
var flow = new Flow( json, optionObj );
//you have to manually pass the context and next into the flow and execute it
//get context obj from somewhere or create it by yourself
//and request and response object should be attached to the context obj
flow.prepare(context, next);
flow.run();
`

#APIs for Task/Policy developer
A task is the unit of the flow assembly. flow-engine leverages tasks to fulfill
a flow assembly. When developing a custom task, flow-engine provides the
following APIs:

- invoke(assembly, options, next):
flow-engine runs the given assembly. When the assembly finishes. the next
callback will be invoked. the options here is the same as the options in
Flow(assembly, optionObj)
- assembly: the flow's assembly and it's JSON object
- next: this callback will be invoked after the assembly finishes
- options: supports the following options:
- paramResolver: a function that is used to resolve the placeholder inside
the flow assembly
- tasks: an object that contains all custom task modules' name and handler
function
If these properties don't exist, the parent's will be used.
- subscribe(event, handler):
Subscribe an event with an event handler.
- event: 'START', 'FINISH', 'ERROR' or 'pre|post:task-name'.
- handler: a handler should be:
function(event, next)
A handler function would get the event name and a done callback(next).
The handler must call next() when it finishes.

- unsubscribe(event, handler):
Remove the handler from the event's subscriber list

- proceed():
When a task finishes its job, it must call this API to tell flow-engine to
run the next task.

- fail(error):
When a task finishes its job with error situation, it must call this API to
tell flow-engine to run the Error handing to process the error.

- stop():
When a task finishes its job and the flow should also end, it must call this
API to tell flow-engine to finish the assembly execution.

- logger:
A bunyan logger object which is used for logging.

These APIs are attached to the third param of task handler interface and could
be used by task developer. Here is a simple task implementation:

`
'use strict';

module.exports = function ( config ) {

return function (props, context, flow) {
var logger = flow.logger;
logger.info('ENTER mypolicy, params:', JSON.stringify(props));
//some business logic

//and register a FINISH event handler
flow.subscribe('FINISH', function(event, next) {
//this would be executed when a flow finishes
next();
});

//call next task
flow.proceed();
};
};
`

#Sample
Use the setup function to create a flow-engine middleware and register this
middleware to handle all of the requests:
`
var createFlow = require('flow-engine');

var flow = createFlow({
flow: "flow.yaml",
tasks: {'activity-log': './myactivity-log.js'},
baseDir: __dirname});

var app = express();
app.post('/*', [ flow ]);

`

#The Design


$3


- The interface is function(request, response, next)
- The flow-engine module supports the middleware interface. It returns the
middleware handler:

`
var middlewareHandler = require('flow-engine')(some_config);
middlewareHandler(request, response, next);
`

$3


- Every task that is defined inside the assembly should have a corresponding
task module.And the task module should provide a factory function -
function(config) and return a task handler function -
function(props, context, flow). Therefore, flow-engine will use the task
module like this:

`
var taskHandler = require('taskModule')(task_config);
taskHandler(props, context, flow);
`

- **Finding task/policy handlers to execute the tasks/policies that defined in
the assembly**:
- tasks under /lib/task are loaded when require('flow-engine').
flow-engine calls these tasks' factory functions with empty config object
and keeps the returned task handler functions
- use task's name to search the task module and see if there is a task module
under options.tasks[taskName].
The options.tasks here is the options that you pass to Flow's constructor or
the setup function.
The difference is the options.tasks that you pass to the setup function will
be processed via the require(), call its factory function by passing empty
config object and get the task handler function. And the options.tasks that
you pass to Flow's constructor function shall be task handler functions
already.
- searching the task module under /lib/task.
- directly use require against the task's name, call its factory function
with empty config and get the handler function

- Execute a task/policy:
- use the Finding task handlers procedure above to get task's handler
function
- get the properties/values that defined in the assemble for the task and use
paramResolver to replace the placeholder into corresponding value if exists
- call the task handler function we get from step 1 and pass the
properties/values, context obj and flow obj into it. the flow here is an
object which contains a set of APIs:
- .proceed(): when a task finishes its job successfully, it must call this
function to tell flow-engine that its job is done, please continue with
next task
- .fail(error): when a task couldn't finish it job, it must call this
function to tell flow-engine that an error is found and ask flow-engine
to handle the error. flow-engine will perform the Error Handling
procedure then.
- .subscribe(event, cb): a task could subscribe some specific events.
flow-engine would invoke the cb when the specific events are triggered.
The cb shall be a function(event, next). Inside the callback, next()
must be called when the callback finishes. Currently, the following events
are supported:
- FINISH: a flow finishes. flow-engine finishes an assembly execution
- ERROR: an error is threw from an assembly execution
- pre|post:taskName: before or after a task's execution
- .unsubscribe(event, cb): unsubscribe an event
- .logger: a bunyan logger object and could be used for logging
- if the task finishes without any error, then the flow engine goes to the
next task

- Error Handling:
- check if there is local error handling for the task:
- if yes, then execute the tasks defined in the error handling:
- when the local error handling finishes and the local handling throws
another error, then go to global error handling
- then the flow ends
- if no, then see if there is any global error handling:
- no global error handling : use default error handling and then ends the
flow
- global error handling exists: execute the global error handling and then
ends the flow

$3

flow-engine supports observable interface and provides subscribe() and
unsubscribe() APIs. Currently, the following events are supported:

- START: flow-engine starts to execute an assembly and none of the policy is
executed yet
- FINISH: all of the policies in an assembly are executed successfully
- ERROR: the flow execution ends with an error
- 'pre|post:task-name': before or after the execution of a task/policy.
The post event is only triggered when a task/policy finishes without any
error, even if the 'catch' clause recovers the error.

In order to keep the flow execution is clean, any error that is thrown/created
by an event handler would only be logged and then ignored. Therefore, an event
handler wouldn't impact the flow execution as well as other event handlers.

You can access the subscribe() and unsubscribe() from a Flow instance if
you use the Flow ctor to create a flow instance. In side a task, the third
param of the task handler function is an flow object which contains
subscribe() and unsubscribe() APIs.

$3

The context object should be created before the flow-engine, probably by
using a context middleware before the flow-engine middleware. flow-engine uses
the context object as one of the arguments when invoking every task function.
A task could access context object, including retrieving and populating
properties. When flow-engine finishes, all the information should be stored into
the context object. Having a middleware after the flow-engine middleware is a
typical approach to produce the output content and maybe flush/write to the
response object at the same time.

Currently, flow-engine provides a context module in ./lib/context which could
be used to create the context object. It provides the following APIs:
- createContext([namespace]):
create a context object with the specified namespace. namespace is optional.

- The context object provides getter, setter and dot notation to access its
variables:
- get(name):
get the variable by name.

- set(name, value[, readOnly]):
add or update a variable with the specified name and value. default value
of readOnly is 'false'.
Use readonly=true to add read-only variable.
Note: updating a read-only variable will cause the exception.

- define(name, getter[,setter, configurable]):
- name: variable name
- getter: getter function
- setter: setter function
- configurable: allow re-define or not. (true|false)

add or update a variable with the specified getter, setter and configurable.
Use configurable=false to avoid variable re-define.
Note: set new value to a getter-only variable will cause the exception.

- del(name):
delete the variable by name.

- subscribe(event, handler):
subscribe an event with the handler function : function(event, next).

- unsubscribe(event, handler):
unsubscribe the event

- notify(event, callback):
fire the event and all of the handlers are invoked sequentially (based on
the registration sequence). Then callback is called with undefined or an
array of errors that subscriber handlers return.

- dot notation:
You can also use dot notation to access the variables of a context object.
``javascript
//If a context is created with a namespace - 'fool',
//you can access its variables like this:
var ctx1 = require('./lib/context').createContext('fool');
ctx1.set('myvar', 'value');
ctx1.fool.myvar === 'value';
ctx1.fool.myvar = 'new-value';
ctx1.get('myvar') === 'new-value';
//
//If there is no namespace, then all of the variables are
//directly attached to the context object:
var ctx2 = require('./lib/context').createContext();
ctx2.set('myvar', 'value');
ctx2.myvar === 'value';
ctx2.myvar = 'new-value';
ctx2.get('myvar') === 'new-value';
``