Heretic

An opinionated background job runner. Job payloads, logs, and metadata are backed
by Postgres. Job queueing and execution are backed by RabbitMQ.

Also, ES6/7.

$3

Lots of job queues are backed by Redis, but in order to leverage Redis's queueing
in a consistent way, you have to write a lot of Lua scripts, and you can still
easily run into issues with jobs getting stuck in weird, inconsistent states.

Redis is also not a great option if you want to be able to search and filter
jobs (by type, by payload, etc). It's up to you (or your queueing library) to
implement your own indexing.

$3

Probably not. It wouldn't be my first Terrible Idea™.

Usage

Not implemented yet:

- Automatic failure retries
- Tests

$3

Feel free to add any other triggers/indexes you'd like.

``sql
CREATE TABLE heretic_jobs (
id SERIAL NOT NULL PRIMARY KEY,
queue_name text NOT NULL,
status text DEFAULT 'pending',
payload jsonb,
attempt_logs jsonb[] DEFAULT '{}',
max_attempts int NOT NULL DEFAULT 1,
created_at timestamptz NOT NULL DEFAULT now(),
updated_at timestamptz NOT NULL DEFAULT now(),
last_attempted_at timestamptz
);

CREATE INDEX ON heretic_jobs (queue_name);
CREATE INDEX ON heretic_jobs (status);

CREATE FUNCTION heretic_updated_at_timestamp() RETURNS TRIGGER AS $$
BEGIN
NEW.updated_at = now();
RETURN NEW;
END;
$$ LANGUAGE 'plpgsql';

CREATE TRIGGER update_heretic_jobs_updated_at
BEFORE UPDATE ON heretic_jobs
FOR EACH ROW EXECUTE PROCEDURE heretic_updated_at_timestamp();
`

$3

`js
import Promise from 'bluebird';
import Heretic from 'heretic';
import knex from 'knex';

const heretic = new Heretic('amqp://127.0.0.1', knex({
client : 'pg',
connection : 'postgres://127.0.0.1',
}));

heretic.process('my.job', function(job, message, done) {
Promise.resolve()
.then(() => {
// do some processing or something. whatever you want!
})
.nodeify(done);
});

heretic.process('other.job', function(job, message, done) {
setTimeout(() => {
done(new Error('job failed'));
}, 1000);
});

// the 'jobError' event happens when a job message is published in RabbitMQ that
// can never be handled correctly (malformed JSON, job id doesn't exist in the
// database, etc.). The message will be dead-lettered for later inspection (by you)
heretic.on('jobError', (err) => {
console.error('Error with job!', err.stack);
});

// the 'jobFailed' event happens when a job fails, but in a recoverable way. it
// will be automatically retried up to the maximum number of retries.
heretic.on('jobFailed', (err) => {
console.error('Job execution failed!', err.stack);
});

// enqueue a job
heretic.enqueue('email.send', {
to : 'bob@example.com',
body : 'Hi',
});
``