Distributed, resilient, Redis‑backed job scheduler for Node.js (cron, multi‑queue, retries + DLQ, definition store, CLI)
npm install node-quartz





A modern, fault‑tolerant job scheduler for Node.js with cron support, multi‑queue workers, retries + DLQ, definition stores (memory/file/custom), and Redis‑based coordination.
BLMOVE (fallback to RPOPLPUSH) for low‑latency pollingscriptsDir or provide in‑memory processors mapHow it compares
- Bull/BullMQ: Great for queue processing with rich features; Node Quartz focuses on cron scheduling + simple job execution with lighter footprint.
- Agenda/Bree: Similar cron scheduling space; Node Quartz adds Redis‑backed definition sync, multi‑queue workers, and a focused CLI.
+--------------------------+
| Job Store (opt) |
| - memory / file / custom|
+------------+-------------+
| load() / upsert
v
+------------------+
| Redis |
|------------------|
| defs:index (SET) |
| defs: (STR) |<-- CLI defs:add/remove/reload
| defs:events (PUB)|----^
| |
| jobs (LIST/KEYS) |<-- enqueue/TTL (:next/:retry)
| processing (LIST)|
| failed (LIST) |<-- CLI failed:list/requeue/delete
| master (KEY) | (pubsub: __keyevent__ expired)
+--------+---------+
^
pubsub (events) | keyspace events (expired)
+---------------------+----------------------+
| |
v v
+--+----------------+ +------+---------------+
| Scheduler A | | Scheduler B |
|-------------------| |---------------------|
| - master election |<-- heartbeat ----->| - standby/worker |
| - schedule cron | | - schedule on events|
| - worker loop |<-- BL/MOVE/RPOP -->| - worker loop |
| - processors | | - processors |
+--+----------------+ +------+---------------+
| processors (scriptsDir) | processors (scriptsDir)
v v
[job module fn] [job module fn]CLI: interacts directly with Redis (defs:*, failed, jobs) to inspect
and control state; changes propagate via defs:events.
`Installation
It's on NPM.
npm install node-quartz
Quick Start
`bash
1) Start Redis with keyspace notifications enabled
Local (requires redis-server installed):
redis-server --notify-keyspace-events ExOr via Docker:
docker run --rm -p 6379:6379 redis:7 redis-server --notify-keyspace-events Ex2) In another terminal, run the demo (closes after ~35s)
Optionally set REDIS_URL if Redis isn't on localhost
export REDIS_URL=redis://127.0.0.1:6379
npm run demo3) Inspect failed jobs (if any) with the CLI
npx quartz failed:list --count 20 --redis "$REDIS_URL" --prefix quartz:testRequeue the first failed job and reset attempts
npx quartz failed:requeue --idx 0 --reset --redis "$REDIS_URL" --prefix quartz:test
`Usage
`javascript
// create an instance
const create = require('node-quartz');
const quartz = create({
scriptsDir: '/my/scripts/path',
prefix: 'quartz', // optional, defaults to 'quartz'
logger: console, // optional, provide your own logger (debug/info/warn/error)
queues: ['high', 'default', 'low'], // optional, defaults to ['default']
heartbeat: { intervalMs: 2000, jitterMs: 500 }, // optional jittered master heartbeat
// Optional: persist and sync job definitions across instances
// Uncomment one of the store options below
// store: { type: 'memory', jobs: [ / preloaded Job objects / ] },
// store: { type: 'file', path: './jobs.json' },
// store: { type: 'custom', impl: myStore },
redis: {
// Prefer url form; legacy host/port also supported
url: process.env.REDIS_URL || 'redis://localhost:6379'
},
// Optional: provide in-memory processors map instead of requiring files
// processors: { 'scriptToRun': async (job) => { / ... / } }
}); // schedule a job (6-field cron with seconds)
const job = {
id: 'job_id',
script: 'scriptToRun', // resolved relative to scriptsDir
cron: '/10 ', // every 10 seconds
data: { any: 'payload' }, // optional payload passed to your script via job.data
queue: 'high', // optional, defaults to 'default'
options: {
currentDate: null,
endDate: new Date(Date.now() + 60 * 1000),
tz: 'America/New_York' // optional, timezone for cron schedule
}
};
quartz.scheduleJob(job);
// Later, shut down cleanly (unsubscribe and quit Redis):
// await quartz.close();
`Your job processor module (at
/my/scripts/path/scriptToRun.js) should export a function
and may be sync, callback-based, or async (Promise):`javascript
// /my/scripts/path/scriptToRun.js
module.exports = function (job, done) {
// access job.data if you provided it
console.log('processing job', job.id, job.data);
// do work, then call done(err?)
done();
};
// or, Promise/async style
// module.exports = async function (job) {
// console.log('processing job', job.id, job.data);
// // await work
// };
`
Requirements
- Node.js >= 14
- Redis with keyspace notifications for expired enabled (notify-keyspace-events Ex).
- Local example: redis-server --notify-keyspace-events Ex
- Docker Compose/service: add args --notify-keyspace-events ExRedis Options
- redis.url: connection string, e.g. redis://localhost:6379
- redis.database: database index (number). Defaults to 0. Keyspace notifications
subscriptions use this DB for __keyevent@.
- Legacy redis.host and redis.port are still accepted and converted to a URLOther Options
- prefix: Redis key prefix (default quartz). Keys: , ,
, , .
- logger: pluggable logger with debug/info/warn/error methods; defaults to console.
- processors: object map of { [scriptName]: processorFn } to avoid dynamic require().
- queues: array of queue names to poll (default ['default']). Jobs with job.queue are pushed
to and processed atomically into .
- heartbeat: { intervalMs?: number, jitterMs?: number } controls master heartbeat frequency
and random jitter (defaults: 2000ms interval, ±500ms jitter).
- store: load and synchronize job definitions across instances.
- Memory: { type: 'memory', jobs: [/ Job /] }
- File: { type: 'file', path: './jobs.json' } (JSON array of Job objects)
- Custom: { type: 'custom', impl } where impl implements load/list/save/remove$3
- Definitions are stored in Redis under:
- Set:
- Keys: (stringified Job)
- PubSub: (JSON messages: {action:'upsert'|'remove'|'reload', id?})
- On startup: loads from configured store (optional), upserts to Redis, loads all Redis definitions, and schedules them.
- CLI can manage definitions; changes propagate via PubSub to all instances.Retries and Failures
- Set retry on the job (top-level or under options.retry):
- maxAttempts: number of retry attempts
- backoff: either a number (base delay ms, exponential) or an object
{ delay: number, factor?: number, maxDelay?: number }
- On failure:
- If attempts remain, the job is scheduled for retry using key expiry ().
- If exhausted, the job is pushed to with minimal error info.
- Cron jobs: on success they reschedule to the next run; on failure they follow the retry policy for the current run, and continue with future schedules after retries are exhausted.Worker Loop
- A background worker loop consumes via BRPOPLPUSH, moves items to , and runs your processor.
- With multiple queues, the worker attempts an atomic BLMOVE from each queue's jobs list to its processing list (Redis >= 6.2),
falling back to round‑robin RPOPLPUSH with short sleeps.
- On startup, it recovers orphaned items from each back to .
- close() stops the loop and quits Redis gracefully.API
- Factory: const quartz = create(options)
- Methods:
- scheduleJob(job): schedule or enqueue a job (supports 6-field cron with seconds)
- getJob(jobId, cb): fetch stored job payload
- removeJob(jobId, cb): delete stored job payload
- listJobsKey(cb): list all persisted job keys for the prefix
- close(cb?): stop worker loop and quit Redis connections
- Events (quartz.events is an EventEmitter):
- scheduled (job, nextDate)
- started (job)
- succeeded (job)
- failed (job, error)
- retryScheduled (job, delayMs)The library uses
node-redis v4 (async).CLI
Install globally or use via npx:- List failed jobs:
quartz failed:list --prefix quartz --redis redis://localhost:6379 --count 20
- Requeue a failed job: quartz failed:requeue --idx 0 --prefix quartz --redis redis://localhost:6379 --reset
- Delete a failed job: quartz failed:delete --idx 0 --prefix quartz --redis redis://localhost:6379
- Purge failed queue: quartz failed:purge --prefix quartz --redis redis://localhost:6379
- Inspect by id: quartz failed:get --id
- Requeue by id: quartz failed:requeue-id --id
- Delete by id: quartz failed:delete-id --id
- Export failed to file: quartz failed:drain-to-file --out failed.json --prefix quartz --redis redis://localhost:6379 --purge
- Import failed from file: quartz failed:import-from-file --in failed.json --prefix quartz --redis redis://localhost:6379
- Requeue from file: quartz failed:import-from-file --in failed.json --requeue --reset- List job definitions:
quartz defs:list --prefix quartz --redis redis://localhost:6379
- Add a job definition from file: quartz defs:add --file job.json --prefix quartz --redis redis://localhost:6379
- Remove a job definition: quartz defs:remove --id job_id --prefix quartz --redis redis://localhost:6379
- Ask instances to reload defs: quartz defs:reload --prefix quartz --redis redis://localhost:6379You can also set env vars:
REDIS_URL and QUARTZ_PREFIX.Testing
- Start Redis with keyspace notifications: redis-server --notify-keyspace-events Ex
- Run tests: npm test
- CI workflow runs tests against Redis (with notifications enabled) on Node 14/16/18/20.$3
- Run tests in containers (spins Redis and a Node runner):
- docker compose up --abort-on-container-exit --exit-code-from test
- Or via npm script: npm run test:compose
- The test runner mounts your working directory and uses REDIS_URL=redis://redis:6379`.