@nm-logger/logger

Minimal JSON logger for Express + Axios with optional S3 sync.

Features

- Logs handled Express requests as success
- Logs Express errors as error
- Logs Axios calls as external
- Optional database error logs as db_error
- Local daily files under configurable baseDir:
- YYYY/MM/DD/daily_logs_success.json
- YYYY/MM/DD/daily_logs_error.json
- YYYY/MM/DD/daily_logs_external.json
- YYYY/MM/DD/daily_logs_db_error.json
- S3: one JSON file per day per category (append by read + merge + put)
- Previous day's local folder is removed when the date changes
- employee_id taken from req.user.employee_id || req.user.emp_code || req.user.id (used internally only, not written to JSON)
- Each log entry includes:
- url, method, type (internal/external/db_error)
- status_code
- user_agent
- project_version (from Logger options)
- error
- optional response (for external errors)
- count (when consecutive identical entries are compacted)
- External API logging:
- Logs response body only for error-like calls (4xx/5xx or when a custom externalErrorDetector marks it as error)
- Long responses are truncated with maxResponseLength

Install

``bash
npm install @nm-logger/logger
`

Basic usage (Express + Axios)

`js
const express = require("express");
const axios = require("axios");
const Logger = require("@nm-logger/logger");

const app = express();

const nm_logger = new Logger(
{
accessKeyId: process.env.AWS_ACCESS_KEY_ID,
secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
region: "us-east-1",
bucket: "project",
},
{
baseDir: "/var/log/nm-logger",
maskFields: ["aadhaar", "pan"],

// Optional metadata for logs
projectVersion: "Project-1.0.0",

// Background S3 uploader (set to false if you prefer cron; see below)
enableDailyWatcher: true,

// Interval for background S3 uploads when enableDailyWatcher = true
uploadIntervalMs: 60 * 1000, // 1 minute

// How many characters of external response to keep (after masking)
maxResponseLength: 4000,

// If true (default), include external response body only for error-like calls
logExternalResponseOnError: true,

// Control logging of successful external calls
// - true (default): log all external calls
// - false: log only error-like calls (4xx/5xx or externalErrorDetector = true)
logExternalSuccess: true,

// Optional semantic error detector for 2xx/3xx external responses
// Example below in "External error-only logging"
externalErrorDetector: null,
}
);

// Express middlewares
app.use(express.json());
app.use(nm_logger.requestLoggerMiddleware());
app.use("/api", require("./routes"));
app.use(nm_logger.expressMiddleware());

// Attach Axios interceptor
nm_logger.attachAxiosLogger(axios);

app.listen(3000, () => {
console.log("Server listening on 3000");
});
`

$3

Make sure the base directory exists and is writable by your Node process:

`bash
mkdir -p /var/log/nm-logger
chmod -R 777 /var/log/nm-logger
`

External error-only logging (recommended for high load)

If your external API returns HTTP 200 but the body contains error info, you can:

- Skip logging successful calls completely
- Still log error-like responses with full (masked + truncated) body

Example:

`js
const nm_logger = new Logger(
{
accessKeyId: process.env.AWS_ACCESS_KEY_ID,
secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
region: "us-east-1",
bucket: "project",
},
{
baseDir: "/var/log/nm-logger",
maskFields: ["aadhaar", "pan"],

enableDailyWatcher: false, // use cron for S3 (see next section)
logExternalSuccess: false, // do NOT log success responses

externalErrorDetector: ({ response }) => {
try {
// Adjust this according to your external API response format
if (response && response.success === false) return true;
if (response && typeof response.status === "string" &&
response.status.toLowerCase() === "error") return true;
if (response && (response.error || response.error_code)) return true;
} catch (e) {
return false;
}
return false;
},
}
);
`

With this config:

- Only external calls that look like errors are logged (4xx/5xx or externalErrorDetector = true).
- For these, the masked + truncated response body is stored in JSON.

S3 upload modes

There are two ways to sync daily logs to S3.

$3

If enableDailyWatcher: true (or omitted), the logger will:

- Start an internal setInterval in the same Node process
- Every uploadIntervalMs:
- Read local daily log JSONs
- Merge them with S3 files
- Upload merged logs back to S3
- Clean up previous day's local folder

This is simple but adds some extra work inside your API process.

$3

To keep your API process as light as possible, you can:

1. Disable the internal watcher in your API
2. Use a separate cron script to upload logs to S3

#### 1. API server: disable watcher

`js
const nm_logger = new Logger(
{
accessKeyId: process.env.AWS_ACCESS_KEY_ID,
secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
region: "us-east-1",
bucket: "project",
},
{
baseDir: "/var/log/nm-logger",
maskFields: ["aadhaar", "pan"],
enableDailyWatcher: false, // 🔴 no setInterval in API process
}
);

app.use(nm_logger.requestLoggerMiddleware());
app.use("/api", routes);
app.use(nm_logger.expressMiddleware());
nm_logger.attachAxiosLogger(axios);
`

This way, the API only writes local JSON files.

$3

If you are already using node-cron in your Node project, you can avoid Linux crontab entirely and schedule uploads from your app code.

1. Disable the internal watcher in your API (same as before):

`js
const nm_logger = new Logger(
{
accessKeyId: process.env.AWS_ACCESS_KEY_ID,
secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
region: "us-east-1",
bucket: "project",
},
{
baseDir: "/var/log/nm-logger",
maskFields: ["aadhaar", "pan"],
enableDailyWatcher: false, // no setInterval
}
);
`

2. Use node-cron in your Node app to trigger uploads:

`js
const cron = require("node-cron");
const Logger = require("@nm-logger/logger");

const nm_logger = new Logger(
{
accessKeyId: process.env.AWS_ACCESS_KEY_ID,
secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
region: "us-east-1",
bucket: "project",
},
{
baseDir: "/var/log/nm-logger",
maskFields: ["aadhaar", "pan"],
enableDailyWatcher: false,
}
);

// Every 5 minutes
cron.schedule("/5 *", async () => {
try {
await nm_logger.uploadDailyLogsOnce();
console.log("[nm-logger] ✅ Daily logs uploaded to S3 (node-cron)");
} catch (err) {
console.error("[nm-logger] ❌ Upload failed (node-cron):", err);
}
});
`

This way you:
- Do not depend on OS-level crontab.
- Still avoid setInterval-based uploads inside the main request flow.
- Keep all scheduling logic inside Node where you are already comfortable.

#### 3. System Crontab entry (optional, Linux)

If you prefer OS-level cron instead of node-cron, you can still use a separate script as before.

Create a separate script (for example in your backend root):

`js
// upload-logs.js
require("dotenv").config();
const Logger = require("@nm-logger/logger");

async function main() {
const logger = new Logger(
{
accessKeyId: process.env.AWS_ACCESS_KEY_ID,
secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
region: "us-east-1",
bucket: "project",
},
{
baseDir: "/var/log/nm-logger",
maskFields: ["aadhaar", "pan"],
enableDailyWatcher: false, // no internal timers here either
}
);

try {
await logger.uploadDailyLogsOnce();
console.log("[nm-logger] ✅ Daily logs uploaded to S3");
} catch (err) {
console.error("[nm-logger] ❌ Failed to upload daily logs:", err);
process.exitCode = 1;
}
}

main();
`

On your server (Linux):

`bash
crontab -e
`

Add a line like:

`cron
/5 * /usr/bin/node /path/to/upload-logs.js >> /var/log/nm-logger-cron.log 2>&1
`

- This will not be created automatically by the package.
- You must add it manually using crontab -e.
- Make sure the node path (/usr/bin/node) and script path (/path/to/upload-logs.js) are correct for your server.

Notes

- The logger does not create or manage cron jobs automatically; you must configure node-cron or crontab yourself if you choose cron mode.
- For lower CPU and disk usage on busy servers:
- Prefer logExternalSuccess: false + externalErrorDetector` to log only problematic external calls.
- Use cron-based upload (Mode 2 / Mode 2b) so S3 sync runs in a separate short-lived context.