A Node-RED node that buffers and transmits data reliably with local file-based ring buffer storage.
npm install node-red-contrib-simple-buSimpleBU is a custom Node-RED node that buffers, formats, and transmits measurement data to a server (e.g. deZem M2M endpoints).
It is designed for robustness in environments with unstable internet connections:
- If the server is reachable → data is sent immediately.
- If the server is not reachable → data is stored locally in a disk-based ring buffer.
- As soon as connectivity returns, the node will drain buffered data in chronological order.
This ensures no data loss, strict CSV format, and predictable uploads.
---
The node accepts Node-RED msg objects. The important fields are:
- msg.payload → numeric value(s)
- msg.topic → ID (required if payload is a single number)
- msg.timestamp (optional) → if not provided, the node auto-generates the current UNIX time.
json
{
"topic": "temp1",
"payload": 22.5
}
`Produces:
temp1,22.5,1757385119
$3
json
{
"payload": [
["temp1", 22.5],
["power", 120]
]
}
`Produces:
temp1,22.5,1757385119
power,120,1757385119
---
Features
- CSV output format
Each line is strictly
.- Timestamp handling
- If
msg.timestamp exists, it is used.
- Supported formats:
- UNIX seconds → 1757385119
- UNIX milliseconds → 1757385119000
- ISO8601 string → "2025-09-17T10:30:00Z"
- JavaScript Date object
- All are normalized to UNIX seconds.- Single-file ring buffer
- Configured by total size in MB (default 100 MB).
- No splitting into multiple files.
- When buffer is full → the oldest data is dropped (with a warning).
- Prevents mixing data or losing order.
- HTTP POST uploads
- Each request sends at most 100 KB.
- Large files are split into 100 KB chunks.
- Successfully sent chunks are removed from the file.
- Resilience
- If server is down, data is kept locally until connectivity returns.
- Buffered data is always sent oldest-first.
- Debug mode
- If enabled, all errors and warnings are logged in Debug sidebar.
- Examples:
- “Missing ID (msg.topic)”
- “Invalid value”
- “Ring cap reached: dropped 512 KB”
- “Cannot write to /data/buffer”
---
Configuration
| Field | Description |
|-------------------------|-------------|
| Endpoint-URL | Target URL where CSV data is POSTed. Required. Example:
https://m2m.dezem.de/services/hpp/simple-bu.php. |
| Simple SN | Serial number of the device/logger. Added to User-Agent header. |
| Simple Type | Device type string. Added to User-Agent header. |
| Ring Buffer Size (MB) | Maximum total size of the local buffer file (default: 100 MB). If exceeded, oldest data is dropped. |
| Storage Path | Directory for storing the buffer file (e.g. /data/buffer). Must be writable. |
| Flush Interval (s) | How often the node attempts to send data (in seconds). Default: 60s. |
| HTTP Timeout (s) | Maximum time to wait for server response (in seconds). Default: 10s. |
| Debug Enabled | If checked, detailed error/warning messages are shown in the Debug window. |
| Name | Optional display name in the flow. |---
Status Indicators
- 🔵 Buffered → Data accepted into RAM buffer.
- 🟢 Data sent → Data successfully uploaded.
- 🟢 Drained file → Buffered file chunk uploaded & removed.
- 🟡 Ring cap reached: dropped X KB → Oldest data was deleted because buffer exceeded configured size.
- 🔴 Send failed / Cannot write → Error writing to disk or sending to server.
---
Server Communication
- Data is sent via HTTP POST with headers:
`
Content-Type: text/csv; charset=utf-8; header=absent
User-Agent: IF:Simple TYPE: SN:
- Server must respond with:
- HTTP 2xx status
- Response body: empty or If not, the data remains in the buffer.
---
- Invalid ID or value → error status (red).
- Non-writable storage → “Cannot write to /path/to/storage”.
- Buffer overflow → “Ring cap reached: dropped N KB”.
- Send failed → data stays in buffer, retry later.
With Debug Enabled, full error details are logged.
---
MIT License
© 2025 deZem GmbH / Emir Dovletyarov