TailFile

At LogDNA, consuming log files and making them searchable is what we do!
It all starts with the ability to efficiently watch log files on a local
host and send new lines up to the LogDNA service. This Node.js class provides
functionality like Unix's tail -f command, and we use it in our
agents to get the job done. Of course, anything needing tail functionality
in Node.js could also benefit from using this.

---

* Features
* Installation
* Usage
* Example consuming data events
* Example using pipe
* Example using readline
* Example for Clean Shutdown
* Events
* Event: 'flush'
* Event: 'renamed'
* Event: 'retry'
* Event: 'tail_error'
* Event: 'truncated'
* Event: (Any Readable event)
* API
* [Constructor: new TailLog(filename[, options])](#constructor-new-tailfilefilename-options)
* tail.start()
* tail.quit()
* Program Flow
* How Log Rolling is Handled
* Backpressure Pauses Polling
* Log Rolling During Backpressure
* Contributing

Features

* Zero dependencies! It's lightweight and uses 100% Node.js core modules.
* It implements a [Readable][] stream, which is efficient
and flexible in terms of being able to pipe to other streams or consume via events.
* Stream backpressure is properly respected, so at no time is data pushed through the
stream unless it is requested.
* It handles log rolling. Renaming files is handled gracefully without losing lines
written to the "old" file, no matter what the poll interval is.
* It handles file truncation, continuing to tail the file despite being truncated.

Installation

``sh
npm install @logdna/tail-file
`

Supported Versions of Node.js


This package works with the following supported versions. Other versions might work, but the ones listed here are officially tested.
If backward-compatibility is lost over time, a major version will be released supporting
the latest version, and deprecating incompatible versions. Users are then encouraged to try
previous versions of this package for those deprecated versions of Node.js.

Node.js versions: 18, 20

Usage

Instantiate an instance by passing the full path of a file to tail.
This will return a stream that can be piped to other streams or consumed
via [data events][data]. To begin the tailing, call the start method.

$3

`js
const TailFile = require('@logdna/tail-file')

const tail = new TailFile('/path/to/your/logfile.txt', {encoding: 'utf8'})
.on('data', (chunk) => {
console.log(Recieved a utf8 character chunk: ${chunk})
})
.on('tail_error', (err) => {
console.error('TailFile had an error!', err)
})
.on('error', (err) => {
console.error('A TailFile stream error was likely encountered', err)
})
.start()
.catch((err) => {
console.error('Cannot start. Does the file exist?', err)
})
`

$3

This example is more realistic. It pipes the output to a transform stream
which breaks the data up by newlines, emitting its own data event for
every line.

`js
const TailFile = require('@logdna/tail-file')
const split2 = require('split2') // A common and efficient line splitter

const tail = new TailFile('/path/to/your/logfile.txt')
tail
.on('tail_error', (err) => {
console.error('TailFile had an error!', err)
throw err
})
.start()
.catch((err) => {
console.error('Cannot start. Does the file exist?', err)
throw err
})

// Data won't start flowing until piping

tail
.pipe(split2())
.on('data', (line) => {
console.log(line)
})
`

$3

This is an easy way to get a "line splitter" by using Node.js core modules.
For tailing files with high throughput, an official Transform stream is
recommended since it will edge out readline slightly in performance.

`js
const readline = require('readline')
const TailFile = require('@logdna/tail-file')

async function startTail() {
const tail = new TailFile('./somelog.txt')
.on('tail_error', (err) => {
console.error('TailFile had an error!', err)
})

try {
await tail.start()
const linesplitter = readline.createInterface({
input: tail
})

linesplitter.on('line', (line) => {
console.log(line)
})
} catch (err) {
console.error('Cannot start. Does the file exist?', err)
}
}

startTail().catch((err) => {
process.nextTick(() => {
throw err
})
})
`

$3

TailFile will call flush() when quit() is called. Therefore, to exit cleanly,
one must simply await the quit call. If the implementation wishes to keep track of
the last position read from the file (for resuming in the same spot later, for example),
then a simple listener can be added to always track the file position. That way, when
quit() is called, it will get properly updated.

`js
const TailFile = require('@logdna/tail-file')
let position // Can be used to resume the last position from a new instance

const tail = new TailFile('./somelog.txt')

process.on('SIGINT', () => {
tail.quit()
.then(() => {
console.log(The last read file position was: ${position})
})
.catch((err) => {
process.nextTick(() => {
console.error('Error during TailFile shutdown', err)
})
})
})

tail
.on('flush', ({lastReadPosition}) => {
position = lastReadPosition
})
.on('data', (chunk) => {
console.log(chunk.toString())
})
.start()
.catch((err) => {
console.error('Cannot start. Does the file exist?', err)
throw err
})
`

Events

TailFile is a [Readable stream][Readable], so it can emit any events from that
superclass. Additionally, it will emit the following custom events.

$3

* [