Fluentd Logger

!GitHub Workflow Status
!GitHub package.json version
!npm bundle size
!NPM
!GitHub package.json dynamic
!node-current

A Logger wrapper that allows to send your application logs to an instance of Fluentd.

The library supports NestJS context, log severity and hostname collection (for distributed systems).

Installation

Install @dynatech-corp/nestjs-fluentd-logger package

``shell


npm installation

npm install @dynatech-corp/nestjs-fluentd-logger

yarn installation

yarn add @dynatech-corp/nestjs-fluentd-logger
`

Usage

This section covers multiple use-cases for this logger.

There are two providers that implement functionality.

Provider FluentConnection handles communication with Fluentd instance.
FluentConnection provider accepts the same connection parameters as
the underlying @fluent-org/logger library.
The connection is lazy-loaded, meaning it is initiated when the first logs appear, not on module initialization.

Provider FluentLogger is an interface that allows to log different severity and context messages to FluentConnection.
FluentLogger is created with Transient scope, which ensures independent logger instance for each injection.

Logging to Fluentd

The basic use case is logging information directly to Fluentd.

$3

`typescript
import { Module } from '@nestjs/common';
import {
FluentLogger,
FluentConnection,
} from '@dynatech-corp/nestjs-fluentd-logger';

@Module({
providers: [
FluentConnection,
FluentLogger,
],
})
export class AppModule {}
`

$3

Option bufferLogs is used to buffer logs that are generated before logger is initialized.

Logger is initialized with await app.resolve, because it had a dependency on another service FluentConnection, that is initialized with Dependency Injection.

Calling app.flushLogs() right after we initialize logger passes the logs to the logger right away. This helps to prevent cases when there's an initialization error and no logs are displayed.

`typescript
import { NestFactory } from '@nestjs/core';
import { AppModule } from './app.module';
import { FluentLogger } from '@dynatech-corp/nestjs-fluentd-logger';

async function bootstrap() {
const app = await NestFactory.create(AppModule, {
// buffer logs untill logger is initialized
bufferLogs: true,
});

// use FluentLogger as the main logger
app.useLogger(await app.resolve(FluentLogger));
// flush logs after we setup the logger
app.flushLogs();

await app.listen(3000);
}
bootstrap();
`

$3

Using Logger proxy-class seems to currently be the cleanest approach to initializing Logger with proper context.

Internally it uses the logger we provided during bootstrap with app.useLogger.

`typescript
import { Controller, Get, Logger } from '@nestjs/common';

@Controller()
export class AppController {
// create logger instance with proper context
private readonly logger = new Logger(AppController.name);

@Get()
getHello(): string {
// log your message
this.logger.log('Log "Hello, world!" message');
// ... do stuff
return 'Hello, world!';
}
}
`

Customize Fluentd connection

There are cases when you want to log data to a custom destination,
e.g. Fluentd that's located at another IP address, listens to a non-default port, etc.

Custom configurations can be passed to FluentdConnection provider's constructor.
The parameters use an interface from @fluent-org/logger.

`typescript
import { Module } from '@nestjs/common';
import {
FluentLogger,
FluentConnection,
} from '@dynatech-corp/nestjs-fluentd-logger';

@Module({
providers: [
// fluentd connection service
{
provide: FluentConnection,
useFactory: () => {
return new FluentConnection({
prefix: 'logs.my_app',
connection: {
socket: {
host: '10.0.2.12',
port: 20022,
timeout: 10000,
},
},
});
},
},
// fluentd logger
FluentLogger,
],
})
export class AppModule {}
`

Customize connection with Environment variables

For cases when you have different environments (dev/stage/prod)
and want to configure your log delivery accordingly,
you can use @nestjs/config library to provide dynamic configurations to the connection.

$3

Get configurations module into your project

`shell
npm install @nestjs/config
`

$3

By injecting ConfigService into provide factory, you can pass custom configurations from your .env file.
Note that you can additionally configure ConfigService to fetch configurations from different sources (like json or yaml files, etc.)

`typescript
import { Module } from '@nestjs/common';
import { ConfigModule } from '@nestjs/config';
import {
FluentLogger,
FluentConnection,
} from '@dynatech-corp/nestjs-fluentd-logger';

@Module({
imports: [
// global config module
ConfigModule.forRoot({ isGlobal: true }),
],
providers: [
// fluentd connection service
{
provide: FluentConnection,
useFactory: (config: ConfigService) => {
return new FluentConnection({
prefix: config.get('LOGS_PROJECT'),
connection: {
socket: {
host: config.get('LOGS_HOST'),
port: config.get('LOGS_PORT'),
timeout: config.get('LOGS_TIMEOUT'),
},
},
});
},
inject: [ConfigService],
},
// fluentd logger
FluentLogger,
],
})
export class AppModule {}
`

$3

Example .env file for FluentConnection provider configuration.

`dotenv
LOGS_PROJECT=logs
LOGS_HOST=127.0.0.1
LOGS_PORT=24224
LOGS_TIMEOUT=10000
`

Configure logs destination with environment variables

Despite the fact that centralized logging with fluentd is crucial for production systems,
it's not convenient for development.

You would want to have logs displayed in console output during development, but have them in your
centralized log analysis system in staging/production.

Luckily, this can be done with provider useFactory and @nestjs/config module.

$3

Create factory, that determines what logger to use based on LOGS_OUTPUT env variable value.

Note that internally FluentConnection lazy-loads the connection.
So in case when we use ConsoleLogger as our main logger,
the connection isn't tried and there are no errors.

`typescript
import { ConsoleLogger, Logger, Module } from '@nestjs/common';
import {
FluentLogger,
FluentConnection,
} from '@dynatech-corp/nestjs-fluentd-logger';
import { ConfigModule, ConfigService } from '@nestjs/config';

@Module({
imports: [ConfigModule.forRoot({ isGlobal: true })],
providers: [
{
provide: FluentConnection,
useFactory: (config: ConfigService) => {
return new FluentConnection({
prefix: config.get('LOGS_PROJECT'),
connection: {
socket: {
host: config.get('LOGS_HOST'),
port: config.get('LOGS_PORT'),
timeout: config.get('LOGS_TIMEOUT'),
},
},
});
},
inject: [ConfigService],
},
{
provide: Logger,
useFactory: (config: ConfigService, fluent: FluentConnection) => {
// get LOGS_OUTPUT variable value
const output = config.get('LOGS_OUTPUT');
// create NestJS ConsoleLogger for development (console)
if (output === 'console') {
return new ConsoleLogger(undefined, { timestamp: true });
}
// create FluentLogger instance for staging / production
if (output === 'fluent') {
return new FluentLogger(fluent);
}
// throw error when the variable is not Configured
throw new Error('LOGS_OUTPUT should be console|fluent');
},
// inject ConfigService - for configuration values
// inject FluentConnection - for when FluentLogger is instantiated
inject: [ConfigService, FluentConnection],
},
],
})
export class AppModule {}
`

$3

Because our factory provides generic NestJS Logger in provider factory provide: Logger,
we need to change our bootstrap signature to use that logger.

Now we use app.useLogger(await app.resolve(Logger)) NestJS generic Logger,
which is dynamically provided with Dependency Injection.

`typescript
import { NestFactory } from '@nestjs/core';
import { AppModule } from './app.module';
import { Logger } from '@nestjs/common';

async function bootstrap() {
const app = await NestFactory.create(AppModule, {
bufferLogs: true,
});

// we use generic nestjs Logger
// that would get resolved to what we configured
app.useLogger(await app.resolve(Logger));
app.flushLogs();

await app.listen(3000);
}
bootstrap();
`

$3

Defing which logger you want to use in this environment with LOGS_OUTPUT variable.

`dotenv


would configure logger to write to flunet

LOGS_OUTPUT=fluent

would configure logger to write to console output

#LOGS_OUTPUT=console
`

Development

The approach for development was taken from Publishing NestJS Packages with npm
by NestJS.


Install dependencies
`shell
npm install
`

Start watching for library files
`shell
npm run start:dev
`

Start docker composer
`shell
docker compose up
`

Install dependencies for test-app
`shell
cd test-app
`

Install test-app dependencies
`shell
npm install
`

Start test-app
`shell
npm run start:dev
``

Stay in touch

* Author Bogdans Ozerkins
* Website https://dynatech.lv