dawntech/tools

Node package for Dawntech services.

Installation

``sh
npm install @dawntech/tools
`

Included modules

- Logger
- Cron
- REST
- Sheet Database
- Kafka

Support

For support inquiries, don't hesitate to get in touch with us at infra+npm@dawntech.dev.

Usage

$3

Provides a standardized logging utility with support for different formats and log levels.

#### Quick Start

The recommended way to use the Logger is to import the root instance directly:

`javascript
import { logger, setLoggerDefaults, getLogger } from '@dawntech/tools'

// Configure the root logger (usually at application startup)
setLoggerDefaults({
format: process.env.NODE_ENV === 'production' ? 'json' : 'simplified',
level: process.env.LOG_LEVEL || 'info',
extras: { service: 'my-app' },
})

// Use the root logger directly
logger.info('Application started')
logger.error('Something went wrong', { errorCode: 500 })

// Create child loggers for specific contexts
const dbLogger = getLogger('database')
dbLogger.info('Connected to database')

const apiLogger = getLogger('api', { extras: { version: 'v1' } })
apiLogger.http('Request received')
`

#### Creating a New Instance

You can also instantiate a new Logger from the class:

`javascript
import { Logger, Types } from '@dawntech/tools'

const logger = new Logger({
format: 'json',
level: process.env.LOG_LEVEL as Types.Logger.LogLevel,
extras: { anyField: 'any value' },
context: 'my-service',
disabledContexts: ['context/*/c', 'context'], // Uses glob patterns
})
`

#### Logger API

##### Configuration Options

| Option | Type | Default | Description |
| ------------------ | ---------------------------------------------------------------------- | -------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| format | 'json' \| 'simplified' \| 'simplified-with-extras' | 'simplified' | Output format. json prints the whole content as a normalized json, simplified prints only the message and simplified-with-extras prints the message with extras. |
| level | 'error' \| 'warn' \| 'info' \| 'http' \| 'debug' \| 'none' | 'info' | Minimum log level. Set to 'none' to disable logging. |
| extras | object | {} | Additional fields included in every log entry. |
| context | string | '' | Identifies the source of logs (e.g., 'api', 'database'). |
| disabledContexts | string[] | [] | Glob patterns to disable logging for specific contexts. |

##### Logging Methods

All logging methods accept a message and an optional extras object:

`javascript
logger.debug(message, extras?) // Detailed debugging information
logger.info(message, extras?) // General application messages
logger.warn(message, extras?) // Unexpected situations that don't stop execution
logger.error(message, extras?) // Serious issues or failed operations
logger.http(message, extras?) // HTTP request/response logging
`

Examples:

`javascript
logger.info('User logged in', { userId: '123' })
logger.error(new Error('Connection failed'))
logger.debug({ query: 'SELECT * FROM users', duration: 45 })
`

##### logger.getLogger(context?, params?)

Creates a child logger inheriting properties from the parent. Child values override parent values.

`javascript
const dbLogger = logger.getLogger('db')
const apiLogger = logger.getLogger('api', { extras: { version: 'v1' } })
const verboseLogger = logger.getLogger({ level: 'debug' })
`

##### logger.updateConfigs(params)

Updates the logger configuration. Child loggers are not affected.

`javascript
logger.updateConfigs({ level: 'debug', format: 'json' })
`

##### setLoggerDefaults(params)

Configures the root logger instance. Equivalent to calling logger.updateConfigs().

`javascript
setLoggerDefaults({
format: 'json',
level: 'info',
extras: { environment: 'production' },
})
`

##### getLogger(context?, params?)

Creates a child logger from the root instance. Equivalent to calling logger.getLogger().

`javascript
const serviceLogger = getLogger('payments', { extras: { currency: 'USD' } })
`

$3

A helper class meant to define cron jobs, handle errors and prevent duplicated executions.

`javascript
import { Cron, Logger, Exceptions } from '@dawntech/tools'

const logger = new Logger()
const cron = new Cron({ logger, timezone: 'America/Los_Angeles' })

const job = cron.defineJob({
pattern: '0 12 *',
job: () => MessageService.sendMessages,
name: 'send customer messages',
description: 'Send message to customers about promotions once a day',
options: {
duplicatedExecution: false
customValidation: async () => {
if (await isAHolidayApi()) {
throw new Exceptions.Cron.CustomRuleExecutionError('Today is holiday')
}
}
cronOptions: { timeZone: 'America/Los_Angeles'}
}
})

cron.startAll()
// or job.start()
`

#### Cron API

##### const cron = new Cron({ logger, timezone: 'America/Los_Angeles' })

Creates a new instance of the helper.

- params An optional object containing options
- logger An Logger instance. Will not log anything if ommited.
- timezone Overrides the timezone used by the service.

##### cron.jobs

The list of all registered jobs.

##### cron.startAll()

Starts all pending jobs. Ignores already running jobs.

##### cron.findJob(jobName)

Returns a Job if it finds one matching the name. Retuns null otherwise.

##### cron.getAllActiveJobs()

Returns a list of all active jobs.

##### cron.getAllRunningJobs()

Returns a list of all jobs that are running right now.

##### const job = cron.defineJob({ pattern, job, name, options: { description, duplicatedExecution, customValidation } })

Defines a new job. Defined jobs don't run unless calling cron.startAll or job.start. Returns the defined job.

- params
- pattern: A cron pattern in the format "\ \ \ \ \*".
- job(job): A method that will be called everytime the cron triggers. Receives a callback function in the params.
- name: A unique name to identify the job.
- description: A description of what the job does. Used for documentation.
- options: An object containing advanced options:
- duplicatedExecution: If true, a new job will start even if the last job of the same name still running. Defaults to false.
- validateCustomRule(job): A function that indicates if the job should run. Ideal for validating holidays. Must throw an error of type Exceptions.Cron CustomRuleExecutionError(reason) to indicate that the cron should not run. Receives the job as a parameter.
- job The job instance:
- start(): Start running the job. Ignored if is already running.
- stop(): Stop running scheduled jobs. Ignored if is already stopped. Job continues available on cron list and can be restarted.
- cancel(): Stops and delete the job. Cannot be called twice.
- specs: The specs of the job. Cannot be updated.
- cron: The internal cron instance. Can be called directly. Check cron for more info.
- logger: The logger instance.
- runningProcessesCount: The amount of processes running by this job.

$3

Creates an API with support for validation schemas and documentation generation.

#### REST API

##### const rest = new REST({ port, logger, serveDocs, docs, servers, maxBodySize })

Creates a new server instance. Call rest.start() afterwards to start server.

- port: The port where the server will be hosted
- logger: A Logger instance. Will be used for internal logging.
- serveDocs: If the docs should be available at /docs route.
- docs: An object containing info that will be used to generate the OpenApi specs:
- title: The API name
- version: The version in semver format
- description: An optional description to add to the specs
- theme: The theme to use for documentation. Defaults to Swagger UI. Options: swagger, scalar.
- publicConfiguration: If the OpenAPI configuration JSON should be available publically. If true, the configuration will be accessible at customConfiguration.url or /openapi.json route.
- customConfiguration: A custom object for customizing documentation, based on used themes. Check theme documentation for options.
- servers: An array containing objects specifing available servers. Those servers will be available in the OpenApi specs and in the Swagger page. Must be an array of objects in the following format:
- url: The url of the server
- description: A short description of the server
- maxBodySize: Controls the maximum request body size. If this is a number, then the value specifies the number of bytes; if it is a string, the value is passed to the bytes library for parsing. Defaults to '100kb'.

##### rest.defineRoute({ path, method, schema, contentType, summary, responses, middlewares, controller })

Register a new route in the server

- path: A pattern used to register routes. Only the first match will be called, so order of declaration of routes matter.
- method: A string or list of strings indicating to which method this route should respond.
- schema: A Zod schema. Will be used for validating input data and to generate docs. The createHttpSchema helper function can be called to help building the schema.
- contentType: The content type that the route will receive. Used only in documentation. Default is application/json.
- summary: A description of the route function. Will be used to generate docs.
- middlewares: An array containing a list of middlewares. Check the middleware section for more info.
- responses: An object containing the schema and example values that will be returned from the route. The schema is required and needs to be specified in Zod format. The schemas for the status 400 and 422 is already defined, but they can be overwriten.
- controller: The callback function that will be called when a request matches the path. The callback receives two parameters: content, which contains the request data after schema validation and parsing, and extras, an object in the format { request, response } (check Express docs for more info about those values). The return value of this function will be used as the response body. If nothing is returned, the status code will be set to 204. You may also respond directly using the response object from extras.

##### rest.start({ skipHosting })

Starts the server. Must be called last.

- skipHosting: If true, will not create a server instance for the REST server. Useful when testing the application.

##### rest.stop()

Stops the server. It's possible to call rest.start() again after the server was stopped.

##### const router = rest.getRouter(path, { middlewares })

Creates a new router. Routes created in a router will be nested together in the URL path.
You can call defineRoute or getRouter from a router instance, being possible to recursively nest routes.
Example:

`js
// Register the route /products
rest.defineRoute({ path: '/products', method: 'GET', controller: () => {} })

const cartRouter = rest.getRouter('/cart')
// Register the route /cart/products
cartRouter.defineRoute({ path: '/products', method: 'GET', controller: () => {} })

const productsRouter = cartRouter.getRouter('/products')
// Register the route /cart/products/last
productsRouter.defineRoute({ path: '/last', method: 'GET', controller: () => {} })
`

##### rest.getOpenApiSpecs()

Generates and return the whole OpenApi doc.

##### rest.getApp()

Returns the express app instance. Use this as a last resort method, since using this can break future implementations.

##### Middlewares

You can define a middleware in the same way an express middleware can be defined. Defining a middleware with 4 parameters will define it as an error handler.

`js
import { Exceptions, Types } from '@dawntech/tools'
function authMiddleware(req: Types.REST.Request, _res: Types.REST.Response, next: Types.REST.NextFunction) {
const requestToken = req.headers['x-api-key']
if (typeof requestToken === 'string') {
if (requestToken.split(' ').at(-1) === process.env.AUTH_TOKEN) {
return next()
}
}
throw new Exceptions.REST.UnauthorizedException('Invalid API key provided')
}
`

###### createMiddleware

createMiddleware is a utility function that helps you define and document middleware. It allows you to specify details such as authentication requirements and possible response bodies, which are automatically reflected in Swagger documentation.

Using this function is optional, but it provides stronger type safety and better integration with your API documentation.

`js
import { Utils } from '@dawntech/utils'

const authMiddleware = Utils.REST.createMiddleware(
(req: Request, _res: Response, next: NextFunction) => {
if (req.headers.auth) next()
throw new UnauthorizedException('Auth failed.')
},
{
docs: {
authMethods: [
{
type: 'apiKey',
in: 'header',
name: 'X-API-Key',
},
],
responses: {
401: {
schema: z.object({
message: z.string(),
}),
example: {
message: 'Failed to authenticate',
},
},
},
},
},
)

// authMiddleware now can be used as a normal middleware.

`

#### Error handling

By default, any error thrown during route execution results in a 500 Internal Server Error response.

You can customize the response status code and body by throwing a class that extends HttpException and overriding the toJson method:

`js
import { Exceptions } from '@dawntech/tools'

class MyCustomError extends Exceptions.REST.HttpException {
status = 418

toJson() {
return {
error: 'Invalid type',
displayError: 'O tipo informado é inválido.',
errorCode: 'ERR-223',
}
}
}
`

The Exceptions module also provides pre-defined classes for common HTTP error codes:

`js
import { Exceptions } from '@dawntech/tools'

/* This line will generate the following response:
Status: 400
Body: { message: 'Invalid value' }
*/
throw new Exceptions.REST.BadRequestException('Invalid value')
`

#### Examples

`javascript
import z from 'zod'
import { REST, Logger, Utils, Exceptions, Types } from '@dawntech/tools'

const logger = new Logger({ isProd: false })

const rest = new REST({
logger,
serveDocs: true,
servers: [{ url: 'https://example.com', description: 'The prod server' }],
port: 3000,
docs: {
title: 'Example API',
version: '1.0.0',
description: 'An example API',
},
})

const authMiddleware = Utils.REST.createMiddleware(
(req: Request, _res: Response, next: NextFunction) => {
if (req.headers.auth) next()
throw new UnauthorizedException('Auth failed.')
},
{
docs: {
authMethods: [
{
type: 'apiKey',
in: 'header',
name: 'X-API-Key',
},
],
responses: {
401: {
schema: z.object({
message: z.string(),
}),
example: {
message: 'Failed to authenticate',
},
},
},
},
},
)

rest.defineRoute({
path: '/test',
method: 'POST',
schema: Utils.REST.createHttpSchema({
body: z.object({
field: z.string().meta({
description: 'Additional info can be added to docs params, check zod-openapi package for more info',
}),
}),
headers: z.object({
'x-custom-header': z.number(),
}),
params: z.object({
id: z.string(),
}),
query: z.object({
filter: z.string().optional(),
}),
}),
controller: (content) => {
return { field: content.body.field }
},
middlewares: [authMiddleware],
contentType: 'application/json',
summary: 'A description explaning what the route does.',
responses: {
'200': {
description: 'The success response.',
schema: z.object({
field: z.string(),
}),
example: {
field: '12',
},
},
},
})

const userRouter = rest.getRouter('/user')
userRouter.defineRoute({
path: '/last',
method: 'GET',
controller: () => {
return { userId: '8479238374' }
},
})

const appUserRouter = userRouter.getRouter('/app', { middlewares: [authMiddleware] })
appUserRouter.defineRoute({
path: '/:id',
method: 'GET',
controller: () => {
return { userId: '8479238374' }
},
})

// Must be called after all routes were defined
rest.start()

`

$3

Utility to handle a Google Sheet as a database. Provides functions such as find or update.

All changes are made in memory and are only applied in the sheet after calling apply for performance.

The apply function does not check for changes made between the data fetching and the update, so be carefull when someone else is updating the sheet between operations.

As a note, every empty value will be parsed as null. Other values depends of the value of valueRenderOption parameter.

#### How to use

`javascript
import { SheetDatabase, Types } from '@dawntech/tools'

interface Data extends Types.SheetDatabase.Row {
col1: string
col2: number
col3: string
}

const sheet = await SheetDatabase.getInstance({
credentials,
sheetPage: 'the-sheet-page-name',
sheetId: 'the-id-of-the-sheet',
valueRenderOption: 'UNFORMATTED_VALUE', // Default value
})

sheet.create({ col1: 'test3', col2: 20, col3: 'value' })
sheet.update({ $and: [{ col1: 'test3' }, { col2: 20 }] }, { col2: 90 })
sheet.delete({ col1: 'test3' })
sheet.find({ col1: /test2/ })
sheet.find({ col1: { $ne: 'test2' } })
sheet.find({ col1: { $in: ['test2'] } })
sheet.find({ col1: { $nin: ['test2'] } })

// Changes are only applied to the sheet after calling apply
await sheet.apply()
`

##### const sheet = await SheetDatabase.getInstance({credentials, sheetPage, sheetId})

Creates an instance of sheet database. Will attempt to connect and fetch all the rows.

- credentials: Credentials is an Google credentials Object. You can get these at
- sheetPage: The name of the page inside the sheed
- sheetId: The sheet ID. You can get this value in the sheet URL.
- valueRenderOption: How values will be parsed
- UNFORMATTED_VALUE (default): Values will be calculated, but not formatted.
- FORMATTED_VALUE: Values will be returned as they are presented on sheets. Numbers will be returned as strings, for example.
- FORMULA: Formulas will not be calculated.

##### sheet.create({data})

Inserts a new entry at the end of the file. data is a plain object.

##### sheet.get({query})

Return a list of matches. Check query section.

##### sheet.update({query, data})

Updates entries in the sheet. data is a plain object. Check query section.

##### sheet.delete({query})

Delete entries. Check query section.

##### sheet.apply()

Apply all changes to the database.

##### sheet.refresh()

Will fetch the sheet data again and reset every pending change.

##### Queries

The following operations are supported by queries:

- By value: { field: 'value' }. Will return exactly matches. The comparations is made using == so will match 1 and "1"
- By regex: { field: /^value/m }: Will test values using regex. Numerical values will be converted to string during the test.
- Boolean: { $or: [ { field: 'value' }, { field: 'something'} ]}/{ $and: [ { field: 'value' }, { field: 'something'} ]}. Applies boolean logic when testing elements. Can be used with the other query types and even recursivelly.

$3

A wrapper around KafkaJS with support for Avro schema registry, JSON, and plain text payloads.

`javascript
import { Kafka, Logger, Types } from '@dawntech/tools'

const logger = new Logger({ isProd: true })

const kafka = new Kafka(
{
kafkaHost: 'localhost:9092',
kafkaApplicationId: 'my-app',
kafkaSaslUsername: 'user',
kafkaSaslPassword: 'password',
kafkaGroupId: 'my-group',
schemaRegistryUrl: 'http://localhost:8081',
schemaRegistryUsername: 'registry-user',
schemaRegistryPassword: 'registry-password',
},
logger,
)

// Subscribe to a topic
await kafka.subscribe(
'my-topic',
async (message) => {
console.log('Received message:', message)
},
{ payload: 'AVRO' },
)

// Connect and start consuming
await kafka.connect()

// Send a message
await kafka.send(
'my-topic',
{ field: 'value' },
{
subject: 'my-topic-value',
payload: 'AVRO',
},
)

// Disconnect
await kafka.disconnect()
`

Development

- npm run build: Build the project.
- npm run lint: Lint the project.
- npm publish`: Publish the package to NPM.