Accept and automatically parse NDJSON stream requests in NestJS with Express!

Description

A lightweight library that enables NestJS applications to accept and process streaming NDJSON (Newline Delimited JSON) requests with Express. Perfect for handling large datasets, real-time data feeds, and streaming APIs where each line contains a valid JSON object.

Features

- ✨ Simple decorator-based API for handling NDJSON streams
- 🚀 Memory-efficient processing using AsyncGenerators
- 🎯 TypeScript support with generic types
- 🔧 Configurable batch processing
- 🛡️ Automatic content-type validation
- ⚡ Zero dependencies (only NestJS peer dependencies)

Requirements

- Node.js >= 20.0.0
- NestJS >= 10.0.0
- Express framework

Installation

``bash
npm install nest-ndjson-req-stream
`

Quick Start

$3

`typescript
import { Controller, Post } from '@nestjs/common';
import { NdJsonStreamReq, NdJsonStreamRequest } from 'nest-ndjson-req-stream';

interface DataItem {
id: number;
name: string;
value: number;
}

@Controller('stream')
export class StreamController {
@Post('process')
async processStream(
@NdJsonStreamReq() request: NdJsonStreamRequest
) {
const results: DataItem[] = [];

// Process each item from the stream
for await (const item of request.body) {
// Process your data here
console.log('Received:', item);
results.push(item);
}

return {
message: 'Stream processed successfully',
itemCount: results.length,
};
}
}
`

$3

Use TypeScript generics for type-safe stream processing:

`typescript
interface User {
id: string;
email: string;
profile: {
name: string;
age: number;
};
}

@Post('users')
async importUsers(
@NdJsonStreamReq() request: NdJsonStreamRequest
) {
for await (const user of request.body) {
// TypeScript knows that user is of type User
console.log(user.email); // ✅ Type safe
console.log(user.profile.name); // ✅ Type safe
}
}
`

API Reference

$3

Parameter decorator for handling NDJSON streaming requests.

#### Options:

- batchSize?: number - The batch size for processing streamed objects (default: 25)

$3

Extended Express Request interface with AsyncGenerator type body:

- body: AsyncGenerator - AsyncGenerator that yields parsed NDJSON objects
- batchSize: number - The configured batch size for processing

Testing

`bash


Run unit tests

npm run test

Run tests with coverage

npm run test:cov
`

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

1. Fork the repository
2. Create your feature branch (git checkout -b feature/AmazingFeature)
3. Commit your changes (git commit -m 'Add some AmazingFeature')
4. Push to the branch (git push origin feature/AmazingFeature`)
5. Open a Pull Request

License

This project is MIT licensed.