Kafka adapter for message-queue-toolkit
npm install @message-queue-toolkit/kafka> _NOTE:_ Check README.md for transport-agnostic library documentation.
Use AbstractKafkaPublisher as a base class for publisher implementation.
See test publisher for an example of implementation.
Use AbstractKafkaConsumer as a base class for consumer implementation.
See test consumer for an example of implementation.
Kafka supports batch processing for improved throughput. To enable it, set batchProcessingEnabled to true and configure batchProcessingOptions.
When batch processing is enabled, message handlers receive an array of messages instead of a single message.
- batchSize - Maximum number of messages per batch
- timeoutMilliseconds - Maximum time to wait for a batch to fill before processing
Messages are buffered per topic-partition combination. Batches are processed when either:
- The buffer reaches the configured batchSize
- The timeoutMilliseconds timeout is reached since the first message was added
After successful batch processing, the offset of the last message in the batch is committed.
See test batch consumer for an example of implementation.