Core chunking library for Chonkie - lightweight and efficient text chunking
npm install @chonkiejs/core_Core chunking library for Chonkie - lightweight and efficient text chunking with zero dependencies._
Install with npm:
``bash`
npm i @chonkiejs/core
Install with pnpm:`bash`
pnpm add @chonkiejs/core
Install with yarn:`bash`
yarn add @chonkiejs/core
Install with bun:`bash`
bun add @chonkiejs/core
`typescript
import { RecursiveChunker } from '@chonkiejs/core';
// Create a chunker
const chunker = await RecursiveChunker.create({
chunkSize: 512,
minCharactersPerChunk: 24
});
// Chunk your text
const chunks = await chunker.chunk('Your text here...');
// Use the chunks
for (const chunk of chunks) {
console.log(chunk.text);
console.log(Tokens: ${chunk.tokenCount});`
}
| Name | Description |
|------|-------------|
| RecursiveChunker | Recursively splits text using hierarchical rules (paragraphs ā sentences ā punctuation ā words ā characters). Each level only activates if chunks exceed the configured size. |TokenChunker
| | Splits text into fixed-size token chunks with optional overlap. Uses character-based tokenization by default, or HuggingFace models with @chonkiejs/token. |
For detailed API documentation, configuration options, and advanced usage, see DOCS.md.
Want to help grow Chonkie? Check out CONTRIBUTING.md to get started! Whether you're fixing bugs, adding features, improving docs, or simply leaving a āļø on the repo, every contribution helps make Chonkie a better CHONK for everyone.
Remember: No contribution is too small for this tiny hippo!
Chonkie would like to CHONK its way through a special thanks to all the users and contributors who have helped make this library what it is today! Your feedback, issue reports, and improvements have helped make Chonkie the CHONKIEST it can be.
And of course, special thanks to Moto Moto for endorsing Chonkie with his famous quote:
> "I like them big, I like them chonkie in TypeScript" ~ Moto Moto... definitely did not say this
If you use Chonkie in your research, please cite it as follows:
`bibtex``
@software{chonkie2025,
author = {Bhavnick Minhas and Shreyash Nigam},
title = {Chonkie: A no-nonsense fast, lightweight, and efficient text chunking library},
year = {2025},
publisher = {GitHub},
howpublished = {\url{https://github.com/chonkie-inc}},
}