ctxsplit

Split large contexts intelligently. Semantic chunking. Overlap control.

Install

``bash
npm install -g ctxsplit
`

Usage

`bash


Split file into chunks

ctxsplit large-doc.txt

Specify chunk size (tokens)

ctxsplit doc.txt --size 4000
ctxsplit doc.txt -s 4000

Set overlap between chunks

ctxsplit doc.txt --overlap 200
ctxsplit doc.txt -o 200

Split by sentences (semantic)

ctxsplit doc.txt --semantic

Split by paragraphs

ctxsplit doc.txt --by paragraph

Split by headers (markdown)

ctxsplit doc.md --by header

Output to files

ctxsplit doc.txt --output chunks/

Output as JSON

ctxsplit doc.txt --json

Show chunk info only

ctxsplit doc.txt --info

Read from stdin

cat doc.txt | ctxsplit -s 2000

Help

ctxsplit --help
`

Strategies

- size (default): Split by token count with overlap
- sentence: Split at sentence boundaries
- paragraph: Split at paragraph boundaries
- header: Split markdown by headers

Output

`
$ ctxsplit doc.txt --info
ctxsplit - pnkd.dev
───────────────────
Input: doc.txt
Size: 45,678 tokens
Strategy: size (4000 tokens)
Overlap: 200 tokens

Chunks: 12
Range: 3,800 - 4,000 tokens
Average: 3,806 tokens

$ ctxsplit doc.txt --output chunks/
✓ Created chunks/chunk-01.txt (3,923 tokens)
✓ Created chunks/chunk-02.txt (4,001 tokens)
✓ Created chunks/chunk-03.txt (3,856 tokens)

3 chunks written to chunks/
``

License

MIT

---

pnkd.dev - glitch the system