Unbind llms-full.txt into individual pages.
npm install llms-full-unbind> Unbind llms-full.txt into individual pages programmatically.
A specialized parser designed to extract pages from the monolithic llms-full.txt format.
![NPM license][npm-package]
![NPM version][npm-package]
This library is primarily intended for use in the llms-full-unbind-mcp package, but can be used in other projects as well.
``bash`
npm install llms-full-unbind
Fetch the text content and unbind it in one go.
`typescript
import { unbind } from "llms-full-unbind";
// 1. Fetch the remote llms-full.txt
const response = await fetch("https://example.com/llms-full.txt");
const text = await response.text();
// 2. Unbind into pages
const pages = Array.from(unbind(text));
console.log(Extracted ${pages.length} pages.);`
For large llms-full.txt files, use unbindStream to process data chunk-by-chunk directly from the network response. This minimizes memory usage.
`typescript
import { unbindStream } from "llms-full-unbind";
const response = await fetch("https://example.com/llms-full.txt");
if (!response.body) {
throw new Error("Response body is empty");
}
// Pipe the Web Stream directly into the parser
for await (const page of unbindStream(response.body)) {
console.log(Processed: ${page.title});`
// e.g. Save to DB or display immediately
}
This library automatically detects and parses five common llms-full.txt formats:
This format wraps each page in the tag with optional metadata attributes.llms_txt2ctx
Generated by the CLI from the llms-txt package.
Used by fastht.ml and similar projects.
`markdown`
Content of the page...
This format wraps each page in the tag.
Used by cloudflare.com project.
`markdown`
Content of the page...
Pages are separated by frontmatter-style metadata blocks.
Generated by vitepress-plugin-llms.
Used by vuejs.org, vitejs.dev, vitepress.dev and similar VitePress-based projects.
`markdownPage Title {#optional-anchor}
Content of the page...
---
url: /optional/metadata.md
---
More content...
`
Each page starts with a markdown header followed by a Source: line indicating the page URL.
Generated by Mintlify.
Used by modelcontextprotocol.io and bun.sh.
`markdownPage Title
Source: https://example.com/path/to/page
Content of the page...
More content...
`
Pages are separated by H1 headers (# Title).
Used by projects like svelte.dev, nuxt.com, and docs.astro.build.
`markdownPage Title
Content of the page...
More content...
`
Parses the entire string synchronously and returns an iterable (Generator) of pages. Use Array.from(unbind(content)) to get an array.
Accepts a standard Web ReadableStream (returned by fetch) or any Async Iterable. Yields pages as soon as they are parsed.
#### Type Definition: Page
`typescript``
export interface Page {
title: string;
content: string; // The extracted text content
metadata?: Record
}
MIT
[npm-package]: https://www.npmjs.com/package/llms-full-unbind