Showing 1-20 of 259,062 packages
Javascript text tokenizer that is easy to use and compose
Javascript text tokenizer that is easy to use and compose
A CJK text tokenizer
Tokenize CSS
A promise based streaming tokenizer
Tokenized zip support
A modern runtime text tokenizer, for modern web.
TypeScript definition for strtok3 token
Simple HTML Tokenizer is a lightweight JavaScript library that can be used to tokenize the kind of HTML normally found in templates.
A pure JavaScript implementation of a BPE tokenizer (Encoder/Decoder) for GPT-2 / GPT-3 / GPT-4 and other OpenAI models
ProseMirror Markdown integration
A tokenzier for Sass' SCSS syntax
Claude tokenizer
Parses and stringifies CSS selectors
Algorithms to help you parse CSS from an array of tokens.
Tokenizes a string that represents a regular expression.
A WebAssembly binding for the charabia multilingual text tokenizer used by Meilisearch.
Solve CSS math expressions
Posts/comments text parser for FreeFeed.net
r/w stream of glsl tokens