Showing 1-20 of 202,319 packages
Simple HTML Tokenizer is a lightweight JavaScript library that can be used to tokenize the kind of HTML normally found in templates.
Small, fast, event-driven, fault-tolerant html tokenizer. Works in node or browsers.
Simple HTML Tokenizer is a lightweight JavaScript library that can be used to tokenize the kind of HTML normally found in templates.
Tokenize CSS
A promise based streaming tokenizer
Tokenized zip support
A markdown-to-HTML tokenizer and parser for rendering Ponyshow files
TypeScript definition for strtok3 token
A tokenzier for Sass' SCSS syntax
A pure JavaScript implementation of a BPE tokenizer (Encoder/Decoder) for GPT-2 / GPT-3 / GPT-4 and other OpenAI models
Parses and stringifies CSS selectors
Algorithms to help you parse CSS from an array of tokens.
Tokenizes a string that represents a regular expression.
Solve CSS math expressions
r/w stream of glsl tokens
Tiny JavaScript tokenizer.
ProseMirror Markdown integration
JS tokenizer for LLaMA-based LLMs