Showing 1-20 of 28,033 packages
Tokenize CSS
A promise based streaming tokenizer
Tokenized zip support
TypeScript definition for strtok3 token
Parses and stringifies CSS selectors
Algorithms to help you parse CSS from an array of tokens.
A tokenzier for Sass' SCSS syntax
A pure JavaScript implementation of a BPE tokenizer (Encoder/Decoder) for GPT-2 / GPT-3 / GPT-4 and other OpenAI models
Solve CSS math expressions
Simple HTML Tokenizer is a lightweight JavaScript library that can be used to tokenize the kind of HTML normally found in templates.
ProseMirror Markdown integration
Common token types for decoding and encoding numeric and string values
tokenizer of source code for jscpd
Claude tokenizer
r/w stream of glsl tokens
Tokenizes a string that represents a regular expression.
Advanced tagging/tokenizing plugin for input fields with a focus on keyboard and copy-paste support.
Parse CSS media query lists.
Multilingual tokenizer that automatically tags each token with its type
Tokenize a shell string into argv array