generic-lexer

Does the lexing (aka tokenizing) step of parsing source code.

This is a generic lexer, it works pretty well for lots of mainstream syntaxes.