Skip to content

Commit 6a1b54d

Browse files
authored
refactor(tokenizer): Introduce sequences & fast forwarding (#1007)
Sequences allow us to remove a lot of repetitive states. Fast forwarding allows us to skip ahead in the buffer, if we are only looking for one specific character. Also fixes a bug where immediately closed comments (eg. `<!-->`) wouldn't be recognised.
1 parent 3adc0e7 commit 6a1b54d

File tree

1 file changed

+294
-415
lines changed

1 file changed

+294
-415
lines changed

0 commit comments

Comments
 (0)