Some questions on the design of elisp-tree-sitter

[aymanosman]

I recently asked Tuấn-Anh Nguyễn a couple of questions regarding the design of the elisp-tree-sitter package, which he kindly answered. I hope it is okay that I reproduce it here @ubolonton.

I'd just like to know about how you came about the architecture of the elisp-tree-sitter package. Whether you based it on an existing package?

It's not based on an existing package.
For parsing, I read the doc for Emacs's change tracking, which is the
important part.
For code highlighting, I studied the source of 'font-lock, 'jit-lock
to understand the display engine's interface.

I see you seem to basically rely on the 'after-change-hook facility.

Yes, incremental parsing relies on that hook, and 'before-change-hook.
Highlighting is a separate library, which overrides 'font-lock.

Any pointers for someone who would like to understand this way of implementing syntax highlighting in emacs without relying on the "font-lock" subsystem?

Basically the low-level mechanism is 'fontification-functions and the
'jit-lock library.
Theoretically alternative code coloring methods can interface with these.
In reality, the non-regexp parts of 'font-lock form an interface that
is used by many other packages.
It's thus more practical to override 'font-lock at appropriate places.
You can start by looking at the function 'tree-sitter-hl--setup.