Support for semantic highlighting is moving on Microsoft part

Theses last few week there is movement on semantic highlighting on LSP.

4 Likes

That's great news! This would solve sooooooooooooo many issues.

I actually implemented a full textmate parser and colouring system once. Besides poor documentation, they're absolutely not wrong about how complex some of these definitions are. You end up with enormous stacks of regexes, some of which need to be edited and compiled on-the-fly, many of which rely on extensions only available in the Oniguruma regex engine. The matching algorithm is all line-based, too, so you end up with limitations on the maximum line-length to stop things spiralling out of control.

Then, once you do all of that, it turns out that the conversion from matched grammars to a particular colour is also not trivial (yes, some of the best textmate documentation is stuff Microsoft reverse-engineered). Textmate was great for its day, but its day has certainly passed.

It's not great for portability that we're moving from parsable grammar definitions to full-on native parser daemons (sometimes you want to edit on a platform that doesn't have its own compiler available - not least on the web), but there are huge wins from the new approach.

1 Like

I totally agree with you there is a lot of benefits into moving to lsp.

Btw semantic token is available in lsp since 3.16.

And sourcekit-lsp has the right dependency now.

And it’s partially already implemented.

It should happen soon-ish now.

3 Likes