Updated syntax highlighting in GitHub & VS Code

Community Showcase
,
1

Hi folks,
I've recently been working on some major updates to the Swift tmLanguage grammar. As of today, GitHub has updated their syntax highlighter to use the latest version of this grammar. It's also included in VS Code's built-in Swift extension (not the vscode-swift extension maintained by the server workgroup).

The grammar now supports regular expression literals (perhaps more features of the proposal than Swift itself supports right now :slight_smile:):

image
image1084×408 57 KB

image

As well as some of the newer ownership features:

image

And macros (this screenshot is from VS Code because there appears to be a bug where GitHub isn't highlighting macros right now):

image
image1718×224 43.2 KB

Hope you enjoy!

22 Likes
2

That's nice. Though, it seems GitHub is having issues with multiline strings now, not sure if that's related.

Screenshot 2023-12-13 at 2.05.07 PM
Screenshot 2023-12-13 at 2.05.07 PM3704×1426 496 KB

1 Like
3

Yep, I saw that too. I'll try to look into it soon – Bug with multiline strings in GitHub · Issue #7 · jtbandes/swift-tmlanguage · GitHub

2 Likes
4

Awesome work, thank you a lot @jtbandes !

I see even distributed functions and actors highlight properly now as well, I think they didn't for the longest time -- thanks!

I spotted a few files where things seem to go slightly wrong still:

https://github.com/apple/swift/pull/68861/files

Should I separate issues for those?

2 Likes
5

Thank you for doing this, @jtbandes! I'd been looking at the sorry situation of Swift syntax highlighting in VS Code just recently, coincidentally, and was really rather depressed about it - by all appearances it was abandoned years ago.

My interest is not actually in VS Code itself, but rather that these grammar files are also used in things like WordPress plug-ins (e.g. Code Block Pro). It's a real struggle to find any syntax highlighting plug-in for WordPress that does even just a half-decent job with Swift.

The other problem with Swift syntax highlighting is poor theme support, even in VS Code surprisingly - I thought there was a non-trivial number of people using Swift in VS Code, but judging from the abysmal support for Swift in all the popular themes (and even supposedly purpose-built-for-Swift themes), that is not the case. Do you have any plans to tackle that as well, perchance?

(I spent several long days trying to build a new theme for VS Code specifically for Swift, and while it didn't take long at all to out-do every existing theme, I realised it'll take way more time to actually complete it… writing good VS Code themes is extremely laborious :pensive:)

6

i gave up on client-side swift highlighting long ago; the language grammar is too complex for anything but SwiftSyntax to parse correctly.

here is a “dark” theme which i use but never published anywhere:

{
	"name": "swift-madrid",
	"semanticHighlighting": true,
	"colors": 
	{
		"editor.background": "#131313",
		"editor.foreground": "#c8c8c8",
		"activityBarBadge.background": "#ff2164",
		"sideBarTitle.foreground": "#bbbbbb"
	},
	"semanticTokenColors": 
	{
		"keyword": "#ff446d",
		"modifier": "#ff446d",
		"class": "#ffcb6b",
		"typeParameter": 
		{
			"foreground": "#ffa734",
			"fontStyle": "italic"
		},
		"interface": "#ffa734",
		"struct": "#ffa734",
		"enum": "#ffa734",
		"type": "#ffa734",
		"number": "#a98aff",
		"string": "#a98aff",
		"variable": 
		{
			"foreground": "#c8c8c8"
		},
		"identifier": "#c8c8c8",
		"enumMember": "#c8c8c8",
		"function": "#c8c8c8",
		"property": "#c8c8c8",
		"method": "#c8c8c8",
		"operator": "#c8c8c8"
	},
	"tokenColors": 
	[
		{
			"name": "keywords",
			"scope": 
			[
				"keyword",
				"keyword.control",
				"keyword.operator.logical.python",
				"storage.type",
				"storage.modifier",
				"support.variable.magic",
				"support.function.magic",

				"variable.language.special.self"
			],
			"settings":
			{
				"foreground": "#ff446d"
			}
		},
		{
			"name": "types",
			"scope": 
			[
				"meta.class"
			],
			"settings":
			{
				"foreground": "#ffa734"
			}
		},
		{
			"name": "identifiers",
			"scope": 
			[
				"support.function.swift", 

				"entity.name.function"
			],
			"settings":
			{
				"foreground": "#c8c8c8"
			}
		},
		{
			"name": "punctuation",
			"scope": 
			[
				"keyword.operator",
				
				"punctuation",

				"source.swift"
			],
			"settings":
			{
				"foreground": "#747474"
			}
		},
		{
			"name": "macros",
			"scope": 
			[
				"support.constant.platform.os.swift",
				"punctuation.definition.preprocessor.swift",
				"keyword.other.condition.swift",
				"keyword.control.preprocessor.conditional.swift"
			],
			"settings":
			{
				"foreground": "#e355ff"
			}
		},
		{
			"name": "anchors",
			"scope": 
			[
				"punctuation.section.embedded.begin.swift",
				"punctuation.section.embedded.end.swift source.swift",

				"constant.character.format"
			],
			"settings":
			{
				"foreground": "#e355ff"
			}
		},
		{
			"name": "literals",
			"scope": 
			[
				"string",
				"punctuation.definition.string",
				"constant"
			],
			"settings":
			{
				"foreground": "#a98aff"
			}
		},
		{
			"name": "comments",
			"scope": [
				"comment",
				"punctuation.definition.comment"
			],
			"settings": {
				"foreground": "#666666"
			}
		},

		{
			"name": "invalid",
			"scope": [
				"invalid",
				"invalid.illegal"
			],
			"settings": {
				"foreground": "#FF5370"
			}
		},
	]
}

here is what it looks like:

image
image544×837 62.3 KB

7

Yes, go ahead and file separate issues for those. Thanks!

1 Like
8

I'm not planning to make any changes to themes, but as part of these updates I did change several of the scopes that the grammar adds, so the coloring looks better in the default themes. There are some screenshots here: Update Swift grammar and upstream repository by jtbandes · Pull Request #197470 · microsoft/vscode · GitHub

Unfortunately each of these tools uses a slightly different implementation and sometimes features of the grammar that work in some places don't work in others (for example, the GitHub bugs above). There are usually ways to tweak/simplify the grammar so it works in more situations but it's hard to do so without an easy way to reproduce the issues.

You might be interested in this script I wrote a while back to translate RTF on the clipboard to HTML — I haven't used it recently so can't say whether it still works in modern versions of Xcode.

1 Like
9

Also worth noting on the VS Code front: if you install the vscode-swift extension it'll take over syntax highlighting.

1 Like
10

Thanks for your work here @jtbandes !

I filed two issues:

I don't know how acurrate my guess-diagnosis is, there could be other reasons why it goes wrong there.

Thank you for your work on this, it'll be awesome to have an accurate syntax representation and highlighting. :pray:

1 Like
11

Thanks for reporting issues and sorry about the disruption from regressions. It seems like there may be a GitHub issue that affects other languages too — the folks who work on this are aware of the issue and planning to follow up in the new year.

1 Like
12

GitHub has finally fixed the highlighting issues on their side, but sadly, they have also stopped using the tmLanguage grammar for Swift. It has been replaced with tree-sitter-swift. VS Code still uses the tmLanguage version.

2 Likes
13

For VS Code I've really enjoyed using this Gruvbox Material theme extension, and then enabling this extension setting:

"gruvboxMaterial.colorfulSyntax": true

Apparently:

Enabling this option will cause more tokens to be colored. By default, only the minimum but necessary tokens will be colored.

Note: This design is deprecated, there are a lot of bugs and that's why I redesigned the syntax highlighting logic and set it as default.

While I do not necessarily love the colors, whatever this deprecated design is in my opinion superior to everything else that exists, for readability first and foremost - I don't really see how it's buggy -- in some cases I notice it is slow, but then again aren't we all a little sometimes? :sweat_smile:

It looks like this:

Gruvbox Material with Swift
Gruvbox Material with Swift1175×1229 207 KB

I wouldn't suppose your work could work with this, or how vs code syntax highlight themes even work exactly -- but it is getting tiresome using the same theme for years now, it'd be nice to base a new one off this deprecated design or anything else that can give me similar extensive highlighting.

14

Indeed they do work together! In fact, the scopes used by the theme extension...

...are precisely meant to correspond with scope regions produced by the grammar's patterns:

A "colorful" or "extensive" design is probably just one which has put a lot more care into specifying colors for many different specialized scopes.

VS Code actually makes it fairly simple to customize your color settings based on scopes, and then package the customizations into your own theme extension. Read more about it here:

The hard part is choosing the colors! :sweat_smile:

2 Likes
15

You are great! This is really insightful, I'm going to check it out now, I really appreciate you showing me where to look, thank you so much!

The hard part is choosing the colors! :sweat_smile:

A simple dynamic interactive syntax highlight color generator app would be epic... :face_in_clouds:

1 Like