Are macros supported on Linux yet?

Development Macros
1

Are macros supported on Linux yet? Or is there a compiler switch I need to enable them? Because in the latest nightly-jammy docker image (2023-06-05) I'm getting this from the compiler

macros are not supported in this compiler
3 Likes
2

No, they are not supported yet, because they require building swift-syntax first with a prebuilt Swift toolchain, which is currently unsupported on the linux CI.

You can figure this out for yourself by taking a look at a recent build on the linux CI, which explicitly notes that Macro support is not built into the linux compiler:

-- Not building SwiftMacros because swift-syntax is not available
-- Not building ObservationMacros because swift-syntax is not available

If you search for Macros/ in that log, you can see that all the tests for it are disabled too.

2 Likes
3

Well I guess the question is will they be available when Swift 5.9 is released?

2 Likes
4

Yes. The background here is that much of the code supporting macros in the compiler is written in Swift, including using the new parser in swift-syntax and all of the macro support libraries there, so we need a Swift host toolchain to build a compiler with macros support. I have all of the compiler bits working locally (and merged), and we're looking to enable this in Swift CI real soon, at which point it will work with the toolchains on swift.org.

Doug

34 Likes
5

Thank you Doug for the detailed answer. Looking forward to seeing this.

6 Likes
6

Is it available yet?

7

No, not yet:

-- Not building SwiftMacros because swift-syntax is not available
-- Not building ObservationMacros because swift-syntax is not available
8

Any update to share?:eyes:

4 Likes
9

BTW do regex literals currently work on Linux (havenā€˜t tried)? Both macros and regex literals currently do not work on Windows, and my amateurish guess is some of the same work has to be done on both Linux and Windows. Would be nice to see both features on ā€œallā€ platforms with Swift 6. Thanks to everyone involved!

1 Like
10

Regex already works on linux :muscle:

2 Likes
11

As I pointed out twice, this is very easy to check. Simply pick a recent passing pull on the linux CI and look for those two lines I pasted in the log output. Nothing has changed:

-- Not building SwiftMacros because swift-syntax is not available
-- Not building ObservationMacros because swift-syntax is not available

As @Genaro-Chris pointed out, regex literals are enabled on linux, as can be seen in that CI log:

-- Using Experimental String Processing library for libswift _RegexParser (/home/build-user/swift-experimental-string-processing).

The reason is that there are two levels to building Swift code into the Swift compiler, either before LLVM is even built, by using a pre-installed full host toolchain including SwiftPM, or after a fresh Swift compiler has been built, by using that new compiler alone with CMake to bootstrap.

Macros are built with a pre-installed toolchain with SwiftPM, whereas regex literals are built by the fresh Swift compiler bootstrap, probably because the Regex code is much easier to grab in CMake while the swift-syntax package manifest used for Macros is much more sprawling and is better handled by SwiftPM.

Since the linux CI has never had a pre-installed Swift toolchain with SwiftPM, regex literals are currently built but Macros are not. Windows has a separate issue, where I believe C++ interop isn't fully working, like on Android, so that compiler has none of these features that are implemented in Swift.

1 Like
12

@Douglas_Gregor is there anything I - or another reader of this post - can do to help move Macros on Linux forward?

2 Likes
13

The main issue is installing a prebuilt Swift toolchain on the linux CI, as already done on the Mac CI, so that the Swift Syntax package is built and linked into the freshly-built Swift 5.9 compiler to support Macros. There has been some work on getting the linux CI ready for the prebuilt toolchain, but obviously that pull will need to be reverted for Macros to start working.

1 Like
14

I found the pull that adds a prebuilt toolchain to the linux Docker images: you could build and test a fresh compiler in the linux distro you use with SwiftSyntax enabled (replace --skip-early-swiftsyntax with --validation-test) and report back your results in this thread. That may help.

15

We're detangling some issues with bootstrapping and host tools, where we (well, mostly I) baked in some assumptions about the ABI stability into the bootstrap process, which of course don't hold on Linux.

We're staging in the changes to get everything building properly, which is a multi-step process:

  1. Explicitly turn off building "early" Swift syntax on the Linux builds temporarily. Right now, they are off implicitly because there is no Swift host toolchain anyway. Make that explicit so that the next step is a no-op...
  2. Start installing a Swift 5.8 toolchain onto all of the CI nodes so it's available as a host compiler.
  3. Get "early" Swift syntax building properly with bootstrapping so we can enable it on Linux CI. Once that happens, we'll get macros (yay).

We've done (1) and are close on (2). It's (3) that's been tricky, because the way the "early" Swift syntax build is setup, it is only built once with the host tools... but we also need to build a second time with the just-built compiler. So we're trying to find the best way to do that.

Doug

14 Likes
16

I would hope that once you figured it out on Linux, the same method would help on Windows too?

17

Yes, most of this work is in the CMake build system, which is shared across the various platforms.

We're not as far along on step (2) for Windows, but we're getting there.

Doug

4 Likes
18

Since some seem to be waiting for this, a prebuilt 5.8.1 toolchain has now been installed on the linux CI and work is ongoing to use it to build Macro support into the Swift compiler for linux.

9 Likes
19

Thanks for the update :grinning:

1 Like
20

Several pulls have now been merged to get Macros working on linux and the CI shows many of the Macro tests passing (search for Macros/ in the build log), so I downloaded the latest Sep. 1 trunk snapshot build for linux and tried to build Doug's macro examples repo. I am still getting errors like this, so it's not fully working yet?

/home/foo/swift-macro-examples/MacroExamplesPlugin/ObservableMacro.swift:110:28: error: cannot find type 'ConformanceMacro' in scope
extension ObservableMacro: ConformanceMacro {
                           ^~~~~~~~~~~~~~~~
/home/foo/swift-macro-examples/MacroExamplesPlugin/OptionSetMacro.swift:121:27: error: cannot find type 'ConformanceMacro' in scope            extension OptionSetMacro: ConformanceMacro {
2 Likes