SwiftSyntax with Swift 5.1

@Xi_Ge created the 0.50100.0 tag, which includes the linux fix.

2 Likes

Thanks, Αργύριε!

3 Likes

I'm trying to use SwiftSyntax on iOS 13 (Simulator) using Xcode 11.0 (11A420a) -- AppStore

I added SwiftSyntax using Xcode's GUI, and it indicates the version is 0.50100.0 as you commented just over a week ago.

-- What's the right tag to use for SwiftSyntax to get exactly the version in my current Xcode? I'm getting the following error:
Could not find or use auto-linked library '_InternalSwiftSyntaxParser'

To be clear: I'm not trying to Build Swift from Source, and my hello world-esque usage of SwiftSyntax successfully compiles, it just doesn't link!

Is it a fools errand to be attempting to get this to work on iOS? (I don't know if all the dependencies are able to compile on iOS at all!)

2 Likes

See my initial response here and recent update here.

Essentially you'd need to build the parser library for simulator (and device). The parser library inside the Xcode toolchain is only built for macOS.
If you checkout the swift-5.1-branch sources you can copy the build-parser-lib script from master and invoke it as I mentioned in the other forum thread.

1 Like

Thanks for the reply!! a primary goal of mine in today's experiments was to prove that it could work at all on iOS. -- I hadn't run across your other thread before now.

I am trying to use the Swift Package Manager to build a macOS command line tool to play with the SwiftSyntax library. But currently I am running into a runtime error "dyld: Library not loaded: @rpath/lib_InternalSwiftSyntaxParser.dylib". I am using Xcode 11.2.1 and pointing SPM at the master branch of SwiftSyntax and tag "0.50100.0" as per the readme.
I see from this thread and others that the dyld error appears to be a known problem. One suggestion is to "put a symlink to lib_InternalSwiftSyntaxParser.dylib in the same directory as the executable" but I am not sure where to find lib_InternalSwiftSyntaxParser.dylib, is that part of the main Swift branch?
Some other comments in this thread indicate that the dyld issue has been fixed in the swift-5.1-branch branch but that branch is now a couple of months old and has not been merged into master. If I point SPM at swift-5.1-branch there are build errors which seem to stem from the fact that the .gyp files have not been processed. Is there a way to add this step as part of my build process?
I apologize in advance if I have missed something obvious and thanks for the help.

1 Like

I am trying to use the Swift Package Manager to build a macOS command line tool to play with the SwiftSyntax library. [...] I am using Xcode 11.2.1 and pointing SPM at the master branch of SwiftSyntax and tag "0.50100.0" as per the readme.

Then everything should already work when you do swift build, swift run, etc. from the command line. It should also work when you do do swift package generate-xcodeproj and then use that generated project.

But currently I am running into a runtime error "dyld: Library not loaded: @rpath/lib_InternalSwiftSyntaxParser.dylib".

You should only run into this if you open the package itself in Xcode by double‐clicking Package.swift. I simply don’t use it that way, because the other two ways are simpler. But if you need to do this, then you need to symlink the library from the corresponding toolchain. You will want to create a script or something that automates it, because it will vanish every time you clean the build directory.

I am not sure where to find lib_InternalSwiftSyntaxParser.dylib

lib_InternalSwiftSyntaxParser.dylib is in the Swift toolchain. With a basic Xcode install from the App Store, that would be /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/macosx/lib_InternalSwiftSyntaxParser.dylib.

Some other comments in this thread indicate that the dyld issue has been fixed in the swift-5.1-branch branch

Ignore that. The very first comment that mentioned the branch is out of date; everything the branch had then is also in 0.50100.0. The last few posts brought the branch up again, but they are talking about something else (building from source for iOS).

3 Likes

Perfect, thank you very much.

I should have been more clear about how I was using SPM. I am using the SPM support built into Xcode 11.2.1 which looks like:

This is the scenario in which the "dyld: Library not loaded: @rpath/lib_InternalSwiftSyntaxParser.dylib" error appears during runtime of the tool. Adding a link to lib_InternalSwiftSyntaxParser.dylib next to the binary fixes this issue. I put the creation of the link into a build script as you suggested.

Since it sounds like using the build system without Xcode does not have this issue perhaps there is a bug in Xcode's SPM support?

Thanks

Exactly.


You could also work around it in your case by doing this:

  1. Check out the SwiftSyntax repository at 0.50100.0 in a neighbouring directory, as a submodule, or any similar method. (Or do it with the top‐level package if SwiftSyntax is actually a transitive dependency of a different package.)
  2. Run swift package generate-xcodeproj once in SwiftSyntax’s repository.
  3. Drag that Xcode project into your workspace.

There are two reasons I suggest this over the symlink strategy:

  1. It will work consistently no matter where Swift happens to be installed, whereas the symlink and its script will break if you check your project out on a machine where Swift is installed elsewhere.
  2. All parts of the workaround live in the repository; nothing is in the products directory. That means you can set it up once and it will work indefinitely. The symlink on the other hand needs to be put in place repeatedly every time.
3 Likes

Hi @fbartho, Are you able to get it working? If yes can you please show the steps for it?
Thanks.

Here's a hacky workaround. You can run this after the first build for testing, then running the tests from within Xcode should work.

for path in ~/Library/Developer/Xcode/DerivedData/*/Build/Products/Debug/; do
    cp "$(xcode-select -p)/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/macosx/lib_InternalSwiftSyntaxParser.dylib" $path
done
2 Likes

Has anybody found a way to do this appropriately now that generate-xcodeproj is being deprecated?

2 Likes

For a long time I have just used swift run and swift test instead as on Linux. The issue only manifests at runtime, so I still use Xcode for editing source files.

Just wondering why this is the solution? Surely Swift's own syntax library should be usable without manually embedding a library from the toolchain. This issue complicates the usage of a variety of tools that build atop of SwiftSyntax.

The swift-syntax repo provides a Swift interface but it doesn't implement its own parser, it uses a library from the toolchain for accessing the parser implementation from the Swift repo.

1 Like

This is a great hack. I've added this in my run script as a pre-build action.

Follow up question, is it possible to add custom run scripts to Swift Packages? In that case, copying over the dylib from the toolchain would become much easier

This can be also solved by copying/symlinking lib_InternalSwiftSyntaxParser.dylib to /usr/local/lib as an alternative that doesn't require a script to keep derived data up to date.

sudo mkdir -p /usr/local/lib
# sudo cp /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/macosx/lib_InternalSwiftSyntaxParser.dylib /usr/local/lib/

The issue itself is out of date now. SwiftSyntax 0.50600.1 includes the library as a binary target on macOS, which works just fine with Xcode.

1 Like

That's weird. I can see it's conditionally added for macOS, but I hit it yesterday using 0.50600.1 and Xcode 13.3.1, building for an M1. What could have caused that? I was writing a plugin (I doubt it matters tho) and added deps like so:

dependencies: [
    .product(name: "SwiftSyntax", package: "swift-syntax"),
    .product(name: "SwiftSyntaxParser", package: "swift-syntax"),
    .product(name: "SwiftSyntaxBuilder", package: "swift-syntax")
]

Strange. @ahoppen?