Python 3 support for building Swift Toolchain

Hey all-

Fedora is actively deprecating Python 2 for Python 3 and as of just a few days ago, the change was made that /usr/bin/python is no longer symlinked to /usr/bin/python2 but /usr/bin/python3. I rebuilt the latest 2019-07-14 snapshot and it fails with, you guessed it, Python errors.

As Python 2 is soon to be EOL'ed (as noted by this amusing site) I was wondering if there was any "official" plan to upgrade the build scripts to Python 3. I was thinking to do it and apply a set of patches to make Swift build on Fedora, but am happy to contribute them back as pull requests. That said, I don't want to duplicate others work, so if nobody's actively working on it, mind if I go ahead and do it?

Thanks,

Ron

15 Likes

Okay, I am able to build Swift with Python 3. Pull requests are here and here. I mentioned that it's probably early to be submitting the pull request insofar as I'm guessing the automated builders are still 2.7; I wanted to make the code available for whenever it's decided to move forward with Python 3.

3 Likes

As a followup, I have a Swift 5.1-2019-07-24 rpm available here for Fedora that was built on a Python 2-free environment; python3 had to be explicitly symlinked to python because that's what build-script expects to find. :upside_down_face:

1 Like

Hey @Ron_Olson, I'd like to thank you for pushing this effort forward. It will likely simplify packaging for many different distributions. In particular, I know that the Arch Linux swift-language package on AUR does a fair bit of regex replacing to patch the build script to explicitly use Python 2. I decided to try out your changes by building swift-DEVELOPMENT-SNAPSHOT-2019-07-22-a, and they have worked smoothly so far, minus one exception—I had to patch import diagnostics in utils/swift_build_support/swift_build_support/host_specific_configuration.py.

Unfortunately, however, I ran into an error a while into the build. I'm not actually certain if this error is related to the Python version, so if it's not, I'll file a bug report. Linked here are some pertinent lines from the crash. The two lines that seem most suspect are (firstly) FAILED: docs/CMakeFiles/docs_html after Building HTML documentation and (once it begins building the Sphinx documentation) Warning, treated as error: /home/aemino/aur/swift-language/src/swift/docs/ARCOptimization.rst:100:Pygments lexer name 'sil' is not known

The following warning:
/home/aemino/aur/swift-language/src/swift/utils/line-directive:709: FutureWarning: Possible nested set at position 36 '(?P<middle>, line |:)(?P<line>[0-9]+)(?P<tail>.*?)\n?$')
also shows up a fair bit.

I have very little knowledge about Swift's build process, so your input would be appreciated here. Thanks!

Yep, I'm very familiar with that error. The issue is that somewhere the python-sphinx tool, using pygments, can't find the custom SIL colorizer; I looked into it a bit and I think the idea is that the build script wants to pass it to pygments as it's not on the known list of colorizers, and it fails. That was going to be next on the list of things to investigate.

In the meantime, the solution is straightforward: remove python-sphinx from your system; the Swift toolchains builds fine without it. I updated the Fedora spec file and removed python-sphinx as a requirement.

The FutureWarning is the thing I'm looking into now; it's not an error as you mention but it would be nice to be as warning-free as possible.

FYI there's a great article on gyb that really helped explain how it's being used as part of the build process; once I read the article I "got it" in terms of what is trying to be accomplished, and actually found that it's a separate project unto itself and have found my new boilerplate-generator tool. :grin:

2 Likes

Glad to hear it's not just me! I'll remove python-sphinx and retry the build tonight. Thanks for your help and good luck fixing the issue!

FYI, I found the problem, but it turns out it was already fixed via this pull request., so it shouldn't be an issue at some point soon (it's still there in the 2019-07-25 build).

Just in case you haven't seen it already, Gwynne did a lot of work in https://github.com/apple/swift/pull/23256 and I did some pieces (some repeated) in https://github.com/apple/swift/pull/23038 (sadly mixed with other changes).

Oh, wow, cool, no, I didn't know about these pull requests. Awesome work!

It'll definitely be interesting to see how the 2->3 transition happens with macOS; it's been the source of a lot of headaches in Fedora (at least for me :upside_down_face:).

Terms of Service

Privacy Policy

Cookie Policy