Swift 5.3 Windows: no such module 'Python'

Using Swift
,
#1

I'm trying to extend the HelloWorld example here:
git://github.com/compnerd/swift-build-examples /SourceCache/swift-build-examples

by simply adding this line at the top of HelloWorldCore.swift:
import Python

building gives this error:
\SourceCache\swift-build-examples\HelloWorld-CMake\Source\HelloWorldCore\HelloWorldCore.swift:2:8: error: no such module 'Python'
import Python
^
ninja: build stopped: subcommand failed.

I'd installed Visual Studio 2019, including Python (and c++ for native tools)
I then installed Swift “VS2019(Google)” by downloading “installer.exe” from
https://dev.azure.com/compnerd/swift-build/_build/results?buildId=32557&view=artifacts&type=publishedArtifacts

Without the "import Python" HelloWorld builds fine:
[2/5] Linking C shared library bin\CCore.dll
Creating library lib\CCore.lib and object lib\CCore.exp
[3/5] Linking Swift shared library bin\HelloWorldCore.dll
Creating library lib\HelloWorldCore.lib and object lib\HelloWorldCore.exp
[4/5] Linking Swift executable bin\TestHelloWorldCore.exe
Creating library bin\TestHelloWorldCore.lib and object bin\TestHelloWorldCore.exp
[5/5] Linking Swift executable bin\HelloWorld.exe
Creating library bin\HelloWorld.lib and object bin\HelloWorld.exp

So the general setup and configuration seems to be all there. Not sure what to do to make Python module visible to the compiler/linker. Python is definitely installed as I'm able to type "python" in the same command prompt from which I'm attempting to build, and that works fine by starting the python interpreter.

Wondered what was a good way to make it possible to import Python into Swift 5.3 running on windows. Is Swift “VS2019(Google)” a good choice, or would it be easier to start off by using a different swift installer which might have python more readily integrated? Or perhaps some other suggestions - thanks!

#2

On reviewing this article:
https://www.tensorflow.org/swift/tutorials/python_interoperability

I've had more success by using:
import PythonKit

That is in conjunction with an install of "VS2019 Swift TensorFlow (Google)" by using installer.exe downloaded from:
https://dev.azure.com/compnerd/swift-build/_build/results?buildId=32329&view=artifacts&type=publishedArtifacts

Making progress now, will post again here with more info should I discover anything else which might help anyone trying to do something similar.

1 Like
#3

I tried to write up all the steps, which I then followed myself when setting up a new VM (window 10) from scratch. It looks like the steps below are complete and should pretty much just work. There are 30 steps in total. I had to break into several replies as I'm only allowed up to 5 links per post being as I'm new on this forum.

Swift – Hello World!
Installing Swift

  1. Install Visual Studio 2019 using MSDN subscription:
    Include C++ so that “x64 Native Tools” command prompt is available.
    Also include Python which is needed for integration with Front Arena.
  2. Install CMake (https://cmake.org/download/) and choose option to add PATH to all users on machine
  3. Install ninja (https://github.com/ninja-build/ninja/releases) unzip ninja.exe and copy it to cmake binaries folder (C:\Program Files\CMake\bin)
  4. Install Git for Windows (https://gitforwindows.org/)
  5. Get the “hello world!” sample source code by running the following from command prompt:
    git clone git://github.com/compnerd/swift-build-examples /SourceCache/swift-build-examples
2 Likes
#4
  1. Install Swift “VS2019 TensorFlow (Google)” by downloading “installer.exe” from latest working build from:
    https://dev.azure.com/compnerd/swift-build/_build
    At the time of writing, latest working build could be found through this link:
    https://dev.azure.com/compnerd/swift-build/_build/results?buildId=32651&view=artifacts&type=publishedArtifactscts
    This TensorFlow build includes the PythonKit module which is essential for Python integration
  2. Unzip intaller.exe, and choose “more” and then select “run anyway” as the publisher is not recognised/certified by Windows OS
  3. Open an elevated Visual Studio “x64 Native Tools” command prompt as Administrator
  4. You may wish to “pin” the command prompt to taskbar as it is needed frequently
1 Like
#5
  1. Run the following in this command prompt:
    curl -L "https://raw.githubusercontent.com/apple/swift/master/stdlib/public/Platform/ucrt.modulemap" -o "%UniversalCRTSdkDir%\Include%UCRTVersion%\ucrt\module.modulemap"
    curl -L "https://raw.githubusercontent.com/apple/swift/master/stdlib/public/Platform/visualc.modulemap" -o "%VCToolsInstallDir%\include\module.modulemap"
    curl -L "https://raw.githubusercontent.com/apple/swift/master/stdlib/public/Platform/visualc.apinotes" -o "%VCToolsInstallDir%\include\visualc.apinotes"
    curl -L "https://raw.githubusercontent.com/apple/swift/master/stdlib/public/Platform/winsdk.modulemap" -o "%UniversalCRTSdkDir%\Include%UCRTVersion%\um\module.modulemap"
#6
  1. Close the command prompt and then reopen the “x64 Native Tools” command prompt again
  2. Run the following in this new command prompt:
    set SDKROOT=%SystemDrive%\Library\Developer\Platforms\Windows.platform\Developer\SDKs\Windows.sdk
    copy "%SDKROOT%\usr\share\ucrt.modulemap" "%UniversalCRTSdkDir%\Include%UCRTVersion%\ucrt\module.modulemap"
    copy "%SDKROOT%\usr\share\visualc.modulemap" "%VCToolsInstallDir%\include\module.modulemap"
    copy "%SDKROOT%\usr\share\visualc.apinotes" "%VCToolsInstallDir%\include\visualc.apinotes"
    copy "%SDKROOT%\usr\share\winsdk.modulemap" "%UniversalCRTSdkDir%\Include%UCRTVersion%\um\module.modulemap"
  3. Setup build parameters:
    set SDKROOT=%SystemDrive%/Library/Developer/Platforms/Windows.platform/Developer/SDKs/Windows.sdk
    set SWIFTFLAGS=-sdk %SDKROOT% -I %SDKROOT%/usr/lib/swift -L %SDKROOT%/usr/lib/swift/windows
  4. Run the following to configure cmake:
    "%ProgramFiles%/CMake/bin/cmake.exe" ^
    -B /BinaryCache/HelloWorld ^
    -D BUILD_SHARED_LIBS=YES ^
    -D CMAKE_BUILD_TYPE=Release ^
    -D CMAKE_Swift_FLAGS="%SWIFTFLAGS%" ^
    -G Ninja ^
    -S /SourceCache/swift-build-examples/HelloWorld-CMake
  5. The output of the above configuration commands should look like this:
    -- The Swift compiler identification is Apple 5.3
    -- Check for working Swift compiler: C:/Library/Developer/Toolchains/unknown-Asserts-development.xctoolchain/usr/bin/swiftc.exe
    -- Check for working Swift compiler: C:/Library/Developer/Toolchains/unknown-Asserts-development.xctoolchain/usr/bin/swiftc.exe - works
    -- Configuring done
    -- Generating done
    -- Build files have been written to: /BinaryCache/HelloWorld
  6. Now build the hello world exe file:
    cmake --build /BinaryCache/HelloWorld
  7. The output of the build command should look like this:
    [2/5] Linking C shared library bin\CCore.dll
    Creating library lib\CCore.lib and object lib\CCore.exp
    [3/5] Linking Swift shared library bin\HelloWorldCore.dll
    Creating library lib\HelloWorldCore.lib and object lib\HelloWorldCore.exp
    [4/5] Linking Swift executable bin\TestHelloWorldCore.exe
    Creating library bin\TestHelloWorldCore.lib and object bin\TestHelloWorldCore.exp
    [5/5] Linking Swift executable bin\HelloWorld.exe
    Creating library bin\HelloWorld.lib and object bin\HelloWorld.exp
  8. To verify, navigate to the newly built exe location:
    cd C:\BinaryCache\HelloWorld\bin
  9. Typing HelloWorld.exe should give this output:
    Hello World!
  10. Swift works – Congratulations!
1 Like
#7

Integrating Python
21. For convenience, you may wish to install Notepad++ (https://notepad-plus-plus.org/downloads/) which does have support for Swift syntax highlighting.
22. Edit this file using Notepad++ or similar:
C:\SourceCache\swift-build-examples\HelloWorld-CMake\Source\HelloWorld\HelloWorld.swift
23. The new content of the file is changed near the top so that the whole file looks like this:
import HelloWorldCore

#if canImport(PythonKit)
import PythonKit
#else
import Python
#endif
print(Python.version)

struct HelloWorldStringAccessor {
etc…

  1. Save the file, and then delete or rename this folder to force a clean rebuild
    C:\BinaryCache\HelloWorld
  2. PATH environment variable will need to be updated to include the path where Python was installed by Visual Studio 2019. If python was installed through other means (e.g. by installing Anaconda) you might be able to skip the next step and go straight to step 27
  3. In Windows start menu, search for “control panel” and launch
    In control panel, search for “environment” and click on “Edit system environment variables”
    In “System Properties” the “Advanced” tab is selected, click on “Environment Variables…” button
    On the lower pane “System variables” select “Path” and then click on the Edit button
    Click on “New” and paste the path where Visual Studio 2019 installed Python to, which is normally:
    C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64
    Click OK on all open dialog boxes
  4. To verify, reopen a new “x64 Native Tools” command prompt
    Type “path” in this new command prompt then hit return to confirm that Python is included in the search paths
    Or, alternatively type “python” to confirm that the interpreter starts (ctrl+z then enter to exit)
  5. Repeat steps 13 to 18 above to rebuild and link etc
  6. Typing HelloWorld.exe should give this output:
    3.7.2 (tags/v3.7.2:9a3ffc0492, Dec 23 2018, 23:09:28) [MSC v.1916 64 bit (AMD64)]
    Hello World!
  7. The first line shows the version of Python which has been integrated – Congratulations!
1 Like
#8

Hi,

Thanks for writing these up! I wonder if it makes sense to do a few things here. First of all, I think it's valuable to split these up into two separate categories:

  • Swift
  • Swift for TensorFlow

For the Swift case (steps 1-20), most of that should be covered in https://github.com/compnerd/swift-build/blob/master/docs/GettingStartedWindows.md. If there are improvements to be made to that, I would appreciate a PR. Once we have nightlies for Windows from swift.org, I think it would make sense to migrate those instructions to the Swift repository.

For the Swift for TensorFlow specific instructions (21-30), perhaps you would be willing to help improve the documentation in the TensorFlow Swift APIs repository?

Saleem

#9

Is there any kind of roadmap for simplifying the steps and reducing their amount? I have no relevant experience with the Windows ecosystem, but maybe Swift for Windows could be packaged and distributed with something like the recently announced Microsoft's winget package manager?

#10

I think that the way the steps are described, it makes it seem more than it is.

Packaging this into winget would basically be just changing steps 6-7 which download the installer and extract it.

For what I mean by the prose makes this seem more involved:

Steps 1-5: Install Visual Studio from https://visualstudio.com
Steps 6-9: Install Swift from https://github.com/compnerd/swift-build/releases
Step 10: Copy files

This is where I think that it would be possible to improve the installer to automate this, but no one has stepped up to do this work, and my Windows knowledge is based on what I've figured out while doing this work :). If someone knows how to handle the required elevation to automate the SDK modifications, that would be a welcome enhancement to the installer.

Steps 11-20: build project

About half the steps here are this is expected output and the other half splits up the configure into 3 steps, build into 1 step, and execution into 1 step.

Steps 21-30: specifics to editing the code, setting up a particular dependency which has additional specifics to system setup (e.g. python).

This is probably the area that really needs improvements. However, some of this is specific to development on Windows and the realities of using Python (such as editing the PATH).

#11

Sure, I'd be happy to help however I can. This might be a basic question, bearing in mind that I'm a total Swift beginner - would one expect "import Python" to work from plain Swift, as opposed to Swift TensorFlow? If so, then it might make sense to address that aspect in "GettingStartedWindows.md" as you mention.

I say this because the notes in GettingStartedWindows do mention including Python support when installing Visual Studio say that this is "needed for Python integration". But by following all the steps in GettingStartedWindows it is not clear how to work with Python with the resulting setup.

To me, making use of Swift TensorFlow was in way of workaround to get Python integration working with Swift, but there might be a more direct way of doing so. If that's the case, that would basically be the real answer to the original question I'd posted here, and if someone is able to provide that real answer I'd be more than happy to update GettingStartedWindows.md accordingly if that helps.

1 Like
#12

No, import Python would not be expected to work in Swift by default. The python interoperability requires PythonKit which is a separate project.

The python integration it is referring to is specific to LLDB. The debugger depends on the python interpreter. That should be rewritten as I originally wrote the instructions bearing Swift for TensorFlow in mind. Things have progressed since that point. The Windows port tends to move faster than the documentation, reminders like this are often the catalyst for improving the documentation.

I’m glad that you found Swift for TensorFlow useful :). There are many other really interesting features in Swift for TensorFlow. For the use of Python interoperability, that is not specific to Swift for TensorFlow, it is just is that the dependency is included in the build. Building the module yourself would allow you to have Python interoperability with Swift.

I think that a general section in the document for adding dependencies seems reasonable in the examples repository. Unfortunately, SwiftPM isn’t yet ready on Windows and so the instructions are slightly different (normally, you would just add the dependency to the manifest). Here you need to build the dependency and add the paths to the build invocation.

1 Like
Terms of Service

Privacy Policy

Cookie Policy