PythonKit not linking?

Hi there. First attempt at embedding Python and have hit a wall, and not really sure how to get around it. Hoping for some tips from the community. Apologies for the long post.

Environment: Ventura13.0.1, Xcode 14.2, MBP Pro M1 Max.

Process:

  1. cloned the PythonKit repo, and downloaded the pre-built binary of Python_Apple_Support for macOS

  2. dragged python-stdlib and Python.xcframework into my Xcode project

  3. added the PythonKit package

  4. Added SystemConfiguration.framework to the Frameworks in Build Phases

  5. Added PythonKit to Target Dependencies in Build Phases

  6. Verified that Python.xcframework, and SystemConfiguration.framework are Do Not Embed, and that python-stdlib is Copied as a Bundle Resource

  7. Created a file in Python.xcframework/macos-arm64_x86_64/Headers/module.modulemap and pasted in

module Python {
 umbrella header "Python.h"
 export *
 link "Python"
}
  1. Added Run Script phase, pasted in the following script
set -e
echo "Signing as $EXPANDED_CODE_SIGN_IDENTITY_NAME ($EXPANDED_CODE_SIGN_IDENTITY)"
find "$CODESIGNING_FOLDER_PATH/Contents/Resources/python-stdlib/lib-dynload" -name "*.so" -exec /usr/bin/codesign — force — sign "$EXPANDED_CODE_SIGN_IDENTITY" -o runtime — timestamp=none — preserve-metadata=identifier,entitlements,flags — generate-entitlement-der {} \;
  1. Modified the default app source
class EmbeddedPython {
    init() {
        guard let stdLibPath = Bundle.main.path(forResource: "python-stdlib", ofType: nil) else { return }
        guard let libDynloadPath = Bundle.main.path(forResource: "python-stdlib/lib-dynload", ofType: nil) else { return }
        setenv("PYTHONHOME", stdLibPath, 1)
        setenv("PYTHONPATH", "\(stdLibPath):\(libDynloadPath)", 1)
        Py_Initialize()
        
        let sys = Python.import("sys")
        
        print("Python Version: \(sys.version_info.major).\(sys.version_info.minor)")
        print("Python Encoding: \(sys.getdefaultencoding().upper())")
        print("Python Path: \(sys.path)")
        
        _ = Python.import("math") // verifies `lib-dynload` is found and signed successfully
    }
}

@main
struct EmbeddedPythonApp: App {
    @State var embeddedPython = EmbeddedPython()
    var body: some Scene {
        WindowGroup {
            ContentView()
        }
    }
}

This does not link, generating a warning "Could not find or use auto-linked library 'Python', and a smattering of Undefined symbol errors which all look like the arm64 binaries are missing. But using otool -D to examine libPython3.11.a shows both architectures are present. I can't seem to find the PythonKit library so haven't checked if it includes the arm64 architecture. Does anyone know where this library is?

Have I missed a step, or am I not correctly configuring my Xcode project. Some help would be deeply appreciated.

I cannot really help you with your specific use case (“embedding” Python), but for using PythonKit see my example (also see some useful hints e.g. how to read UTF-8 encoded files when using PythonKit).

Thanks for your reply. I did manage to get PythonKit integrated into my Xcode project. It builds and the resulting app seems to be able to run python code fine.

The problem I'm having now is not knowing how to install third-party modules, since I can't call the embedded interpreter from the command-line, or invoke pip. I've tried just copying numpy from the system installation, but though it loaded, the embedded interpreter had a problem loading some modules: specifically _multiarray_umath, which I believe is compiled c/c++ code. I do codesign these binaries as the last phase of the build.

I've written up a tutorial on the precise steps I took to properly include PythonKit in an Xcode project. Here's a link:

Edited to correct mistaken link.

If the Python modules are installed (via conda or pip), you just need to reference the according Python library (via environment variable PYTHON_LIBRARY when working the “normal” way with PythonKit), e.g. the Python library from your according Python environment (i.e. the environment you activate e.g. via conda activate). So I suppose you just need to include such an environment where the modules are installed in your project,

1 Like

I will give that a try, but does that mean that I'll be trying to load binaries external to the app? I think I can only do that if I disable Sandboxing and Hardened Runtime, which I'd prefer not to do.

To embed Python using PythonKit, I have been following this article: Embedding a Python interpreter inside a MacOS / iOS app, and publishing to the App Store successfully. | by Eldar Eliav | Swift2Go | Dec, 2022 | Medium.

It outlines a way to retain Sandboxing and Hardened Runtime, by embedding a self-contained installation of Python in the Resource fork of the target application. All the c/c++ compiled binaries have to be code signed. Unfortunately, there doesn't seem to be any documentation on how to install third-party modules with this method.