PythonKit integration with Swift - Crashing on release due to module import

I have a project that is using Swift (for interface) and Python (for function) for a Mac application.

I added PythonKit as a dependency and after: • removing "App Sandbox" from "Signing & Capabilities" • turning "Enable Hardened Runtime" off • Setting environment variable "PYTHON_LIBRARY" to "/opt/homebrew/bin/python3.11"

Now everything works as hoped and when I run it works great.

However, now I archive the Xcode project -> Distribute -> Custom -> Copy App. It creates an instance of the application to where I save it and as soon as I open it it opens for a few milliseconds (the window never opens) and it immediately crashes with this warning window:

-------------------------------------
Translated Report (Full Report Below)
-------------------------------------

Version:               1.0 (1)
Code Type:             ARM-64 (Native)
Parent Process:        launchd [1]

OS Version:            macOS 14.0 (23A344)
Report Version:        12

Exception Type:        EXC_BREAKPOINT (SIGTRAP)
Exception Codes:       0x0000000000000001, 0x000000019f534658

Termination Reason:    Namespace SIGNAL, Code 5 Trace/BPT trap: 5
Terminating Process:   exc handler [97502]

Thread 0 Crashed::  Dispatch queue: com.apple.main-thread
0   libswiftCore.dylib          0x19f534658 _assertionFailure(_:_:file:line:flags:) + 268
1   libswiftCore.dylib          0x19f5a0cf8 swift_unexpectedError + 516
2   Filex                       0x102576428 PythonInterface.import(_:) + 148 (Python.swift:707)
3   Filex                       0x10256f7e8 PythonAPI.init() + 568 (PythonAPI.swift:24)
4   Filex                       0x10256f5a4 PythonAPI.__allocating_init() + 44
5   Filex                       0x102541a30 implicit closure #2 in implicit closure #1 in variable initialization expression of Main._pythonAPI + 28
6   SwiftUI                     0x1bb15ab5c 0x1ba412000 + 13929308

Notes: It looks like it's crashing at this function:

public init() {        
   // Initialize Python
   system = Python.import("sys")
   system.path.append("/Users/reidhtaylor/Documents/Projects/Python/Filex/FilexBackend")
        
   // Get main file
   main = Python.import("main") //!!!! AT THIS LINE #24 !!!!//
}

First of all I don't understand why it works fine when I "Run" the project but "Archive doesn't work". Also I don't get how system.path.append should work cause won't everyones path be different when I build the app? And yes the main.py file is in the FilexBackend folder and it works when Running the project.

I tried settings "Strip Style" to only "Debugging Symbols" (didn't work) and I couldn't find the settings "Enable Bitcode" to try turning it off (which is a fix I saw online)

1 Like

If you intend to distribute this program, you're going to want to embed a copy of Python in it; you can't really use one from Homebrew because not everyone will have that installed, and even people who are using Homebrew won't necessarily have python 3.11. Also, PYTHON_LIBRARY surely shouldn't point at the python3.11 binary? It's presumably supposed to point at Python inside Python.framework (on macOS, at least).

As for the crash itself, it's a being caused by the try! in Python.swift on line 707. You should probably take a look at the assertion failure message, as that will likely identify the error that's being thrown.

It should point to the according library file cf. this description. There it is also noted:

You may not need to set this environment variable on macOS. But you might have to set "Enable Hardened Runtime" to "No" in the macOS settings when using a Python version installed by you.

Thanks for the response!! @al45tair

1:
Ok that’s what I was thinking about the python library. How do I go about embedding one?

2:
I know I thought the path I gave it was weird, but if I didn’t put a path or tried one of the paths I would’ve expected I kept getting a “Fatal Error: Python Library not found” and o did set “Enable Hardened Runtime” to false @sspringer

3:
Yah so I did look at where the error is pointing and it’s cause by the import module line I have above. I understand that adding the system path won’t work cause everyone’s path will be different to my application but how can I embed python files in my Xcode project and then have python import them?

Thanks

I suspect this might help as it describes how to do what you're after, I think (including using PythonKit).

Ok. Thanks for the Response! @al45tair

I got python all embedded and my python files are now built as bundle resources so everything is working when I archive and then export as copy build.

Big problem though is whenever I send the app to another computer to test it, it crashes immediately upon opening with this error:

Translated Report (Full Report Below)
-------------------------------------

Process:               Filex [27455]
Path:                  /private/var/folders/*/Filex.app/Contents/MacOS/Filex
Identifier:            com.scorpionstudios.Filex
Version:               1.0 (1)
Code Type:             X86-64 (Native)
Parent Process:        launchd [1]
User ID:               501

Date/Time:             2024-06-29 13:55:12.8411 -0700
OS Version:            macOS 14.5 (23F79)
Report Version:        12
Bridge OS Version:     8.5 (21P5077)
Anonymous UUID:        0BCB58F4-9F99-4BF4-18CD-32398D822138

Sleep/Wake UUID:       2CD8D2B8-6C7E-4A0A-9B76-EFCB335AD370

Time Awake Since Boot: 1200000 seconds
Time Since Wake:       121 seconds

System Integrity Protection: enabled

Crashed Thread:        0

Exception Type:        EXC_BAD_INSTRUCTION (SIGILL)
Exception Codes:       0x0000000000000001, 0x0000000000000000

Termination Reason:    Namespace SIGNAL, Code 4 Illegal instruction: 4
Terminating Process:   exc handler [27455]

Kernel Triage:
VM - (arg = 0x3) mach_vm_allocate_kernel failed within call to vm_map_enter
VM - (arg = 0x3) mach_vm_allocate_kernel failed within call to vm_map_enter
VM - (arg = 0x3) mach_vm_allocate_kernel failed within call to vm_map_enter
VM - (arg = 0x3) mach_vm_allocate_kernel failed within call to vm_map_enter
VM - (arg = 0x3) mach_vm_allocate_kernel failed within call to vm_map_enter

Thread 0 Crashed:
0   libswiftCore.dylib                    0x7ff81796cd9f _assertionFailure(_:_:file:line:flags:) + 351
1   libswiftCore.dylib                    0x7ff8179c4a48 swift_unexpectedError + 840
2   Filex                                    0x10442930f PythonInterface.import(_:) + 111
3   Filex                                    0x1044238e4 PythonAPI.init() + 5796
4   Filex                                    0x104422231 PythonAPI.__allocating_init() + 33
5   Filex                                    0x1043f5918 implicit closure #2 in implicit closure #1 in variable initialization expression of Main._pythonAPI + 24
6   SwiftUI                               0x7ff9148f840f 0x7ff913d73000 + 12080143

I can't figure out why this is happening??? If I comment out this line of code the error doesn't happen:

var main : PythonObject = Python.import("main")

"main" is a file (main.py) that is is included in the apps bundle resources and I added the resource path Bundle.main.path(forResource: "FilexResources", ofType: nil). It works on my computer when I build it so I don't know why it doesn't work on others??? Any ideas?

Does

let sys = Python.import("sys")
print("Python Version: \(sys.version_info.major).\(sys.version_info.minor)")
print("Python Encoding: \(sys.getdefaultencoding().upper())")
print("Python Path: \(sys.path)")

work from your program?

If so, it suggests that maybe your Python path is wrong and it just can't find main.py. Printing sys.path as shown above might be a good first step in figuring this out. I'd also look in Console to see if there are any interesting messages from your program. Finally, you could try it on another machine in the debugger, maybe?

Yes I have that in the program and it the program doesn't crash if it's just that. The program only crashes as soon as I add this line: var main : PythonObject = Python.import("main").

I am not sure why this would happen though. I added the folder "Backed" as a group folder reference to Xcode and when I build the app I can see the "Backend" folder in resources in "Show package contents" so I know it exists. The Backend folder contains a bunch of normal python files.

I make it so I can import python files from the folder with Bundle.main.path(forResource: "Backend", ofType: nil) and add it to the PYTHONPATH environment variable at runtime with the : seperator. The most confusing thing is why it works as a release build on my computer but not on others... I would think that implies that I am drawing from some global resource on my computer that a normal person would not have? Thanks for the help

My guess is that you might be importing packages that aren't installed in your bundle, but that are installed on your machine. If your Python code relies on packages you would normally install using e.g. pip, you need to make sure that you install them into a packages directory that is part of your bundle, and that directory needs to be in sys.path/PYTHONPATH at runtime.

Interesting. You're correct in that when I erase everything in the .py files in "Backend" and just do a short test everything works fine. It starts crashing as soon as I start importing third party modules like import PIL. I used this command

pip install --target "PATH_TO_EMBEDDED_PYTHON_REFERENCE/python-stdlib" pillow

This work and it creates the PIL library in the embedded python:

So I have no clue why the importing is still not working... FYI I also added the path of PIL to PYTHONPATH environment variable as instructed here Python-Apple-support/USAGE.md at main · beeware/Python-Apple-support · GitHub

Bundle.main.path(forResource: "python-stdlib/PIL", ofType: nil)

How do I go about adding third party modules to the embedded python? I tried doing things like this as well

python3.9 -m pip download -d "PATH_TO_EMBEDDED_PYTHON_REFERENCE/python-stdlib" --platform=macosx_11_0_arm64 --only-binary=:all: pillow 

I don't think the python-stdlib directory is the right place for third-party modules. The platform-site directory seems like a better bet, and you'll want the platform-site directory in your PYTHONPATH too.

Also, Pillow has lots of binary code, so you'll want to make sure that's getting code signed properly. You might try using DYLD_PRINT_SEARCHING or DYLD_PRINT_LIBRARIES to see whether the embedded dylib from the PIL module is actually getting loaded (or whether it's being rejected for some reason).