PythonKit not finding Python 3 library

Hi. :wave:

I'm experimenting with @pvieito's great looking PythonKit. I'm having a weird dlopen/search path issue.
I'm wondering if anyone can advise?

Problem

Following the project README I have a very simple hello world setup that imports PythonKit, sets sys.path to find my local script, imports that, and calls a function that prints a Hello, World!.

Thus far, all great. Out of the box, PythonKit is finding Python 2.7, so (again following the README) I try setting the PYTHON_VERSION and PYTHON_LIBRARY environment variables. (I also set PYTHON_LOADER_LOGGING for good measure.)

The issue is PYTHON_LIBRARY seems correct — it reports as loading correctly — but Python 2.7 is still being loaded.

This is the sample code from the README:

let sys = Python.import("sys")
print("Python \(sys.version_info.major).\(sys.version_info.minor)")
print("Python Version: \(sys.version)")
print("Python Encoding: \(sys.getdefaultencoding().upper())")

Which (with PYTHON_LIBRARY set) outputs:

Loading symbol 'Py_Initialize' from the Python library...
Trying to load library at '/opt/boxen/homebrew/Cellar/python@3.8/3.8.7/Frameworks/Python.framework/Python'...
Library at '/opt/boxen/homebrew/Cellar/python@3.8/3.8.7/Frameworks/Python.framework/Python' was sucessfully loaded.
Loading symbol 'PyEval_GetBuiltins' from the Python library...
Loading symbol 'Py_IncRef' from the Python library...
Loading symbol 'PyRun_SimpleString' from the Python library...
Loading symbol 'PyImport_ImportModule' from the Python library...
Loading symbol 'PyObject_GetAttrString' from the Python library...
Loading symbol 'Py_DecRef' from the Python library...
Loaded legacy Python library, using legacy symbols...
Loading symbol 'PyString_FromStringAndSize' from the Python library...
Loading symbol 'PyObject_GetItem' from the Python library...
Loading symbol 'PyErr_Occurred' from the Python library...
Loading symbol 'PyTuple_New' from the Python library...
Loading symbol 'PyTuple_SetItem' from the Python library...
Loading symbol 'PyObject_CallObject' from the Python library...
Loading symbol 'PyString_AsString' from the Python library...
Python 2.7
Python Version: 2.7.16 (default, Jan 22 2021, 05:09:26) 
[GCC Apple LLVM 12.0.5 (clang-1205.0.19.9) [+internal-os, ptrauth-isa=deploymen
Python Encoding: ASCII

With the Library at <PATH> was sucessfully loaded., which is the 3rd line of the output above , I'm expecting Python 3.8, but 2.7 is in fact what we get.

With a test script such as:

print("Hello, world! (From Python 💃)")

... we get a crash, without declaring the file encoding — Ah, yes, definitely Python 2… :slightly_smiling_face:

I'm not quite sure (yet) how to get the planets aligned here. How can I get PythonKit to pick up the right Python version? Is there something obvious I'm missing? (Quite likely.)

Many thanks if you're to advise! I know this sort of thing is difficult to comment on.
Kind Regards,

Carlton

Possibly a related post in another thread suggesting the need to edit the PythonLibrary.swift file to set the search paths correctly. :thinking:

Hi! That may be related to the Python 2 library being linked to the executable, you can check that with otool:

$ otool -L path_to_your_executable
	/System/Library/Frameworks/Foundation.framework/Versions/C/Foundation (compatibility version 300.0.0, current version 1770.255.0)
	/usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 228.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1292.60.1)
	/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1770.255.0)
	@rpath/libswiftCore.dylib (compatibility version 1.0.0, current version 1200.2.41)
	@rpath/libswiftCoreFoundation.dylib (compatibility version 1.0.0, current version 1.6.0, weak)
	@rpath/libswiftCoreGraphics.dylib (compatibility version 1.0.0, current version 2.0.0, weak)
	@rpath/libswiftDarwin.dylib (compatibility version 1.0.0, current version 0.0.0)
	@rpath/libswiftDispatch.dylib (compatibility version 1.0.0, current version 4.40.2, weak)
	@rpath/libswiftFoundation.dylib (compatibility version 1.0.0, current version 20.0.0)
	@rpath/libswiftIOKit.dylib (compatibility version 1.0.0, current version 1.0.0, weak)
	@rpath/libswiftObjectiveC.dylib (compatibility version 1.0.0, current version 1.0.0, weak)
	@rpath/libswiftXPC.dylib (compatibility version 1.0.0, current version 1.1.0, weak)

Make sure no references are made to Python and that you are not linking nor importing the Python module in your code.

1 Like

Hi @pvieito — thanks for the quick response!

I can't see that I'm linking against Python...

$ otool -L /Users/carlton/Library/Developer/Xcode/DerivedData/FromPython-axlvqoykyiwdfwddzgxlgofxeral/Build/Products/Debug/FromPython
/Users/carlton/Library/Developer/Xcode/DerivedData/FromPython-axlvqoykyiwdfwddzgxlgofxeral/Build/Products/Debug/FromPython:
	/System/Library/Frameworks/Foundation.framework/Versions/C/Foundation (compatibility version 300.0.0, current version 1770.255.0)
	/usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 228.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1292.60.1)
	/usr/lib/swift/libswiftCore.dylib (compatibility version 1.0.0, current version 1200.2.41)
	/usr/lib/swift/libswiftCoreFoundation.dylib (compatibility version 1.0.0, current version 1.6.0, weak)
	/usr/lib/swift/libswiftCoreGraphics.dylib (compatibility version 1.0.0, current version 2.0.0, weak)
	/usr/lib/swift/libswiftDarwin.dylib (compatibility version 1.0.0, current version 0.0.0)
	/usr/lib/swift/libswiftDispatch.dylib (compatibility version 1.0.0, current version 4.40.2, weak)
	/usr/lib/swift/libswiftFoundation.dylib (compatibility version 1.0.0, current version 20.0.0, weak)
	/usr/lib/swift/libswiftIOKit.dylib (compatibility version 1.0.0, current version 1.0.0, weak)
	/usr/lib/swift/libswiftObjectiveC.dylib (compatibility version 1.0.0, current version 1.0.0, weak)
	/usr/lib/swift/libswiftXPC.dylib (compatibility version 1.0.0, current version 1.1.0, weak)

:thinking: Still thinking.

Hey @pvieito — it's always SIP :grinning:

Set "Enable Hardened Runtime" to "No" and I get the correct Python load.
:dancer: — I'm off. :tada:

I'll experiment but, this makes me wonder if static linking the libpython would work (despite the dlopen)... — that would be quite handy. Gonna have a read of Use RTDL_SELF to load embedded symbols by kewlbear · Pull Request #30 · pvieito/PythonKit · GitHub.

Thanks so much for your efforts on this lib. Very exciting! (I see you don't have Issues or Discussions enabled, so you prefer it here yes?)

Have a great rest of your weekend, and thank you for the help! :medal_sports:

Oh, gotcha! Yes, with the “Hardened Runtime” enabled the process won't load libraries not signed by Apple or the developer. You can embed the Python framework in the app and sign it with your developer certificate.

1 Like

Do you know the exact steps to accomplish that (embed the Python framework in the app)?

Terms of Service

Privacy Policy

Cookie Policy