Unable to use python modules that are installed on my mac with PythonKit

Related Projects
, , , ,
1

So, I'm trying to use pyserial and regular expressions with PythonKit, which I have pip installed onto my mac, but I cannot use it, and it runs into the following error:

PythonKit/Python.swift:707: Fatal error: 'try!' expression unexpectedly raised an error: Python exception: No module named 'serial'
Traceback:
  File "/Users/username/Library/CloudStorage/OneDrive-Personal/project/projapp/projapp/pycode.py", line 9, in <module>
    import serial
    ^^^^^^^^^^^^^

The error found No module named 'serial' even though I have it pip installed

Please help, there was no support anywhere online!

2

If you enter the Python interpreter via the command line

python

Then try importing it there does it work? (without the >>>)

>>> import serial

if so, what happens if you type:

>>> import sys
>>> sys.version
>>> sys.path

And does that match in your Swift code if you add this code before you import serial:

let sys = Python.import("sys")
print("Python \(sys.version_info.major).\(sys.version_info.minor)")
print("Python Version: \(sys.version)")
print(sys.path)
3

as stated, I already have serial and such pip installed and they work without any problems when working on python idle or vs code, but when using PythonKit I don’t think it can see everything I have pip installed

I don’t know if there is even something like import pip

4

I found out that PythonKit is using

3.12.3 (main, Apr  9 2024, 08:09:14) [Clang 15.0.0 (clang-1500.3.9.4)]

while idle & vs code are using

3.10.5 (v3.10.5:f377153967, Jun  6 2022, 12:36:10) [Clang 13.0.0 (clang-1300.0.29.30)]

How can I force PythonKit to use Python 3.10.5 with clang 13.0.0 (1300) to match the standard python? (IF THAT SOLVES THE ISSUE*)

5

It appears that you have two different versions of Python installed on your machine.

Would I be right in assuming that you installed pyserial using the command line inside IDLE?

I am not very familiar with Python, but it would seem that you have two options:

  1. Try and get PythonKit to use the same version of Python as IDLE uses (as you mentioned in your last post)

  2. Install the packages your require via the version of Python that PythonKit is using.

In case 1) - The PythonKit Package Github Page states that it tries to use the most recent version of Python, but you can tell it to use another via environment variables. (You can set environment variables in your Swift project in Xcode.) PYTHON_VERSION=3.10.5

In case 2) - You could try to install pyserial etc using the more recent version of Python. Try opening the macOS Terminal app (not IDLE) and try using pip to install the required packages.

I'm curious how you have two version of Python installed. Did you perhaps use brew (Homebrew) to install Python?

6

I used terminal initially already.

Yes, back in 2022 I used brew to do it.

I tried, but I never found anywhere which helped with doing so.

7

Do you try setting the environment variable in your projects target?

PYTHON_VERSION=3.10.5

8

Just did that, but now it gives the error:

PythonKit/PythonLibrary.swift:59: Fatal error: 'try!' expression unexpectedly raised an error: Python library not found. Set the PYTHON_LIBRARY environment variable with the path to a Python library.

I'm assuming it wants me to find the directory of the python libraries

9

I'm wondering if there is any way to pip install to PythonKit?

10

I'm pretty sure that @pvieito 's PythonKit supports modules.

Although, the website doesn't show anything related to it.

11

So I tried what the console error told me and

Set the PYTHON_LIBRARY environment variable with the path to a Python library.

Screenshot 2024-05-14 at 10.43.14 AM
Screenshot 2024-05-14 at 10.43.14 AM1312×304 30.7 KB

it still errors :frowning:

12

I found out that PythonKit is directly compatible with NumPy through a file in it for conversion. I wonder if there is possibility to create something like that for serial and re (regular expressions)

13

I think that the error isn't saying what you think it's saying. I think that PYTHON_LIBRARY isn't the path to the external pip package libraries - it's looking for the path to the library for Python itself.

Anyway - you may have better luck trying the other approach (use the version of Python that PythonKit is already finding). It worked for me.

When I installed Python via brew and attempted to install some packages via pip I was warned that:

× This environment is externally managed
    If you wish to install a Python library that isn't in Homebrew,
    use a virtual environment:

So I made a virtual environment (which is a separate location for installed packages etc...)

Then I needed to add that location to the PATH for the instance of Python that PythonKit was using.

In my Swift test app I added the following:

let os = Python.import("os")
let sys = Python.import("sys")
sys.path.append(os.path.abspath("/Users/diggory/python/venv/lib/python3.12/site-packages"))

That path is only valid on my machine (obviously). If you open the terminal and in the Python interpreter type

import sys 
sys.path

It will show you the paths that it has. One of which is where your pip packages have been installed.

n.b. This isn't very portable as you are hard-coding a local path into your code (but then you are relying on pip installed packages anyway, so I assume that this project is just for your own machine and you're not expecting anyone else to run your app.)

It's also fragile as the python version is hard-coded into the path.

Also - this means that you are using packages installed by a different version of Python that the one that is using them, which could cause problems...

14

I was thinking - one way of solving your issue is to remove one of the installed versions of Python...

15

If I do, would that uninstall the packages though? I could update the homebrew python to match, (although I have to find out how)

16

I can try this and see if it works

17

I tried this and it still errors, and I can provide code because I think I did something wrong:

import Foundation
import PythonKit


func runPyCode() -> PythonObject {
    // File
    let os = Python.import("os")
    let sys = Python.import("sys")
    sys.path.append(os.path.abspath("/Users/username/Library/CloudStorage/OneDrive-Personal/project/PKT/PKT/"))
    let file = Python.import("pycode")
    
    // Libraries
    let serial = Python.import("serial")
    serial.path.append(os.path.abspath("/Users/username/Library/Python/3.10/lib/python/site-packages/"))
    let re = Python.import("re")
    re.path.append(os.path.abspath("/Users/username/Library/Python/3.10/lib/python/site-packages/"))
    
    let pyFunc = file.runCode()
    
    print(pyFunc)
    
    return pyFunc
}

Although PythonKit does not show where the error occurred, it just throws an error to something in its own code.

18
19

I deleted the old version of python and defaulted pip to the system version.

Whilst that didn't fix the issue, I don't know exactly what I did but now instead of asking for PYTHON_LIBRARY, it now says that

Thread 1: Fatal error: 'try!' expression unexpectedly raised an error: Python exception: No module named 'serial'
and

PythonKit/Python.swift:621: Fatal error: 'try!' expression unexpectedly raised an error: Python exception: No module named 'serial'
Traceback:
  File "/Users/aravprasad/Library/CloudStorage/OneDrive-Personal/crisislab/PKT/PKT/pycode.py", line 12, in runCode
    import serial # pyserial
    ^^^^^^^^^^^^^
20

Let's go back to basics.

Here's a basic demo package using PythonKit:

If I run that swift code, it prints the following:

Python 3.12
Python Version: 3.12.3 (main, Apr  9 2024, 08:09:14) [Clang 15.0.0 (clang-1500.3.9.4)]
Python PATH
['/opt/homebrew/Cellar/python@3.12/3.12.3/Frameworks/Python.framework/Versions/3.12/lib/python312.zip', '/opt/homebrew/Cellar/python@3.12/3.12.3/Frameworks/Python.framework/Versions/3.12/lib/python3.12', '/opt/homebrew/Cellar/python@3.12/3.12.3/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload', '/opt/homebrew/Cellar/python@3.12/3.12.3/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages']
Error importing module 'camelcase': Python exception: No module named 'camelcase'
Program ended with exit code: 0

As expected it shows the only version of Python (3.12.3) that I have on my machine (installed via homebrew.)

The import fails because there is no Python module named camelCase in any of the paths listed.

In that package there is also a Python script (pythonScript.py) that does the same thing.

If I chmod +x that script and run it from the terminal it outputs pretty much the same thing: (same version of Python and same PATH values)

Python 3.12
Python Version: 3.12.3 (main, Apr  9 2024, 08:09:14) [Clang 15.0.0 (clang-1500.3.9.4)]
Python PATH: ['/Users/diggory/Code/OpenSource/PythonKitSimpleTest', '/opt/homebrew/Cellar/python@3.12/3.12.3/Frameworks/Python.framework/Versions/3.12/lib/python312.zip', '/opt/homebrew/Cellar/python@3.12/3.12.3/Frameworks/Python.framework/Versions/3.12/lib/python3.12', '/opt/homebrew/Cellar/python@3.12/3.12.3/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload', '/opt/homebrew/lib/python3.12/site-packages']
Traceback (most recent call last):
  File "/Users/diggory/Code/OpenSource/PythonKitSimpleTest/./pythonScript.py", line 15, in <module>
    import camelcase
ModuleNotFoundError: No module named 'camelcase'

So far so good. In fact we can confirm in the terminal that camelCase isn't installed.

pip list

Package Version
------- -------
pip     24.0
wheel   0.43.0

Now, if I swap to my virtual environment that I made to install pip packages (as recommended by brew) you can see that camelCase is installed.

source ~/python/venv/bin/activate

pip list

Package   Version
--------- -------
camelcase 0.2
pip       24.0

And I can see that the pure python script can now import the module.

./pythonScript.py 

Python 3.12
Python Version: 3.12.3 (main, Apr  9 2024, 08:09:14) [Clang 15.0.0 (clang-1500.3.9.4)]
Python PATH: ['/Users/diggory/Code/OpenSource/PythonKitSimpleTest', '/opt/homebrew/Cellar/python@3.12/3.12.3/Frameworks/Python.framework/Versions/3.12/lib/python312.zip', '/opt/homebrew/Cellar/python@3.12/3.12.3/Frameworks/Python.framework/Versions/3.12/lib/python3.12', '/opt/homebrew/Cellar/python@3.12/3.12.3/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload', '/Users/diggory/python/venv/lib/python3.12/site-packages']

No error and you can see that PATH includes my virtual environment's 'site-packages' path.

Now if I add the following line to my swift code (line 16 in main.swift)

 sys.path.append(os.path.abspath("/Users/diggory/python/venv/lib/python3.12/site-packages"))

The Swift code also works.

Python 3.12
Python Version: 3.12.3 (main, Apr  9 2024, 08:09:14) [Clang 15.0.0 (clang-1500.3.9.4)]
Python PATH
['/opt/homebrew/Cellar/python@3.12/3.12.3/Frameworks/Python.framework/Versions/3.12/lib/python312.zip', '/opt/homebrew/Cellar/python@3.12/3.12.3/Frameworks/Python.framework/Versions/3.12/lib/python3.12', '/opt/homebrew/Cellar/python@3.12/3.12.3/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload', '/opt/homebrew/Cellar/python@3.12/3.12.3/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages']
Program ended with exit code: 0