I am trying to call two different python function using PythonKit from macOs command line application's two different (parallel) threads. whenever both of them starts to run in parallel the later one gets an error of EXC_BAD_ACCESS from internal code pythonkit :
1 Like
I'm having similar problems. Did you find a workaround?
Python has a global interpreter lock, so it shouldn't be possible to use it from multiple threads. PythonKit does not have built-in thread safety because it assumes you will only use it from a single-threaded context.
Perhaps use a synchronous Swift dispatch queue to coordinate operations onto one thread.
2 Likes
Yup, this is just what I've been working on setting up, thanks!
Hi @jbmaxwell and @philipturner! Do either of you have resources or tips about how to set up such a "synchronous Swift dispatch queue"? In my Swift app with PythonKit I'm currently running everything on the main thread, which is suboptimal for operations that can take many seconds to complete. I have only rudimentary experience with DispatchQueue and Swift async/await, so this might be a real beginner question!
Sadly, "life" (ML) has pushed me into writing Python pretty much exclusively for the last year, or so, so I haven't been working on that project. I just opened it up and it looks like I never got around to setting up that synchronous call, but if I were to do it now I'd do it like this:
Using the async/await feature.
Thanks, @jbmaxwell. I'm not quite sure how to adapt that API call / network request example to PythonKit, but I've been looking into creating a custom DispatchQueue and these articles make me think I could do something like this:
let pythonQueue = DispatchQueue(
label: "com.myapp.queue.python",
qos: .background
)
Then I'd keep a reference to that custom queue at all times—probably in a singleton like MyPythonFunctions, from which I call any function that uses PythonKit. I haven't tested this approach, and I might need to initialize the queue with a DispatchQueue.Attributes value that turns off concurrency—I'm not sure. I also don't know if/how this could work with async/await... but even if it doesn't it would be an improvement by moving the time-consuming Python functions off the main thread. If I eventually have time to test this I'll report back, and I welcome any thoughts in the meantime!
DispatchQueues are serial by default, so what you have there is not parallel. I wouldn't use an explicit QoS, especially .background, as this doesn't seem like low priority work, and it will likely be promoted anyway. And on iOS .background priority becomes optional under certain conditions, like low power mode.
1 Like
Well, this is what you currently want because of CPython‘s GIL. But the GIL might be removed in due time, see e.g. this current discussion. (Or Mojo by Chris Lattner and others might become an option…)
1 Like
Right, I was just responding to this:
I got the same error. And I think pythonkit don't support subthread, because I really create a serial thread for it. Here is my code :
lazy var thread:Thread = {
var t = Thread(target: self, selector: #selector(begin), object: nil)
t.name = "com.python"
t.qualityOfService = .default // i tested all of qos enum, but all of them not work
return t
}()
.....
@objc
func begin() {
print("pre start")
let example = Python.import("my_python_module")
print("start")
let result = example.start()
print(result)
}
PS: I'm already call self.thread.start() to run the thread.
It's always _pthread_cond_wait () for something
