I forgot to mention that Violet only implements the core language itself without any additional modules. This means that importing anything except for most basic modules (like: builtins, sys, _imp, _os, _warnings and importlib) is not supported.
Although you can import other Python files, so technically you could import Python standard library. This is what RustPython (Python interpreter written in Rust) does. Beware: I have not tested it, and also:
- in some cases they implement some things in C so we would have to implement them in Swift (this is the RustPython version of this).
- they sometimes use things that we do not have, like comprehensions.
Also, Violet is not CPython API compatible, as in: we do not expose all of the CPython functions like Py_IncRef etc. This means that the special adapter would be needed for numpy etc.
PythonKit
Have you thought about making a version of PythonKit that supports Violet instead of CPython?
Most of the interpreters have a separate objects and execution (this is not exactly 'runtime') layers. In Violet all of our objects are implemented in Sources/Objects and execution in Sources/VM. In Python objects are stored in Objects and execution in Python.
The main goal of 'execution' is manipulating objects to get specific goal. So, for example this is how in stack based VMs implement 2+2:
/// Implements `TOS = TOS1 + TOS`.
internal func binaryAdd() -> InstructionResult {
let right = self.stack.pop()
let left = self.stack.top
switch self.py.add(left: left, right: right) {
case let .value(result):
self.stack.top = result
return .ok
case let .error(e):
return .exception(e)
}
}
To answer your question: Violet has already built-in PythonKit inside Objects module. You can use and create Python objects without using the VM as execution engine. This is what we do in out unit tests:
func test__eq__() {
// Create Python context
let py = Py(config: self.config,
delegate: self.delegate,
fileSystem: self.fileSystem)
// 'asObject' converts from PyInt -> PyObject
let leftObject = py.newInt(3).asObject
let rightObject = py.newInt(4).asObject
let result = py.isEqual(leftObject, rightObject)
// Check if 'result' is correct.
}
Compilation
Depending on how quickly it compiles, I would like to experiment with using it for Swift-Colab.
While Violet implements a sizeable portion of the language, I think that CPython is much… much… better choice.
If the compilation performance is a problem then:
- try to use newer Python versions - for example PEP 617 added PEG parser, so while it is not an holy grail, it is a tiny bit faster than the previous one (from what I have seen).
- check if the
.pyc files are used - they contain compiled bytecode, so the interpreter can skip the compilation altogether.
Conversions Swift <-> Python
That could provide tighter low-level integration with the Python interpreter and prevent a lot of conversions between Swift and Python objects.
I don't think marshaling can be avoided regardless of the choice of the underlying Python interpreter. Well… unless you have an interpreter build specially for this, but I think that Python as a language is too complex/big for this.
For example this is what lua does (based on this github mirror, side note: I have never used lua, I just looked up their sources to answer this question, I may be totally wrong):
/* Union of all Lua values */
typedef union Value {
struct GCObject *gc; /* collectable objects */
void *p; /* light userdata */
lua_CFunction f; /* light C functions */
lua_Integer i; /* integer numbers */
lua_Number n; /* float numbers */
} Value;
/* Tagged Values. This is the basic representation of values in Lua: an actual value plus a tag with its type. */
#define TValuefields Value value_; lu_byte tt_
typedef struct TValue {
TValuefields;
} TValue;
#define LUA_INTEGER long long
typedef LUA_INTEGER lua_Integer;
LUA_API void lua_pushinteger(lua_State *L, lua_Integer n) {
lua_lock(L);
// Expanding 'setivalue(s2v(L->top), n);' macro:
TValue *io=((&(L->top)->val));
val_(io).i=(n);
settt_(io, LUA_VNUMINT);
// Increments the top of the register stack or whatever…
// From what I see registers are kept in the run-time stack (array).
api_incr_top(L);
lua_unlock(L);
}
Basically in the register they store TValue { Value.i = n, tt_ = LUA_VNUMINT }.
So, yeah… they store what is essentially C long long directly in register without the need to wrap it inside the Lua/Python object (like PyInt in Violet or PyLongObject in CPython). But the tradeoff is that (and I may be wrong on this, so Lua people don't shout at me) lua is much simpler than Python.
EDIT: 'lua is much simpler than Python' is not exactly correct. I think that the better expression would be: Lua was not designed to be (or compete with) Python. Just like Python was not designed to more-readable/better C. Python is not slow, it is just that it was not designed to be fast. In real life I would rather write a server in Python/TypeScript than in C (unless it is totally necessary).