superMac & Massively Pararell Swift

I am new to this august body. I have a great deal of respect for the work that has been achieved thus far in SSWG community. I have experience in mathematics, computation and cell biochemistry. I have a scientific computing problem that I would like to achieve using the Apple ecosystem.

Here is the pitch:
Create a "superMac" by physically connecting many "fully loaded" MacPro "nodes" and have a layer of Swift software the enables Swift application development on each node (Swift and using Metal on the GPUs) and horizontally across the different nodes. Perhaps the SSWG might be interested in this. My application by the way is a simulator of the human cell at the molecular level.

Apple already owns the creative industry professional world; it apparently now has its eye on scientific computing: which nVidia currently owns. Creating "superMac" could help in this regard. Very keen to receive any feedback before I purchase my first 2 "fully loaded" Mac Pros.

Thank you in advance for letting me pitch this and for any feedback; even if you think it is a crazy idea.

1 Like

It's actually a good idea. Apple used to have a system called Xgrid that supported this type of application execution across machines. There are several open-source systems (OpenMPI, for example) that can be used both across processes on the same machine, and across machines over the network. None of this is in Swift, but, OpenMPI, for example, has a C interface that could the basis for a Swift port. On-machine, you can already use Metal within Swift to get SIMD. You have dispatch queues within Swift, and you can get to Posix threads via the C interface until Swift get's its own native concurrency infrastructure to provide thread-level parallelism. LLVM/clang has OpenMP already which could be used with Swift with work potentially allowing parallel directives within the source code like C/C++/Fortran.

Depends on your time frame and expectations. Going to take a bunch of work to develop a truly native experience.

1 Like

You might look at Boinc from Berkeley. It's been used in some large scale grid computing and distributed computing projects, including SETI@Home. Again, not in Swift, but accessible by C interface. Probably good for short/medium term.

There are a bunch of grid computing systems out there. Most will work on Mac OS X (or, at least used to, haven't checked in a while after I retired, and my super-computing days were over).

Tensorflow folks might have some thoughts as well.

Thank you Mr. Jon Prescott for your insightful and pragmatic advice.