Realtime threads with Swift

Evolution Discussion
#62

Some interesting discussion. Chipping in as the voice of someone a bit at the coalface of real-time (I think), adding an attribute appeals to me, a bit like @convention(c) which I often use in my API. I think I’ll have more to say after I’ve had time to play with adding this pass in my compiler (having trouble building on my Mac M1). It still feels a big ask to add to the main line compiler at this early stage though. It’s ok for custom compiler builds like my forks. I think the need is real but we might find a more complete engineering approach in due course that might make more sense to add to mainline. I want something the wwdc engineers would be comfortable introducing in “what’s new in swift”! :slight_smile:

Another approach might be adding a command line flag to the compiler like —enforce-real-time=true that indicates intent. At first it could add an llvm pass similar to the @realtime above, to make sure only a subset of runtime marked as “real-time safe” (ie predictable, stable and proportionate in execution time... O(1) ish) was allowed and others caused a compiler error. Later that flag could be expanded in use to disable COW somehow, maybe load a slightly different standard library for your target, etc. It would have the advantage of making a catch all and you should know when you’re compiling if a file is intended to produce hard realtime code or not.

For me, I think the fundamental question is “how good is swift at writing bare metal, type, system type code?”.. where you’re comparing to C or Rust. I would love the answer to be “just as good, in realistic use”. So if someone wants to write linux device drivers in swift and they have never touched ios or macs or even UI code themselves, it’s still a natural and painless choice... they gain the defined behaviour and modern syntax of swift and much of the clean architecture of the swift standard library... their code is more accessible to a new generation of coders.

I meant this post to be short... typical me. :slight_smile:

2 Likes
#63

with @realtime or a similar construct it would not be just "as good as C"... it would be better! e.g. one would be able to use malloc, or mutex lock, or dispatch async, etc in C, but not in realtime swift function...

(footgun is always possible though) 
while someCondition {
   ... some long loop here where you spend more than allowed time ...
}
#64

@tera late comer to this thread... .

I believe the "no Swift for Audio" is still policy. Probably the most recent sign of this is the compiler error you get if you want to use the new (since iOS 14) Audio Workgroups in Swift.

#65

Yeah. I also have it on good authority that it's still the case. Wish I could say more.

#66

Hi @audulus. I'm patching my local tree with your llvm pass, as it's useful info. Great work!

Looking at https://github.com/apple/swift/compare/main...audulus:realtime, I saw commits from 4f8f6b9 to 3ef8bd8. I put them into a diff then tried to apply it to my tree. They were for 5.5 or newer and I'm still on 5.3 so I had to do a bit of tweaking but I got it working.

I'm getting output like...

main.swift:26:9: warning: variable 'timer' was never mutated; consider changing to 'let' constant
    var timer = timers[timerNumber]
    ~~~ ^
    let
Validating function main.testAllocations() -> () for realtime safety.
Function main.testAllocations() -> () contains swift runtime calls.

Which is a great start. Your question to @Joe_Groff on June 4th: "How can I report nice errors/warnings to the user from my LLVM pass (including source line numbers)? "

Did you make any progress with his suggestions for adding it to diag so I can report a proper warning or error to the user/developer?

I couldn't see that in the tree I'm looking at?

Carl

1 Like
#67

I'm trying to work out how to output diagnostics,

F.getParent()->getContext().Diags.diagnose(SourceLoc(), diag::realtime_detected), name);

I can't get progress. Is it possible to output diagnostics from the llvm passes or is it too degenerate?

(specifically, in other places I think it's normal to use the ASTContext but I suspect it's not available in LLVM passes?)

#68

@audulus I watched the AudioKit talk you posted, thanks for the great demo!

I wonder if the thing you want is a fixed size array at compile time? Maybe this would be useful: penguin/FixedSizeArray.swift at main · saeta/penguin · GitHub

1 Like
#69

Hey Carl, I'm afraid not. I found building and testing the Swift compiler so unpleasant that I kinda gave up. I may get back to it. Also thinking about writing a clang plugin for realtime verification of C/C++ code.

1 Like
#70

Yeah. That's reasonable. I maintain my own variant swift compiler and standard lib. One of the common questions is something like "why isn't your product open source?" (Set aside for a minute that mostly they are furious at the notion of maybe paying even a cent for all the years of work I've put in...) As I tell people... "compiling a compiler is like compiling a kernel... you might be able to do it but you probably don't want to... trust me."

Your patch seemed to work well enough as a first. The suggestions to put scanning code in IRGen or as a SIL pass seemed to miss some important points...

The compiler liberally creates runtime calls in many places, for many, many reasons. I feel it would be a big job catching them all in IRGen. And in SIL, it's too high level. Many SIL features map to a lot of IR code. The only way to be sure is your approach. Run an LLVM pass on all functions in the IR during emission, so there's nowhere to hide.

Unfortunately at that point, in the LLVM pass, there seems to be very little you can do to get to the normal diagnostics machinery. Quite possibly it has already shut down at that point (after IRGen).

I've tried a few things but generally hit dead ends. And I'm no swift compiler expert.

I just started an attempt to instrument when getXYZRUNTIMEFn() is called, which could work too. At least we have ASTContext there to gain access to diagnostics. But at that point, how do we decide if we are emitting a runtime function call from a function marked as @realtime? I don't think we have that context?

Also, in whole-module-compilation the LLVM steps are run multi threaded for performance, which might complicate it further.

#71

p.s. And I realised... I haven't yet said THANK YOU so much for your work on this and your patch! :slight_smile: Good stuff!

#72

Sure thing. I hope the idea of automatic realtime safety verification ends up realized at some point.

(I just checked out the Swift code from GitHub and quickly determined that building is still broken and hard. One would think a compiler could be made to be a pretty easy thing to build. After all, it takes text files in, and spits out other files. I think the Swift project isn't really open source, insofar as it's too hard to work on it outside of Apple)

2 Likes
#73

It took quite a bit of practice to get swift compiling for me. I’m using a Mac, which means it’s tied Tolbert tightly to your xcode version. I have an older laptop where the build works and I pretty much don’t touch it! My new M1 Mac mini I started to get it working but got stuck and it was too much effort. The readme on swift is more or less right but I think there are many cases where it doesn’t work.

One particular thing I think is nowhere near made clear enough, each branch and build of swift requires a specific version of Xcode to build. Some versions either side will work. But for example I would bet if you get the latest head release of swift from github.com/Apple/swift it will probably only build if you have the latest version of xcode from the store. And sometimes it even needs the latest xcode beta to build!

This should be much more clear.

To some extent it can bootstrap. So I know it’s sometimes capable of checking out and building ninja as a first step. But it seems unreliable to me.

See my webpage: https://www.swiftforarduino.com/openSource.php for some partial instructions and ideas but it’s not a proper set of instructions, more like ores and ideas.

The only way I’ve got it working is…
0) get Cmakeand set it up

  1. make a folder and open a terminal session in that folder
  2. checkout a branch of swift (eg Swift 5.3) into a sub folder
  3. using swift/utilise/update-checkout —clone —scheme X (see my page for examples)
  4. that should check out llvm, ninja, etc at the right versions needed next to your swift folder
  5. use Swift/Utils/build-script to control the build. I use -S so the build script creates all the build files for ninja but doesn’t do the build
  6. go into your build folder, Ninja si directory then go into each of cmake, llvm and lastly Swift and do a ninja in each. It takes an hour on my machine!
Huge performance hit from exclusive memory access checks
#74

Would apple/swift#39902 by @Erik_Eckstein be relevant to this topic?

8 Likes
#75

That looks really great. Is it accurate though without looking at the LLVM IR?

#76

i am really pleased someone is looking at it. if this is proceed to its logical end one day swift could become "safer than C" irt realtime programming. kudos to @Erik_Eckstein

re: the comment in the PR:

unless proven absolutely unusable, i'd suggest to make it as strict as possible.. if objc_msgSend can ever take a lock (slow path, etc) - it is not realtime safe. the law is the law. cc @Mike_Ash

5 Likes
#77

Totally agreed that this should be strict. My comment is really just about how it's described. The description currently says "unpredictable performance," which encompasses the whole problem, but isn't very specific.

4 Likes
#78

Erik's patch... I should call it a tree more like... looks amazing. And coming from someone on the (Apple) Swift core team, it's going to be better than anything we mere mortals can produce. :) I also loved the "tiny swift" Erik came up with... but this patch looks more realistic for something that could go into a pitch and end up in the mainline.

@Mike_Ash @audulus @Erik_Eckstein ... is there any suggestion to move this to a proper SE pitch? Then the community could thrash out these ideas?

I think there is serious interest in having something like this available in core swift. I feel like the SE process would be the way to move it forward. Also @Erik_Eckstein if you needed to justify your time spent on this project, this thread shows serious interest.

@Mike_Ash ... speaking as someone who is very interested in one type of "realtime" on microcontrollers* I feel like I can sort of answer a bit. The worst thing for microcontroller code is not just slowness... it's unpredictability... you need to know when writing statements in swift that a given statement will take a given amount of time. Because you need to write code that controls hard real-time events, like emitting a servo controller pulse of exactly the right width, or bit banging a port protocol for I2C.

So if someone is writing something like...

for n in 0...myArray.count {
  // do some hardware checking
  myArray[n].wasChecked = true
}

...then it can be unpredictable, I think. If myArray is a local array, not aliased, the memory is probably pretty straightforward, this is probably a single write to memory in a single location. If myArray is passed in as a parameter to a function that does this, but the function is inlined, then it might be basically the same. But if myArray is in a function and sometimes causes a C.O.W. then there could be a very large difference in execution time.

I think a reasonable rule of thumb would be statements should never vary in execution time more than 10% or something like that.

*This is just my opinion and just one case of realtime. If you're writing linux device drivers, or audio, or writing OpenGL pipeline then you'd probably have a different view from me!

Edit: Even if code is a bit inefficient, if you can measure for example that a loop always takes 120us then you can allow for it and debug it but if the language has internal, invisible structure that suddenly makes something take 20 times as long, or even 3 times as long and it's not obvious to people who are not compiler/optimiser experts then that makes it difficult or impossible to use for many "realtime" uses.

I hope my thoughts help and aren't annoying anyone.

2 Likes
#79

dynamic structures like swift's array / dictionary / set are outlawed in real time safe code as they have internal reference types which could take locks (and cause memory allocation sometimes). taking a lock can take an unbounded amount of time, and real time code's main concern is hard limit on the time in the worst case scenario. (by the same logic if realtime code uses quicksort on arbitrary data it must assume its worst case O(n^2) performance, or switch to a true O(log(n)) worst case performance algorithm. swift "POD" structs only consisting of other POD structs and simple fixed types like bool / int / double / etc (but not dynamic types like string, data / url / etc) are realtime safe, and in swift we don't have the equivalent of C arrays, any algorithm that needs arrays would have to emulate them somehow (e.g. using preallocated memory + memmove).

Given an enumeration such as MPMusicShuffleMode, which has 4 cases, what's the best way *cycle* through them?
#80

Interesting, these are some of the constraints required to run Swift in a GPU compute kernel.

3 Likes
#81

Interesting spot. I hope I can write Metal kernels in Swift with autodiff soon...

5 Likes