I need to calculate some complex derivatives for the app I am building. Figuring out how to do these calculations manually is beyond my capabilities.
Recently, I discovered Swift’s AutoDiff library. With some playing around and a few customizations, I succeeded in getting it to do what I needed. Awesome!
AutoDiff is not yet released - available in a toolchain. I am unfamiliar with tool chains or the open source of swift and so I have a few questions.
Will apps using toolchains be able to pass Apple’s review process when releasing?
If no, is it possible/easy enough to extract and port a feature branch from a toolchain into my project/workspace? Or is there some other approach I can take to still utilize AutoDiff?
Are there any public timelines for when to expect future tool chains, or features within, to be officially included in a version of Swift?
Failing all of the above, and returning to my main purpose - are there any other autodiff libraries available for Swift? Or any other suggestions for how to proceed with figuring out some difficult derivative calculations?
Will apps using toolchains be able to pass Apple’s review process when releasing?
No, Xcode will refuse to upload them or App Store Connect will fail their validation. Only Xcode's toolchain can ship apps to the stores.
If no, is it possible/easy enough to extract and port a feature branch from a toolchain into my project/workspace? Or is there some other approach I can take to still utilize AutoDiff?
Given that autodiff is a whole language feature it seems unlikely there's anything to extract into your project. You'd want an external library instead.
Are there any public timelines for when to expect future tool chains, or features within, to be officially included in a version of Swift?
No, Apple doesn't talk timelines. Given the end of most projects using the autodiff feature (despite the recent maintenance by @philipturner) the future of the feature is very uncertain. Personally, I think it's unlikely the feature will ever be publicly available in Apple's toolchains unless Apple has a specific use for it. In general, they have an unfortunate habit of turning off compiler features in their builds of compilers, such as OpenMP support in their version of clang, so I'm not too hopeful.
For the autodiff work in my group I have considered (but not tried yet) the following approach for potentially getting apps that use autodiff into the app store:
Use the toolchain that has autodiff to build a library that exports functions via a C interface (using @_cdecl). Then build the remaining app with the official toolchain and link in the library as you would do with any other library that has a C interface.
Please, correct me if there is some flaw in that strategy.
If you come across my Differentiation package that enables AutoDiff in release toolchains, note that Apple will almost certainly not permit it on the App Store. Plus, the code there is already obsolete with modifications recently made to the Standard Library, such as support for tgmath differentiation of Double-precision numbers.
@Jon_Shier I am trying to get AutoDiff to the point where Apple will put it the next minor release toolchain. I need to do this so that I can test out new AutoDiff modifications on Swift-Colab and other projects. If not by WWDC, then I need it to happen by Swift 5.8. As you mentioned, this is just my timeline and not Apple's.
Oh, I was under the impression that Apple didn't include the _Differentiation module in Xcode toolchains. If it's there then I'm not quite as certain as I was originally that they wouldn't let you ship it in an app. As long as it's not considered private symbols by the App Store Connect scanner, there's probably no issue. Someone would have to try to be sure.
They don't ship the _Differentiation module. That's what my Differentiation package solves - it copies the entire _Differentiation module from the standard library. It has to declare a module named _Differentiation inside an encapsulating package, which I just name Differentiation (without the underscore). The reason it works it that AutoDiff is built into the compiler, even if it's omitted from the standard library.
Specifically, all you need is a few functions in the Builtin module that map to the proper SIL instructions. I'm pretty sure that's abstracted away and invisible in the final assembly language.
Personally, I think that it might actually slip by Apple's reviewers given its peculiarity. But, I STRONGLY discourage doing so because it's likely a violation of legal agreements you sign as an Apple developer. Although I'm free of liability because I license my Differentiation package under MIT, I do not intend for it to be used that way.
Thank you for making this, @philipturner. I hope you find success getting it into one of the future release toolchains. At the very least, getting a clearer picture of what is happening for solving the derivatives gives me some hope of maybe piecing together the formula, in the meantime. Thanks again!
I spent two months learning knowledge once just so I could find the derivative of the process of solving a system of equations. You can see the final product in my MultiPendulum repository. I definitely understand how difficult it is to create your own math formula.
I thought I would update everyone that I overcame the hurdle about a month ago. For anybody in a similar predicament in the future, the Python library SymPy was a godsend and helped take care of some heavy lifting. There was still a lot of playing around and getting creative with things on my part and then doing a final simplification of the algorithm, but I wouldn't have gotten past this without its help.