I'd like to add some observations about the timeline of when AutoDiff might be approved for Swift Evolution.
- Correct me if I'm mistaken, but AutoDiff was first revealed in 2018. The YouTube video introducing Swift for TensorFlow was posted then.
- AutoDiff has been in progress for 4 years (2018 - 2022), giving it the longest development time of any feature in the Swift compiler.
- It seems that it's going to take years to get Differentiable programming for gradient-based machine learning through Swift Evolution and officially part of Swift release toolchains.
- The proposal linked above is only for bare-bones differentiation. Differentiability of Standard Library types will be an entirely separate proposal, and we have a long way to go on that front. I don't mean there hasn't been much effort there already. But imagine developing a derivative for almost every API-public function in the Swift Standard Library (or even 10% of it). That task has such a massive scope and has been barely discussed.
Based on past precedents, we have every reason to assume AutoDiff will be perpetually restricted to development toolchains. I hope that's not the case. But PassiveLogic is now the only primary contributor to AutoDiff, with much less people than Google's S4TF team. I don't see myself making major contributions to AutoDiff anymore, unless it blocks S4TF from compiling. If a new person in the Swift community came along and contributed, it would really speed things up. Even if you have little or no experience with the C++ side of the Swift compiler (like me a few months ago), there are areas to help. I would even volunteer a great deal of time to guide anyone through getting started with contributing to AutoDiff.
Recently, I have tried to use AutoDiff in Swift Playgrounds. With the recent activity around AutoDiff in the Swift compiler, Swift 5.7 should be the first time S4TF can compile on a release toolchain (using philipturner/differentiation to do so). However, my repo uses a compiler flag that's forbidden on Swift Playgrounds: -parse-stdlib. I tried every workaround possible, but this is a rare situation where I can't bypass a restriction on technology. If you own an iPad, you can't run libraries depending on AutoDiff natively unless there's a Mac tethered to it. In the link below, I explain the problem in much greater detail.
https://github.com/vojtamolda/tensor-thumbnail/issues/2#issuecomment-1144289998
Previously, most users of AutoDiff could get away with it being experimental. That includes Google with the original S4TF, PassiveLogic, me with the new S4TF (mostly), and others in the community. But Swift Playgrounds is the first time I encountered a use case where the only option is getting AutoDiff all the way through Swift Evolution. Does anyone else have a use case that requires AutoDiff no longer being an "experimental" feature?