I got automatic differentiation working on iOS and made an AR sample project with it. Doing this requires a workaround in the 5.5.1 release toolchain, where an imported package (Differentiation) includes files from the Swift Standard Library. Compiler crashes and some missing features prevented me from differentiating optionals and SIMD vectors while creating the project.
import Differentiation
let cube = cubes[...]
@differentiable(reverse)
func getLocationY(t: Float) -> Float {
let acceleration: SIMD3<Float> = [0, -9.8, 0]
let a_contribution = acceleration.y / 2 * t * t
return a_contribution + cube.velocity0.y * t + cube.location0.y
}
let dydt = gradient(at: cube.timeSinceCollision, of: getLocationY(t:))
let velocityText = String(format: "%.1f", dydt)
let messageText = "Velocity (Y): \(velocityText) m/s"
This idea was suggested by @fan, and we are discussing a possible resurrection of Swift for TensorFlow. In order for that to happen, autodiff needs to be completely stable in the compiler. Along with iOS/tvOS/watchOS support, I plan to bring a Metal GPU backend to S4TF (or a spinoff project) and possibly a DirectX backend if time allows for it. The estimated completion time for this is summer 2022, before the release of Swift 5.6.
Just FYI, Swift 5.6 will likely be this spring's release, similar to 5.4 in March of 2021. The summer's release will likely be Swift 6, or 5.7 if they decide to hold it back. Of course, this is just speculation based on Apple's past releases, but it's been accurate for several years now, and it's not like you'll hear anything from them anyway.
@Jon_Shier if this coincides with both the completion of C++ interop (see my idea) and the resurrection of S4TF and also a massive DirectX project with Swift on Windows, Swift 6 could very well be summer or fall 2022.
If Swift 6 isn't the summer of '22 release (announced in June at WWDC, released in September with some set of new OSes) then, if Apple follows the pattern, the next release would be spring of '23, if they want to release such a major version alongside a minor OS release.
I agree. It makes more sense to announce Swift 6 at a WWDC event, and June is pushing it. I also think we should have good Android support, and other people in the community working on S4TF 24/7 after I am no longer able to do that. I think summer 2023 is a more realistic date.
@fan this will require some pull requests to swift-apis, as the reason it won’t compile in Xcode 13 is some types not conforming to Sendable (not because of compiler segfaults). Thus, we have to agree to pull swift-apis out of archive mode to build it on main.
In addition, we have to restructure swift-apis so it can run as an independent package. It will be intractable to have to compile and download a new Swift toolchain each time we want to debug it.
@compnerd@rxwei@BradLarson I’m mentioning the former S4TF team because it is vital that the message above reaches you. I also made a PR to swift-apis about the situation with autodiff, which should commence the resurrection of S4TF. We can keep tensorflow/swift as it is for the moment.
I would love to hear something from the Core Team about the effort @philipturner is putting in autodiff. Is this something that is still worth pursuing? This silence doesn't give me too much hope, unfortunately.
We are still working on autodiff. There is someone besides me who already works on fixing the bugs full time, but acquiring the skill to do so would require a lot of time for me. I figured that it would be better to focus on Colab support right now with vanilla Swift. When I absolutely need to get S4TF building to start the Metal backend, then I’ll need to fix whatever autodiff bugs prevent it from building.
Autodiff is amazing and has very useful applications outside of S4TF (e.g. biomedicine). I wish I could devote more time to fixing it, but I have to sacrifice some stuff for my main goal of resurrecting S4TF and meeting the time deadline of synchronizing release of Metal support with PyTorch. Also, I’m a high schooler while still doing this, so I’m not working full time and have a massive time constraint.
