I can't speak much for linux, or even Android cross-compilation, as my work has been to get a Swift toolchain natively running on an Android device. The goal is to get it to the point where you can just type pkg install swift and start writing Swift packages on your Android tablet, but for now you will have to build the Swift toolchain from source, with a few simple steps. The advantage of getting everything running natively is that I can run all the tests easily and I reported last month that I was down to only 16 test failures across all the tests from the Swift compiler validation suite up through the SwiftPM and sourcekit-lsp tests.
Since then, the last test fix for Foundation has been merged, so the only failures are 11 in Swift validation, 1 in XCTest, and 3 in SwiftPM, largely inconsequential issues like some validation tests not having RUNPATH set like Android requires or some LLVM sanitizers not working on Android. Of course, I'm guessing the tests don't cover everything and there is some functionality that hasn't been implemented for Android, such as Bundle support (ironically, @pvieito, who started this thread a couple years ago, tried to fix it and then abandoned it), but I haven't looked into how much of that there is.
I tried building and testing some Swift packages on an Android device, such as SwiftyJSON, the most-starred Swift package on github that advertises linux support. While other packages worked great, with swift build; swift test just working and passing all their tests, SwiftyJSON has been broken on linux itself with Swift 5.1. It's a simple fix though, and with a couple other tweaks for the lack of Bundle support, I was able to get the same test results on both linux x86_64 and Android aarch64, around 5 test failures, will probably submit a pull with those tweaks. Another package that had problems was swift-nio, which supposedly worked at some point for cross-compiling to Android, but will require some patching to build natively.
I have not tried building an Android app like the flowkey guys or writing much of anything myself with this new Swift toolchain on Android, so I can only report test results. Jeremy has another perspective just above.
I think it might be possible to set up a native Android CI now using the new Anbox Cloud service I mentioned in my post last month. I'll reach out to them and see how it works.