What can be done to slim them down further?
I don't think all of those will be used by your geometry library, for example, I'm not sure where those libcrypto.so and libssl.so dependencies are coming from. I believe libswiftSwiftOnoneSupport.so is only needed when compiling with -Onone and libswiftRemoteMirror.so for runtime reflection or something. I doubt your geometry library is calling FoundationNetworking and probably not FoundationXML either, which means you can remove those and libcurl.so/libxml2.so also.
Libicudata is the biggest hog, there was talk a couple years ago of slimming it down. You may want to explore that.
Using the Android-embedded ICU somehow instead of these libs is the clear #1 option, but a bit out of my league.
There is only a Java interface for the Android system ICU, not a native one. Google says not to link against these system libraries as they will vary by system. Since there is no support in the NDK for linking against the system libicu, they say you should supply your own and that you should not link native code against system libraries (couldn't find the link where they said this).
Any other thoughts to reduce their size?
You could try static linking with -static-stdlib and then try running some tool to strip out unused code. If you want to stick with shared libraries, there may be some tool that audits the function dependency graph and will help you similarly strip out unused functions (I don't know of any such tool, never looked for it).