Regarding .o files: XCFrameworks do not currently support .o files, but they do support static archives (which are really just ar archives containing .o files). But static archives have library semantics, meaning that the linker will only use those .o files that provide the symbols needed to resolve unresolved symbol references from other .o files. This means that if the package code contains, for example, a class whose name is only mentioned in a storyboard or xib file, or is in some other way only looked up at runtime, then that code won't be linked into the product.
In the short term it might be possible to convince Xcode to use the -force_load flag when linking against the library, so that the linker bypasses the usual static library semantics and includes all the .o files, but in the long run it seems cleaner to lobby for an extension of the XCFrameworks format to also allow it to contain .o files (with non-library semantics, i.e. that all the .o files are always linked in).
It's a bit unfortunate to have different formats for the binary artifacts on different platforms, but perhaps that will be inevitable anyway as support for new platforms are added.