'swift test' sometimes fails with 'error: malformed'

clackary · August 21, 2023, 4:42pm

One of our main packages is consistently failing with a vague error: malformed message when doing anything with swift test (i.e. list, --filter, --skip). It completes building/linking successfully, but fails immediately after.

[7053/7053] Linking FooPackageTests.xctest
Build complete! (351.75s)
error: malformed

I found an existing issue that seems somewhat similar to what we're experiencing, but our package doesn't contain LinuxMain.swift directly (some dependencies do contain a LinuxMain.swift).

I'm looking for a way to get more information about the error. Where is it coming from? What exactly is malformed?

clackary · August 21, 2023, 4:47pm

I'm working on trimming down our package to a minimal reproducer, but poking around in sources a bit, I was able to find this piece in apple/swift...

github.com

apple/swift/blob/3113e83c4ec26513c38f772ddb97a243e63e8e92/include/swift/Serialization/Validation.h#L100


      
          struct ValidationInfo {
            StringRef name = {};
            StringRef targetTriple = {};
            StringRef shortVersion = {};
            StringRef miscVersion = {};
            version::Version compatibilityVersion = {};
            llvm::VersionTuple userModuleVersion;
            StringRef sdkName = {};
            StringRef problematicRevision = {};
            size_t bytes = 0;
            Status status = Status::Malformed;
            std::vector<StringRef> allowableClients;
          };
          
          /// A collection of options that can be used to set up a new AST context
          /// before it has been created.
          ///
          /// Note that this is intended to be a transient data structure; as such,
          /// <strong>none of the string values added to it are copied</strong>.
          ///
          /// \sa validateSerializedAST()

Does swiftpm attempt to validate a serialized AST somewhere in the swift test process?

Finagolfin · August 23, 2023, 2:28pm

I've seen this cryptic error before, which as you note, comes from the Swift compiler. Try running the tests in verbose mode by adding the -v flag, that will give you more info.

SwiftPM should be modified to never pass along that cryptic compiler error message alone. Right, @NeoNacho?

NeoNacho · August 23, 2023, 2:54pm

Agreed, I am not quite sure when specifically it happens. From the output it looks like it happens during test discovery maybe?

clackary · August 23, 2023, 4:59pm

Try running the tests in verbose mode by adding the -v flag, that will give you more info.

We get some absurdly long data back, but nothing that looks particularly interesting, at least to my eyes. Guessing that we'll benefit a lot from a reduced package. I think we're pretty close to a minimally reproducing package (big thanks to my colleague @fibrechannelscsi). I'll either update the post or hijack that issue when we have a bit more info.

happens during test discovery maybe?

That would be my guess as well. I've been able to do swift build without issue, but anything swift test seems to trip up on this. I should also mention that we're reproducing via the 07-10 toolchain and on Linux, so I'll include a bit more detail there as well once we have it.

NeoNacho · August 23, 2023, 8:06pm

One very low-tech idea for getting more info:

Create a script that does something like this:

echo "$@" >> /tmp/log.txt
swiftc "$@"

Run SWIFT_EXEC=/path/to/script swift test
The log file should contain info for all swiftc CLI invocations, presumably the last one is the one that errors.

clackary · August 24, 2023, 10:58pm

Thanks for the suggestion @NeoNacho! A new bug has been filed with a minimal reproducer, as well as a log with the output from the low-tech script: An error message is generated in certain cases when running `swift test --filter`. · Issue #6856 · apple/swift-package-manager · GitHub