Requesting help with stdlib testing

Hi all,
I am trying to make a change to the stdlib and test it as well.
But I am stuck with the whole lit.py system and getting output from my test.

I am following the guide at:


Using the latest toolchain to build the stdlib and start the tests.
I am running the suggested command:
./swift/utils/build-script --preset=$PRESET_NAME toolchain_path=$TOOLCHAIN_BIN_DIR

Where my preset is a debug build that includes tests.

It works perfectly, but I can't find the output of the tests - other than the 'FAIL: ...' message. I would like to know what actually failed...
Also I would like to only run one single test and have tried setting the environment variable LIT_FILTER=stdlib/CodableTests.swift, but the build script command above still runs all stdlib tests.

I have also tried running lit.py manually as such:

llvm-project/llvm/utils/lit/lit.py -sv build/stdlib_DA_standalone/swift-macosx-x86_64/test-macosx-x86_64/stdlib

But then all the tests fail to find the StdlibUnittest library.

I am hoping that someone could point me in the correct direction for where to look for the test output. :-)

Have you tried using ./swift/utils/run-test? You can pass the path to one or more tests to run only those tests.

I've never seen that QuickStart contribution page before.

Do you know where it's linked from, and who's responsible for updating it?

It contains some outdated information, such as:

  • the macOS 10.14.4 and Xcode 11 beta 1 requirements,
  • the 2019-06-06 snapshot,
  • the CMake and Sccache versions.

Should the page link to the new How-To Guides instead?

2 Likes

Hi @nnnnnnnn and @benrimmington,

Thanks for your replies!
The ./swift/utils/run-test gives me the same issue as running lit.py manually. Namely that the imported library StdlibUnitTest can't be found.
I am running it as follows:

./swift/utils/run-test swift/test/stdlib/ --build-dir=./build/stdlib_DA_standalone/swift-macosx-x86_64/ --build=skip

I'm sure it can be fixed by adding environmen or parameters for pointing to the library:

find . -name "StdlibUnittest.*"
./swift/stdlib/private/StdlibUnittest/StdlibUnittest.swift
./build/stdlib_DA_standalone/swift-macosx-x86_64/lib/swift/macosx/StdlibUnittest.swiftmodule
./build/stdlib_DA_standalone/swift-macosx-x86_64/lib/swift/macosx/StdlibUnittest.swiftmodule/StdlibUnittest.opt.bitstream
./build/stdlib_DA_standalone/swift-macosx-x86_64/stdlib/private/StdlibUnittest/OSX/x86_64/StdlibUnittest.o
./build/stdlib_DA_standalone/swift-macosx-x86_64/stdlib/private/StdlibUnittest/OSX/x86_64/StdlibUnittest.opt.bitstream

I'm just not certain how to specify the path.

@benrimmington: No, I don't know where it is linked from - I found it by googling the specific thing that I was trying to achieve: Building (and testing) stdlib using the latest development toolchain so that I don't have to compile the entire toolchain again. I can't find a similar guide in the documentation in the swift repo.
I am of course using the latest development toolchain and not the one referred to in the guide. :-) And besides from me not being able to find the test output - the guide did work out well for me.

Sincerely,
/morten

@Varun_Gandhi you previously worked on How-To Guides, do you know if it would be possible to update the page on swift.org as well?

1 Like

The full test output is dumped to an XML file in the build directory, something like build/Ninja-Release/swift-android-aarch64/swift-test-results/aarch64-unknown-linux-android/lit-tests.xml, which is what I get when running the compiler testsuite natively on Android. As for running a single stdlib test, you could try modifying the filter line in the preset to be more specific.

1 Like

I believe it was prepared for last year’s Try Swift San Jose event. @Michael_Gottesman prepared a new standard library development workflow for that event; I’m not sure if things have changed since. Michael?

1 Like

Yes, we have an Apple-internal process for making changes to the website. Fixing this particular issue has been on my Todo list but I've been occupied with some other work so I haven't gotten around to it yet...

Terms of Service

Privacy Policy

Cookie Policy