Many tests failing when I build from head `8ae3c9ed005bee1b257269f60bce02c75d0b3486`?

Development Compiler
1

Sorry if I'm being stupid here. I synched up my swift directory with GitHub - apple/swift: The Swift Programming Language main and tried to do a naive build, using utils/update-checkout to get all the relevant dependencies then tried build and test with utils/build-script --clean -R --skip-ios --skip-watchos --skip-tvos -t.

It builds cleanly but a lot of tests fail (400 or so). Is that expected or am I doing something wrong? Many of them error with <unknown>:0: error: unexpected remark produced: unable to perform implicit import of "_StringProcessing" module: no such module found

macOS is Sonoma (14.1.2), Xcode is 15.0.1, ninja and cmake are installed. I did a spot check of dependencies (python3, cmake, ninja) as suggested in the help and everything looks fine.

I assumed that if I just pulled the latest main (last night) it would have all tests passing, but maybe that's not always true? Happy to go back to a snapshot if that's better?

(I'm trying to get a green slate of tests so I can check for regressions on the patch I'm submitting and so I can add new tests for it.)

Cheers,
Carl

2 
Testing Time: 802.22s
  Unsupported      :  572
  Passed           : 9451
  Expectedly Failed:   34
  Failed           :  470
3

Ideally to get tests in the exactly same state passing as on CI, you have to build the exactly same preset that a CI job matching your machine uses. IIRC utils/build-script --clean -R --skip-ios --skip-watchos --skip-tvos -t doesn't match any of the existing presets.

4

Ok cool. Where do I find them?

5

I suspect there’s some kind of cmake configuration issue where it cannot find the swift-experimental-string-processing repo. Is it checked out alongside swift?

The tests usually pass on main because of presubmit PR testing. Occasionally something breaks but I assume in this case it’s a local issue.

6

Yep! That's it. I was using a dodgy version of the update-checkouts command line...

utils/update-checkout --clone-with-ssh --skip-repository swift-nio \
--skip-repository swift-nio-ssh --skip-repository swift-lmdb --skip-repository swift-docc \
--skip-repository swift-docc-render-artifact --skip-repository swift-docc-symbolkit \
--skip-repository swift-markdown --skip-repository swift-experimental-string-processing \
--skip-repository swift-llvm-bindings --skip-repository swift-xcode-playground-support \
--skip-repository swift-corelibs-libdispatch --skip-repository swift-corelibs-foundation \
--skip-repository swift-corelibs-xctest --skip-repository swift-stress-tester \
--skip-repository swift-crypto --skip-repository swift-atomics \
--skip-repository swift-nio-ssl --skip-repository sourcekit-lsp \
--skip-repository indexstore-db --skip-repository swiftpm \
--skip-repository swift-numerics --skip-repository swift

I'm trying to run the script again like this...

utils/update-checkout --clone-with-ssh --skip-repository swift-nio \
--skip-repository swift-nio-ssh --skip-repository swift-lmdb --skip-repository swift-docc \
--skip-repository swift-docc-render-artifact --skip-repository swift-docc-symbolkit \
--skip-repository swift-markdown \
--skip-repository swift-llvm-bindings --skip-repository swift-xcode-playground-support \
--skip-repository swift-corelibs-libdispatch --skip-repository swift-corelibs-foundation \
--skip-repository swift-corelibs-xctest --skip-repository swift-stress-tester \
--skip-repository swift-crypto \
--skip-repository swift-nio-ssl --skip-repository sourcekit-lsp

...we'll see if that gives a working build. I don't guess I need any of those packages to get the swift tests passing, swift NIO, corelibs-foundation, etc.?

7

No, you don't need those, the tests invoked by the -t option of build-script run on the Swift stdlib. Swift NIO, corelibs-foundation etc are built after the stdlib compilation steps and its test run have completed.

1 Like
8

That got rid of most test failures. It just left...

Failed Tests (2):
  Swift(macosx-arm64) :: ClangImporter/SceneKit_test.swift
  Swift(macosx-arm64) :: Python/python_lint.swift

Neither of which worry me.

Thank you guys. I can work with this and I'll just ignore these two test fails for the purposes of preparing my patch.

FYI The SceneKit failure seems based around...

/Users/carl/Documents/Code/swift/swift/test/ClangImporter/SceneKit_test.swift:6:8: error: unexpected error produced: failed to build module 'SceneKit'; this SDK is not supported by the compiler (the SDK is built with 'Apple Swift version 5.9 (swiftlang-5.9.0.123.203 clang-1500.0.37.1)', while this compiler is 'Swift version 5.11-dev (LLVM e2b71339d3295a4, Swift 8ae3c9ed005bee1)'). Please select a toolchain which matches the SDK.
import SceneKit

So then expected errors aren't produced.

And python lint is expecting flake and flake8 to be installed...

The flake8 and flake8-import-order Python packages are required for linting,
but these were not found on your system.
1 Like