Compilation speed: help test batch mode!

Hi,

The Swift compiler has recently seen some changes to its work-processing model to improve compile times in debug configuration, and in particular eliminate the tradeoff between the standard and whole-module compilation modes.

Currently only standard (single-file) mode makes adequate use of multiple cores, or supports incremental recompilation; yet whole-module mode can often be faster, even on a single core, since it does less repeated work overall. Ideally, this tradeoff would not exist.

To this end, the Swift compiler has acquired a new mode called "batch" or "multiple file" mode. This new mode takes the set of files that would normally be (incrementally) compiled with individual subprocesses, and assigns them dynamically to multiple batches (one per core), compiling each batch with a single process. This means that incremental mode is still operative, and compilation makes use of multiple cores, but the benefits of whole-module mode (fewer processes, less redundant work) also apply.

In short, the new mode should be faster than either existing mode, while keeping the benefits of both.

Batch mode appears to be working on the codebases it has been tested against so far, but it could use more testing against other codebases, especially large projects. It should provide a greater margin of speedup on larger modules, so modules with hundreds or even thousands of files are especially interesting to test against.

If you're feeling adventurous and want to give it a try, here are some instructions. Please read them carefully, and feel free to ask questions if anything is confusing:

  1. Download this "no assertions" development snapshot compiler from swift.org.

  2. Install and activate the compiler. Please make sure you activate the 2018-03-19 "no assertions" compiler:

    • On macOS this involves double-clicking the .pkg file you downloaded, completing the installer, and then in Xcode, selecting the menu item Xcode => Toolchains => Swift Development Snapshot (no assertions + LTO) 2018-05-02 (no assertions + LTO) to activate it.
    • On Linux this involves extracting the .tar.gz file you downloaded and adding the swift-DEVELOPMENT-SNAPSHOT-.../usr/bin directory within it to your PATH environment variable, ahead of any other current Swift installation you have.
  3. Build your project once in debug configuration, with the snapshot compiler, to ensure that it works "normally", without turning on batch mode.

  4. If present, remove any -whole-module-optimization or -wmo arguments from your project build flags.

  5. Clean and rebuild your project, passing the flag -enable-batch-mode to enable batch mode.

    • On macOS, this can be done by adding or adjusting the build setting OTHER_SWIFT_FLAGS to have the value -enable-batch-mode
    • On Linux, this can be done by running swift build with the extra arguments -Xswiftc -enable-batch-mode
  6. If you're willing, please post back to this thread the following information:

    • The size of your project, in terms of files and/or number of lines of Swift code.
    • The number of cores on your build machine.
    • Whether batch mode worked for you (and if it failed, whether you'd be willing to file a bug describing how).
    • The time taken to build your project in each of 3 modes on the snapshot compiler: standard (single-file) mode, whole-module mode (if you use it), and batch mode.

If all goes well, you should see your compilation running a smaller number of processes (about one per core, for each module) and compiling faster with batch mode enabled. If you see dozens or hundreds of processes compiling individual Swift files, or only a single whole-module process building the entire module, you may not have configured the flags correctly.

Please also note that an earlier version of this post described using standard development snapshot compilers that have assertions enabled; these snapshots were more prone to crashing and had very different performance characteristics to release compilers. Please use the "no assertions" snapshot linked above.

Thanks!

16 Likes

I tested this on SwiftPM using the generated Xcode project (Utilities/bootstrap --generate-xcodeproj):

cores: 4
single-file: 27.247s
batch-mode: 25.472s
wmo: 17.695s

Interesting! On that codebase at 4-way concurrency I get 31s single-file, 25s batch, 26s wmo (not a big win since most of the modules are quite small, but still). I wonder what the differences are between our environments! Can you re-run it a few times to be sure?

You can find a list of open source iOS apps here I hope it helps.

I am trying to run it against my company app I will report soon.

It does not compile on my normally working project. Can I disable assertion ?

Hi,

I get the following error, but i don't know if it's related with this compilation mode or because assertions are enabled.

Note, i get this error in a module without the -enable-batch-mode flag

Assertion failed: (newSig->getCanonicalSignature() == getCanonicalSignature() && "signature differencing removed too many requirements"), function requirementsNotSatisfiedBy, file /Users/buildnode/jenkins/workspace/oss-swift-package-osx/swift/lib/AST/GenericSignature.cpp, line 838.
0  swift                    0x00000001062347a8 llvm::sys::PrintStackTrace(llvm::raw_ostream&) + 40
1  swift                    0x0000000106234eb6 SignalHandler(int) + 694
2  libsystem_platform.dylib 0x00007fff63e13f5a _sigtramp + 26
3  libsystem_platform.dylib 0x00007fc04d825992 _sigtramp + 3919649362
4  libsystem_c.dylib        0x00007fff63c3e312 abort + 127
5  libsystem_c.dylib        0x00007fff63c06368 basename_r + 0
6  swift                    0x0000000103c6b51a swift::GenericSignature::requirementsNotSatisfiedBy(swift::GenericSignature*) + 858
7  swift                    0x0000000103bcd2be swift::Mangle::ASTMangler::appendGenericSignature(swift::GenericSignature const*, swift::GenericSignature*) + 302
8  swift                    0x0000000103bcb422 swift::Mangle::ASTMangler::appendContext(swift::DeclContext const*) + 1474
9  swift                    0x0000000103bca843 swift::Mangle::ASTMangler::appendEntity(swift::ValueDecl const*) + 179
10 swift                    0x0000000103bca700 swift::Mangle::ASTMangler::mangleEntity(swift::ValueDecl const*, bool, swift::Mangle::ASTMangler::SymbolKind) + 48
11 swift                    0x0000000103712995 swift::SILDeclRef::mangle(swift::SILDeclRef::ManglingKind) const + 1845
12 swift                    0x00000001037629d7 swift::SILModule::getOrCreateFunction(swift::SILLocation, swift::SILDeclRef, swift::ForDefinition_t, swift::ProfileCounter) + 55
13 swift                    0x00000001031af959 swift::Lowering::SILGenModule::getFunction(swift::SILDeclRef, swift::ForDefinition_t) + 329
14 swift                    0x00000001032872e7 swift::Lowering::SILGenFunction::emitGlobalFunctionRef(swift::SILLocation, swift::SILDeclRef, swift::Lowering::SILConstantInfo) + 391
15 swift                    0x00000001031c1a29 (anonymous namespace)::Callee::getFnValue(swift::Lowering::SILGenFunction&, bool, llvm::Optional<swift::Lowering::ManagedValue>) const & + 1321
16 swift                    0x00000001031be203 (anonymous namespace)::CallEmission::apply(swift::Lowering::SGFContext) + 3475
17 swift                    0x00000001031bcdb3 swift::Lowering::SILGenFunction::emitApplyExpr(swift::Expr*, swift::Lowering::SGFContext) + 51
18 swift                    0x0000000103221af8 swift::ASTVisitor<(anonymous namespace)::RValueEmitter, swift::Lowering::RValue, void, void, void, void, void, swift::Lowering::SGFContext>::visit(swift::Expr*, swift::Lowering::SGFContext) + 88
19 swift                    0x00000001032116e3 swift::Lowering::SILGenFunction::emitRValue(swift::Expr*, swift::Lowering::SGFContext) + 35
20 swift                    0x0000000103283ca5 swift::Lowering::SILGenFunction::emitReturnExpr(swift::SILLocation, swift::Expr*) + 469
21 swift                    0x0000000103280ea2 swift::ASTVisitor<(anonymous namespace)::StmtEmitter, void, void, void, void, void, void>::visit(swift::Stmt*) + 4274
22 swift                    0x0000000103281fa8 swift::ASTVisitor<(anonymous namespace)::StmtEmitter, void, void, void, void, void, void>::visit(swift::Stmt*) + 8632
23 swift                    0x000000010327fde5 swift::Lowering::SILGenFunction::emitStmt(swift::Stmt*) + 21
24 swift                    0x00000001032398dd swift::Lowering::SILGenFunction::emitFunction(swift::FuncDecl*) + 509
25 swift                    0x00000001031b82e7 swift::Lowering::SILGenModule::emitFunction(swift::FuncDecl*)::$_1::operator()(swift::SILFunction*) const + 311
26 swift                    0x00000001031b02ed swift::Lowering::SILGenModule::emitFunction(swift::FuncDecl*) + 733
27 swift                    0x00000001031b529b swift::Lowering::SILGenModule::emitSourceFile(swift::SourceFile*, unsigned int) + 811
28 swift                    0x00000001031b6050 swift::SILModule::constructSIL(swift::ModuleDecl*, swift::SILOptions&, swift::FileUnit*, llvm::Optional<unsigned int>, bool) + 352
29 swift                    0x00000001031b663f swift::performSILGeneration(swift::FileUnit&, swift::SILOptions&, llvm::Optional<unsigned int>) + 95
30 swift                    0x0000000102aa34e6 performCompile(swift::CompilerInstance&, swift::CompilerInvocation&, llvm::ArrayRef<char const*>, int&, swift::FrontendObserver*, swift::UnifiedStatsReporter*) + 8598
31 swift                    0x0000000102aa033e swift::performFrontend(llvm::ArrayRef<char const*>, char const*, void*, swift::FrontendObserver*) + 3310
32 swift                    0x0000000102a56e93 main + 2051
33 libdyld.dylib            0x00007fff63b92115 start + 1

It looks like that assertion was removed recently, so you might want to try a new snapshot in a couple of days.

Totally unscientific benchmarks

384.655s Clean Build
297.418s Clean batch mode (enabled only on app target)
Failed Clean WMO (enable only on app target)
303.012s Clean, batch mode on all dependencies <- this is unexpected

System: Macbook Pro 2016 13", 3,1 GHz Intel Core i5, 16 Go 2133 MHz LPDDR3

Project metrics:

find . -name *.swift -exec cat '{}' + | grep -v -e '^$' | wc -l

49kloc *.swift
258kloc *.m (mostly generated code)

For folks just tuning in: we ought to have a downloadable no-assertions compiler to try soon, if you're running into too many errors while testing the assertion-enabled snapshots linked above.

I just tried the 3-14 snapshot with -enable-batch-mode set and ran into an error early on building a 3rd-party lib. At PlanGrid, we use this library here: GitHub - mikaoj/BSGridCollectionViewLayout: A simple grid collection view layout, which failed to build. Here's the error output:

Ld /Users/basher/Library/Developer/Xcode/DerivedData/PlanGrid-    gqlncmhpixhqencipfdoevzawlxo/Build/Intermediates.noindex/Pods.build/Debug-iphoneos/BSGridCollectionViewLayout.build/Objects-normal/armv7/BSGridCollectionViewLayout normal armv7
        cd /Users/basher/Code/plangrid/Pods
        export IPHONEOS_DEPLOYMENT_TARGET=9.0
        /Applications/Xcode-beta.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -arch armv7 -dynamiclib -isysroot /Applications/Xcode-beta.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS11.3.sdk -L/Users/basher/Library/Developer/Xcode/DerivedData/PlanGrid-gqlncmhpixhqencipfdoevzawlxo/Build/Products/Debug-iphoneos/BSGridCollectionViewLayout -F/Users/basher/Library/Developer/Xcode/DerivedData/PlanGrid-gqlncmhpixhqencipfdoevzawlxo/Build/Products/Debug-iphoneos/BSGridCollectionViewLayout -filelist /Users/basher/Library/Developer/Xcode/DerivedData/PlanGrid-gqlncmhpixhqencipfdoevzawlxo/Build/Intermediates.noindex/Pods.build/Debug-iphoneos/BSGridCollectionViewLayout.build/Objects-normal/armv7/BSGridCollectionViewLayout.LinkFileList -install_name @rpath/BSGridCollectionViewLayout.framework/BSGridCollectionViewLayout -Xlinker -rpath -Xlinker @executable_path/Frameworks -Xlinker -rpath -Xlinker @loader_path/Frameworks -miphoneos-version-min=9.0 -dead_strip -Xlinker -object_path_lto -Xlinker /Users/basher/Library/Developer/Xcode/DerivedData/PlanGrid-gqlncmhpixhqencipfdoevzawlxo/Build/Intermediates.noindex/Pods.build/Debug-iphoneos/BSGridCollectionViewLayout.build/Objects-normal/armv7/BSGridCollectionViewLayout_lto.o -Xlinker -export_dynamic -Xlinker -no_deduplicate -fembed-bitcode-marker -fobjc-arc -fobjc-link-runtime -L/Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2018-03-14-a.xctoolchain/usr/lib/swift/iphoneos -Xlinker -add_ast_path -Xlinker /Users/basher/Library/Developer/Xcode/DerivedData/PlanGrid-gqlncmhpixhqencipfdoevzawlxo/Build/Intermediates.noindex/Pods.build/Debug-iphoneos/BSGridCollectionViewLayout.build/Objects-normal/armv7/BSGridCollectionViewLayout.swiftmodule -framework Foundation -compatibility_version 1 -current_version 1 -Xlinker -dependency_info -Xlinker /Users/basher/Library/Developer/Xcode/DerivedData/PlanGrid-gqlncmhpixhqencipfdoevzawlxo/Build/Intermediates.noindex/Pods.build/Debug-iphoneos/BSGridCollectionViewLayout.build/Objects-normal/armv7/BSGridCollectionViewLayout_dependency_info.dat -o /Users/basher/Library/Developer/Xcode/DerivedData/PlanGrid-gqlncmhpixhqencipfdoevzawlxo/Build/Intermediates.noindex/Pods.build/Debug-iphoneos/BSGridCollectionViewLayout.build/Objects-normal/armv7/BSGridCollectionViewLayout
    ld: warning: Auto-Linking '/Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2018-03-14-a.xctoolchain/usr/lib/swift/iphoneos/libswiftSwiftOnoneSupport.dylib' does not contain bitcode. You must rebuild it with bitcode enabled (Xcode setting ENABLE_BITCODE), obtain an updated library from the vendor, or disable bitcode for this target.
    ld: warning: Auto-Linking '/Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2018-03-14-a.xctoolchain/usr/lib/swift/iphoneos/libswiftCore.dylib' does not contain bitcode. You must rebuild it with bitcode enabled (Xcode setting ENABLE_BITCODE), obtain an updated library from the vendor, or disable bitcode for this target.
    ld: warning: Auto-Linking '/Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2018-03-14-a.xctoolchain/usr/lib/swift/iphoneos/libswiftQuartzCore.dylib' does not contain bitcode. You must rebuild it with bitcode enabled (Xcode setting ENABLE_BITCODE), obtain an updated library from the vendor, or disable bitcode for this target.
    ld: warning: Auto-Linking '/Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2018-03-14-a.xctoolchain/usr/lib/swift/iphoneos/libswiftCoreImage.dylib' does not contain bitcode. You must rebuild it with bitcode enabled (Xcode setting ENABLE_BITCODE), obtain an updated library from the vendor, or disable bitcode for this target.
    ld: warning: Auto-Linking '/Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2018-03-14-a.xctoolchain/usr/lib/swift/iphoneos/libswiftCoreGraphics.dylib' does not contain bitcode. You must rebuild it with bitcode enabled (Xcode setting ENABLE_BITCODE), obtain an updated library from the vendor, or disable bitcode for this target.
    ld: warning: Auto-Linking '/Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2018-03-14-a.xctoolchain/usr/lib/swift/iphoneos/libswiftDispatch.dylib' does not contain bitcode. You must rebuild it with bitcode enabled (Xcode setting ENABLE_BITCODE), obtain an updated library from the vendor, or disable bitcode for this target.
    ld: warning: Auto-Linking '/Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2018-03-14-a.xctoolchain/usr/lib/swift/iphoneos/libswiftObjectiveC.dylib' does not contain bitcode. You must rebuild it with bitcode enabled (Xcode setting ENABLE_BITCODE), obtain an updated library from the vendor, or disable bitcode for this target.
    ld: warning: Auto-Linking '/Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2018-03-14-a.xctoolchain/usr/lib/swift/iphoneos/libswiftCoreFoundation.dylib' does not contain bitcode. You must rebuild it with bitcode enabled (Xcode setting ENABLE_BITCODE), obtain an updated library from the vendor, or disable bitcode for this target.
    ld: warning: Auto-Linking '/Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2018-03-14-a.xctoolchain/usr/lib/swift/iphoneos/libswiftUIKit.dylib' does not contain bitcode. You must rebuild it with bitcode enabled (Xcode setting ENABLE_BITCODE), obtain an updated library from the vendor, or disable bitcode for this target.
    ld: warning: Auto-Linking '/Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2018-03-14-a.xctoolchain/usr/lib/swift/iphoneos/libswiftDarwin.dylib' does not contain bitcode. You must rebuild it with bitcode enabled (Xcode setting ENABLE_BITCODE), obtain an updated library from the vendor, or disable bitcode for this target.
    ld: warning: Auto-Linking '/Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2018-03-14-a.xctoolchain/usr/lib/swift/iphoneos/libswiftMetal.dylib' does not contain bitcode. You must rebuild it with bitcode enabled (Xcode setting ENABLE_BITCODE), obtain an updated library from the vendor, or disable bitcode for this target.
    ld: warning: Auto-Linking '/Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2018-03-14-a.xctoolchain/usr/lib/swift/iphoneos/libswiftFoundation.dylib' does not contain bitcode. You must rebuild it with bitcode enabled (Xcode setting ENABLE_BITCODE), obtain an updated library from the vendor, or disable bitcode for this target.
    Undefined symbols for architecture armv7:
      "_$SSis10ComparablesWP", referenced from:
          _$S26BSGridCollectionViewLayout04GridbcD0C16indexPathsInRectySay10Foundation9IndexPathVGSo6CGRectVF in GridCollectionViewLayout.o
          _$Ss5RangeVySiGMa in GridCollectionViewLayout.o
          _$Ss11ClosedRangeVySiGMa in GridCollectionViewLayout.o
      "_$SSiN", referenced from:
          _$S26BSGridCollectionViewLayout04GridbcD0C22estimatedNumberOfItemsSiyF in GridCollectionViewLayout.o
          _$S26BSGridCollectionViewLayout04GridbcD0C16indexPathsInRectySay10Foundation9IndexPathVGSo6CGRectVF in GridCollectionViewLayout.o
          _$S26BSGridCollectionViewLayout04GridbcD0C9flatIndexySi10Foundation0G4PathVF in GridCollectionViewLayout.o
          _$Ss5RangeVySiGMa in GridCollectionViewLayout.o
          _$Ss11ClosedRangeVySiGMa in GridCollectionViewLayout.o
      "_$SBOWV", referenced from:
          _$S26BSGridCollectionViewLayout04GridbcD0CMf in GridCollectionViewLayout.o
      "__swift_FORCE_LOAD_$_swiftCoreGraphics", referenced from:
          __swift_FORCE_LOAD_$_swiftCoreGraphics_$_BSGridCollectionViewLayout in GridCollectionViewLayout.o
         (maybe you meant: __swift_FORCE_LOAD_$_swiftCoreGraphics_$_BSGridCollectionViewLayout)
      "__swift_FORCE_LOAD_$_swiftCoreImage", referenced from:
          __swift_FORCE_LOAD_$_swiftCoreImage_$_BSGridCollectionViewLayout in GridCollectionViewLayout.o
         (maybe you meant: __swift_FORCE_LOAD_$_swiftCoreImage_$_BSGridCollectionViewLayout)
      "__swift_FORCE_LOAD_$_swiftDispatch", referenced from:
          __swift_FORCE_LOAD_$_swiftDispatch_$_BSGridCollectionViewLayout in GridCollectionViewLayout.o
         (maybe you meant: __swift_FORCE_LOAD_$_swiftDispatch_$_BSGridCollectionViewLayout)
      "_$SSis10StrideablesWP", referenced from:
          _$Ss5RangeVySiGAByxGs8Sequencess10StrideableRzs13SignedInteger6StrideRpzrlWl in GridCollectionViewLayout.o
          _$Ss11ClosedRangeVySiGAByxGs10Collectionss10StrideableRzs13SignedInteger6StrideRpzrlWl in GridCollectionViewLayout.o
      "__swift_FORCE_LOAD_$_swiftCoreFoundation", referenced from:
          __swift_FORCE_LOAD_$_swiftCoreFoundation_$_BSGridCollectionViewLayout in GridCollectionViewLayout.o
         (maybe you meant: __swift_FORCE_LOAD_$_swiftCoreFoundation_$_BSGridCollectionViewLayout)
      "_swift_getForeignTypeMetadata", referenced from:
          _$SSo6CGSizeVMa in GridCollectionViewLayout.o
      "_swift_unknownRelease", referenced from:
          _$Ss13_StringObjectV8_VariantOWe in GridCollectionViewLayout.o
      "_$Ss5RangeVyxGs8Sequencess10StrideableRzs13SignedInteger6StrideRpzrlWa", referenced from:
          _$Ss5RangeVySiGAByxGs8Sequencess10StrideableRzs13SignedInteger6StrideRpzrlWl in GridCollectionViewLayout.o
      "_$Ss11ClosedRangeVyxGs10Collectionss10StrideableRzs13SignedInteger6StrideRpzrlWa", referenced from:
          _$Ss11ClosedRangeVySiGAByxGs10Collectionss10StrideableRzs13SignedInteger6StrideRpzrlWl in GridCollectionViewLayout.o
      "_$SSayxGs8SequencesWa", referenced from:
          _$SSaySo32UICollectionViewLayoutAttributesCSgGSayxGs8SequencesWl in GridCollectionViewLayout.o
      "__swift_FORCE_LOAD_$_swiftFoundation", referenced from:
          __swift_FORCE_LOAD_$_swiftFoundation_$_BSGridCollectionViewLayout in GridCollectionViewLayout.o
         (maybe you meant: __swift_FORCE_LOAD_$_swiftFoundation_$_BSGridCollectionViewLayout)
      "_swift_getInitializedObjCClass", referenced from:
          _$SSo32UICollectionViewLayoutAttributesCMa in GridCollectionViewLayout.o
          _$S26BSGridCollectionViewLayout04GridbcD0CMa in GridCollectionViewLayout.o
      "_swift_getObjCClassFromMetadata", referenced from:
          _$SSo32UICollectionViewLayoutAttributesC11forCellWithAB10Foundation9IndexPathV_tcfCTO in GridCollectionViewLayout.o
      "_$SSqMa", referenced from:
          _$SSo32UICollectionViewLayoutAttributesCSgMa in GridCollectionViewLayout.o
      "_$SSS10FoundationE36_unconditionallyBridgeFromObjectiveCySSSo8NSStringCSgFZ", referenced from:
          _$S26BSGridCollectionViewLayout04GridbcD0C029layoutAttributesForDecorationC06ofKind2atSo012UICollectioncdG0CSgSS_10Foundation9IndexPathVtFTo in GridCollectionViewLayout.o
          _$S26BSGridCollectionViewLayout04GridbcD0C032layoutAttributesForSupplementaryC06ofKind2atSo012UICollectioncdG0CSgSS_10Foundation9IndexPathVtFTo in GridCollectionViewLayout.o
      "_$SSaMa", referenced from:
          _$SSay10Foundation9IndexPathVGMa in GridCollectionViewLayout.o
          _$SSaySo32UICollectionViewLayoutAttributesCSgGMa in GridCollectionViewLayout.o
      "__swift_FORCE_LOAD_$_swiftMetal", referenced from:
          __swift_FORCE_LOAD_$_swiftMetal_$_BSGridCollectionViewLayout in GridCollectionViewLayout.o
         (maybe you meant: __swift_FORCE_LOAD_$_swiftMetal_$_BSGridCollectionViewLayout)
      "_$S10ObjectiveC22_convertBoolToObjCBoolyAA0eF0VSbF", referenced from:
          _$S26BSGridCollectionViewLayout04GridbcD0C016shouldInvalidateD015forBoundsChangeSbSo6CGRectV_tFTo in GridCollectionViewLayout.o
      "_$SSi2geoiySbSi_SitFZ", referenced from:
          _$S26BSGridCollectionViewLayout04GridbcD0C23layoutAttributesForItem2atSo012UICollectioncdG0CSg10Foundation9IndexPathV_tF in GridCollectionViewLayout.o
          _$S26BSGridCollectionViewLayout04GridbcD0C22indexPathFromFlatIndexy10Foundation0jG0VSiF in GridCollectionViewLayout.o
      "__swift_FORCE_LOAD_$_swiftUIKit", referenced from:
          __swift_FORCE_LOAD_$_swiftUIKit_$_BSGridCollectionViewLayout in GridCollectionViewLayout.o
         (maybe you meant: __swift_FORCE_LOAD_$_swiftUIKit_$_BSGridCollectionViewLayout)
      "_$S10Foundation9IndexPathV36_unconditionallyBridgeFromObjectiveCyACSo07NSIndexC0CSgFZ", referenced from:
          _$S26BSGridCollectionViewLayout04GridbcD0C23layoutAttributesForItem2atSo012UICollectioncdG0CSg10Foundation9IndexPathV_tFTo in GridCollectionViewLayout.o
          _$S26BSGridCollectionViewLayout04GridbcD0C029layoutAttributesForDecorationC06ofKind2atSo012UICollectioncdG0CSgSS_10Foundation9IndexPathVtFTo in GridCollectionViewLayout.o
          _$S26BSGridCollectionViewLayout04GridbcD0C032layoutAttributesForSupplementaryC06ofKind2atSo012UICollectioncdG0CSgSS_10Foundation9IndexPathVtFTo in GridCollectionViewLayout.o
      "_swift_deallocObject", referenced from:
          l_objectdestroy in GridCollectionViewLayout.o
          l_objectdestroy.2 in GridCollectionViewLayout.o
          l_objectdestroy.5 in GridCollectionViewLayout.o
          l_objectdestroy.8 in GridCollectionViewLayout.o
          l_objectdestroy.11 in GridCollectionViewLayout.o
          l_objectdestroy.14 in GridCollectionViewLayout.o
          l_objectdestroy.17 in GridCollectionViewLayout.o
          ...
      "_$S10Foundation9IndexPathV19_bridgeToObjectiveCSo07NSIndexC0CyF", referenced from:
          _$S26BSGridCollectionViewLayout04GridbcD0C9flatIndexySi10Foundation0G4PathVF in GridCollectionViewLayout.o
          _$SSo32UICollectionViewLayoutAttributesC11forCellWithAB10Foundation9IndexPathV_tcfCTO in GridCollectionViewLayout.o
      "_$Ss5RangeVMa", referenced from:
          _$Ss5RangeVySiGMa in GridCollectionViewLayout.o
      "_$SSa10FoundationE19_bridgeToObjectiveCSo7NSArrayCyF", referenced from:
          _$S26BSGridCollectionViewLayout04GridbcD0C27layoutAttributesForElements2inSaySo012UICollectioncdG0CGSgSo6CGRectV_tFTo in GridCollectionViewLayout.o
      "_$Ss3minyxx_xts10ComparableRzlF", referenced from:
          _$S26BSGridCollectionViewLayout04GridbcD0C16indexPathsInRectySay10Foundation9IndexPathVGSo6CGRectVF in GridCollectionViewLayout.o
      "_$Ss8SequencePsE6reduceyqd__qd___qd__qd___7ElementQztKXEtKlF", referenced from:
          _$S26BSGridCollectionViewLayout04GridbcD0C22estimatedNumberOfItemsSiyF in GridCollectionViewLayout.o
          _$S26BSGridCollectionViewLayout04GridbcD0C9flatIndexySi10Foundation0G4PathVF in GridCollectionViewLayout.o
      "_swift_getObjCClassMetadata", referenced from:
          _$SSo32UICollectionViewLayoutAttributesCMa in GridCollectionViewLayout.o
      "_$Ss3maxyxx_xts10ComparableRzlF", referenced from:
          _$S26BSGridCollectionViewLayout04GridbcD0C16indexPathsInRectySay10Foundation9IndexPathVGSo6CGRectVF in GridCollectionViewLayout.o
      "_$Ss11ClosedRangeV15uncheckedBoundsAByxGx5lower_x5uppert_tcfCSi_Tg5", referenced from:
          _$S26BSGridCollectionViewLayout04GridbcD0C16indexPathsInRectySay10Foundation9IndexPathVGSo6CGRectVF in GridCollectionViewLayout.o
      "_$SSi1goiySbSi_SitFZ", referenced from:
          _$S26BSGridCollectionViewLayout04GridbcD0C7prepareyyF in GridCollectionViewLayout.o
          _$S26BSGridCollectionViewLayout04GridbcD0C010collectionC11ContentSizeSo6CGSizeVvg in GridCollectionViewLayout.o
          _$S26BSGridCollectionViewLayout04GridbcD0C16indexPathsInRectySay10Foundation9IndexPathVGSo6CGRectVF in GridCollectionViewLayout.o
          _$S26BSGridCollectionViewLayout04GridbcD0C14lastIndexInRow_012withItemsPerI008numberOfK0S2i_S2itFZ in GridCollectionViewLayout.o
      "_$SSayxGs10CollectionsWa", referenced from:
          _$SSay10Foundation9IndexPathVGSayxGs10CollectionsWl in GridCollectionViewLayout.o
      "_swift_bridgeObjectRetain", referenced from:
          _$S26BSGridCollectionViewLayout04GridbcD0C16indexPathsInRectySay10Foundation9IndexPathVGSo6CGRectVF in GridCollectionViewLayout.o
      "_$Ss11ClosedRangeVMa", referenced from:
          _$Ss11ClosedRangeVySiGMa in GridCollectionViewLayout.o
      "_swift_bridgeObjectRelease", referenced from:
          _$S26BSGridCollectionViewLayout04GridbcD0C16indexPathsInRectySay10Foundation9IndexPathVGSo6CGRectVF in GridCollectionViewLayout.o
          _$S26BSGridCollectionViewLayout04GridbcD0C27layoutAttributesForElements2inSaySo012UICollectioncdG0CGSgSo6CGRectV_tFTo in GridCollectionViewLayout.o
          _$SSay10Foundation9IndexPathVGWh0_ in GridCollectionViewLayout.o
          _$SSaySo32UICollectionViewLayoutAttributesCSgGWh0_ in GridCollectionViewLayout.o
      "__swift_FORCE_LOAD_$_swiftQuartzCore", referenced from:
          __swift_FORCE_LOAD_$_swiftQuartzCore_$_BSGridCollectionViewLayout in GridCollectionViewLayout.o
         (maybe you meant: __swift_FORCE_LOAD_$_swiftQuartzCore_$_BSGridCollectionViewLayout)
      "_$S10Foundation9IndexPathVMa", referenced from:
          _$S26BSGridCollectionViewLayout04GridbcD0C16indexPathsInRectySay10Foundation9IndexPathVGSo6CGRectVF in GridCollectionViewLayout.o
          _$S26BSGridCollectionViewLayout04GridbcD0C27layoutAttributesForElements2inSaySo012UICollectioncdG0CGSgSo6CGRectV_tFAGSg10Foundation9IndexPathVXEfU_ in GridCollectionViewLayout.o
          _$S26BSGridCollectionViewLayout04GridbcD0C23layoutAttributesForItem2atSo012UICollectioncdG0CSg10Foundation9IndexPathV_tF in GridCollectionViewLayout.o
          _$S26BSGridCollectionViewLayout04GridbcD0C9flatIndexySi10Foundation0G4PathVF in GridCollectionViewLayout.o
          _$SSo32UICollectionViewLayoutAttributesC11forCellWithAB10Foundation9IndexPathV_tcfCTO in GridCollectionViewLayout.o
          _$S26BSGridCollectionViewLayout04GridbcD0C23layoutAttributesForItem2atSo012UICollectioncdG0CSg10Foundation9IndexPathV_tFTo in GridCollectionViewLayout.o
          _$S26BSGridCollectionViewLayout04GridbcD0C029layoutAttributesForDecorationC06ofKind2atSo012UICollectioncdG0CSgSS_10Foundation9IndexPathVtF in GridCollectionViewLayout.o
          ...
      "_$Ss10CollectionPsE3mapySayqd__Gqd__7ElementQzKXEKlF", referenced from:
          _$S26BSGridCollectionViewLayout04GridbcD0C27layoutAttributesForElements2inSaySo012UICollectioncdG0CGSgSo6CGRectV_tF in GridCollectionViewLayout.o
          _$S26BSGridCollectionViewLayout04GridbcD0C16indexPathsInRectySay10Foundation9IndexPathVGSo6CGRectVF in GridCollectionViewLayout.o
      "_$S10Foundation9IndexPathV5UIKitE7sectionSivg", referenced from:
          _$S26BSGridCollectionViewLayout04GridbcD0C23layoutAttributesForItem2atSo012UICollectioncdG0CSg10Foundation9IndexPathV_tF in GridCollectionViewLayout.o
      "_$SSis13SignedIntegersWP", referenced from:
          _$Ss5RangeVySiGAByxGs8Sequencess10StrideableRzs13SignedInteger6StrideRpzrlWl in GridCollectionViewLayout.o
          _$Ss11ClosedRangeVySiGAByxGs10Collectionss10StrideableRzs13SignedInteger6StrideRpzrlWl in GridCollectionViewLayout.o
      "_$SSa12arrayLiteralSayxGxd_tcfC", referenced from:
          _$S26BSGridCollectionViewLayout04GridbcD0C16indexPathsInRectySay10Foundation9IndexPathVGSo6CGRectVF in GridCollectionViewLayout.o
      "_swift_willThrow", referenced from:
          _$S10Foundation9IndexPathVSo32UICollectionViewLayoutAttributesCSgs5Error_pIgiozo_AcFsAG_pIegirzo_TR in GridCollectionViewLayout.o
          _$SSo32UICollectionViewLayoutAttributesCSgACs5Error_pIgxozo_A2CsAD_pIegirzo_TR in GridCollectionViewLayout.o
          _$SSi10Foundation9IndexPathVs5Error_pIgyrzo_SiACsAD_pIegirzo_TR in GridCollectionViewLayout.o
          _$SS3is5Error_pIgyydzo_S3isAA_pIegiirzo_TR in GridCollectionViewLayout.o
      "_swift_endAccess", referenced from:
          _$S26BSGridCollectionViewLayout04GridbcD0C11itemSpacing12CoreGraphics7CGFloatVvg in GridCollectionViewLayout.o
          _$S26BSGridCollectionViewLayout04GridbcD0C11itemSpacing12CoreGraphics7CGFloatVvs in GridCollectionViewLayout.o
          _$S26BSGridCollectionViewLayout04GridbcD0C11itemSpacing12CoreGraphics7CGFloatVvW in GridCollectionViewLayout.o
          _$S26BSGridCollectionViewLayout04GridbcD0C17estimatedItemSizeSo6CGSizeVyF in GridCollectionViewLayout.o
          _$S26BSGridCollectionViewLayout04GridbcD0C11itemsPerRowSivg in GridCollectionViewLayout.o
          _$S26BSGridCollectionViewLayout04GridbcD0C11itemsPerRowSivs in GridCollectionViewLayout.o
          _$S26BSGridCollectionViewLayout04GridbcD0C11itemsPerRowSivW in GridCollectionViewLayout.o
          ...
      "__swift_FORCE_LOAD_$_swiftObjectiveC", referenced from:
          __swift_FORCE_LOAD_$_swiftObjectiveC_$_BSGridCollectionViewLayout in GridCollectionViewLayout.o
         (maybe you meant: __swift_FORCE_LOAD_$_swiftObjectiveC_$_BSGridCollectionViewLayout)
      "_swift_allocObject", referenced from:
          _$S26BSGridCollectionViewLayout04GridbcD0C22estimatedNumberOfItemsSiyF in GridCollectionViewLayout.o
          _$S26BSGridCollectionViewLayout04GridbcD0C27layoutAttributesForElements2inSaySo012UICollectioncdG0CGSgSo6CGRectV_tF in GridCollectionViewLayout.o
          _$S26BSGridCollectionViewLayout04GridbcD0C16indexPathsInRectySay10Foundation9IndexPathVGSo6CGRectVF in GridCollectionViewLayout.o
          _$S26BSGridCollectionViewLayout04GridbcD0C9flatIndexySi10Foundation0G4PathVF in GridCollectionViewLayout.o
      "_$Ss8SequencePsE7flatMapySayqd__Gqd__Sg7ElementQzKXEKlF", referenced from:
          _$S26BSGridCollectionViewLayout04GridbcD0C27layoutAttributesForElements2inSaySo012UICollectioncdG0CGSgSo6CGRectV_tF in GridCollectionViewLayout.o
      "_$Ss5RangeV15uncheckedBoundsAByxGx5lower_x5uppert_tcfCSi_Tg5", referenced from:
          _$S26BSGridCollectionViewLayout04GridbcD0C22estimatedNumberOfItemsSiyF in GridCollectionViewLayout.o
          _$S26BSGridCollectionViewLayout04GridbcD0C9flatIndexySi10Foundation0G4PathVF in GridCollectionViewLayout.o
      "_swift_release", referenced from:
          _$S26BSGridCollectionViewLayout04GridbcD0C22estimatedNumberOfItemsSiyF in GridCollectionViewLayout.o
          _$S26BSGridCollectionViewLayout04GridbcD0C27layoutAttributesForElements2inSaySo012UICollectioncdG0CGSgSo6CGRectV_tF in GridCollectionViewLayout.o
          _$S26BSGridCollectionViewLayout04GridbcD0C16indexPathsInRectySay10Foundation9IndexPathVGSo6CGRectVF in GridCollectionViewLayout.o
          _$S26BSGridCollectionViewLayout04GridbcD0C9flatIndexySi10Foundation0G4PathVF in GridCollectionViewLayout.o
      "_$SSi2leoiySbSi_SitFZ", referenced from:
          _$S26BSGridCollectionViewLayout04GridbcD0C16indexPathsInRectySay10Foundation9IndexPathVGSo6CGRectVF in GridCollectionViewLayout.o
      "__swift_FORCE_LOAD_$_swiftDarwin", referenced from:
          __swift_FORCE_LOAD_$_swiftDarwin_$_BSGridCollectionViewLayout in GridCollectionViewLayout.o
         (maybe you meant: __swift_FORCE_LOAD_$_swiftDarwin_$_BSGridCollectionViewLayout)
      "_$Ss27_allocateUninitializedArrayySayxG_BptBwlF", referenced from:
          _$S26BSGridCollectionViewLayout04GridbcD0C16indexPathsInRectySay10Foundation9IndexPathVGSo6CGRectVF in GridCollectionViewLayout.o
      "_$Ss18_fatalErrorMessage__4file4line5flagss5NeverOs12StaticStringV_A2HSus6UInt32VtF", referenced from:
          _$S26BSGridCollectionViewLayout04GridbcD0C7prepareyyF in GridCollectionViewLayout.o
          _$S26BSGridCollectionViewLayout04GridbcD0C22estimatedNumberOfItemsSiyF in GridCollectionViewLayout.o
          _$S26BSGridCollectionViewLayout04GridbcD0C16indexPathsInRectySay10Foundation9IndexPathVGSo6CGRectVF in GridCollectionViewLayout.o
          _$S26BSGridCollectionViewLayout04GridbcD0C23layoutAttributesForItem2atSo012UICollectioncdG0CSg10Foundation9IndexPathV_tF in GridCollectionViewLayout.o
          _$S26BSGridCollectionViewLayout04GridbcD0C9flatIndexySi10Foundation0G4PathVF in GridCollectionViewLayout.o
      "_$S10Foundation9IndexPathV5UIKitE4item7sectionACSi_SitcfC", referenced from:
          _$S26BSGridCollectionViewLayout04GridbcD0C22indexPathFromFlatIndexy10Foundation0jG0VSiF in GridCollectionViewLayout.o
      "_$S10Foundation9IndexPathV5UIKitE3rowSivg", referenced from:
          _$S26BSGridCollectionViewLayout04GridbcD0C23layoutAttributesForItem2atSo012UICollectioncdG0CSg10Foundation9IndexPathV_tF in GridCollectionViewLayout.o
      "_swift_beginAccess", referenced from:
          _$S26BSGridCollectionViewLayout04GridbcD0C11itemSpacing12CoreGraphics7CGFloatVvg in GridCollectionViewLayout.o
          _$S26BSGridCollectionViewLayout04GridbcD0C11itemSpacing12CoreGraphics7CGFloatVvs in GridCollectionViewLayout.o
          _$S26BSGridCollectionViewLayout04GridbcD0C11itemSpacing12CoreGraphics7CGFloatVvW in GridCollectionViewLayout.o
          _$S26BSGridCollectionViewLayout04GridbcD0C17estimatedItemSizeSo6CGSizeVyF in GridCollectionViewLayout.o
          _$S26BSGridCollectionViewLayout04GridbcD0C11itemsPerRowSivg in GridCollectionViewLayout.o
          _$S26BSGridCollectionViewLayout04GridbcD0C11itemsPerRowSivs in GridCollectionViewLayout.o
          _$S26BSGridCollectionViewLayout04GridbcD0C11itemsPerRowSivW in GridCollectionViewLayout.o
          ...
    ld: symbol(s) not found for architecture armv7
    clang: error: linker command failed with exit code 1 (use -v to see invocation)

The only difference between the build command we normally use and this was are these added args (using Xcode 9.3b4): -toolchain org.swift.3020180314a OTHER_SWIFT_FLAGS='$(inherited) -enable-batch-mode' (and disabled warnings as errors). Let me know if I'm missing something, and I'm looking forward to helping test here!

I can guess at some other flags to try, but I bet folks who know for sure will chime in very soon. In the meantime, it it's easy, it would be interesting to see if you get the same issues without -enable-batch-mode but with -toolchain org.swift.3020180314a. (That might be the first thing I would check.)

Thanks!

  • David Ungar (senderPath)

PS: Sorry you ran into trouble. It can be frustrating.

No worries! There will be bugs at first. Will let ya know.

Good call! I get the same errors with that snapshot but without -enable-batch-mode.

1 Like

Perhaps you tried it already, but when switching toolchains / swift version I often faced issues with symbols being linked from wrong build artifacts. Perhaps dumping derived data may help.

Thanks for the suggestion! I neglected to mention that my build script starts by doing this to ensure a more accurate timing each run, so that shouldn't be an issue:

rm -rf ~/Library/Developer/Xcode/DerivedData/ModuleCache/*
rm -rf ~/Library/Developer/Xcode/DerivedData/PlanGrid-*
1 Like

Manage to build the lib locally on xcode 9.2 with latest toolchain (just to make sure I wasn't crazy) :smile:

Found the issue: these errors go away when I switch back to the old build system. Will post timings once I've had a chance to gather everything.

1 Like

Thanks!

One of the projects I help maintain (RxSwift) isn't compiling with the latest snapshot with a "expression too complex to infer" error (works fine on the Xcode 9.2 toolchain, takes about 1m40s to build). More specifically it's a piece of the example application, which is fairly trivial. You can see a screenshot of the error here (it doesn't let me attach more than a single screenshot to this post).

Results

When commenting out the offending sample code, here are the results I get for building the entire project (with the framework itself) on 2018-03-14 (a) :

  • No batch, Debug (-Onone) - 88.6s
  • No batch, Debug (-O -whole-module-optimization) - 66.8s
  • No batch, Debug (-O, e.g. single file optimization) - 99.2s, 146.s (two attempts)
  • Batch, Debug (-Onone optimization) - 70.098s, 65.1s (two attempts)

I was surprised the build time was pretty similar with batch turned on, so I'm posting a screenshot to confirm I have the flag set up correctly:

Additional information:

  • Size: The project is fairly large. Building the example project builds all 4 subsequent frameworks as a subproject. Output of find . -name "*.swift" | wc -l is 977 files (if there's a more accurate way to count Swift files, I'd love to know). You can see the built branch here: https://github.com/ReactiveX/RxSwift/tree/develop

  • Setup: I've changed the appropriate flags (optimization and additional flags) on the example project, and the subproject (meaning all of the framework dependencies as well). I've started each build with a "Clean Build Folder" (e.g. Cmd+Opt+Shift+K).

  • Number of Cores: 4 (2.5 GHz Intel Core i7)

Let me know if there's anything you'd like me to try. I'll also try building our actual main production app with the snapshot later.

Shai.

I ran into another build issue with the 3-14 snapshot; filed it here https://bugs.swift.org/browse/SR-7206. Will try again when a new snapshot is out.

Also went to give this a whirl and got this assertion failure [SR-7207] Assertion failure in getOrCreateKeyPathEqualsAndHash · Issue #49755 · apple/swift · GitHub