-whole-module-optimization with -Onone


(Ben Asher) #1

Hello! Someone recently tipped me off to using -whole-module-optimization
flag with -Onone for use during debug builds to speed up compile times. In
our project, the speedup feels quite dramatic, but when it gets to the
linking step (after compiling both Swift and Obj-C in the project) it fails
because ld can't find the individual object files that normally get emitted
during the debug-type build presumably because -whole-module-optimization
only emits one (and this isn't a normal "-Owholemodule"-type build which
works fine).

I can't seem to reproduce this outside of Xcode, but I was curious if
anyone has tried this and knows of a workaround to get
-whole-module-optimization to work with -Onone in Xcode?

I'm currently using Xcode 8.1 (App Store build) and Swift 3 on macOS Sierra.

Thanks!

Ben


(Jordan Rose) #2

Xcode needs to know that you're building in WMO mode, so rather than putting -whole-module-optimization in your "Other Swift Flags", put -Onone there. It's an ugly hack but it should work in the near term.

We do want to work to make this drastic speed difference go away, so if you're able we (at Apple) would love to have a source drop of your Swift 3 project, for additional data on where the problems are. Mind filing a Radar?

Best,
Jordan

···

On Dec 1, 2016, at 11:51, Ben Asher via swift-dev <swift-dev@swift.org> wrote:

Hello! Someone recently tipped me off to using -whole-module-optimization flag with -Onone for use during debug builds to speed up compile times. In our project, the speedup feels quite dramatic, but when it gets to the linking step (after compiling both Swift and Obj-C in the project) it fails because ld can't find the individual object files that normally get emitted during the debug-type build presumably because -whole-module-optimization only emits one (and this isn't a normal "-Owholemodule"-type build which works fine).

I can't seem to reproduce this outside of Xcode, but I was curious if anyone has tried this and knows of a workaround to get -whole-module-optimization to work with -Onone in Xcode?

I'm currently using Xcode 8.1 (App Store build) and Swift 3 on macOS Sierra.

Thanks!

Ben
_______________________________________________
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev


(Ben Asher) #3

Okay I think that worked! And just to clarify, you meant set
SWIFT_OPTIMIZATION_LEVEL = -Owholemodule and OTHER_SWIFT_FLAGS = -Onone ?

I'll file a radar this afternoon with some details and DM you the number.

Thanks again!

Ben

···

On Thu, Dec 1, 2016 at 1:10 PM, Jordan Rose <jordan_rose@apple.com> wrote:

Xcode needs to know that you're building in WMO mode, so rather than
putting -whole-module-optimization in your "Other Swift Flags", put -Onone
there. It's an ugly hack but it should work in the near term.

We do want to work to make this drastic speed difference go away, so if
you're able we (at Apple) would love to have a source drop of your Swift 3
project, for additional data on where the problems are. Mind filing a Radar?

Best,
Jordan

> On Dec 1, 2016, at 11:51, Ben Asher via swift-dev <swift-dev@swift.org> > wrote:
>
> Hello! Someone recently tipped me off to using
-whole-module-optimization flag with -Onone for use during debug builds to
speed up compile times. In our project, the speedup feels quite dramatic,
but when it gets to the linking step (after compiling both Swift and Obj-C
in the project) it fails because ld can't find the individual object files
that normally get emitted during the debug-type build presumably because
-whole-module-optimization only emits one (and this isn't a normal
"-Owholemodule"-type build which works fine).
>
> I can't seem to reproduce this outside of Xcode, but I was curious if
anyone has tried this and knows of a workaround to get
-whole-module-optimization to work with -Onone in Xcode?
>
> I'm currently using Xcode 8.1 (App Store build) and Swift 3 on macOS
Sierra.
>
> Thanks!
>
> Ben
> _______________________________________________
> swift-dev mailing list
> swift-dev@swift.org
> https://lists.swift.org/mailman/listinfo/swift-dev

--
Ben


(Ben Asher) #4

Just running a quick trial before and after I made this change in our
project, we were previously seeing builds of our main target that took just
under 13min. With this hack, a clean debug build takes about 4.5min.

Ben

···

On Thu, Dec 1, 2016 at 1:33 PM, Ben Asher <benasher44@gmail.com> wrote:

Okay I think that worked! And just to clarify, you meant set
SWIFT_OPTIMIZATION_LEVEL = -Owholemodule and OTHER_SWIFT_FLAGS = -Onone ?

I'll file a radar this afternoon with some details and DM you the number.

Thanks again!

Ben

On Thu, Dec 1, 2016 at 1:10 PM, Jordan Rose <jordan_rose@apple.com> wrote:

Xcode needs to know that you're building in WMO mode, so rather than
putting -whole-module-optimization in your "Other Swift Flags", put -Onone
there. It's an ugly hack but it should work in the near term.

We do want to work to make this drastic speed difference go away, so if
you're able we (at Apple) would love to have a source drop of your Swift 3
project, for additional data on where the problems are. Mind filing a Radar?

Best,
Jordan

> On Dec 1, 2016, at 11:51, Ben Asher via swift-dev <swift-dev@swift.org> >> wrote:
>
> Hello! Someone recently tipped me off to using
-whole-module-optimization flag with -Onone for use during debug builds to
speed up compile times. In our project, the speedup feels quite dramatic,
but when it gets to the linking step (after compiling both Swift and Obj-C
in the project) it fails because ld can't find the individual object files
that normally get emitted during the debug-type build presumably because
-whole-module-optimization only emits one (and this isn't a normal
"-Owholemodule"-type build which works fine).
>
> I can't seem to reproduce this outside of Xcode, but I was curious if
anyone has tried this and knows of a workaround to get
-whole-module-optimization to work with -Onone in Xcode?
>
> I'm currently using Xcode 8.1 (App Store build) and Swift 3 on macOS
Sierra.
>
> Thanks!
>
> Ben
> _______________________________________________
> swift-dev mailing list
> swift-dev@swift.org
> https://lists.swift.org/mailman/listinfo/swift-dev

--
Ben

--
Ben


(Mark Lacey) #5

Just running a quick trial before and after I made this change in our project, we were previously seeing builds of our main target that took just under 13min. With this hack, a clean debug build takes about 4.5min.

You may find that recent snapshot builds from swift.org <http://swift.org/> help with your build times even without enabling -Owholemodule. The redundant type checking of synthesized accessors which we talked about a month or two should now be fixed on master, and it would be great to verify that’s the case with your code and to get an idea of how much it improves your build times.

Mark

···

On Dec 1, 2016, at 3:13 PM, Ben Asher via swift-dev <swift-dev@swift.org> wrote:

Ben

On Thu, Dec 1, 2016 at 1:33 PM, Ben Asher <benasher44@gmail.com <mailto:benasher44@gmail.com>> wrote:
Okay I think that worked! And just to clarify, you meant set SWIFT_OPTIMIZATION_LEVEL = -Owholemodule and OTHER_SWIFT_FLAGS = -Onone ?

I'll file a radar this afternoon with some details and DM you the number.

Thanks again!

Ben

On Thu, Dec 1, 2016 at 1:10 PM, Jordan Rose <jordan_rose@apple.com <mailto:jordan_rose@apple.com>> wrote:
Xcode needs to know that you're building in WMO mode, so rather than putting -whole-module-optimization in your "Other Swift Flags", put -Onone there. It's an ugly hack but it should work in the near term.

We do want to work to make this drastic speed difference go away, so if you're able we (at Apple) would love to have a source drop of your Swift 3 project, for additional data on where the problems are. Mind filing a Radar?

Best,
Jordan

> On Dec 1, 2016, at 11:51, Ben Asher via swift-dev <swift-dev@swift.org <mailto:swift-dev@swift.org>> wrote:
>
> Hello! Someone recently tipped me off to using -whole-module-optimization flag with -Onone for use during debug builds to speed up compile times. In our project, the speedup feels quite dramatic, but when it gets to the linking step (after compiling both Swift and Obj-C in the project) it fails because ld can't find the individual object files that normally get emitted during the debug-type build presumably because -whole-module-optimization only emits one (and this isn't a normal "-Owholemodule"-type build which works fine).
>
> I can't seem to reproduce this outside of Xcode, but I was curious if anyone has tried this and knows of a workaround to get -whole-module-optimization to work with -Onone in Xcode?
>
> I'm currently using Xcode 8.1 (App Store build) and Swift 3 on macOS Sierra.
>
> Thanks!
>
> Ben
> _______________________________________________
> swift-dev mailing list
> swift-dev@swift.org <mailto:swift-dev@swift.org>
> https://lists.swift.org/mailman/listinfo/swift-dev

--
Ben

--
Ben
_______________________________________________
swift-dev mailing list
swift-dev@swift.org <mailto:swift-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-dev


(Ben Asher) #6

Sure! Thanks for reminding me. I'll follow up on that, test, and get back
to you.

Ben

···

On Thu, Dec 1, 2016 at 2:48 PM, Mark Lacey <mark_lacey@apple.com> wrote:

On Dec 1, 2016, at 3:13 PM, Ben Asher via swift-dev <swift-dev@swift.org> > wrote:

Just running a quick trial before and after I made this change in our
project, we were previously seeing builds of our main target that took just
under 13min. With this hack, a clean debug build takes about 4.5min.

You may find that recent snapshot builds from swift.org help with your
build times even without enabling -Owholemodule. The redundant type
checking of synthesized accessors which we talked about a month or two
should now be fixed on master, and it would be great to verify that’s the
case with your code and to get an idea of how much it improves your build
times.

Mark

Ben

On Thu, Dec 1, 2016 at 1:33 PM, Ben Asher <benasher44@gmail.com> wrote:

Okay I think that worked! And just to clarify, you meant set
SWIFT_OPTIMIZATION_LEVEL = -Owholemodule and OTHER_SWIFT_FLAGS = -Onone ?

I'll file a radar this afternoon with some details and DM you the number.

Thanks again!

Ben

On Thu, Dec 1, 2016 at 1:10 PM, Jordan Rose <jordan_rose@apple.com> w
rote:

Xcode needs to know that you're building in WMO mode, so rather than
putting -whole-module-optimization in your "Other Swift Flags", put -Onone
there. It's an ugly hack but it should work in the near term.

We do want to work to make this drastic speed difference go away, so if
you're able we (at Apple) would love to have a source drop of your Swift 3
project, for additional data on where the problems are. Mind filing a Radar?

Best,
Jordan

> On Dec 1, 2016, at 11:51, Ben Asher via swift-dev <swift-dev@swift.org> >>> wrote:
>
> Hello! Someone recently tipped me off to using
-whole-module-optimization flag with -Onone for use during debug builds to
speed up compile times. In our project, the speedup feels quite dramatic,
but when it gets to the linking step (after compiling both Swift and Obj-C
in the project) it fails because ld can't find the individual object files
that normally get emitted during the debug-type build presumably because
-whole-module-optimization only emits one (and this isn't a normal
"-Owholemodule"-type build which works fine).
>
> I can't seem to reproduce this outside of Xcode, but I was curious if
anyone has tried this and knows of a workaround to get
-whole-module-optimization to work with -Onone in Xcode?
>
> I'm currently using Xcode 8.1 (App Store build) and Swift 3 on macOS
Sierra.
>
> Thanks!
>
> Ben
> _______________________________________________
> swift-dev mailing list
> swift-dev@swift.org
> https://lists.swift.org/mailman/listinfo/swift-dev

--
Ben

--
Ben
_______________________________________________
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

--
Ben


(Ben Asher) #7

I'm trying out DEVELOPMENT-SNAPSHOT-2016-11-29-a now. It's still going, so
I don't have any time results yet. But, I did notice something new (or
maybe existed before but didn't have this warning to expose it). For every
Swift file that's compiled (only using -Onone here), it outputs the same 2
warnings (easy to fix, but not related to any of this) from the same method
in the same Obj-C header referring to the arguments and the return types in
that method:

- "array parameter is missing a nullability type specifier (_Nonnull,
_Nullable, or _Null_unspecified)"
- "inferring '_Nonnull' for pointer type within array is deprecated"

Here's an example of what this method looks like:

+ (NSSet<SomeObject *> *)setWithSELs:(SEL[])sels count:(NSUInteger)count;

Could this point to more duplicated work?

Ben

···

On Thu, Dec 1, 2016 at 2:50 PM, Ben Asher <benasher44@gmail.com> wrote:

Sure! Thanks for reminding me. I'll follow up on that, test, and get back
to you.

Ben

On Thu, Dec 1, 2016 at 2:48 PM, Mark Lacey <mark_lacey@apple.com> wrote:

On Dec 1, 2016, at 3:13 PM, Ben Asher via swift-dev <swift-dev@swift.org> >> wrote:

Just running a quick trial before and after I made this change in our
project, we were previously seeing builds of our main target that took just
under 13min. With this hack, a clean debug build takes about 4.5min.

You may find that recent snapshot builds from swift.org help with your
build times even without enabling -Owholemodule. The redundant type
checking of synthesized accessors which we talked about a month or two
should now be fixed on master, and it would be great to verify that’s the
case with your code and to get an idea of how much it improves your build
times.

Mark

Ben

On Thu, Dec 1, 2016 at 1:33 PM, Ben Asher <benasher44@gmail.com> wrote:

Okay I think that worked! And just to clarify, you meant set
SWIFT_OPTIMIZATION_LEVEL = -Owholemodule and OTHER_SWIFT_FLAGS = -Onone ?

I'll file a radar this afternoon with some details and DM you the number.

Thanks again!

Ben

On Thu, Dec 1, 2016 at 1:10 PM, Jordan Rose <jordan_rose@apple.com> w
rote:

Xcode needs to know that you're building in WMO mode, so rather than
putting -whole-module-optimization in your "Other Swift Flags", put -Onone
there. It's an ugly hack but it should work in the near term.

We do want to work to make this drastic speed difference go away, so if
you're able we (at Apple) would love to have a source drop of your Swift 3
project, for additional data on where the problems are. Mind filing a Radar?

Best,
Jordan

> On Dec 1, 2016, at 11:51, Ben Asher via swift-dev < >>>> swift-dev@swift.org> wrote:
>
> Hello! Someone recently tipped me off to using
-whole-module-optimization flag with -Onone for use during debug builds to
speed up compile times. In our project, the speedup feels quite dramatic,
but when it gets to the linking step (after compiling both Swift and Obj-C
in the project) it fails because ld can't find the individual object files
that normally get emitted during the debug-type build presumably because
-whole-module-optimization only emits one (and this isn't a normal
"-Owholemodule"-type build which works fine).
>
> I can't seem to reproduce this outside of Xcode, but I was curious if
anyone has tried this and knows of a workaround to get
-whole-module-optimization to work with -Onone in Xcode?
>
> I'm currently using Xcode 8.1 (App Store build) and Swift 3 on macOS
Sierra.
>
> Thanks!
>
> Ben
> _______________________________________________
> swift-dev mailing list
> swift-dev@swift.org
> https://lists.swift.org/mailman/listinfo/swift-dev

--
Ben

--
Ben
_______________________________________________
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

--
Ben

--
Ben


(Ben Asher) #8

The build failed compiling 3 files in our project. Our app is crashing that
snapshot; here is some output from the failures:

- Assertion failed: (newType == type || (isa<TupleType>(newType) &&
cast<TupleType>(newType)->getNumElements() == 1 &&
cast<TupleType>(newType).getElementType(0) == type)), function rewriteType,
file /Users/buildn
ode/jenkins/workspace/oss-swift-package-osx/swift/lib/SILGen/RValue.h, line
207.

- SIL verification failed: method's Self parameter should be constrained by
protocol: selfRequirement.getKind() == RequirementKind::Conformance &&
selfRequirement.getFirstType()->isEqual(selfGenericParam) && selfR
equirement.getSecondType()->getAs<ProtocolType>() ->getDecl() == protocol

We're not getting all the way to linking, but compiling everything for that
snapshot took ~11min45s. Based on that, it appears that about a minute is
saved with the new fixes since the Swift 3.0 that shipped with Xcode 8.1
(App Store).

With Xcode 8.2 Beta 2, the app builds fine, and it takes about 12m15s. If
Xcode 8.2 Beta 2 also has those fixes that we expect to speed up the build,
the extra time here (compared to the snapshot) could be explained by
linking, signing, and a few other custom builds scripts that run after
compiling.

Thanks everyone for your help!

Ben

···

On Thu, Dec 1, 2016 at 4:16 PM, Ben Asher <benasher44@gmail.com> wrote:

I'm trying out DEVELOPMENT-SNAPSHOT-2016-11-29-a now. It's still going,
so I don't have any time results yet. But, I did notice something new (or
maybe existed before but didn't have this warning to expose it). For every
Swift file that's compiled (only using -Onone here), it outputs the same 2
warnings (easy to fix, but not related to any of this) from the same method
in the same Obj-C header referring to the arguments and the return types in
that method:

- "array parameter is missing a nullability type specifier (_Nonnull,
_Nullable, or _Null_unspecified)"
- "inferring '_Nonnull' for pointer type within array is deprecated"

Here's an example of what this method looks like:

+ (NSSet<SomeObject *> *)setWithSELs:(SEL[])sels count:(NSUInteger)count;

Could this point to more duplicated work?

Ben

On Thu, Dec 1, 2016 at 2:50 PM, Ben Asher <benasher44@gmail.com> wrote:

Sure! Thanks for reminding me. I'll follow up on that, test, and get back
to you.

Ben

On Thu, Dec 1, 2016 at 2:48 PM, Mark Lacey <mark_lacey@apple.com> wrote:

On Dec 1, 2016, at 3:13 PM, Ben Asher via swift-dev <swift-dev@swift.org> >>> wrote:

Just running a quick trial before and after I made this change in our
project, we were previously seeing builds of our main target that took just
under 13min. With this hack, a clean debug build takes about 4.5min.

You may find that recent snapshot builds from swift.org help with your
build times even without enabling -Owholemodule. The redundant type
checking of synthesized accessors which we talked about a month or two
should now be fixed on master, and it would be great to verify that’s the
case with your code and to get an idea of how much it improves your build
times.

Mark

Ben

On Thu, Dec 1, 2016 at 1:33 PM, Ben Asher <benasher44@gmail.com> wrote:

Okay I think that worked! And just to clarify, you meant set
SWIFT_OPTIMIZATION_LEVEL = -Owholemodule and OTHER_SWIFT_FLAGS = -Onone ?

I'll file a radar this afternoon with some details and DM you the
number.

Thanks again!

Ben

On Thu, Dec 1, 2016 at 1:10 PM, Jordan Rose <jordan_rose@apple.com> w
rote:

Xcode needs to know that you're building in WMO mode, so rather than
putting -whole-module-optimization in your "Other Swift Flags", put -Onone
there. It's an ugly hack but it should work in the near term.

We do want to work to make this drastic speed difference go away, so
if you're able we (at Apple) would love to have a source drop of your Swift
3 project, for additional data on where the problems are. Mind filing a
Radar?

Best,
Jordan

> On Dec 1, 2016, at 11:51, Ben Asher via swift-dev < >>>>> swift-dev@swift.org> wrote:
>
> Hello! Someone recently tipped me off to using
-whole-module-optimization flag with -Onone for use during debug builds to
speed up compile times. In our project, the speedup feels quite dramatic,
but when it gets to the linking step (after compiling both Swift and Obj-C
in the project) it fails because ld can't find the individual object files
that normally get emitted during the debug-type build presumably because
-whole-module-optimization only emits one (and this isn't a normal
"-Owholemodule"-type build which works fine).
>
> I can't seem to reproduce this outside of Xcode, but I was curious
if anyone has tried this and knows of a workaround to get
-whole-module-optimization to work with -Onone in Xcode?
>
> I'm currently using Xcode 8.1 (App Store build) and Swift 3 on macOS
Sierra.
>
> Thanks!
>
> Ben
> _______________________________________________
> swift-dev mailing list
> swift-dev@swift.org
> https://lists.swift.org/mailman/listinfo/swift-dev

--
Ben

--
Ben
_______________________________________________
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

--
Ben

--
Ben

--
Ben


(Mark Lacey) #9

The build failed compiling 3 files in our project. Our app is crashing that snapshot; here is some output from the failures:

It would be awesome if you could come up with small reproducers for these and open bugs for them. Alternately, opening a radar with the full project attached will give us an opportunity to get these fixed ASAP!

Mark

···

On Dec 1, 2016, at 5:48 PM, Ben Asher via swift-dev <swift-dev@swift.org> wrote:

- Assertion failed: (newType == type || (isa<TupleType>(newType) && cast<TupleType>(newType)->getNumElements() == 1 && cast<TupleType>(newType).getElementType(0) == type)), function rewriteType, file /Users/buildn
ode/jenkins/workspace/oss-swift-package-osx/swift/lib/SILGen/RValue.h, line 207.

- SIL verification failed: method's Self parameter should be constrained by protocol: selfRequirement.getKind() == RequirementKind::Conformance && selfRequirement.getFirstType()->isEqual(selfGenericParam) && selfR
equirement.getSecondType()->getAs<ProtocolType>() ->getDecl() == protocol

We're not getting all the way to linking, but compiling everything for that snapshot took ~11min45s. Based on that, it appears that about a minute is saved with the new fixes since the Swift 3.0 that shipped with Xcode 8.1 (App Store).

With Xcode 8.2 Beta 2, the app builds fine, and it takes about 12m15s. If Xcode 8.2 Beta 2 also has those fixes that we expect to speed up the build, the extra time here (compared to the snapshot) could be explained by linking, signing, and a few other custom builds scripts that run after compiling.

Thanks everyone for your help!

Ben

On Thu, Dec 1, 2016 at 4:16 PM, Ben Asher <benasher44@gmail.com <mailto:benasher44@gmail.com>> wrote:
I'm trying out DEVELOPMENT-SNAPSHOT-2016-11-29-a now. It's still going, so I don't have any time results yet. But, I did notice something new (or maybe existed before but didn't have this warning to expose it). For every Swift file that's compiled (only using -Onone here), it outputs the same 2 warnings (easy to fix, but not related to any of this) from the same method in the same Obj-C header referring to the arguments and the return types in that method:

- "array parameter is missing a nullability type specifier (_Nonnull, _Nullable, or _Null_unspecified)"
- "inferring '_Nonnull' for pointer type within array is deprecated"

Here's an example of what this method looks like:

+ (NSSet<SomeObject *> *)setWithSELs:(SEL[])sels count:(NSUInteger)count;

Could this point to more duplicated work?

Ben

On Thu, Dec 1, 2016 at 2:50 PM, Ben Asher <benasher44@gmail.com <mailto:benasher44@gmail.com>> wrote:
Sure! Thanks for reminding me. I'll follow up on that, test, and get back to you.

Ben

On Thu, Dec 1, 2016 at 2:48 PM, Mark Lacey <mark_lacey@apple.com <mailto:mark_lacey@apple.com>> wrote:

On Dec 1, 2016, at 3:13 PM, Ben Asher via swift-dev <swift-dev@swift.org <mailto:swift-dev@swift.org>> wrote:

Just running a quick trial before and after I made this change in our project, we were previously seeing builds of our main target that took just under 13min. With this hack, a clean debug build takes about 4.5min.

You may find that recent snapshot builds from swift.org <http://swift.org/> help with your build times even without enabling -Owholemodule. The redundant type checking of synthesized accessors which we talked about a month or two should now be fixed on master, and it would be great to verify that’s the case with your code and to get an idea of how much it improves your build times.

Mark

Ben

On Thu, Dec 1, 2016 at 1:33 PM, Ben Asher <benasher44@gmail.com <mailto:benasher44@gmail.com>> wrote:
Okay I think that worked! And just to clarify, you meant set SWIFT_OPTIMIZATION_LEVEL = -Owholemodule and OTHER_SWIFT_FLAGS = -Onone ?

I'll file a radar this afternoon with some details and DM you the number.

Thanks again!

Ben

On Thu, Dec 1, 2016 at 1:10 PM, Jordan Rose <jordan_rose@apple.com <mailto:jordan_rose@apple.com>> wrote:
Xcode needs to know that you're building in WMO mode, so rather than putting -whole-module-optimization in your "Other Swift Flags", put -Onone there. It's an ugly hack but it should work in the near term.

We do want to work to make this drastic speed difference go away, so if you're able we (at Apple) would love to have a source drop of your Swift 3 project, for additional data on where the problems are. Mind filing a Radar?

Best,
Jordan

> On Dec 1, 2016, at 11:51, Ben Asher via swift-dev <swift-dev@swift.org <mailto:swift-dev@swift.org>> wrote:
>
> Hello! Someone recently tipped me off to using -whole-module-optimization flag with -Onone for use during debug builds to speed up compile times. In our project, the speedup feels quite dramatic, but when it gets to the linking step (after compiling both Swift and Obj-C in the project) it fails because ld can't find the individual object files that normally get emitted during the debug-type build presumably because -whole-module-optimization only emits one (and this isn't a normal "-Owholemodule"-type build which works fine).
>
> I can't seem to reproduce this outside of Xcode, but I was curious if anyone has tried this and knows of a workaround to get -whole-module-optimization to work with -Onone in Xcode?
>
> I'm currently using Xcode 8.1 (App Store build) and Swift 3 on macOS Sierra.
>
> Thanks!
>
> Ben
> _______________________________________________
> swift-dev mailing list
> swift-dev@swift.org <mailto:swift-dev@swift.org>
> https://lists.swift.org/mailman/listinfo/swift-dev

--
Ben

--
Ben
_______________________________________________
swift-dev mailing list
swift-dev@swift.org <mailto:swift-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-dev

--
Ben

--
Ben

--
Ben
_______________________________________________
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev


(Ben Asher) #10

Sure thing! I’ll see if I can put small reproducers together tomorrow.

Ben

···

On Thu, Dec 1, 2016 at 7:08 PM, Mark Lacey <mark_lacey@apple.com> wrote:

On Dec 1, 2016, at 5:48 PM, Ben Asher via swift-dev <swift-dev@swift.org> > wrote:

The build failed compiling 3 files in our project. Our app is crashing
that snapshot; here is some output from the failures:

It would be awesome if you could come up with small reproducers for these
and open bugs for them. Alternately, opening a radar with the full project
attached will give us an opportunity to get these fixed ASAP!

Mark

- Assertion failed: (newType == type || (isa<TupleType>(newType) &&
cast<TupleType>(newType)->getNumElements() == 1 &&
cast<TupleType>(newType).getElementType(0) == type)), function
rewriteType, file /Users/buildn
ode/jenkins/workspace/oss-swift-package-osx/swift/lib/SILGen/RValue.h,
line 207.

- SIL verification failed: method's Self parameter should be constrained
by protocol: selfRequirement.getKind() == RequirementKind::Conformance &&
selfRequirement.getFirstType()->isEqual(selfGenericParam) && selfR
equirement.getSecondType()->getAs<ProtocolType>() ->getDecl() == protocol

We're not getting all the way to linking, but compiling everything for
that snapshot took ~11min45s. Based on that, it appears that about a minute
is saved with the new fixes since the Swift 3.0 that shipped with Xcode 8.1
(App Store).

With Xcode 8.2 Beta 2, the app builds fine, and it takes about 12m15s. If
Xcode 8.2 Beta 2 also has those fixes that we expect to speed up the build,
the extra time here (compared to the snapshot) could be explained by
linking, signing, and a few other custom builds scripts that run after
compiling.

Thanks everyone for your help!

Ben

On Thu, Dec 1, 2016 at 4:16 PM, Ben Asher <benasher44@gmail.com> wrote:

I'm trying out DEVELOPMENT-SNAPSHOT-2016-11-29-a now. It's still going,
so I don't have any time results yet. But, I did notice something new (or
maybe existed before but didn't have this warning to expose it). For every
Swift file that's compiled (only using -Onone here), it outputs the same 2
warnings (easy to fix, but not related to any of this) from the same method
in the same Obj-C header referring to the arguments and the return types in
that method:

- "array parameter is missing a nullability type specifier (_Nonnull,
_Nullable, or _Null_unspecified)"
- "inferring '_Nonnull' for pointer type within array is deprecated"

Here's an example of what this method looks like:

+ (NSSet<SomeObject *> *)setWithSELs:(SEL[])sels count:(NSUInteger)count;

Could this point to more duplicated work?

Ben

On Thu, Dec 1, 2016 at 2:50 PM, Ben Asher <benasher44@gmail.com> wrote:

Sure! Thanks for reminding me. I'll follow up on that, test, and get
back to you.

Ben

On Thu, Dec 1, 2016 at 2:48 PM, Mark Lacey <mark_lacey@apple.com> wrote:

On Dec 1, 2016, at 3:13 PM, Ben Asher via swift-dev < >>>> swift-dev@swift.org> wrote:

Just running a quick trial before and after I made this change in our
project, we were previously seeing builds of our main target that took just
under 13min. With this hack, a clean debug build takes about 4.5min.

You may find that recent snapshot builds from swift.org help with your
build times even without enabling -Owholemodule. The redundant type
checking of synthesized accessors which we talked about a month or two
should now be fixed on master, and it would be great to verify that’s the
case with your code and to get an idea of how much it improves your build
times.

Mark

Ben

On Thu, Dec 1, 2016 at 1:33 PM, Ben Asher <benasher44@gmail.com> wrote:

Okay I think that worked! And just to clarify, you meant set
SWIFT_OPTIMIZATION_LEVEL = -Owholemodule and OTHER_SWIFT_FLAGS = -Onone ?

I'll file a radar this afternoon with some details and DM you the
number.

Thanks again!

Ben

On Thu, Dec 1, 2016 at 1:10 PM, Jordan Rose <jordan_rose@apple.com> w
rote:

Xcode needs to know that you're building in WMO mode, so rather than
putting -whole-module-optimization in your "Other Swift Flags", put -Onone
there. It's an ugly hack but it should work in the near term.

We do want to work to make this drastic speed difference go away, so
if you're able we (at Apple) would love to have a source drop of your Swift
3 project, for additional data on where the problems are. Mind filing a
Radar?

Best,
Jordan

> On Dec 1, 2016, at 11:51, Ben Asher via swift-dev < >>>>>> swift-dev@swift.org> wrote:
>
> Hello! Someone recently tipped me off to using
-whole-module-optimization flag with -Onone for use during debug builds to
speed up compile times. In our project, the speedup feels quite dramatic,
but when it gets to the linking step (after compiling both Swift and Obj-C
in the project) it fails because ld can't find the individual object files
that normally get emitted during the debug-type build presumably because
-whole-module-optimization only emits one (and this isn't a normal
"-Owholemodule"-type build which works fine).
>
> I can't seem to reproduce this outside of Xcode, but I was curious
if anyone has tried this and knows of a workaround to get
-whole-module-optimization to work with -Onone in Xcode?
>
> I'm currently using Xcode 8.1 (App Store build) and Swift 3 on
macOS Sierra.
>
> Thanks!
>
> Ben
> _______________________________________________
> swift-dev mailing list
> swift-dev@swift.org
> https://lists.swift.org/mailman/listinfo/swift-dev

--
Ben

--
Ben
_______________________________________________
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

--
Ben

--
Ben

--
Ben
_______________________________________________
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

--
Ben


(Ben Asher) #11

Filed the issues I ran into with repro cases here:

https://bugs.swift.org/browse/SR-3319
https://bugs.swift.org/browse/SR-3321

SR-3321 covers the SIL verification failure. The assertion failure was in
the same file as the one that repro'd the SIL verification failure, but I'm
not seeing it anymore. If/when SR-3321 gets fixed, I'll follow up again and
see if it shows up.

Thanks!

Ben

···

On Thu, Dec 1, 2016 at 7:10 PM, Ben Asher <benasher44@gmail.com> wrote:

Sure thing! I’ll see if I can put small reproducers together tomorrow.

Ben

On Thu, Dec 1, 2016 at 7:08 PM, Mark Lacey <mark_lacey@apple.com> wrote:

On Dec 1, 2016, at 5:48 PM, Ben Asher via swift-dev <swift-dev@swift.org> >> wrote:

The build failed compiling 3 files in our project. Our app is crashing
that snapshot; here is some output from the failures:

It would be awesome if you could come up with small reproducers for these
and open bugs for them. Alternately, opening a radar with the full project
attached will give us an opportunity to get these fixed ASAP!

Mark

- Assertion failed: (newType == type || (isa<TupleType>(newType) &&
cast<TupleType>(newType)->getNumElements() == 1 &&
cast<TupleType>(newType).getElementType(0) == type)), function
rewriteType, file /Users/buildn
ode/jenkins/workspace/oss-swift-package-osx/swift/lib/SILGen/RValue.h,
line 207.

- SIL verification failed: method's Self parameter should be constrained
by protocol: selfRequirement.getKind() == RequirementKind::Conformance &&
selfRequirement.getFirstType()->isEqual(selfGenericParam) && selfR
equirement.getSecondType()->getAs<ProtocolType>() ->getDecl() == protocol

We're not getting all the way to linking, but compiling everything for
that snapshot took ~11min45s. Based on that, it appears that about a minute
is saved with the new fixes since the Swift 3.0 that shipped with Xcode 8.1
(App Store).

With Xcode 8.2 Beta 2, the app builds fine, and it takes about 12m15s. If
Xcode 8.2 Beta 2 also has those fixes that we expect to speed up the build,
the extra time here (compared to the snapshot) could be explained by
linking, signing, and a few other custom builds scripts that run after
compiling.

Thanks everyone for your help!

Ben

On Thu, Dec 1, 2016 at 4:16 PM, Ben Asher <benasher44@gmail.com> wrote:

I'm trying out DEVELOPMENT-SNAPSHOT-2016-11-29-a now. It's still going,
so I don't have any time results yet. But, I did notice something new (or
maybe existed before but didn't have this warning to expose it). For every
Swift file that's compiled (only using -Onone here), it outputs the same 2
warnings (easy to fix, but not related to any of this) from the same method
in the same Obj-C header referring to the arguments and the return types in
that method:

- "array parameter is missing a nullability type specifier (_Nonnull,
_Nullable, or _Null_unspecified)"
- "inferring '_Nonnull' for pointer type within array is deprecated"

Here's an example of what this method looks like:

+ (NSSet<SomeObject *> *)setWithSELs:(SEL[])sels count:(NSUInteger)count;

Could this point to more duplicated work?

Ben

On Thu, Dec 1, 2016 at 2:50 PM, Ben Asher <benasher44@gmail.com> wrote:

Sure! Thanks for reminding me. I'll follow up on that, test, and get
back to you.

Ben

On Thu, Dec 1, 2016 at 2:48 PM, Mark Lacey <mark_lacey@apple.com> >>>> wrote:

On Dec 1, 2016, at 3:13 PM, Ben Asher via swift-dev < >>>>> swift-dev@swift.org> wrote:

Just running a quick trial before and after I made this change in our
project, we were previously seeing builds of our main target that took just
under 13min. With this hack, a clean debug build takes about 4.5min.

You may find that recent snapshot builds from swift.org help with
your build times even without enabling -Owholemodule. The redundant type
checking of synthesized accessors which we talked about a month or two
should now be fixed on master, and it would be great to verify that’s the
case with your code and to get an idea of how much it improves your build
times.

Mark

Ben

On Thu, Dec 1, 2016 at 1:33 PM, Ben Asher <benasher44@gmail.com> w
rote:

Okay I think that worked! And just to clarify, you meant set
SWIFT_OPTIMIZATION_LEVEL = -Owholemodule and OTHER_SWIFT_FLAGS = -Onone ?

I'll file a radar this afternoon with some details and DM you the
number.

Thanks again!

Ben

On Thu, Dec 1, 2016 at 1:10 PM, Jordan Rose <jordan_rose@apple.com> w
rote:

Xcode needs to know that you're building in WMO mode, so rather than
putting -whole-module-optimization in your "Other Swift Flags", put -Onone
there. It's an ugly hack but it should work in the near term.

We do want to work to make this drastic speed difference go away, so
if you're able we (at Apple) would love to have a source drop of your Swift
3 project, for additional data on where the problems are. Mind filing a
Radar?

Best,
Jordan

> On Dec 1, 2016, at 11:51, Ben Asher via swift-dev < >>>>>>> swift-dev@swift.org> wrote:
>
> Hello! Someone recently tipped me off to using
-whole-module-optimization flag with -Onone for use during debug builds to
speed up compile times. In our project, the speedup feels quite dramatic,
but when it gets to the linking step (after compiling both Swift and Obj-C
in the project) it fails because ld can't find the individual object files
that normally get emitted during the debug-type build presumably because
-whole-module-optimization only emits one (and this isn't a normal
"-Owholemodule"-type build which works fine).
>
> I can't seem to reproduce this outside of Xcode, but I was curious
if anyone has tried this and knows of a workaround to get
-whole-module-optimization to work with -Onone in Xcode?
>
> I'm currently using Xcode 8.1 (App Store build) and Swift 3 on
macOS Sierra.
>
> Thanks!
>
> Ben
> _______________________________________________
> swift-dev mailing list
> swift-dev@swift.org
> https://lists.swift.org/mailman/listinfo/swift-dev

--
Ben

--
Ben
_______________________________________________
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

--
Ben

--
Ben

--
Ben
_______________________________________________
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

--
Ben

--
Ben


(Ben Asher) #12

I've also come up with a small repro case that shows the duplicated
bridging header work: https://bugs.swift.org/browse/SR-3338.

Thanks!

Ben

···

On Fri, Dec 2, 2016 at 3:28 PM, Ben Asher <benasher44@gmail.com> wrote:

Filed the issues I ran into with repro cases here:

https://bugs.swift.org/browse/SR-3319
https://bugs.swift.org/browse/SR-3321

SR-3321 covers the SIL verification failure. The assertion failure was in
the same file as the one that repro'd the SIL verification failure, but I'm
not seeing it anymore. If/when SR-3321 gets fixed, I'll follow up again and
see if it shows up.

Thanks!

Ben

On Thu, Dec 1, 2016 at 7:10 PM, Ben Asher <benasher44@gmail.com> wrote:

Sure thing! I’ll see if I can put small reproducers together tomorrow.

Ben

On Thu, Dec 1, 2016 at 7:08 PM, Mark Lacey <mark_lacey@apple.com> wrote:

On Dec 1, 2016, at 5:48 PM, Ben Asher via swift-dev <swift-dev@swift.org> >>> wrote:

The build failed compiling 3 files in our project. Our app is crashing
that snapshot; here is some output from the failures:

It would be awesome if you could come up with small reproducers for
these and open bugs for them. Alternately, opening a radar with the full
project attached will give us an opportunity to get these fixed ASAP!

Mark

- Assertion failed: (newType == type || (isa<TupleType>(newType) &&
cast<TupleType>(newType)->getNumElements() == 1 &&
cast<TupleType>(newType).getElementType(0) == type)), function
rewriteType, file /Users/buildn
ode/jenkins/workspace/oss-swift-package-osx/swift/lib/SILGen/RValue.h,
line 207.

- SIL verification failed: method's Self parameter should be constrained
by protocol: selfRequirement.getKind() == RequirementKind::Conformance &&
selfRequirement.getFirstType()->isEqual(selfGenericParam) && selfR
equirement.getSecondType()->getAs<ProtocolType>() ->getDecl() ==
protocol

We're not getting all the way to linking, but compiling everything for
that snapshot took ~11min45s. Based on that, it appears that about a minute
is saved with the new fixes since the Swift 3.0 that shipped with Xcode 8.1
(App Store).

With Xcode 8.2 Beta 2, the app builds fine, and it takes about 12m15s.
If Xcode 8.2 Beta 2 also has those fixes that we expect to speed up the
build, the extra time here (compared to the snapshot) could be explained by
linking, signing, and a few other custom builds scripts that run after
compiling.

Thanks everyone for your help!

Ben

On Thu, Dec 1, 2016 at 4:16 PM, Ben Asher <benasher44@gmail.com> wrote:

I'm trying out DEVELOPMENT-SNAPSHOT-2016-11-29-a now. It's still
going, so I don't have any time results yet. But, I did notice something
new (or maybe existed before but didn't have this warning to expose it).
For every Swift file that's compiled (only using -Onone here), it outputs
the same 2 warnings (easy to fix, but not related to any of this) from the
same method in the same Obj-C header referring to the arguments and the
return types in that method:

- "array parameter is missing a nullability type specifier (_Nonnull,
_Nullable, or _Null_unspecified)"
- "inferring '_Nonnull' for pointer type within array is deprecated"

Here's an example of what this method looks like:

+ (NSSet<SomeObject *> *)setWithSELs:(SEL[])sels
count:(NSUInteger)count;

Could this point to more duplicated work?

Ben

On Thu, Dec 1, 2016 at 2:50 PM, Ben Asher <benasher44@gmail.com> wrote:

Sure! Thanks for reminding me. I'll follow up on that, test, and get
back to you.

Ben

On Thu, Dec 1, 2016 at 2:48 PM, Mark Lacey <mark_lacey@apple.com> >>>>> wrote:

On Dec 1, 2016, at 3:13 PM, Ben Asher via swift-dev < >>>>>> swift-dev@swift.org> wrote:

Just running a quick trial before and after I made this change in our
project, we were previously seeing builds of our main target that took just
under 13min. With this hack, a clean debug build takes about 4.5min.

You may find that recent snapshot builds from swift.org help with
your build times even without enabling -Owholemodule. The redundant type
checking of synthesized accessors which we talked about a month or two
should now be fixed on master, and it would be great to verify that’s the
case with your code and to get an idea of how much it improves your build
times.

Mark

Ben

On Thu, Dec 1, 2016 at 1:33 PM, Ben Asher <benasher44@gmail.com> w
rote:

Okay I think that worked! And just to clarify, you meant set
SWIFT_OPTIMIZATION_LEVEL = -Owholemodule and OTHER_SWIFT_FLAGS = -Onone ?

I'll file a radar this afternoon with some details and DM you the
number.

Thanks again!

Ben

On Thu, Dec 1, 2016 at 1:10 PM, Jordan Rose <jordan_rose@apple.com> >>>>>>> wrote:

Xcode needs to know that you're building in WMO mode, so rather
than putting -whole-module-optimization in your "Other Swift Flags", put
-Onone there. It's an ugly hack but it should work in the near term.

We do want to work to make this drastic speed difference go away,
so if you're able we (at Apple) would love to have a source drop of your
Swift 3 project, for additional data on where the problems are. Mind filing
a Radar?

Best,
Jordan

> On Dec 1, 2016, at 11:51, Ben Asher via swift-dev < >>>>>>>> swift-dev@swift.org> wrote:
>
> Hello! Someone recently tipped me off to using
-whole-module-optimization flag with -Onone for use during debug builds to
speed up compile times. In our project, the speedup feels quite dramatic,
but when it gets to the linking step (after compiling both Swift and Obj-C
in the project) it fails because ld can't find the individual object files
that normally get emitted during the debug-type build presumably because
-whole-module-optimization only emits one (and this isn't a normal
"-Owholemodule"-type build which works fine).
>
> I can't seem to reproduce this outside of Xcode, but I was
curious if anyone has tried this and knows of a workaround to get
-whole-module-optimization to work with -Onone in Xcode?
>
> I'm currently using Xcode 8.1 (App Store build) and Swift 3 on
macOS Sierra.
>
> Thanks!
>
> Ben
> _______________________________________________
> swift-dev mailing list
> swift-dev@swift.org
> https://lists.swift.org/mailman/listinfo/swift-dev

--
Ben

--
Ben
_______________________________________________
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

--
Ben

--
Ben

--
Ben
_______________________________________________
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

--
Ben

--
Ben

--
Ben