Need help with compile times (type inference?)

David_Turnbull · 2016-01-22T20:25:31.000Z

I'm trying to improve compile times in my project. I've been using
"-Xfrontend -debug-time-function-bodies" to find to sore spots. Here's
something that takes 1300ms to compile:

public func ortho<T:FloatingPointScalarType>

(left:T, _ right:T, _ bottom:T, _ top:T) -> Matrix4x4<T>

{

let t0:T = T(0)

let t1:T = T(1)

let t2:T = T(2)

let r00:T = t2 / (right - left)

let r11:T = t2 / (top - bottom)

let r30:T = -(right + left) / (right - left)

let r31:T = -(top + bottom) / (top - bottom)

return Matrix4x4<T>(

r00, t0, t0, t0,

t0, r11, t0, t0,

t0, t0, -t1, t0,

r30, r31, t0, t1

)

}

I'm not sure what to do from here. My program is unavoidably math heavy.
Every new line of code adds about 60ms of compile time. And that's after I
spend time typing in hints and splitting up the arithmetic.

-david GitHub - AE9RB/SwiftGL: This project has moved.

ddunbar · 2016-01-22T22:23:04.000Z

It doesn't help with this problem, but I use:
-Xfrontend -solver-memory-threshold -Xfronted <SOME-BIG-NUMBER-BUT-LESS-THAN-THE-DEFAULT>
as a way to cause the compiler to give up sooner.

This helps some to identify new troublesome code as it is written, but unfortunately doesn't really help with speeding up code like you posted.

- Daniel

···

On Jan 22, 2016, at 12:25 PM, David Turnbull via swift-users <swift-users@swift.org> wrote:

I'm trying to improve compile times in my project. I've been using "-Xfrontend -debug-time-function-bodies" to find to sore spots. Here's something that takes 1300ms to compile:

public func ortho<T:FloatingPointScalarType>
    (left:T, _ right:T, _ bottom:T, _ top:T) -> Matrix4x4<T>
{
    let t0:T = T(0)
    let t1:T = T(1)
    let t2:T = T(2)

    let r00:T = t2 / (right - left)
    let r11:T = t2 / (top - bottom)

    let r30:T = -(right + left) / (right - left)
    let r31:T = -(top + bottom) / (top - bottom)

    return Matrix4x4<T>(
        r00, t0, t0, t0,
        t0, r11, t0, t0,
        t0, t0, -t1, t0,
        r30, r31, t0, t1
    )
}

I'm not sure what to do from here. My program is unavoidably math heavy. Every new line of code adds about 60ms of compile time. And that's after I spend time typing in hints and splitting up the arithmetic.

-david https://github.com/AE9RB/SwiftGL_______________________________________________
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users

David_Turnbull · 2016-01-23T23:56:06.000Z

I get the impression that there's no immediate solution. Will Swift 3.0
improve build times?

-david

···

On Fri, Jan 22, 2016 at 2:23 PM, Daniel Dunbar <daniel_dunbar@apple.com> wrote:

It doesn't help with this problem, but I use:
  -Xfrontend -solver-memory-threshold -Xfronted
<SOME-BIG-NUMBER-BUT-LESS-THAN-THE-DEFAULT>
as a way to cause the compiler to give up sooner.

This helps some to identify new troublesome code as it is written, but
unfortunately doesn't really help with speeding up code like you posted.

- Daniel

On Jan 22, 2016, at 12:25 PM, David Turnbull via swift-users < > swift-users@swift.org> wrote:

I'm trying to improve compile times in my project. I've been using
"-Xfrontend -debug-time-function-bodies" to find to sore spots. Here's
something that takes 1300ms to compile:

public func ortho<T:FloatingPointScalarType>
    (left:T, _ right:T, _ bottom:T, _ top:T) -> Matrix4x4<T>
{
    let t0:T = T(0)
    let t1:T = T(1)
    let t2:T = T(2)

    let r00:T = t2 / (right - left)
    let r11:T = t2 / (top - bottom)

    let r30:T = -(right + left) / (right - left)
    let r31:T = -(top + bottom) / (top - bottom)

    return Matrix4x4<T>(
        r00, t0, t0, t0,
        t0, r11, t0, t0,
        t0, t0, -t1, t0,
        r30, r31, t0, t1
    )
}

I'm not sure what to do from here. My program is unavoidably math heavy.
Every new line of code adds about 60ms of compile time. And that's after I
spend time typing in hints and splitting up the arithmetic.

-david GitHub - AE9RB/SwiftGL: This project has moved.
_______________________________________________
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users

Joe_Groff · 2016-01-24T00:03:43.000Z

We hope so. Joe Pamer just landed some improvements into the master branch that should improve a lot of common cases, and I believe more improvements are on the way. In the meantime, Bryan Irace recently wrote an article that might help, about using some of the compiler's internal debugging flags to find and improve problem points: http://irace.me/swift-profiling/

-Joe

···

On Jan 23, 2016, at 3:56 PM, David Turnbull via swift-users <swift-users@swift.org> wrote:

I get the impression that there's no immediate solution. Will Swift 3.0 improve build times?

ddunbar · 2016-01-24T04:21:38.000Z

It occurs to me that we should just get SwiftPM to have a feature to aggregate and report that data automatically:
[SR-608] Provide automatic features for exposing Swift function profiling data · Issue #5323 · apple/swift-package-manager · GitHub

This would be a decent package manager starter bug, I'm guessing it isn't all that much work.

- Daniel

···

On Jan 23, 2016, at 4:03 PM, Joe Groff <jgroff@apple.com> wrote:

On Jan 23, 2016, at 3:56 PM, David Turnbull via swift-users <swift-users@swift.org> wrote:

I get the impression that there's no immediate solution. Will Swift 3.0 improve build times?

We hope so. Joe Pamer just landed some improvements into the master branch that should improve a lot of common cases, and I believe more improvements are on the way. In the meantime, Bryan Irace recently wrote an article that might help, about using some of the compiler's internal debugging flags to find and improve problem points: http://irace.me/swift-profiling/

-Joe

David_Turnbull · 2016-01-24T05:34:07.000Z

I've exhausted the method described in that Irace post. He seemed happy
with a 60% improvement. I'm seeing build times 10000% slower than C++. Like
5 minutes vs 3 seconds.

-david

···

On Sat, Jan 23, 2016 at 4:03 PM, Joe Groff <jgroff@apple.com> wrote:

> On Jan 23, 2016, at 3:56 PM, David Turnbull via swift-users < > swift-users@swift.org> wrote:
>
> I get the impression that there's no immediate solution. Will Swift 3.0
improve build times?

We hope so. Joe Pamer just landed some improvements into the master branch
that should improve a lot of common cases, and I believe more improvements
are on the way. In the meantime, Bryan Irace recently wrote an article that
might help, about using some of the compiler's internal debugging flags to
find and improve problem points: http://irace.me/swift-profiling/

-Joe

Chris_Lattner · 2016-01-25T05:55:40.000Z

I've exhausted the method described in that Irace post. He seemed happy with a 60% improvement. I'm seeing build times 10000% slower than C++. Like 5 minutes vs 3 seconds.

Are you willing/able to share the code for your project? That definitely sounds strange,

-Chris

···

On Jan 23, 2016, at 9:34 PM, David Turnbull via swift-users <swift-users@swift.org> wrote:

-david

On Sat, Jan 23, 2016 at 4:03 PM, Joe Groff <jgroff@apple.com <mailto:jgroff@apple.com>> wrote:

> On Jan 23, 2016, at 3:56 PM, David Turnbull via swift-users <swift-users@swift.org <mailto:swift-users@swift.org>> wrote:
>
> I get the impression that there's no immediate solution. Will Swift 3.0 improve build times?

We hope so. Joe Pamer just landed some improvements into the master branch that should improve a lot of common cases, and I believe more improvements are on the way. In the meantime, Bryan Irace recently wrote an article that might help, about using some of the compiler's internal debugging flags to find and improve problem points: http://irace.me/swift-profiling/

-Joe

_______________________________________________
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users

jrose · 2016-01-25T19:41:43.000Z

I commented in the bug, but I'll reply here too: this was a debugging tool added for us, and isn't really ready for the average Swift user. In particular:

- It does a pretty lousy job of identifying functions without names
- It doesn't include type-checking of expressions not in function bodies, like initial values for properties
- It only has a (very basic, unsorted) human-readable output form, instead of something properly tabular
- It can only dump to stderr
- It's only function-granular, not statement-granular
- It only includes statement/expression type-checking (though admittedly that's the part of the compiler most likely to have runaway growth)
- It doesn't provide any guidance on how to make things better

I'd rather not commit to providing this feature "publicly" until these problems are at least addressed, if not fixed.

(I'm fine with letting people know about the debug option. I just don't want to consider it an official, supported feature of Swift.)

Jordan

···

On Jan 23, 2016, at 20:21, Daniel Dunbar via swift-users <swift-users@swift.org> wrote:

It occurs to me that we should just get SwiftPM to have a feature to aggregate and report that data automatically:
[SR-608] Provide automatic features for exposing Swift function profiling data · Issue #5323 · apple/swift-package-manager · GitHub

This would be a decent package manager starter bug, I'm guessing it isn't all that much work.

- Daniel

On Jan 23, 2016, at 4:03 PM, Joe Groff <jgroff@apple.com> wrote:

On Jan 23, 2016, at 3:56 PM, David Turnbull via swift-users <swift-users@swift.org> wrote:

I get the impression that there's no immediate solution. Will Swift 3.0 improve build times?

We hope so. Joe Pamer just landed some improvements into the master branch that should improve a lot of common cases, and I believe more improvements are on the way. In the meantime, Bryan Irace recently wrote an article that might help, about using some of the compiler's internal debugging flags to find and improve problem points: http://irace.me/swift-profiling/

-Joe

_______________________________________________
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users

David_Turnbull · 2016-01-25T08:59:50.000Z

Soitenly: GitHub - AE9RB/SwiftGL: This project has moved.

The 28,000 lines of loader code are fine. The 6,000 lines of math libraries
are the problem.

I'm sure it's something to do with prototypes and generics. You can change
in Types.swift:
public protocol FloatingPointScalarType : ScalarType
to:
public protocol FloatingPointScalarType : ScalarType, FloatingPointType
and make the problem a bit worse. This is something I'd actually like to
use, except I don't because a few "where constraints" do what I need
without the build slowdown.

Swift 2.1 or 2.2-dev doesn't make a difference. The C++ compiler I bench
against is also llvm. The compiled binaries are truly fast (with WMO). It's
only the development process that's too slow because of build times.

-David "nyuk nyuk nyuk" Turnbull

···

On Sun, Jan 24, 2016 at 9:55 PM, Chris Lattner <clattner@apple.com> wrote:

Are you willing/able to share the code for your project? That definitely
sounds strange,

ddunbar · 2016-01-25T22:52:22.000Z

Understood. I had presumed designing a proper interface for Swift to provide that information out as a part of the higher level feature (although I guess I didn't make that clear in the bug).

- Daniel

···

On Jan 25, 2016, at 11:41 AM, Jordan Rose <jordan_rose@apple.com> wrote:

I commented in the bug, but I'll reply here too: this was a debugging tool added for us, and isn't really ready for the average Swift user. In particular:

- It does a pretty lousy job of identifying functions without names
- It doesn't include type-checking of expressions not in function bodies, like initial values for properties
- It only has a (very basic, unsorted) human-readable output form, instead of something properly tabular
- It can only dump to stderr
- It's only function-granular, not statement-granular
- It only includes statement/expression type-checking (though admittedly that's the part of the compiler most likely to have runaway growth)
- It doesn't provide any guidance on how to make things better

I'd rather not commit to providing this feature "publicly" until these problems are at least addressed, if not fixed.

(I'm fine with letting people know about the debug option. I just don't want to consider it an official, supported feature of Swift.)

Jordan

On Jan 23, 2016, at 20:21, Daniel Dunbar via swift-users <swift-users@swift.org <mailto:swift-users@swift.org>> wrote:

It occurs to me that we should just get SwiftPM to have a feature to aggregate and report that data automatically:
[SR-608] Provide automatic features for exposing Swift function profiling data · Issue #5323 · apple/swift-package-manager · GitHub

This would be a decent package manager starter bug, I'm guessing it isn't all that much work.

- Daniel

On Jan 23, 2016, at 4:03 PM, Joe Groff <jgroff@apple.com <mailto:jgroff@apple.com>> wrote:

On Jan 23, 2016, at 3:56 PM, David Turnbull via swift-users <swift-users@swift.org <mailto:swift-users@swift.org>> wrote:

I get the impression that there's no immediate solution. Will Swift 3.0 improve build times?

We hope so. Joe Pamer just landed some improvements into the master branch that should improve a lot of common cases, and I believe more improvements are on the way. In the meantime, Bryan Irace recently wrote an article that might help, about using some of the compiler's internal debugging flags to find and improve problem points: http://irace.me/swift-profiling/

-Joe

_______________________________________________
swift-users mailing list
swift-users@swift.org <mailto:swift-users@swift.org>
https://lists.swift.org/mailman/listinfo/swift-users

Joe_Pamer · 2016-01-25T18:49:12.000Z

Thanks for sharing, David!

As JoeG mentions below, I’ve been rolling out a series of major improvements in this area. I’ll take a look at SwiftGL to see what else can be done.

- Joe

···

On Jan 25, 2016, at 12:59 AM, David Turnbull via swift-users <swift-users@swift.org> wrote:

On Sun, Jan 24, 2016 at 9:55 PM, Chris Lattner <clattner@apple.com <mailto:clattner@apple.com>> wrote:
Are you willing/able to share the code for your project? That definitely sounds strange,

Soitenly: GitHub - AE9RB/SwiftGL: This project has moved.

The 28,000 lines of loader code are fine. The 6,000 lines of math libraries are the problem.

I'm sure it's something to do with prototypes and generics. You can change in Types.swift:
public protocol FloatingPointScalarType : ScalarType
to:
public protocol FloatingPointScalarType : ScalarType, FloatingPointType
and make the problem a bit worse. This is something I'd actually like to use, except I don't because a few "where constraints" do what I need without the build slowdown.

Swift 2.1 or 2.2-dev doesn't make a difference. The C++ compiler I bench against is also llvm. The compiled binaries are truly fast (with WMO). It's only the development process that's too slow because of build times.

-David "nyuk nyuk nyuk" Turnbull

_______________________________________________
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users

David_Turnbull · 2016-01-26T17:57:11.000Z

Awesome! Any attention is appreciated.

I tried the 3.0-dev snapshot from yesterday. The ortho example is slower
than 2.2-dev which is slower than 2.1.

If you get compiler crashes, comment out the any() and all() functions. I
opened a bug on this.

github.com/apple/swift

[SR-622] SIL verification failed on reduce for CollectionType

opened 05:27PM - 26 Jan 16 UTC

closed 08:31PM - 25 Feb 16 UTC

swift-ci

bug compiler crash

| | | |------------------|-----------------|… |Previous ID | SR-622 | |Radar | None | |Original Reporter | dturnbull (JIRA User) | |Type | Bug | |Status | Closed | |Resolution | Done | <details> <summary>Environment</summary> Apple Swift version 3.0-dev (LLVM f95d47afa7, Clang f66c5bb67b, Swift b745691a38) January 25, 2016 Snapshot </details> <details> <summary>Additional Detail from JIRA</summary> | | | |------------------|-----------------| |Votes | 1 | |Component/s | Compiler | |Labels | Bug, CompilerCrash | |Assignee | None | |Priority | Medium | md5: 46e0e2577b19e4e04826e324f60fc6f2 </details> **Issue Description:** This code works on Swift 2.1. On 3.0-dev the compiler crashes. The log says: "SIL verification failed: entry point has wrong number of arguments: entry-\>bbarg_size() == ti-\>getParameters().size()" ``` swift public protocol GLmathType : MutableCollectionType { typealias Element } // This is ok public func any<bvec:GLmathType>(x:bvec) -> Bool { return x.reduce(false) { $0 || $1 as! Bool } } // SIL verification failed: entry point has wrong number of arguments public func any<bvec:GLmathType where bvec.Generator.Element:BooleanType, bvec.Element == Bool >(x:bvec) -> bvec.Element { return x.reduce(false) { $0 || $1 } } ```

If I can help, please let me know.

-david

···

On Mon, Jan 25, 2016 at 10:49 AM, Joe Pamer <jpamer@apple.com> wrote:

Thanks for sharing, David!

As JoeG mentions below, I’ve been rolling out a series of major
improvements in this area. I’ll take a look at SwiftGL to see what else can
be done.

- Joe

On Jan 25, 2016, at 12:59 AM, David Turnbull via swift-users < > swift-users@swift.org> wrote:

On Sun, Jan 24, 2016 at 9:55 PM, Chris Lattner <clattner@apple.com> wrote:

Are you willing/able to share the code for your project? That definitely
sounds strange,

Soitenly: GitHub - AE9RB/SwiftGL: This project has moved.

The 28,000 lines of loader code are fine. The 6,000 lines of math
libraries are the problem.

I'm sure it's something to do with prototypes and generics. You can change
in Types.swift:
public protocol FloatingPointScalarType : ScalarType
to:
public protocol FloatingPointScalarType : ScalarType, FloatingPointType
and make the problem a bit worse. This is something I'd actually like to
use, except I don't because a few "where constraints" do what I need
without the build slowdown.

Swift 2.1 or 2.2-dev doesn't make a difference. The C++ compiler I bench
against is also llvm. The compiled binaries are truly fast (with WMO). It's
only the development process that's too slow because of build times.

-David "nyuk nyuk nyuk" Turnbull

_______________________________________________
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users

Joe_Pamer · 2016-01-26T21:54:52.000Z

Ok, thanks, David - I’ll take a look and see what’s going on here.

- Joe

···

On Jan 26, 2016, at 9:57 AM, David Turnbull <dturnbull@gmail.com> wrote:

Awesome! Any attention is appreciated.

I tried the 3.0-dev snapshot from yesterday. The ortho example is slower than 2.2-dev which is slower than 2.1.

If you get compiler crashes, comment out the any() and all() functions. I opened a bug on this.
[SR-622] SIL verification failed on reduce for CollectionType · Issue #43239 · apple/swift · GitHub

If I can help, please let me know.

-david

On Mon, Jan 25, 2016 at 10:49 AM, Joe Pamer <jpamer@apple.com <mailto:jpamer@apple.com>> wrote:
Thanks for sharing, David!

As JoeG mentions below, I’ve been rolling out a series of major improvements in this area. I’ll take a look at SwiftGL to see what else can be done.

- Joe

On Jan 25, 2016, at 12:59 AM, David Turnbull via swift-users <swift-users@swift.org <mailto:swift-users@swift.org>> wrote:

On Sun, Jan 24, 2016 at 9:55 PM, Chris Lattner <clattner@apple.com <mailto:clattner@apple.com>> wrote:
Are you willing/able to share the code for your project? That definitely sounds strange,

Soitenly: GitHub - AE9RB/SwiftGL: This project has moved.

The 28,000 lines of loader code are fine. The 6,000 lines of math libraries are the problem.

I'm sure it's something to do with prototypes and generics. You can change in Types.swift:
public protocol FloatingPointScalarType : ScalarType
to:
public protocol FloatingPointScalarType : ScalarType, FloatingPointType
and make the problem a bit worse. This is something I'd actually like to use, except I don't because a few "where constraints" do what I need without the build slowdown.

Swift 2.1 or 2.2-dev doesn't make a difference. The C++ compiler I bench against is also llvm. The compiled binaries are truly fast (with WMO). It's only the development process that's too slow because of build times.

-David "nyuk nyuk nyuk" Turnbull

_______________________________________________
swift-users mailing list
swift-users@swift.org <mailto:swift-users@swift.org>
https://lists.swift.org/mailman/listinfo/swift-users