Benchmarks game?

Using Swift
21

Thanks

add the following line to the dependencies in your Package.swift file

I don't have a Package.swift file.

Is there something that doesn't require more than import Dispatch or import Foundation

22

The problem is that I don't see how those facts have any bearing on your intention, which could be clarified if you would answer the questions I asked here.

The larger issue is that these questions aren't answered by the website, but the language used there suggests possible interpretations (which can of course be different for different readers). Depending on what you actually mean, those interpretations could be wildly off-base. Right now it's very unclear what makes an acceptable #8 program.

1 Like
23

fannkuch-redux Swift #8 is apparently "Naive transliteration from Rex Kerr's Scala program".

n-body Swift #8 is "Naive transliteration from Michael Ferguson's Chapel program"

etc

24

More line-by-line literal.

As-it-says:

When programs are "idiomatic" someone will say they are not "comparable".

25

That's a start. Now we know different programs can be transliterated from different languages.

It doesn't answer this:

Nor does it give me an answer to this:

which I think is at the heart of the issue.

As-it-says:

Quoting the website isn't helpful here, because the problem is I'm having trouble interpreting what the website says.

1 Like
26

That was not in reply to you.

27

This works for me:

#if os(Linux) || os(FreeBSD)
import Glibc
#else
import Darwin
#endif

let f = fopen("./one", "w")
fclose(f)

Unfortunately conditional import is needed for reasons explained in this thread.

1 Like
29

Thanks. Of course short-term-ism : fclose(fopen("./one", "w"))

30

Or was there something about Swift that could be "improved" ?

This is not true. We can hit this case. It requires further work in the ARC optimizer. We some time ago actually did have the functionality to hit this case, but I had to disable it b/c of some correctness issues.
File a bug with this test case and assign to me.

31

Now the phrase is: "transliterated line-by-line literal style".

Maybe that will be less contentious.

Transliterated line-by-line seems to have worked-out OK for this Swift fannkuch-redux program.

33

That has less of a chance of being misunderstood, thank you. But there's still no indication (that I could find) of which language's version forms the basis for the translations. Making that very clear would keep someone like me from assuming that the best-performing implementation—or whatever other mistaken criterion you choose—is the basis, seeing that some version doesn't look like a literal translation of that mistaken basis, and wasting time submitting a version that will be rejected. The reference to the C program here still can be taken to imply that the C code is the basis for translation of that benchmark.

It seems to me that choosing a relatively high-level language as the basis for any of these tests is naturally biased towards lower-level languages. Because they have no abstractions corresponding to those in the higher-level language, no translation can truly be literal and you could write almost any code. I do realize that the description pages tend to mitigate against this bias by constraining the benchmarks. Anyway, that's just an observation; nothing to be done about it.

For everyone else, the source of @Isaac_Gouy 's quote is here. It seems they are implying that there is an optimization someone should have made to the swift backend, essentially turning that pass-by-value in to a pass-by-borrowing.

Follow-up: making the methods static so I could control the convention of the self parameter and changing it to borrowing made no difference. And using inout slowed it down considerably.

34

Not that it would likely change the results of your test, but don't we support explicitly borrowing methods even on copyable types?

35

I don't know. For class types? We don't support mutating so I just assumed. Feel free to try yourself, obviously.

36

It works for me, but I haven't benchmarked it. Here's the original class based version with ownership applied: Compiler Explorer

Source code: 
// adapted from original implementation submitted by Isaac Gouy:
// https://benchmarksgame-team.pages.debian.net/benchmarksgame/program/binarytrees-swift-8.html
final class Tree {
  private var left, right: Tree?

  init(_ left: Tree?, _ right: Tree?) {
    self.left = left
    self.right = right
  }

  static func with(_ depth: Int) -> Tree {
    return (depth == 0)
      ? Tree(nil, nil)
      : Tree(with(depth - 1), with(depth - 1))
  }

  borrowing func nodeCount() -> Int {
    return (left == nil)
      ? 1
      : 1 + left!.nodeCount() + right!.nodeCount()
  }

  // Honestly, we should just go ahead and allow mutating methods
  // directly on classes. It's niche, but we can already implement
  // them with protocol extensions, and we allow the other two
  // ownership specifiers.
  consuming func clear() {
    // this deletes all of the nodes in the tree, since we never
    // escaped any of them.
  }
}

func main(_ n: Int) {
  let minDepth = 4
  let maxDepth = minDepth + 2 > n ? minDepth + 2 : n
  let stretchDepth = maxDepth + 1

  stretch(stretchDepth)
  let longLivedTree = Tree.with(maxDepth)

  for depth in stride(from: minDepth, to: stretchDepth, by: 2) {
    let iterations = 1 << (maxDepth - depth + minDepth)
    var sum = 0
    for _ in 1...iterations {
      sum += count(depth)
    }
    print("\(iterations)\t trees of depth \(depth)\t check: \(sum)")
  }
  let count = longLivedTree.nodeCount()
  longLivedTree.clear()
  print("long lived tree of depth \(maxDepth)\t check: \(count)")
}

func stretch(_ depth: Int) {
  print("stretch tree of depth \(depth)\t check: \(count(depth))")
}

func count(_ depth: Int) -> Int {
  let t = Tree.with(depth)
  let c = t.nodeCount()
  t.clear()
  return c
}

do {
  let n =
    if CommandLine.argc > 1 {
      Int(CommandLine.arguments[1])!
    } else {
      10
    }
  main(n)
}

Edit: It seems like the implicit recursive deinit scales very poorly, and it crashes Compiler Explorer when n >= 21.

Edit 2: It actually might be the recursive constructor.

Edit 3: Tried it out on my machine, as both recursive and iterative, and it's actually just so slow that Godbolt kills it when n >= 21. I'm not even sure if the iterative version is faster than the recursive version.

Edit 4:
Is there any reason your enum version didn't go with much simpler definition of:

enum Tree {
    case leaf
    indirect case node(Tree, Tree)
    init(with depth: Int) {
        self = if depth == 0 {
            .leaf
        } else {
            .node(Self(with: depth - 1), Self(with: depth - 1))
        }
    }
    borrowing func nodeCount() -> Int {
        switch self {
        case .leaf: 1
        case .node(let left, let right):
            left.nodeCount() + right.nodeCount() + 1
        }
    }
    mutating func clear() {
        self = .leaf
    }
}
37

No good reason; I think I didn't observe the invariant that there were no nodes with one child until late, and then didn't think to go back and update the data structure.

You can't really drop the body of clear() per the rules of the game.

38

Funnily enough, this doesn't actually enforce that invariant. Each Tree can be a .leaf, which means that a node could still contain a single leaf and a single node. It's just that we never construct any such nodes.

When you say we aren't allowed drop the body of clear(), I'm assuming you mean we can't just make it a consuming function, but the mutating version that sets self to .leaf is just fine, yes?

Edit: Actually, according to the highest ranking (C++) submission, we not only don't need clear() at all we can use memory pools? binary-trees C++ g++ #7 program (Benchmarks Game)

The second place submission (in Rust) also uses bump allocators. binary-trees Rust #5 program (Benchmarks Game)

I thought these were against the rules? Or is that only true when the language doesn't have them in the standard library?

39

As I said above, I suggest you don't waste your time with this benchmark. I know other open-source PL devs who have completely given up on it.

2 Likes
40

Sure, but we should try and convince those users to look elsewhere.

I don't have anything handy, as I pretty much stopped looking at such multi-language benchmark sites more than a decade ago. I remember liking Rosetta code back then though, as it just has a bunch of code samples that you can compare for a ton of languages, many more languages than any benchmark I've ever seen, and you can easily compare the language itself for many functions. If you're worried about speed or other benchmarks, you can simply manually build whatever interests you and compare the metrics you care about yourself.

41

Stumbled upon this, another example of what seems to be someone attempting intuitive coding in various languages and having terrible performance in Swift: https://www.youtube.com/watch?v=m4c38NS43cE.

1 Like
42

This code is basically Swift's worst nightmare. Xcode would die if it had to open it. https://raw.githubusercontent.com/hcsalmon1/Chess-Engine-Test/refs/heads/main/Swift/main.swift

2 Likes