How to best fix this old Random API bug?

Hi!

Almost two years ago I started a thread about a design issue in the Random API. A couple of fixes were discussed: One wasn't really solving the problem, one was AFAICS forgotten and one was pitched as a fix but turned out to be source breaking and not motivated.

Although the old thread and related links contain all the background, I will try to give a complete description of the situation here.


This program

struct MyGenerator : RandomNumberGenerator {}
var generator = MyGenerator()
print(Int.random(in: 1 ... 6, using: &generator))

will (somewhat unexpectedly) compile fine, but it will crash at runtime (debug build) or execute forever at 100% CPU (release build).

This is because the Random API defines a default implementation of its only requirement, and this default implementation ends up calling that same requirement.


RandomNumberGenerator is a very simple protocol:

public protocol RandomNumberGenerator {
  /// Returns a value from a uniform, independent distribution of binary data.
  mutating func next() -> UInt64
}

And as far as I can see, this requirement was never meant to have a default implementation (why should it?). It seems to have ended up being there by accident, and is now a cause of (at least mild) confusion/inconvenience, that is: Anyone who writes a custom pseudorandom number generator in Swift will be slightly surprised and/or annoyed that the usual compile time error and "Do you want to add protocol stubs?" won't be there, and then they might wonder why the API has been designed this way, etc.


The following (self contained) program demonstrates the issue in a reduced form and makes it easy for anyone to quickly try out the solutions below.

// Corresponding stdlib code here: https://github.com/apple/swift/blob/master/stdlib/public/core/Random.swift
protocol Rng {
    mutating func next() -> UInt64
}
extension Rng {
    public mutating func next<T>() -> T
        where T : FixedWidthInteger & UnsignedInteger
    {
        return T._random(using: &self)
    }
}

// Corresponding stdlib code here: https://github.com/apple/swift/blob/d0af9eee9611046bc70094f6af437a4a0f733ecd/stdlib/public/core/Integers.swift#L3359
extension FixedWidthInteger {
    static func _random<R: Rng>(using generator: inout R) -> Self {
        if bitWidth <= UInt64.bitWidth {
            return Self(truncatingIfNeeded: generator.next())
        }
        fatalError("support for type \(self) has been cut out from this demo")
    }
}

// We'd like to implement a simple pseudo random number generator:
struct SplitMix64 : Rng {
    // But here, we notice that the type unexpectedly just conforms,
    // ie no error and no "add protocol stubs", just unexpected acceptance.
}

Solution / workaround 1:

I'd say no, because this wouldn't solve the design error, and as already mentioned above:

which means that the next fix is:

Solution 2:

This, IMHO, is both the simplest and best solution that has been proposed.

Solution 3

This seems to work, or does work, in current Swift, but it is an open bug that makes it work:

Solution 4:

But:

The core team discussed this and while the motivation is sound, this would be a source-breaking change so not sufficiently motivated. The issue with accidental infinite recursion occurs in other APIs and can't be fixed so easily in those cases, so ideally we'd find a way of enhancing the language to help users avoid this situation in a more general way.


Question:

Is there any reason why solution 2 won't work?

3 Likes

Unfortunately renaming a protocol requirement breaks ABI :slightly_frowning_face:

I was afraid that was the case : /

We don't have the feature yet, but it would be reasonable to have an attribute to allow a declaration to have a different ABI name from its source level name.

3 Likes

Although, perhaps I could suggest another solution that (could?) potentially work. We could rename the generic version (will cause source breakage), but add a new attribute that mangles the function to its old ABI name. Looks like @Joe_Groff beat me to it. Edit: perhaps with this new attribute we could instead just rename the requirement.

I agree with the core team's appraisal here that you quoted above. There are a variety of situations where naturally we would have a non-generic requirement and a generic API with a default implementation based on that requirement, where the more logical name for both would be the same.

It would be nice if, even as a private underscored attribute for now, we could mark such situations so that the compiler could diagnose it (to throw out a name for ease of reference: @_requiresConcreteImplementation). I would think it suboptimal to have to rename one or the other API just because the compiler doesn't do the right thing currently, since that is a surmountable problem. I think it'd be even more suboptimal to have source-breaking changes for that reason (or, for that matter, to invent a new attribute to allow ABI-compatible source-breaking renamings when the alternative is to invent a new attribute not to have to rename things at all).

In the meantime, we could improve the documentation to call out this potential pitfall; there is already a section titled "Conforming to the RandomNumberGenerator Protocol," and that should absolutely tell users what they need to do. It isn't enough but it's an imminently doable first step that hasn't been done yet.

6 Likes
6 Likes