SE-0202 Amendment Proposal: Rename Random to DefaultRandomNumberGenerator

Evolution Proposal Reviews
82

If your reply was to my above post (forgive me but I'm not really sure) then I was not worried about finding the RandomNumberGenerator protocol. I'm worried that performance issues and over all design issues, like @xwu just posted about, will stop me from using the Random API, which means I'll just have to continue using my own.

83

Yeah I was, sorry I didn't see @xwu's addition until after I had replied to you ā€” I can see why it now looks muddled in the thread. I don't have serious knowledge of random generation algorithms etc, so am I largely deferring to others with more experience ā€” like yourself ā€” on those areas.

84

I donā€˜t have numbers, but imho it doesnā€™t matter much wether the default generator is cryptographically secure:
A competent expert will make sure not to use an insecure generator - and crypto written by amateurs is always a bad idea, even if the library gives them a perfect generator.

1 Like
85

Well most areas where randomness is needed lie outside the area of cryptography...

86

To echo what Xiaodi stated, it feels there are different reasons for this amendment from Benjamin and Ben. I feel Benjamin's reasons for changing the name is because of the ambiguity of the name, rather than a misuse. As Xiaodi points out, this name is justified for a number of reasons:

As for Ben's justification for this rename, I feel this is the only good argument for changing the name. Implementations like:

let x = Int(Random.default.next() % 10)

are incorrect as Ben points out; this introduces modulo bias. Now for the idea that users shouldn't use Random.default directly (or any generator), I don't entirely agree with.

let x: UInt8 = Random.default.next()
let y: UInt32 = Random.default.next(upperBound: 10)

Implementations like these are not preferred as you can use the higher level APIs, but that doesn't mean there's anything wrong with them. There's nothing inherently wrong with using generators this way, it's just not preferred as to something like:

let x = UInt8.random(in: .min ... .max)
let y = UInt32.random(in: 0 ..< 10)

I don't believe renaming the stdlib rng would prevent implementations like Ben point out. The underlying issue would be the design of RandomNumberGenerator allowing for such implementations. We could redesign the protocol to only have _fill(bytes:) (or however we bikeshed it to...), but it would most likely become a very cumbersome protocol to conform to. This would prevent implementations where users are tempted to use % while at the same time keeping the justified name Random for the stdlib rng.

2 Likes
87

The naming war leads to methods like this:

let result = Measurement(value: 45, unit: UnitAngle.degrees)
        .converted(to: .radians).value

This is madness because the direct conversion by hand is only half the length:

let radians = degrees / 180 * 3.1415

Let's face it - We have a bad naming problem in many parts of Swift 8(

88

Iā€˜d rather say the used framework was not designed for Swift... but this is derailing to much ;-)

89

Okay, I have never posted here before (just signed up) but having read through this very interesting thread, I thought I'd add my two cents (though I am not sure it really matters that much at this late stage).

I was happy to see this addition to Swift, since I've been using a home-grown wrapper to accomplish essentially the same goals for some time. Mine is a singleton with the name Randomizer, which uses the singleton reference format Randomizer.sharedInstance. I suppose a simple shared would have been adequate.

I mention this only because I wanted to point out a few things about the naming:

  1. The word "random" is an adjective, not a noun. Types should have noun names, unless they are protocols which are essentially a type of descriptor (such as Encodable).
  2. Names for common utilities should be short if possible.
  3. The word "default" does carry with it an implication that it's an inferior reference implementation, meant to be replaced in "real" use.

My humble opinion is that there is no such thing as a truly perfect name, since any choice is a balance between readability and clarity. Random certainly achieves high marks on both of these, but fails a bit in the sense that it doesn't read like a type name.

I therefore suggest a name such as RandomSource (or Randomizer) which preserves the clarity and readability while also reading more like a type name.

1 Like
90

Part of Foundation:

https://developer.apple.com/documentation/foundation/measurement

91

Jens, I agree with all the facts you brought up, but disagree on their interpretation. I know that performance concerns are close to your heart, so let me walk you through my understanding of the issues you raised and letā€™s see if we agree in the end.

In Swift Evolution we are trying to design perfect API in an imperfect language. Which means that while we are aiming for an ideal solution, we have to make compromises to accommodate the reality of Swift compiler and language as it stands today. It would be nice if we could hide all the accidental warts of the implementation behind a clean surface, but leaky abstractions do not let us achieve this utopia. Iā€™ll attempt to come up with my own, incomplete, second amendment of SE-0202 to separate the clean interface we strive for, from the accidental details of the implementation that fall out of a drive for maximum performance. Iā€™ll point those out separately at the end. Keep in mind that naming and usage patterns established in standard library set a strong precedent for the whole language, so we have a tough balancing act in front of us. Also note that all of what Iā€™ll write (sans renaming) is already present in SE-0202 and itā€™s current implementation, whether it is readily apparent or not. Obviously, the discussion here shows that all of this subtlety lies in the eye of the beholder.

Random Unification

Main focus of this proposal is to create a unified random API, and access to secure source of randomness for all platforms. On top of this core it introduces utilities for generating random numbers and collection extensions for sampling and shuffling.

Randomness Generators

There are various techniques for generating random data, ranging from hardware generators to an ever-growing number of pseudorandom generators. They are making different trade-offs between computational difficulty and quality of the random distribution. Choosing the right kind of randomness generator depends on the application domain, there is no one-size-fits-all solution. To accommodate this we introduce:

public protocol RandomnessGenerator {
    mutating func bits() -> UInt64
}

Default Randomness Generator

The standard library will provide single, platform specific implementation of randomness generator. Platform vendors are free to choose the most appropriate solution. They should aim to provide cryptographically secure and reasonably performant implementation. Specifics of their implementation must be clearly documented (especially if they cannot meet the cryptographic security goal).

Access to a thread-safe instance of the default randomness generator is provided through the .default property on the type Random.

Custom Randomness Generators

The random API is designed to support custom randomness generator implementations. We expect that various implementations of seedable pseudorandom number generators will exist outside of standard library. To accommodate implementations based on classes as well as value types, the random API methods are declared mutable and take an inout using: argument of a type conforming to the RandomnessGenerator protocol.

Details about obtaining instances of custom generators are left intentionally unspecified, because we consider them to be out of scope for this proposal.

Type Random

To allow for implementation flexibility of the platform provided default randomness generator, its thread-safe instance is obtained from a static getter on the type Random. Because using type erasure would have prohibitive impact on performance, the precise return type is intentionally left out of this proposal, beyond the requirement that it returns a mutable and thread-safe instance of a type conforming to the RandomnessGenerator protocol.

let generator = Random.default

This also opens up a room for anchoring user extensions in a central and easily discoverable place. For example: a custom, seedable, pseudorandom number generator might provide a convenience initialization of an instance seeded from the default randomness generator as an extension on the type Random.

Applied Randomness

Randomness generators are used by higher-level APIs to turn the random bits into concrete applications. Intended usage pattern is to supply the method with randomness generator through the using: argument. When no custom randomness generator is provided, the Random.default will be used.

Other clients of RandomGenerator are expected to adopt the same pattern:

TK example from @nnnnnnnn that uses RG to build Gaussian distributions

Examples below assume an instance of custom randomness generator is available in the context:

// hypothetical generator, not provided by standard library
var customGenerator = Xoroshiro(seededWith: &Random.default)

Following, most common use cases are provided by the standard library.

Random Number Generation

Extensions methods for FixedWidthInteger and BinaryFloatingPoint for generating numbers in a given range. These implementations avoid the modulo bias commonly introduced in naive hand-rolled solutions.

Double.random(in: 0 ... .pi, using: &customGenerator)
Float.random(in: 0 ..< 1)
Int.random(in: 0 ..< 10, using: &customGenerator)
UInt.random(in: .min ... .max) // Full width

Coin Flipping

Bool.random()
Bool.random(using: &customGenerator)

Collection Sampling

New randomElement extension method on Collection protocol returns a random element from the collection. To account for empty collection case this method returns an Optional, modeled after collection methods like .first and .last.

let dice = 1...6 
// Rolling the šŸŽ²; returns Optional<Int>
dice.randomElement()!
dice.randomElement(using: &customGenerator)!

Shuffling

New shuffled extension method on Sequence is added that returns the elements from sequence in random order as a new Array.

let dnaFragment = "AUGAAATGAACGUAG"
// Simulate DNA mutations
dnaFragment.shuffled() // New scrambled sequence: `[Character]`
dnaFragment.shuffled(using: &customGenerator)

For a MutableCollection weā€™ve added a mutating method shuffle, which randomly reorders the collection elements in place.

var cardDeck = "ā™„ļøā™£ļøā™¦ļøā™ ļø".map { 
   suit in "A23456789šŸ”ŸJQK".map {
       rank in "\(rank)\(suit)" }}
// Reorder the array elements in place
cardDeck.shuffle()
cardDeck.shuffle(using: &customGenerator)

Iā€™ve tried to extract what I saw as the essence of the SE-0202, removing unnecessary implementation details from the proposal, so that the resulting specification leaves us more wiggle room with final implementation. At the same time I took a liberty to rename certain parts of the API to further separate concepts whose conflation leads to misuse potential @Ben_Cohenā€™s concerned about:

  • RandomnessGenerator (instead of RandomNumberGenerator) that vends .bits() to further separate the two concepts and steer users that need numbers towards the proper ranged API on the numeric types. I think that let i = Int(Random.default.bits() % n) is plenty of warning for anybody not seeking to intentionally abuse the API. Omitting the Numbers from the core protocol name could help steer learners towards the right place. In worst case we could still fall back to .defaultGenerator as was suggested before by @gwendal.roue.
  • I speak generically about type Random to avoid committing the implementation to specific choice between struct or enum and I leave out the concrete return type of the .default getter in case we need to adjust that due to memory ownership requirements as @Joe_Groff mentioned. Main point was to focus on its role as facade, rather then on the details of how thatā€™s achieved in practice (we can have struct Random do double duty for minimal stdlib footprint, or have enum Random and SystemDefaultRandomnessGenerator split the duties ā€” all just an implementation detail).

My point is that these aspects should be discussed in code review on GitHub PR, not on SE. We went too deep down the implementation rabbit hole, because SE-0202 as it stands commits to too many accidental implementation details, yet still leaves some important questions unaddressed. I didnā€™t go into the Detailed Design section, but if we really want to properly amend SE-0202, that part would definitely need more detailed specification of the guaranties provided by the higher-level RNG methods as most-clearly summarized by @jawbroken, which youā€™ve raised in that other thread:

This isnā€™t meant as a critique of excellent job @Alejandro did driving the original proposal forward. Itā€™s just a symptom of how it was lifted from the implementation. It is an aspect our process, which now requires working implementation for SE proposals, we need to be more cognizant of in the future. I hope I didnā€™t mess up too badly in my reinterpretation and @Alejandro, @benrimmington, @lorentey and others who drove and reviewed the implementation together, would correct me if I did.

While searching online for the Xoroshiro initializer for my example, I came across the RandomKit library by Nikolai Vazquez (who was last active on SE back in Dec '15). I havenā€™t seen it mentioned anywhere on the pitch thread, but the final SE-0202 looks like a stripped down, minimal version of RandomKit. The similarity is remarkable, down to defining:

Int.random(in: 0 ..< 1, using: &randomGenerator)

But his version returns optional. His core protocol is named RandomGenerator and itā€™s also passaed as inout. Iā€™m assuming independent invention here, showing how performance demands ultimately drive the implementation design. I think it is also worth exploring RandomKit for future directions and get the author involved in SE process again. :v:

92

I could probably agree with most of the above. But I don't think the topic of this thread (the name change) is that important, and I agree with @xwu's post. And more generally, I still stand by ā€¦

ā€¦ most of what I said ā€¦

ā€¦ in the proposal review.

And more importantly ā€¦

ā€¦ I think it would have been better to simultaneously discuss, design, use, test and benchmark this Random API as a stand alone implementation (not a framework, just a couple of source files to be used module internally (for simplicity and optimizability-reasons) within a set of non-trivial sample-projects that would put it to actual and heavy use), before adding it to do standard library.

This would have given its design/implementation some valuable time without having to be incrementally evolved, making it possible to evaluate instead of just speculate about the pros and cons of various designs and implementations.

Big changes to the Random API might be hard to get through, once it has landed in the standard library, which I think is unfortunate.

I'd for example like to see if would be possible to increase both composability AND (I'm quite sure) performance by factoring out some of the range conversion aspects of the current Random API to a separate Range Conversion API (I'm not sure if range conversion is the proper term here), ie I have this UInt64 value (could be random, doesn't matter), and I want to convert it from the full range of its type (UInt64) to the full range of some other fixed width uint type, or to the full range of a certain number of bits, or to a unit range Float, Double or Float80, or to any range (eg -1 ..< 23), etc.

A Random API is (or will at least eventually be) a big thing, and it's important to get it right, especially if its API is to be spread across many other types of the standard library (via eg static .random(in: using:).

I will, if I may, probably continue to ask questions about and point out possible opportunities of improvements in the Random API on these forums (though not here, but in Using Swift or Dev / Std Lib), rather than in code review on GitHub.

ā€¦ any reply to this post, should probably go in a new separate thread. : )

2 Likes
93

I have to strongly agree with @xwu and @Alejandro here. The default rng is called Random in just about every other language. The longer name here doesn't really add clarity.

3 Likes
94

Isn't this really an argument for renaming or changing next()?

I would really like to be able to extend Random with other generators. My vote would be to make Random an empty struct/enum with an internal public type of 'Default' that provides the default implementation (still exposed on Random as Random.default).

95

What is going on with this, @Ben_Cohen?
Does formally proposing a second amendment as outlined above make sense, or is this issue resolved in another way?

96

@Ben_Cohen I'm asking specifically if its worth my time to invest more into this, doing a quick implementation of the renaming or has the ship for 4.2 already sailed?

97

Sorry for the stalling of the thread. The core team is actively discussing the feedback and weā€™ll post something soon. However, a significant redesign of the API is not something weā€™d consider at this stage. The intention is only to make modest amendments related to the naming of the default generator ā€” nothing more than that. So it is not worth putting up any PRs at this stage.

1 Like
98

Thanks everyone for the discussion. The core team has reviewed the feedback from this thread and other input, and has come to the following conclusions:

The core team continues to believe that the Random.default type should be renamed before SE-0202 is released in Swift 4.2. The type name conflicts with the naming guidelines, which recommend that types should read as nouns and that names include all the words needed to avoid ambiguity. Random is not sufficiently descriptive of what this type is. Deviation from the naming guidelines is allowed under some circumstances, such as for terms of art, in order to improve things like discoverability and readability. However, these motivations don't apply in this case: improving the discoverability of this type is actively harmful ā€” users should be guided towards more appropriate methods such as Int.random(in:) or Bool.random().

In considering the naming, the core team felt that SystemRandomNumberGenerator was the most descriptive type name. While DefaultRandomNumberGenerator matches the precedent of DefaultIndices, the term "system" accurately describes the nature of the type.

In discussing the naming of the .default property, the team came to the conclusion that this property is not sufficiently justified, and that the difficulty in naming it is a symptom of that. It is a computed property, rather than an actual instance, that generates a fresh instance each time. As such, the property is essentially sugar for creating an instance and using it. Given the only recommended use of the system generator is to default the generator for other methods, this comes down to the difference between:

extension MyType {
    func random() -> MyType {
        return random(using: &SystemRandomNumberGenerator.default)
    }
}

and

extension MyType {
    func random() -> MyType {
        var g = SystemRandomNumberGenerator()
        return random(using: &g)
    }
}

Since in addition to the property syntax being misleading, the need to explicitly specify a generator is rare, and therefore doesn't justify increased library surface area, the team resolved the best approach was to remove the computed property.

The team still feel there is one issue with this type: that it is in the global namespace. We feel there is a better long term solution: to create a System namespace, into which can go the a RandomNumberGenerator type, as well as other useful system information such as the OS name, current PID etc. The core team would welcome pitches for such a namespace, for the Swift 5 timeframe.

15 Likes