[Proposal] Random Unification

Glibc random is not cryptographically secure, nor is it thread-safe.
getrandom() fails in more ways than arc4random and is therefore a primitive
on which we can base a shim but isn't suitable by itself. Swift used to use
libbsd internally on Linux but that dependency was removed. Currently, it
uses the C++ Mersenne twister engine on all platforms, which is not
cryptographically secure, with a device random seed that is not guaranteed
to be random by the C++ standard if no device is available. Likely,
therefore, a shim will have to be manually implemented.

Ā·Ā·Ā·

On Wed, Oct 4, 2017 at 23:17 Alejandro Alonso via swift-evolution < swift-evolution@swift.org> wrote:

I think this is a good idea. I start asking questions about what our
default generator for linux will be if we use Darwin’s arc4random(3). Do we
use Glibc’s random()? If so, what do we seed it with?

- Alejandro

On Oct 4, 2017, 6:26 PM -0500, Ben Cohen via swift-evolution < > swift-evolution@swift.org>, wrote:

On Sep 30, 2017, at 3:23 PM, Chris Lattner via swift-evolution < > swift-evolution@swift.org> wrote:

On Sep 11, 2017, at 9:43 PM, Brent Royal-Gordon <brent@architechies.com> > wrote:

On Sep 9, 2017, at 10:31 PM, Chris Lattner via swift-evolution < > swift-evolution@swift.org> wrote:

- I’d love to see several of the most common random kinds supported, and I
agree it would be nice (but not required IMO) for the default to be
cryptographically secure.

I would be very careful about choosing a "simple" solution. There is a
log, sad history of languages trying to provide a "simple" random number
generator and accidentally providing a powerful footgun instead. But:

- We should avoid the temptation to nuke this mosquito with a heavy handed
solution designed to solve all of the world’s problems: For example, the
C++ random number stuff is crazily over-general. The stdlib should aim to
solve (e.g.) the top 3 most common cases, and let a more specialized
external library solve the fully general problem (e.g. seed management,
every distribution imaginable, etc).

That's not to say we need to have seven engines and twenty distributions
like C++ does. The standard library is not a statistics package; it exists
to provide basic abstractions and fundamental functionality. I don't think
it should worry itself with distributions at all. I think it needs to
provide:

1. The abstraction used to plug in different random number generators
(i.e. an RNG protocol of some kind).

2. APIs on existing standard library types which perform basic
randomness-related functions correctly—essentially, encapsulating Knuth.
(Specifically, I think selecting a random element from a collection (which
also covers generating a random integer in a range), shuffling a mutable
collection, and generating a random float will do the trick.)

3. A default RNG with a conservative design that will sometimes be too
slow, but will never be insufficiently random.

If you want to pick elements with a Poisson distribution, go get a
statistics framework; if you want repeatable random numbers for testing,
use a seedable PRNG from XCTest or some other test tools package. These can
leverage the standard library's RNG protocol to work with existing random
number generators or random number consumers.

+1 to this general plan!

This pretty much exactly matches my preferences.

If random numbers go into the std lib, they should being able to
customize the source of randomness for speed or test reproducibility, but
default to something sensible without the user having to know it’s
configurable. On Darwin that default should be based on arc4random(3). The
std lib doesn’t need to provide other non-default random sources.
Non-random sources for testing should be part of test frameworks and plug
in easily.

The proposal should include shuffle and random element from collection,
which are much-requested and not really the controversial part so won't
hold up the overall progress of the proposal.

(and no need for distributions other than uniform IMO, :fr::fish: or otherwise)

-Chris

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

I can agree with mostly everything in here. I think `.random` on
`RandomAccessCollection` should mimic the current design with `.first` and
`.last` by returning an optional. In terms of the naming of this, we have
to look at how python structures the call site. `random.choice([1, 2, 3,
4])` To me this reads, random choice within this array. This works because
of how it’s called. With the proposed solution, we are calling to get a
random element directly from the array. So I stick by with naming this
random.

On the subject of bike shedding the names, I can agree to use
`RandomNumberGenerator` whole heartily. As for `Randomizable`, I agree
there might be a better name for this, but the question is what?

That is a good question. I don't like `RandomlySamplable`; it might be that
`Randomizable` could be the least bad option.

One more comment that I neglected to make. You don't need to use an `enum`
in order to make a type non-initializable. A `struct` can do the same with
a private init. There are established singleton patterns for Swift and an
`enum` with a single case is...atypical. I'm also not sure why this should
be a public type at all. Why expose `Random.default.next()`? Shouldn't the
user always use `T.random`? Afaict, `Random` can be an internal type
`_Random` that the user never sees at all.

Ā·Ā·Ā·

On Sun, Nov 5, 2017 at 8:21 PM, Alejandro Alonso <aalonso128@outlook.com> wrote:

On Nov 5, 2017, 7:56 PM -0600, Xiaodi Wu <xiaodi.wu@gmail.com>, wrote:

I like this particular version. In particular, the choice of algorithms
is, afaict, correct and that is incredibly important. I had overlooked that
`arc4random` is cryptographically secure past a certain version of macOS,
but you are absolutely right. I am also on board with the fatal error
suggestion if random entropy is unavailable; I think it must be amply
documented, though.

I do think, however, that you're overloading too many things into the word
"random" when they're not the same. Take a look at Python, which is pretty
widely used for numerics. There's `rand` and `random` for getting a random
integer or floating-point value, and there's `choice` and `sample` for
choosing one or more values out of a collection without replacement. These
are sufficiently different tasks and don't all need to be called "random"
or satisfy the same requirement of the same protocol. Put another way, it's
absolutely *not* inconsistent for numeric types to have `random()` while
collection types have a differently named method.

By contrast, I think the great length of text trying to justify naming all
of these facilities `random` in order to parallel `first` and `last` shows
how the proposed design is comparatively weaker. You have to argue that (a)
`Int.random` shouldn't return an optional value because it'd be unwieldy,
and therefore `(0..<5).random` shouldn't either because it would then be
inconsistent; but (b) that `(0..<5).random` should be spelled and behave
like `(0..<5).first` and `(0..<5).last` even though the user must handle
empty collections totally differently because the return types are not the
same. Either `(0..<5).random` should behave analogously to `first` and
`last` or it should not. If it should, it only makes sense to return a
result of type `T?`. After all, if a collection doesn't have a `first`
item, then it can't have a `random` item. Put another way, having a `first`
item is a prerequisite to having a randomly selectable item. The behavior
of the Swift APIs would be very consistent if `first` returns `T?` but
`random` returns `T`. However, I agree that unwrapping `Int.random` every
time would be burdensome, and it would not make sense to have a type
support `random` but not have any instantiable values; therefore, returning
an optional value doesn't make sense, and it follows that `Int.random`
*shouldn't* behave like `first` or `last`.

Once you stop trying to make what Python calls `rand/randint` and
`choice/sample` have the same names, then finding a Swifty design for the
distinct facilities becomes much easier, and it suggests a pretty elegant
result (IMO):

[1, 2, 3, 4].choice // like `first` or `last`, this gets you a value of
type Int?
[1, 2, 3, 4].sampling(2) // like `prefix(2)` or `suffix(2)`, this gets you
a subsequence with at most two elements

Int.random // this gets you a random Int; or it may trap
Float.random // this gets you a random Float; or it may trap

With that, it also becomes clear why--and I agree with you--an independent
`Int.random(in: 0..<5)` is not necessary. `(0..<5).choice` is fine, and it
can now appropriately return a value of type `T?` because it no longer
needs to parallel `Int.random`.

* * *

More in the bikeshedding arena, I take issue with some of the names:

- I reiterate my comment that `Randomizable` is not the best name. There
are multiple dictionary definitions of "randomize" and one is "make
unpredictable, unsystematic, or random in order or arrangement." Wikipedia
gives at least five different contextual meanings for the word. What you're
doing here is specifically **random sampling** and we can do better to
clarify that, I think.

- While I agree that `RNG` can be cryptic, the alternative should be
`RandomNumberGenerator` (as it's called in other languages);
`RandomGenerator` is not quite accurate. Again, we're _consuming_
randomness to _generate_ numbers (or values of other type, based on the
result of a generated number). We're not _generating_ randomness.

On Sun, Nov 5, 2017 at 6:33 PM, Alejandro Alonso <aalonso128@outlook.com> > wrote:

[Proposal] Random Unification by Azoy Ā· Pull Request #760 Ā· apple/swift-evolution Ā· GitHub is the current API and
proposed solution.

- Alejandro

On Nov 5, 2017, 6:18 PM -0600, Xiaodi Wu <xiaodi.wu@gmail.com>, wrote:

My comments are directed to the "more up-to-date" document that you just
linked to in your reply to Jon. Is that one outdated? If so, can you send a
link to the updated proposal and implementation for which you're soliciting
feedback?

On Sun, Nov 5, 2017 at 6:12 PM, Alejandro Alonso <aalonso128@outlook.com> >> wrote:

The proposal and implementation have the current updated API. The link I
sent Jon was the one I brought up a few weeks ago which is outdated now.
The proposal answers all of your questions. As for `.random` being a
function, some would argue that it behaves in the same way as `.first` and
`.last` which are properties.

- Alejandro

On Nov 5, 2017, 6:07 PM -0600, Xiaodi Wu <xiaodi.wu@gmail.com>, wrote:

A few quick thoughts:

I know that there's been some discussion that `(1...10).random` is the
best spelling, but I'd like to push back on that suggestion. When I want a
random number, I tend to think of the type I want first ("I want a random
integer") and then a range ("I want a random integer between a and b"), not
the other way around. My intuition is that `Int.random(in:)` will be more
discoverable, both on that basis and because it is more similar to other
languages' syntax (`Math.random` in JavaScript and `randint` in NumPy, for
example). It also has the advantage that the type is explicit, which I
think is particularly useful in this case because the value itself is,
well, random.

I would also argue that, `random` is most appropriately a method and not
a property; there's no hard and fast rule for this, but the fact that the
result is stochastic suggests (to me) that it's not a "property" of the
range (or, for that matter, of the type).

I would reiterate here my qualms about `Source` being the term used for
a generator. These types are not a _source_ of entropy but rather a
_consumer_ of entropy.

`UnsafeRandomSource` needs to be renamed; "unsafe" has a specific
meaning in Swift--that is, memory safety, and this is not it. Moreover,
it's questionable whether this protocol is useful in any sense. What useful
generic algorithms can one write with such a protocol?

`XoroshiroRandom` cannot be seeded by any `Numeric` value; depending on
the specific algorithm it needs a seed of a specific bit width. If you
default the shared instance to being seeded with an `Int` then you will
have to have distinct implementations for 32-bit and 64-bit platforms. This
is unadvisable. On that note, your `UnsafeRandomSource` needs to have an
associated type and not a generic `<T : Numeric>` for the seed.

The default random number generator should be cryptographically secure;
however, it's not clear to me that it should be device random.

I agree with others that alternative random number generators other than
the default RNG (and, if not default, possibly also the device RNG) should
be accommodated by the protocol hierarchy but not necessarily supplied in
the stdlib.

The term `Randomizable` means something specific which is not how it's
used in your proposed protocol.

There's still the open question, not answered, about how requesting an
instance of the hardware RNG behaves when there's insufficient or no
entropy. Does it return nil, throw, trap, or wait? The proposed API does
not clarify this point, although based on the method signature it cannot
return nil or throw. Trapping might be acceptable but I'd be interested to
hear your take as to why it is preferable.

On Sun, Nov 5, 2017 at 4:43 PM, Alejandro Alonso via swift-evolution < >>> swift-evolution@swift.org> wrote:

For the proof of concept, I had accidentally deleted that one. I have a
more up to date one which was discussed a few weeks later.
Swift Random Unification Design Ā· GitHub

- Alejandro

On Nov 5, 2017, 4:37 PM -0600, Jonathan Hull <jhull@gbis.com>, wrote:

Is there a link to the writeup? The one in the quote 404s.

Thanks,
Jon

On Nov 5, 2017, at 2:10 PM, Alejandro Alonso via swift-evolution < >>>> swift-evolution@swift.org> wrote:

Hello once again Swift evolution community. I have taken the time to
write up the proposal for this thread, and have provided an implementation
for it as well. I hope to once again get good feedback on the overall
proposal.

- Alejandro

On Sep 8, 2017, 11:52 AM -0500, Alejandro Alonso via swift-evolution < >>>> swift-evolution@swift.org>, wrote:

Hello swift evolution, I would like to propose a unified approach to
`random()` in Swift. I have a simple implementation here
https://gist.github.com/Azoy/5d294148c8b97d20b96ee64f434bb4f5\. This
implementation is a simple wrapper over existing random functions so
existing code bases will not be affected. Also, this approach introduces a
new random feature for Linux users that give them access to upper bounds,
as well as a lower bound for both Glibc and Darwin users. This change would
be implemented within Foundation.

I believe this simple change could have a very positive impact on new
developers learning Swift and experienced developers being able to write
single random declarations.

I’d like to hear about your ideas on this proposal, or any
implementation changes if need be.

- Alejando

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Yes, the user should only use `T.random`. I will go ahead and make `Random` internal and as a struct.

- Alejandro

Ā·Ā·Ā·

On Nov 5, 2017, 9:08 PM -0600, Xiaodi Wu <xiaodi.wu@gmail.com>, wrote:
On Sun, Nov 5, 2017 at 8:21 PM, Alejandro Alonso <aalonso128@outlook.com<mailto:aalonso128@outlook.com>> wrote:
I can agree with mostly everything in here. I think `.random` on `RandomAccessCollection` should mimic the current design with `.first` and `.last` by returning an optional. In terms of the naming of this, we have to look at how python structures the call site. `random.choice([1, 2, 3, 4])` To me this reads, random choice within this array. This works because of how it’s called. With the proposed solution, we are calling to get a random element directly from the array. So I stick by with naming this random.

On the subject of bike shedding the names, I can agree to use `RandomNumberGenerator` whole heartily. As for `Randomizable`, I agree there might be a better name for this, but the question is what?

That is a good question. I don't like `RandomlySamplable`; it might be that `Randomizable` could be the least bad option.

One more comment that I neglected to make. You don't need to use an `enum` in order to make a type non-initializable. A `struct` can do the same with a private init. There are established singleton patterns for Swift and an `enum` with a single case is...atypical. I'm also not sure why this should be a public type at all. Why expose `Random.default.next()`? Shouldn't the user always use `T.random`? Afaict, `Random` can be an internal type `_Random` that the user never sees at all.

On Nov 5, 2017, 7:56 PM -0600, Xiaodi Wu <xiaodi.wu@gmail.com<mailto:xiaodi.wu@gmail.com>>, wrote:
I like this particular version. In particular, the choice of algorithms is, afaict, correct and that is incredibly important. I had overlooked that `arc4random` is cryptographically secure past a certain version of macOS, but you are absolutely right. I am also on board with the fatal error suggestion if random entropy is unavailable; I think it must be amply documented, though.

I do think, however, that you're overloading too many things into the word "random" when they're not the same. Take a look at Python, which is pretty widely used for numerics. There's `rand` and `random` for getting a random integer or floating-point value, and there's `choice` and `sample` for choosing one or more values out of a collection without replacement. These are sufficiently different tasks and don't all need to be called "random" or satisfy the same requirement of the same protocol. Put another way, it's absolutely *not* inconsistent for numeric types to have `random()` while collection types have a differently named method.

By contrast, I think the great length of text trying to justify naming all of these facilities `random` in order to parallel `first` and `last` shows how the proposed design is comparatively weaker. You have to argue that (a) `Int.random` shouldn't return an optional value because it'd be unwieldy, and therefore `(0..<5).random` shouldn't either because it would then be inconsistent; but (b) that `(0..<5).random` should be spelled and behave like `(0..<5).first` and `(0..<5).last` even though the user must handle empty collections totally differently because the return types are not the same. Either `(0..<5).random` should behave analogously to `first` and `last` or it should not. If it should, it only makes sense to return a result of type `T?`. After all, if a collection doesn't have a `first` item, then it can't have a `random` item. Put another way, having a `first` item is a prerequisite to having a randomly selectable item. The behavior of the Swift APIs would be very consistent if `first` returns `T?` but `random` returns `T`. However, I agree that unwrapping `Int.random` every time would be burdensome, and it would not make sense to have a type support `random` but not have any instantiable values; therefore, returning an optional value doesn't make sense, and it follows that `Int.random` *shouldn't* behave like `first` or `last`.

Once you stop trying to make what Python calls `rand/randint` and `choice/sample` have the same names, then finding a Swifty design for the distinct facilities becomes much easier, and it suggests a pretty elegant result (IMO):

[1, 2, 3, 4].choice // like `first` or `last`, this gets you a value of type Int?
[1, 2, 3, 4].sampling(2) // like `prefix(2)` or `suffix(2)`, this gets you a subsequence with at most two elements

Int.random // this gets you a random Int; or it may trap
Float.random // this gets you a random Float; or it may trap

With that, it also becomes clear why--and I agree with you--an independent `Int.random(in: 0..<5)` is not necessary. `(0..<5).choice` is fine, and it can now appropriately return a value of type `T?` because it no longer needs to parallel `Int.random`.

* * *

More in the bikeshedding arena, I take issue with some of the names:

- I reiterate my comment that `Randomizable` is not the best name. There are multiple dictionary definitions of "randomize" and one is "make unpredictable, unsystematic, or random in order or arrangement." Wikipedia gives at least five different contextual meanings for the word. What you're doing here is specifically **random sampling** and we can do better to clarify that, I think.

- While I agree that `RNG` can be cryptic, the alternative should be `RandomNumberGenerator` (as it's called in other languages); `RandomGenerator` is not quite accurate. Again, we're _consuming_ randomness to _generate_ numbers (or values of other type, based on the result of a generated number). We're not _generating_ randomness.

On Sun, Nov 5, 2017 at 6:33 PM, Alejandro Alonso <aalonso128@outlook.com<mailto:aalonso128@outlook.com>> wrote:
[Proposal] Random Unification by Azoy Ā· Pull Request #760 Ā· apple/swift-evolution Ā· GitHub is the current API and proposed solution.

- Alejandro

On Nov 5, 2017, 6:18 PM -0600, Xiaodi Wu <xiaodi.wu@gmail.com<mailto:xiaodi.wu@gmail.com>>, wrote:
My comments are directed to the "more up-to-date" document that you just linked to in your reply to Jon. Is that one outdated? If so, can you send a link to the updated proposal and implementation for which you're soliciting feedback?

On Sun, Nov 5, 2017 at 6:12 PM, Alejandro Alonso <aalonso128@outlook.com<mailto:aalonso128@outlook.com>> wrote:
The proposal and implementation have the current updated API. The link I sent Jon was the one I brought up a few weeks ago which is outdated now. The proposal answers all of your questions. As for `.random` being a function, some would argue that it behaves in the same way as `.first` and `.last` which are properties.

- Alejandro

On Nov 5, 2017, 6:07 PM -0600, Xiaodi Wu <xiaodi.wu@gmail.com<mailto:xiaodi.wu@gmail.com>>, wrote:
A few quick thoughts:

I know that there's been some discussion that `(1...10).random` is the best spelling, but I'd like to push back on that suggestion. When I want a random number, I tend to think of the type I want first ("I want a random integer") and then a range ("I want a random integer between a and b"), not the other way around. My intuition is that `Int.random(in:)` will be more discoverable, both on that basis and because it is more similar to other languages' syntax (`Math.random` in JavaScript and `randint` in NumPy, for example). It also has the advantage that the type is explicit, which I think is particularly useful in this case because the value itself is, well, random.

I would also argue that, `random` is most appropriately a method and not a property; there's no hard and fast rule for this, but the fact that the result is stochastic suggests (to me) that it's not a "property" of the range (or, for that matter, of the type).

I would reiterate here my qualms about `Source` being the term used for a generator. These types are not a _source_ of entropy but rather a _consumer_ of entropy.

`UnsafeRandomSource` needs to be renamed; "unsafe" has a specific meaning in Swift--that is, memory safety, and this is not it. Moreover, it's questionable whether this protocol is useful in any sense. What useful generic algorithms can one write with such a protocol?

`XoroshiroRandom` cannot be seeded by any `Numeric` value; depending on the specific algorithm it needs a seed of a specific bit width. If you default the shared instance to being seeded with an `Int` then you will have to have distinct implementations for 32-bit and 64-bit platforms. This is unadvisable. On that note, your `UnsafeRandomSource` needs to have an associated type and not a generic `<T : Numeric>` for the seed.

The default random number generator should be cryptographically secure; however, it's not clear to me that it should be device random.

I agree with others that alternative random number generators other than the default RNG (and, if not default, possibly also the device RNG) should be accommodated by the protocol hierarchy but not necessarily supplied in the stdlib.

The term `Randomizable` means something specific which is not how it's used in your proposed protocol.

There's still the open question, not answered, about how requesting an instance of the hardware RNG behaves when there's insufficient or no entropy. Does it return nil, throw, trap, or wait? The proposed API does not clarify this point, although based on the method signature it cannot return nil or throw. Trapping might be acceptable but I'd be interested to hear your take as to why it is preferable.

On Sun, Nov 5, 2017 at 4:43 PM, Alejandro Alonso via swift-evolution <swift-evolution@swift.org<mailto:swift-evolution@swift.org>> wrote:
For the proof of concept, I had accidentally deleted that one. I have a more up to date one which was discussed a few weeks later. Swift Random Unification Design Ā· GitHub

- Alejandro

On Nov 5, 2017, 4:37 PM -0600, Jonathan Hull <jhull@gbis.com<mailto:jhull@gbis.com>>, wrote:
Is there a link to the writeup? The one in the quote 404s.

Thanks,
Jon

On Nov 5, 2017, at 2:10 PM, Alejandro Alonso via swift-evolution <swift-evolution@swift.org<mailto:swift-evolution@swift.org>> wrote:

Hello once again Swift evolution community. I have taken the time to write up the proposal for this thread, and have provided an implementation for it as well. I hope to once again get good feedback on the overall proposal.

- Alejandro

On Sep 8, 2017, 11:52 AM -0500, Alejandro Alonso via swift-evolution <swift-evolution@swift.org<mailto:swift-evolution@swift.org>>, wrote:
Hello swift evolution, I would like to propose a unified approach to `random()` in Swift. I have a simple implementation here https://gist.github.com/Azoy/5d294148c8b97d20b96ee64f434bb4f5\. This implementation is a simple wrapper over existing random functions so existing code bases will not be affected. Also, this approach introduces a new random feature for Linux users that give them access to upper bounds, as well as a lower bound for both Glibc and Darwin users. This change would be implemented within Foundation.

I believe this simple change could have a very positive impact on new developers learning Swift and experienced developers being able to write single random declarations.

I’d like to hear about your ideas on this proposal, or any implementation changes if need be.

- Alejando

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org<mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org<mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

An error that came up in development, default arguments cannot use internal types, `shuffle(using:)` and `shuffled(using:)`. To remedy this issue I would have to add another method requirement with no arguments that gives a default implementation of using `_Random`. This would work, but would appear in stdlib documentation as two separate functions.

- Alejandro

Ā·Ā·Ā·

On Nov 5, 2017, 9:08 PM -0600, Xiaodi Wu <xiaodi.wu@gmail.com>, wrote:
On Sun, Nov 5, 2017 at 8:21 PM, Alejandro Alonso <aalonso128@outlook.com<mailto:aalonso128@outlook.com>> wrote:
I can agree with mostly everything in here. I think `.random` on `RandomAccessCollection` should mimic the current design with `.first` and `.last` by returning an optional. In terms of the naming of this, we have to look at how python structures the call site. `random.choice([1, 2, 3, 4])` To me this reads, random choice within this array. This works because of how it’s called. With the proposed solution, we are calling to get a random element directly from the array. So I stick by with naming this random.

On the subject of bike shedding the names, I can agree to use `RandomNumberGenerator` whole heartily. As for `Randomizable`, I agree there might be a better name for this, but the question is what?

That is a good question. I don't like `RandomlySamplable`; it might be that `Randomizable` could be the least bad option.

One more comment that I neglected to make. You don't need to use an `enum` in order to make a type non-initializable. A `struct` can do the same with a private init. There are established singleton patterns for Swift and an `enum` with a single case is...atypical. I'm also not sure why this should be a public type at all. Why expose `Random.default.next()`? Shouldn't the user always use `T.random`? Afaict, `Random` can be an internal type `_Random` that the user never sees at all.

On Nov 5, 2017, 7:56 PM -0600, Xiaodi Wu <xiaodi.wu@gmail.com<mailto:xiaodi.wu@gmail.com>>, wrote:
I like this particular version. In particular, the choice of algorithms is, afaict, correct and that is incredibly important. I had overlooked that `arc4random` is cryptographically secure past a certain version of macOS, but you are absolutely right. I am also on board with the fatal error suggestion if random entropy is unavailable; I think it must be amply documented, though.

I do think, however, that you're overloading too many things into the word "random" when they're not the same. Take a look at Python, which is pretty widely used for numerics. There's `rand` and `random` for getting a random integer or floating-point value, and there's `choice` and `sample` for choosing one or more values out of a collection without replacement. These are sufficiently different tasks and don't all need to be called "random" or satisfy the same requirement of the same protocol. Put another way, it's absolutely *not* inconsistent for numeric types to have `random()` while collection types have a differently named method.

By contrast, I think the great length of text trying to justify naming all of these facilities `random` in order to parallel `first` and `last` shows how the proposed design is comparatively weaker. You have to argue that (a) `Int.random` shouldn't return an optional value because it'd be unwieldy, and therefore `(0..<5).random` shouldn't either because it would then be inconsistent; but (b) that `(0..<5).random` should be spelled and behave like `(0..<5).first` and `(0..<5).last` even though the user must handle empty collections totally differently because the return types are not the same. Either `(0..<5).random` should behave analogously to `first` and `last` or it should not. If it should, it only makes sense to return a result of type `T?`. After all, if a collection doesn't have a `first` item, then it can't have a `random` item. Put another way, having a `first` item is a prerequisite to having a randomly selectable item. The behavior of the Swift APIs would be very consistent if `first` returns `T?` but `random` returns `T`. However, I agree that unwrapping `Int.random` every time would be burdensome, and it would not make sense to have a type support `random` but not have any instantiable values; therefore, returning an optional value doesn't make sense, and it follows that `Int.random` *shouldn't* behave like `first` or `last`.

Once you stop trying to make what Python calls `rand/randint` and `choice/sample` have the same names, then finding a Swifty design for the distinct facilities becomes much easier, and it suggests a pretty elegant result (IMO):

[1, 2, 3, 4].choice // like `first` or `last`, this gets you a value of type Int?
[1, 2, 3, 4].sampling(2) // like `prefix(2)` or `suffix(2)`, this gets you a subsequence with at most two elements

Int.random // this gets you a random Int; or it may trap
Float.random // this gets you a random Float; or it may trap

With that, it also becomes clear why--and I agree with you--an independent `Int.random(in: 0..<5)` is not necessary. `(0..<5).choice` is fine, and it can now appropriately return a value of type `T?` because it no longer needs to parallel `Int.random`.

* * *

More in the bikeshedding arena, I take issue with some of the names:

- I reiterate my comment that `Randomizable` is not the best name. There are multiple dictionary definitions of "randomize" and one is "make unpredictable, unsystematic, or random in order or arrangement." Wikipedia gives at least five different contextual meanings for the word. What you're doing here is specifically **random sampling** and we can do better to clarify that, I think.

- While I agree that `RNG` can be cryptic, the alternative should be `RandomNumberGenerator` (as it's called in other languages); `RandomGenerator` is not quite accurate. Again, we're _consuming_ randomness to _generate_ numbers (or values of other type, based on the result of a generated number). We're not _generating_ randomness.

On Sun, Nov 5, 2017 at 6:33 PM, Alejandro Alonso <aalonso128@outlook.com<mailto:aalonso128@outlook.com>> wrote:
[Proposal] Random Unification by Azoy Ā· Pull Request #760 Ā· apple/swift-evolution Ā· GitHub is the current API and proposed solution.

- Alejandro

On Nov 5, 2017, 6:18 PM -0600, Xiaodi Wu <xiaodi.wu@gmail.com<mailto:xiaodi.wu@gmail.com>>, wrote:
My comments are directed to the "more up-to-date" document that you just linked to in your reply to Jon. Is that one outdated? If so, can you send a link to the updated proposal and implementation for which you're soliciting feedback?

On Sun, Nov 5, 2017 at 6:12 PM, Alejandro Alonso <aalonso128@outlook.com<mailto:aalonso128@outlook.com>> wrote:
The proposal and implementation have the current updated API. The link I sent Jon was the one I brought up a few weeks ago which is outdated now. The proposal answers all of your questions. As for `.random` being a function, some would argue that it behaves in the same way as `.first` and `.last` which are properties.

- Alejandro

On Nov 5, 2017, 6:07 PM -0600, Xiaodi Wu <xiaodi.wu@gmail.com<mailto:xiaodi.wu@gmail.com>>, wrote:
A few quick thoughts:

I know that there's been some discussion that `(1...10).random` is the best spelling, but I'd like to push back on that suggestion. When I want a random number, I tend to think of the type I want first ("I want a random integer") and then a range ("I want a random integer between a and b"), not the other way around. My intuition is that `Int.random(in:)` will be more discoverable, both on that basis and because it is more similar to other languages' syntax (`Math.random` in JavaScript and `randint` in NumPy, for example). It also has the advantage that the type is explicit, which I think is particularly useful in this case because the value itself is, well, random.

I would also argue that, `random` is most appropriately a method and not a property; there's no hard and fast rule for this, but the fact that the result is stochastic suggests (to me) that it's not a "property" of the range (or, for that matter, of the type).

I would reiterate here my qualms about `Source` being the term used for a generator. These types are not a _source_ of entropy but rather a _consumer_ of entropy.

`UnsafeRandomSource` needs to be renamed; "unsafe" has a specific meaning in Swift--that is, memory safety, and this is not it. Moreover, it's questionable whether this protocol is useful in any sense. What useful generic algorithms can one write with such a protocol?

`XoroshiroRandom` cannot be seeded by any `Numeric` value; depending on the specific algorithm it needs a seed of a specific bit width. If you default the shared instance to being seeded with an `Int` then you will have to have distinct implementations for 32-bit and 64-bit platforms. This is unadvisable. On that note, your `UnsafeRandomSource` needs to have an associated type and not a generic `<T : Numeric>` for the seed.

The default random number generator should be cryptographically secure; however, it's not clear to me that it should be device random.

I agree with others that alternative random number generators other than the default RNG (and, if not default, possibly also the device RNG) should be accommodated by the protocol hierarchy but not necessarily supplied in the stdlib.

The term `Randomizable` means something specific which is not how it's used in your proposed protocol.

There's still the open question, not answered, about how requesting an instance of the hardware RNG behaves when there's insufficient or no entropy. Does it return nil, throw, trap, or wait? The proposed API does not clarify this point, although based on the method signature it cannot return nil or throw. Trapping might be acceptable but I'd be interested to hear your take as to why it is preferable.

On Sun, Nov 5, 2017 at 4:43 PM, Alejandro Alonso via swift-evolution <swift-evolution@swift.org<mailto:swift-evolution@swift.org>> wrote:
For the proof of concept, I had accidentally deleted that one. I have a more up to date one which was discussed a few weeks later. Swift Random Unification Design Ā· GitHub

- Alejandro

On Nov 5, 2017, 4:37 PM -0600, Jonathan Hull <jhull@gbis.com<mailto:jhull@gbis.com>>, wrote:
Is there a link to the writeup? The one in the quote 404s.

Thanks,
Jon

On Nov 5, 2017, at 2:10 PM, Alejandro Alonso via swift-evolution <swift-evolution@swift.org<mailto:swift-evolution@swift.org>> wrote:

Hello once again Swift evolution community. I have taken the time to write up the proposal for this thread, and have provided an implementation for it as well. I hope to once again get good feedback on the overall proposal.

- Alejandro

On Sep 8, 2017, 11:52 AM -0500, Alejandro Alonso via swift-evolution <swift-evolution@swift.org<mailto:swift-evolution@swift.org>>, wrote:
Hello swift evolution, I would like to propose a unified approach to `random()` in Swift. I have a simple implementation here https://gist.github.com/Azoy/5d294148c8b97d20b96ee64f434bb4f5\. This implementation is a simple wrapper over existing random functions so existing code bases will not be affected. Also, this approach introduces a new random feature for Linux users that give them access to upper bounds, as well as a lower bound for both Glibc and Darwin users. This change would be implemented within Foundation.

I believe this simple change could have a very positive impact on new developers learning Swift and experienced developers being able to write single random declarations.

I’d like to hear about your ideas on this proposal, or any implementation changes if need be.

- Alejando

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org<mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org<mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

Sorry I’ve been gone for a while, I had to do a lot of traveling.

1. Initially I made this thinking that developers had the power to
determine their own lower bound. The current implementation uses the
integer’s min value as a lower bound. If it makes sense to only allow
unsigned integers from an RNG, then I’m perfectly fine with. I do disagree
when you say that it should only generate UInt32s. The current approach
allows, lets say mt19337 and mt19337-64, to be used within one generator.
So if you wanted a UInt32, mt19337 would be used, and if you asked for a
UInt64, mt19337-64 would be used.

I agree with Nate that this doesn't need to be--and is better off not
being--generic. The two different Mersenne Twisters are properly two
distinct PRNGs. A user sufficiently sophisticated to ask for their own PRNG
would expect the same algorithm to be used to generate a value of any
bitwidth, by concatenation of successively generated random bits if
necessary.

Instead, the return type should be an associated type. 32-bit PRNG
algorithms would always return values of type UInt32 and 64-bit algorithms
would always return values of type UInt64.

2. The Randomizable protocol isn’t always used with integers. Think

Date.random or Color.random. These types of values are difficult to express
with ranges. Randomizable solves this issue.

3. I’ve made the adjustment necessary for this.

4. So while I can see your point for this, it would break the consistency
with Randomizable’s random property. You could argue that we could make
this property a function itself, but I think most will agree that
Int.random is a cleaner api than Int.random().

5. I’ve made the adjustment necessary for this.

6. I actually forgot to implement the random api for the ranges where
Bound: BinaryFloatingPoint. While implementing this, I realized these would
never fail and would always return a non-optional.

Sure it can. 0.0..<0.0 is an empty range, just like 0..<0. Any collection
type really should return an Optional for 'random' because collections can
be empty.

So, I decided making the other Countable ranges non-optional. (0 ..<

10).random would return a non-optional, (0.0 ..< 10.0).random would return
a non-optional, and Array(0 ..< 10).random would return an optional. I can
agree that something like (0 ..< 10).random is hard to discover, so I added
Int.random(in: 0 ..< 10) (along with BinaryFloatingPoint).

This is unsatisfying (to me, at least). The proposed design should offer
one very good way to accomplish something, not two spellings for the same
thing.

However, these are not requirements of Randomizable. I think these methods

Ā·Ā·Ā·

On Sun, Nov 12, 2017 at 19:47 Alejandro Alonso <aalonso128@outlook.com> wrote:

would benefit more if they were extension methods:

extension Randomizable where Self: FixedWidthInteger, Self.Stride:
SignedInteger {
public static func random(
in range: Countable{Closed}Range,
using generator: RandomNumberGenerator
) -> Self {
return range.random(using: generator)
}
}

extension Randomizable where Self: BinaryFloatingPointer {
public static func random(
in range: {Closed}Range,
using generator: RandomNumberGenerator
) -> Self {
return range.random
}
}

I think external types that wish to do something similar, like
Data.random(bytes: 128), could extend Randomizable with their own custom
needs. The stdlib would at this point provide all the features needed to
make this happen very simply for something like Data.random(bytes: 128).

- Alejandro

On Nov 5, 2017, 10:44 PM -0600, Nate Cook <natecook@apple.com>, wrote:

Thanks for continuing to push this forward, Alejandro! I’m excited about
the potential of having access to these APIs as part of the standard
library. Here are a few comments on some different parts of the proposal:

1) For your RandomGenerator protocol, I’m not totally clear on the
semantics of the next(_:) and next(_:upperBound:) methods. Do they both
have zero as their lower bound, for example? I’m not sure it makes sense to
have signed integers generated directly by an RNG—perhaps T:
FixedWidthInteger & UnsignedInteger would be a more useful constraint.
(Does it even need to be generic? What if RNGs just generate UInt32s?)

2) Can you say more about the purpose of the Randomizable protocol? How
would we use that protocol in useful ways that we wouldn’t get from being
able to select random values from ranges (half-open and closed) of
FixedWidthInteger / BinaryFloatingPoint? My experience has been that a
full-width random value is rarely what a user needs.

3) I agree with Xiaodi that Random should probably be a struct with a
single shared instance, but I don’t think it should be internal. Hiding
that shared RNG would make it hard for non-stdlib additions to have the
same usage, as they would need to have completely separate implementations
for the ā€œdefaultā€ and custom RNG versions.

4) I would also still suggest that the simplest version of random (that
you use to get a value from a range or an element from a collection) should
be a function, not a property. Collection properties like first, last,
and count all represent facts that already exist about a collection, and
don’t change unless the collection itself changes. Choosing a random
element, on the other hand, is clearly going to be freshly performed on
each call. In addition, with the notable exception of count, we try to
ensure O(1) performance for properties, while random will be O(n) except
in random-access collections. Finally, if it is a method, we can unify the
two versions by providing a single method with the shared RNG as the
default parameter.

5) To match the sorted() method, shuffled() should be on Sequence instead
of Collection. I don’t think either shuffled() or shuffle() needs to be a
protocol requirement, since there isn’t really any kind of customization
necessary for different kinds of collections. Like the sorting algorithms,
both could be regular extension methods.

6) I don’t know whether or not a consensus has formed around the correct
spelling of the APIs for generating random values. From the proposal it
looks like the preferred ways of getting a random value in a range would be
to use the random property (or method) on a range or closed range:

    (0..<10).random // 7
    (0.0 ... 5.0).random // 4.112312

If that’s the goal, and we don’t want those values to be optional, we’ll
need an implementation of random for floating-point ranges and an overload
for fixed-width integer ranges. That said, I don’t think that style is as
discoverable as having static methods or initializers available on the
different types:

    Int.random(in: 0..<10)
    Double.random(in: 0.0 ... 5.0)
    // or maybe
    Int(randomIn: 0..<10)
    Double(randomIn: 0.0 ... 5.0)

(My only quibble with the initializer approach is that Bool would be
awkward.)

In addition, this alternative approach could make creating random values
more consistent with types that don’t work well in ranges:

    Data.random(bytes: 128)
    Color.random(r: 0...0, g: 0...1, b: 0...1, a: 1...1)

————

Thanks again!
Nate

On Nov 5, 2017, at 6:33 PM, Alejandro Alonso via swift-evolution < > swift-evolution@swift.org> wrote:

[Proposal] Random Unification by Azoy Ā· Pull Request #760 Ā· apple/swift-evolution Ā· GitHub is the current API and
proposed solution.

- Alejandro

On Nov 5, 2017, 6:18 PM -0600, Xiaodi Wu <xiaodi.wu@gmail.com>, wrote:

My comments are directed to the "more up-to-date" document that you just
linked to in your reply to Jon. Is that one outdated? If so, can you send a
link to the updated proposal and implementation for which you're soliciting
feedback?

On Sun, Nov 5, 2017 at 6:12 PM, Alejandro Alonso <aalonso128@outlook.com> > wrote:

The proposal and implementation have the current updated API. The link I
sent Jon was the one I brought up a few weeks ago which is outdated now.
The proposal answers all of your questions. As for `.random` being a
function, some would argue that it behaves in the same way as `.first` and
`.last` which are properties.

- Alejandro

On Nov 5, 2017, 6:07 PM -0600, Xiaodi Wu <xiaodi.wu@gmail.com>, wrote:

A few quick thoughts:

I know that there's been some discussion that `(1...10).random` is the
best spelling, but I'd like to push back on that suggestion. When I want a
random number, I tend to think of the type I want first ("I want a random
integer") and then a range ("I want a random integer between a and b"), not
the other way around. My intuition is that `Int.random(in:)` will be more
discoverable, both on that basis and because it is more similar to other
languages' syntax (`Math.random` in JavaScript and `randint` in NumPy,
for example). It also has the advantage that the type is explicit, which
I think is particularly useful in this case because the value itself is,
well, random.

I would also argue that, `random` is most appropriately a method and not
a property; there's no hard and fast rule for this, but the fact that the
result is stochastic suggests (to me) that it's not a "property" of the
range (or, for that matter, of the type).

I would reiterate here my qualms about `Source` being the term used for a
generator. These types are not a _source_ of entropy but rather a
_consumer_ of entropy.

`UnsafeRandomSource` needs to be renamed; "unsafe" has a specific meaning
in Swift--that is, memory safety, and this is not it. Moreover, it's
questionable whether this protocol is useful in any sense. What useful
generic algorithms can one write with such a protocol?

`XoroshiroRandom` cannot be seeded by any `Numeric` value; depending on
the specific algorithm it needs a seed of a specific bit width. If you
default the shared instance to being seeded with an `Int` then you will
have to have distinct implementations for 32-bit and 64-bit platforms. This
is unadvisable. On that note, your `UnsafeRandomSource` needs to have an
associated type and not a generic `<T : Numeric>` for the seed.

The default random number generator should be cryptographically secure;
however, it's not clear to me that it should be device random.

I agree with others that alternative random number generators other than
the default RNG (and, if not default, possibly also the device RNG) should
be accommodated by the protocol hierarchy but not necessarily supplied in
the stdlib.

The term `Randomizable` means something specific which is not how it's
used in your proposed protocol.

There's still the open question, not answered, about how requesting an
instance of the hardware RNG behaves when there's insufficient or no
entropy. Does it return nil, throw, trap, or wait? The proposed API does
not clarify this point, although based on the method signature it cannot
return nil or throw. Trapping might be acceptable but I'd be interested to
hear your take as to why it is preferable.

On Sun, Nov 5, 2017 at 4:43 PM, Alejandro Alonso via swift-evolution < >> swift-evolution@swift.org> wrote:

For the proof of concept, I had accidentally deleted that one. I have a
more up to date one which was discussed a few weeks later.
Swift Random Unification Design Ā· GitHub

- Alejandro

On Nov 5, 2017, 4:37 PM -0600, Jonathan Hull <jhull@gbis.com>, wrote:

Is there a link to the writeup? The one in the quote 404s.

Thanks,
Jon

On Nov 5, 2017, at 2:10 PM, Alejandro Alonso via swift-evolution < >>> swift-evolution@swift.org> wrote:

Hello once again Swift evolution community. I have taken the time to
write up the proposal for this thread, and have provided an implementation
for it as well. I hope to once again get good feedback on the overall
proposal.

- Alejandro

On Sep 8, 2017, 11:52 AM -0500, Alejandro Alonso via swift-evolution < >>> swift-evolution@swift.org>, wrote:

Hello swift evolution, I would like to propose a unified approach to
`random()` in Swift. I have a simple implementation here
https://gist.github.com/Azoy/5d294148c8b97d20b96ee64f434bb4f5\. This
implementation is a simple wrapper over existing random functions so
existing code bases will not be affected. Also, this approach introduces a
new random feature for Linux users that give them access to upper bounds,
as well as a lower bound for both Glibc and Darwin users. This change would
be implemented within Foundation.

I believe this simple change could have a very positive impact on new
developers learning Swift and experienced developers being able to write
single random declarations.

I’d like to hear about your ideas on this proposal, or any
implementation changes if need be.

- Alejando

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Sorry I’ve been gone for a while, I had to do a lot of traveling.

1. Initially I made this thinking that developers had the power to determine their own lower bound. The current implementation uses the integer’s min value as a lower bound. If it makes sense to only allow unsigned integers from an RNG, then I’m perfectly fine with. I do disagree when you say that it should only generate UInt32s. The current approach allows, lets say mt19337 and mt19337-64, to be used within one generator. So if you wanted a UInt32, mt19337 would be used, and if you asked for a UInt64, mt19337-64 would be used.

2. The Randomizable protocol isn’t always used with integers. Think Date.random or Color.random. These types of values are difficult to express with ranges. Randomizable solves this issue.

I don’t think these examples explain how the Randomizable protocol is useful. As currently proposed, types that are Randomizable can provide a random value, but the details of that are left up to the type, and vary widely. When you use .random on an integer type, you get any value between .min and .max, but on a floating-point type, it’s a value between 0.0 and 1.0. What would the range be when you use Date.random? What about Color.random?

Here’s an implementation using the Randomizable protocol as a constraint:

    extension Array where Element: Randomizable {
        init(randomElements: Int) {
            self = (0..<randomElements).map({ _ in Element.random })
        }
    }

Is this useful? Note that there’s no way to constrain these random values to be in a particular range or any other criteria—they’re only going to be random values in whatever the range is that a specific types supplies.

The point I’m trying to make is that although all these types can have random values generated, their similarity is mostly coincidental. As long as we provide good ways of generating random values at the appropriate level, we don’t need the Randomizable protocol. (To me, the appropriate extension points are FixedWidthInteger, BinaryFloatingPoint, and Bool.) Without Randomizable, we can still write things like this, which are the kinds of functionality we want to add anyway:

    extension Array where Element: FixedWidthInteger {
        init(randomElements: Int, in bounds: Range<Element>) {
            self = (0..<randomElements).map({ _ in bounds.random })
        }
    }

-Nate

Ā·Ā·Ā·

On Nov 12, 2017, at 7:47 PM, Alejandro Alonso <aalonso128@outlook.com> wrote:

3. I’ve made the adjustment necessary for this.

4. So while I can see your point for this, it would break the consistency with Randomizable’s random property. You could argue that we could make this property a function itself, but I think most will agree that Int.random is a cleaner api than Int.random().

5. I’ve made the adjustment necessary for this.

6. I actually forgot to implement the random api for the ranges where Bound: BinaryFloatingPoint. While implementing this, I realized these would never fail and would always return a non-optional. So, I decided making the other Countable ranges non-optional. (0 ..< 10).random would return a non-optional, (0.0 ..< 10.0).random would return a non-optional, and Array(0 ..< 10).random would return an optional. I can agree that something like (0 ..< 10).random is hard to discover, so I added Int.random(in: 0 ..< 10) (along with BinaryFloatingPoint). However, these are not requirements of Randomizable. I think these methods would benefit more if they were extension methods:

extension Randomizable where Self: FixedWidthInteger, Self.Stride: SignedInteger {
public static func random(
in range: Countable{Closed}Range,
using generator: RandomNumberGenerator
) -> Self {
return range.random(using: generator)
}
}

extension Randomizable where Self: BinaryFloatingPointer {
public static func random(
in range: {Closed}Range,
using generator: RandomNumberGenerator
) -> Self {
return range.random
}
}

I think external types that wish to do something similar, like Data.random(bytes: 128), could extend Randomizable with their own custom needs. The stdlib would at this point provide all the features needed to make this happen very simply for something like Data.random(bytes: 128).

- Alejandro

On Nov 5, 2017, 10:44 PM -0600, Nate Cook <natecook@apple.com>, wrote:

Thanks for continuing to push this forward, Alejandro! I’m excited about the potential of having access to these APIs as part of the standard library. Here are a few comments on some different parts of the proposal:

1) For your RandomGenerator protocol, I’m not totally clear on the semantics of the next(_:) and next(_:upperBound:) methods. Do they both have zero as their lower bound, for example? I’m not sure it makes sense to have signed integers generated directly by an RNG—perhaps T: FixedWidthInteger & UnsignedInteger would be a more useful constraint. (Does it even need to be generic? What if RNGs just generate UInt32s?)

2) Can you say more about the purpose of the Randomizable protocol? How would we use that protocol in useful ways that we wouldn’t get from being able to select random values from ranges (half-open and closed) of FixedWidthInteger / BinaryFloatingPoint? My experience has been that a full-width random value is rarely what a user needs.

3) I agree with Xiaodi that Random should probably be a struct with a single shared instance, but I don’t think it should be internal. Hiding that shared RNG would make it hard for non-stdlib additions to have the same usage, as they would need to have completely separate implementations for the ā€œdefaultā€ and custom RNG versions.

4) I would also still suggest that the simplest version of random (that you use to get a value from a range or an element from a collection) should be a function, not a property. Collection properties like first, last, and count all represent facts that already exist about a collection, and don’t change unless the collection itself changes. Choosing a random element, on the other hand, is clearly going to be freshly performed on each call. In addition, with the notable exception of count, we try to ensure O(1) performance for properties, while random will be O(n) except in random-access collections. Finally, if it is a method, we can unify the two versions by providing a single method with the shared RNG as the default parameter.

5) To match the sorted() method, shuffled() should be on Sequence instead of Collection. I don’t think either shuffled() or shuffle() needs to be a protocol requirement, since there isn’t really any kind of customization necessary for different kinds of collections. Like the sorting algorithms, both could be regular extension methods.

6) I don’t know whether or not a consensus has formed around the correct spelling of the APIs for generating random values. From the proposal it looks like the preferred ways of getting a random value in a range would be to use the random property (or method) on a range or closed range:

    (0..<10).random // 7
    (0.0 ... 5.0).random // 4.112312

If that’s the goal, and we don’t want those values to be optional, we’ll need an implementation of random for floating-point ranges and an overload for fixed-width integer ranges. That said, I don’t think that style is as discoverable as having static methods or initializers available on the different types:

    Int.random(in: 0..<10)
    Double.random(in: 0.0 ... 5.0)
    // or maybe
    Int(randomIn: 0..<10)
    Double(randomIn: 0.0 ... 5.0)

(My only quibble with the initializer approach is that Bool would be awkward.)

In addition, this alternative approach could make creating random values more consistent with types that don’t work well in ranges:

    Data.random(bytes: 128)
    Color.random(r: 0...0, g: 0...1, b: 0...1, a: 1...1)

————

Thanks again!
Nate

On Nov 5, 2017, at 6:33 PM, Alejandro Alonso via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

[Proposal] Random Unification by Azoy Ā· Pull Request #760 Ā· apple/swift-evolution Ā· GitHub is the current API and proposed solution.

- Alejandro

On Nov 5, 2017, 6:18 PM -0600, Xiaodi Wu <xiaodi.wu@gmail.com <mailto:xiaodi.wu@gmail.com>>, wrote:

My comments are directed to the "more up-to-date" document that you just linked to in your reply to Jon. Is that one outdated? If so, can you send a link to the updated proposal and implementation for which you're soliciting feedback?

On Sun, Nov 5, 2017 at 6:12 PM, Alejandro Alonso <aalonso128@outlook.com <mailto:aalonso128@outlook.com>> wrote:
The proposal and implementation have the current updated API. The link I sent Jon was the one I brought up a few weeks ago which is outdated now. The proposal answers all of your questions. As for `.random` being a function, some would argue that it behaves in the same way as `.first` and `.last` which are properties.

- Alejandro

On Nov 5, 2017, 6:07 PM -0600, Xiaodi Wu <xiaodi.wu@gmail.com <mailto:xiaodi.wu@gmail.com>>, wrote:

A few quick thoughts:

I know that there's been some discussion that `(1...10).random` is the best spelling, but I'd like to push back on that suggestion. When I want a random number, I tend to think of the type I want first ("I want a random integer") and then a range ("I want a random integer between a and b"), not the other way around. My intuition is that `Int.random(in:)` will be more discoverable, both on that basis and because it is more similar to other languages' syntax (`Math.random` in JavaScript and `randint` in NumPy,
for example). It also has the advantage that the type is explicit, which I think is particularly useful in this case because the value itself is, well, random.

I would also argue that, `random` is most appropriately a method and not a property; there's no hard and fast rule for this, but the fact that the result is stochastic suggests (to me) that it's not a "property" of the range (or, for that matter, of the type).

I would reiterate here my qualms about `Source` being the term used for a generator. These types are not a _source_ of entropy but rather a _consumer_ of entropy.

`UnsafeRandomSource` needs to be renamed; "unsafe" has a specific meaning in Swift--that is, memory safety, and this is not it. Moreover, it's questionable whether this protocol is useful in any sense. What useful generic algorithms can one write with such a protocol?

`XoroshiroRandom` cannot be seeded by any `Numeric` value; depending on the specific algorithm it needs a seed of a specific bit width. If you default the shared instance to being seeded with an `Int` then you will have to have distinct implementations for 32-bit and 64-bit platforms. This is unadvisable. On that note, your `UnsafeRandomSource` needs to have an associated type and not a generic `<T : Numeric>` for the seed.

The default random number generator should be cryptographically secure; however, it's not clear to me that it should be device random.

I agree with others that alternative random number generators other than the default RNG (and, if not default, possibly also the device RNG) should be accommodated by the protocol hierarchy but not necessarily supplied in the stdlib.

The term `Randomizable` means something specific which is not how it's used in your proposed protocol.

There's still the open question, not answered, about how requesting an instance of the hardware RNG behaves when there's insufficient or no entropy. Does it return nil, throw, trap, or wait? The proposed API does not clarify this point, although based on the method signature it cannot return nil or throw. Trapping might be acceptable but I'd be interested to hear your take as to why it is preferable.

On Sun, Nov 5, 2017 at 4:43 PM, Alejandro Alonso via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:
For the proof of concept, I had accidentally deleted that one. I have a more up to date one which was discussed a few weeks later. Swift Random Unification Design Ā· GitHub

- Alejandro

On Nov 5, 2017, 4:37 PM -0600, Jonathan Hull <jhull@gbis.com <mailto:jhull@gbis.com>>, wrote:

Is there a link to the writeup? The one in the quote 404s.

Thanks,
Jon

On Nov 5, 2017, at 2:10 PM, Alejandro Alonso via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Hello once again Swift evolution community. I have taken the time to write up the proposal for this thread, and have provided an implementation for it as well. I hope to once again get good feedback on the overall proposal.

- Alejandro

On Sep 8, 2017, 11:52 AM -0500, Alejandro Alonso via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>>, wrote:

Hello swift evolution, I would like to propose a unified approach to `random()` in Swift. I have a simple implementation here https://gist.github.com/Azoy/5d294148c8b97d20b96ee64f434bb4f5\. This implementation is a simple wrapper over existing random functions so existing code bases will not be affected. Also, this approach introduces a new random feature for Linux users that give them access to upper bounds, as well as a lower bound for both Glibc and Darwin users. This change would be implemented within Foundation.

I believe this simple change could have a very positive impact on new developers learning Swift and experienced developers being able to write single random declarations.

I’d like to hear about your ideas on this proposal, or any implementation changes if need be.

- Alejando

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

What I was trying to respond to, by contrast, is the design of a hierarchy of protocols CSPRNG : PRNG (or, in Alejandro's proposal, UnsafeRandomSource : RandomSource) and the appropriate APIs to expose on each. This is entirely inapplicable to your examples. It stands to reason that a non-instantiable source of random numbers does not require a protocol of its own (a hypothetical RNG : CSPRNG), since there is no reason to implement (if done correctly) more than a single publicly non-instantiable singleton type that could conform to it. For that matter, the concrete type itself probably doesn't need *any* public API at all. Instead, extensions to standard library types such as Int that implement conformance to the protocol that Alejandro names "Randomizable" could call internal APIs to provide all the necessary functionality, and third-party types that need to conform to "Randomizable" could then in turn use `Int.random()` or `Double.random()` to implement their own conformance. In fact, the concrete random number generator type doesn't need to be public at all. All public interaction could be through APIs such as `Int.random()`.

If there is globally-available CSPRNG that people are encouraged to use, what is the benefit of a CSPRNG : PRNG hierarchy? What is the benefit of clearly identifying an algorithm as crypto-secure if the recommendation is that you use *the* crypto-secure object/functions?

Again, I'm not only talking about urandom. As far as I'm aware, every API to retrieve cryptographically secure sequences of random bits on every platform for which Swift is distributed can potentially return an error instead of random bits. The question is, what design for our API is the most sensible way to deal with this contingency? On rethinking, I do believe that consistently returning an Optional is the best way to go about it, allowing the user to either (a) supply a deterministic fallback; (b) raise an error of their own choosing; or (c) trap--all with a minimum of fuss. This seems very Swifty to me.

With Linux's getrandom, if you read from urandom (the default) and ask for as much or less than 256 bytes, the only possible error is that urandom hasn't been seeded <http://man7.org/linux/man-pages/man2/getrandom.2.html&gt;\. (With more than 256 bytes, it also becomes possible for the system call to be interrupted by a signal.) OpenBSD's getentropy is literally just arc4random running in the kernel <https://github.com/openbsd/src/blob/master/sys/dev/rnd.c#L889&gt;, and will only fail if you ask for more than 256 bytes because it is a hard-coded limit.

Ironically, I don't know how you get entropy on Darwin. If it's more failable that this, I'd argue that it's time to improve a bit...

FƩlix

Ā·Ā·Ā·

Le 27 sept. 2017 Ơ 17:29, Xiaodi Wu <xiaodi.wu@gmail.com> a Ʃcrit :

I really like the schedule here. After reading for a while, I do agree with Brent that stdlib should very primitive in functionality that it provides. I also agree that the most important part right now is designing the internal crypto on which the numeric types use to return their respected random number. On the discussion of how we should handle not enough entropy with the device random, from a users perspective it makes sense that calling .random should just give me a random number, but from a developers perspective I see Optional being the best choice here. While I think blocking could, in most cases, provide the user an easier API, we have to do this right and be safe here by providing a value that indicates that there is room for error here. As for the generator abstraction, I believe there should be a bare basic protocol that sets a layout for new generators and should be focusing on its requirements.

Whether or not RandomAccessCollection and MutableCollection should get .random and .shuffle/.shuffled in this first proposal is completely up in the air for me. It makes sense, to me, to include the .random in this proposal and open another one .shuffle/.shuffled, but I can see arguments that should say we create something separate for these two, or include all of it in this proposal.

- Alejandro

Ā·Ā·Ā·

On Sep 27, 2017, 7:29 PM -0500, Xiaodi Wu <xiaodi.wu@gmail.com>, wrote:

On Wed, Sep 27, 2017 at 00:18 FƩlix Cloutier <felixcloutier@icloud.com<mailto:felixcloutier@icloud.com>> wrote:
Le 26 sept. 2017 Ơ 16:14, Xiaodi Wu <xiaodi.wu@gmail.com<mailto:xiaodi.wu@gmail.com>> a Ʃcrit :

On Tue, Sep 26, 2017 at 11:26 AM, FƩlix Cloutier <felixcloutier@icloud.com<mailto:felixcloutier@icloud.com>> wrote:

It's possible to use a CSPRNG-grade algorithm and seed it once to get a reproducible sequence, but when you use it as a CSPRNG, you typically feed entropy back into it at nondeterministic points to ensure that even if you started with a bad seed, you'll eventually get to an alright state. Unless you keep track of when entropy was mixed in and what the values were, you'll never get a reproducible CSPRNG.

We would give developers a false sense of security if we provided them with CSPRNG-grade algorithms that we called CSPRNGs and that they could seed themselves. Just because it says "crypto-secure" in the name doesn't mean that it'll be crypto-secure if it's seeded with time(). Therefore, "reproducible" vs "non-reproducible" looks like a good distinction to me.

I disagree here, in two respects:

First, whether or not a particular PRNG is cryptographically secure is an intrinsic property of the algorithm; whether it's "reproducible" or not is determined by the published API. In other words, the distinction between CSPRNG vs. non-CSPRNG is important to document because it's semantics that cannot be deduced by the user otherwise, and it is an important one for writing secure code because it tells you whether an attacker can predict future outputs based only on observing past outputs. "Reproducible" in the sense of seedable or not is trivially noted by inspection of the published API, and it is rather immaterial to writing secure code.

Cryptographically secure is not a property that I'm comfortable applying to an algorithm. You cannot say that you've made a cryptographically secure thing just because you've used all the right algorithms: you also have to use them right, and one of the most critical components of a cryptographically secure PRNG is its seed.

A cryptographically secure algorithm isn’t sufficient, but it is necessary. That’s why it’s important to mark them as such. If I'm a careful developer, then it is absolutely important to me to know that I’m using a PRNG with a cryptographically secure algorithm, and that the particular implementation of that algorithm is correct and secure.

It is a *feature* of a lot of modern CSPRNGs that you can't seed them:

Ā Ā * You cannot seed or add entropy to std::random_device

Although std::random_device may in practice be backed by a software CSPRNG, IIUC, the intention is that it can provide access to a hardware non-deterministic source when available.

Ā Ā * You cannot seed or add entropy to CryptGenRandom
Ā Ā * You can only add entropy to /dev/(u)random
Ā Ā * You can only add entropy to BSD's arc4random

Ah, I see. I think we mean different things when we say PRNG. A PRNG is an entirely deterministic algorithm; the output is non-random and the algorithm itself requires no entropy. If a PRNG is seeded with a random sequence of bits, its output can "appear" to be random. A CSPRNG is a PRNG that fulfills certain criteria such that its output can be appropriate for use in cryptographic applications in place of a truly random sequence *if* the input to the CSPRNG is itself random.

The examples you give above *incorporate* a CSPRNG, environment entropy, and a set of rules about when to mix in additional entropy in order to produce output indistinguishable from a random sequence, but they are *not* themselves really *pseudorandom* generators because they are not deterministic. Not only do such sources of random numbers not require an interface to allow seeding, they do not even have to be publicly instantiable: Swift need only expose a single thread-safe instance (or an instance per thread) of a single type that provides access to CryptGenRandom/urandom/arc4random, since after all the output of multiple instances of that type should be statistically indistinguishable from the output of only one.

What I was trying to respond to, by contrast, is the design of a hierarchy of protocols CSPRNG : PRNG (or, in Alejandro's proposal, UnsafeRandomSource : RandomSource) and the appropriate APIs to expose on each. This is entirely inapplicable to your examples. It stands to reason that a non-instantiable source of random numbers does not require a protocol of its own (a hypothetical RNG : CSPRNG), since there is no reason to implement (if done correctly) more than a single publicly non-instantiable singleton type that could conform to it. For that matter, the concrete type itself probably doesn't need *any* public API at all. Instead, extensions to standard library types such as Int that implement conformance to the protocol that Alejandro names "Randomizable" could call internal APIs to provide all the necessary functionality, and third-party types that need to conform to "Randomizable" could then in turn use `Int.random()` or `Double.random()` to implement their own conformance. In fact, the concrete random number generator type doesn't need to be public at all. All public interaction could be through APIs such as `Int.random()`.

Just because we can expose a seed interface doesn't mean we should, and in this case I believe that it would go against the prime objective of providing secure random numbers.

If we're talking about a Swift interface to a non-deterministic source of random numbers like urandom or arc4random, then, as I write above, not only do I agree that it doesn't need to be seedable, it also does not need to be instantiable at all, does not need to conform to a protocol that specifically requires the semantics of a non-deterministic source, does not need to expose any public interface whatsoever, and doesn't itself even need to be public. (Does it even need to be a type, as opposed to simply a free function?)

In fact, having reasoned through all of this, we can split the design task into two. The most essential part, which definitely should be part of the stdlib, would be an internal interface to a cryptographically secure platform-specific entropy source, a public protocol named something like Randomizable (to be bikeshedded), and the appropriate implementations on Boolean, binary integer, and floating point types to conform them to Randomizable so that users can write `Bool.random()` or `Int.random()`. The second part, which can be a separate proposal or even a standalone core library or third-party library, would be the protocols and concrete types that implement pseudorandom number generators, allowing for reproducible pseudorandom sequences. In other words, instead of PRNGs and CSPRNGs being the primitives on which `Int.random()` is implemented; `Int.random()` should be the standard library primitive which allows PRNGs and CSPRNGs to be seeded.
If your attacker can observe your seeding once, chances are that they can observe your reseeding too; then, they can use their own implementation of the PRNG (whether CSPRNG or non-CSPRNG) and reproduce your pseudorandom sequence whether or not Swift exposes any particular API.

On Linux, the random devices are initially seeded with machine-specific but rather invariant data that makes /dev/urandom spit out predictable numbers. It is considered "seeded" after a root process writes POOL_SIZE bytes to it. On most implementations, this initial seed is stored on disk: when the computer shuts down, it reads POOL_SIZE bytes from /dev/urandom and saves it in a file, and the contents of that file is loaded back into /dev/urandom when the computer starts. A scenario where someone can read that file is certainly not less likely than a scenario where /dev/urandom was deleted. That doesn't mean that they have kernel code execution or that they can pry into your process, but they have a good shot at guessing your seed and subsequent RNG results if no stirring happens.

Sorry, I don't understand what you're getting at here. Again, I'm talking about deterministic algorithms, not non-deterministic sources of random numbers.

Secondly, I see no reason to justify the notion that, simply because a PRNG is cryptographically secure, we ought to hide the seeding initializer (because one has to exist internally anyway) from the public. Obviously, one use case for a deterministic PRNG is to get reproducible sequences of random-appearing values; this can be useful whether the underlying algorithm is cryptographically secure or not. There are innumerably many ways to use data generated from a CSPRNG in non-cryptographically secure ways and omitting or including a public seeding initializer does not change that; in other words, using a deterministic seed for a CSPRNG would be a bad idea in certain applications, but it's a deliberate act, and someone who would mistakenly do that is clearly incapable of *using* the output from the PRNG in a secure way either; put a third way, you would be hard pressed to find a situation where it's true that "if only Swift had not made the seeding initializer public, this author would have written secure code, but instead the only security hole that existed in the code was caused by the availability of a public seeding initializer mistakenly used." The point of having both explicitly instantiable PRNGs and a layer of simpler APIs like "Int.random()" is so that the less experienced user can get the "right thing" by default, and the experienced user can customize the behavior; any user that instantiates his or her own ChaCha20Random instance is already calling for the power user interface; it is reasonable to expose the underlying primitive operations (such as seeding) so long as there are legitimate uses for it.

Nothing prevents us from using the same algorithm for a CSPRNG that is safely pre-seeded and a PRNG that people seed themselves, mind you. However, especially when it comes to security, there is a strong responsibility to drive developers into a pit of success: the most obvious thing to do has to be the right one, and suggesting to cryptographically-unaware developers that they have everything they need to manage their own seed is not a step in that direction.

I'm not opposed to a ChaCha20Random type; I'm opposed to explicitly calling it cryptographically-secure, because it is not unless you know what to do with it. It is emphatically not far-fetched to imagine a developer who thinks that they can outdo the standard library by using their own ChaCha20Random instance after it's been seeded with time() if we let them know that it's "cryptographically secure". If you're a power user and you don't like the default, known-good CSPRNG, then you're hopefully good enough to know that ChaCha20 is considered a cryptographically-secure algorithm without help labels from the language, and you know how to operate it.

I'm fully aware of the myths surrounding /dev/urandom and /dev/random. /dev/urandom might never run out, but it is also possible for it not to be initialized at all, as in the case of some VM setups. In some older versions of iOS, /dev/[u]random is reportedly sandboxed out. On systems where it is available, it can also be deleted, since it is a file. The point is, all of these scenarios cause an error during seeding of a CSPRNG. The question is, how to proceed in the face of inability to access entropy. We must do something, because we cannot therefore return a cryptographically secure answer. Rare trapping on invocation of Int.random() or permanently waiting for a never-to-be-initialized /dev/urandom would be terrible to debug, but returning an optional or throwing all the time would be verbose. How to design this API?

If the only concern is that the system might not be initialized enough, I'd say that whatever returns an instance of a global, framework-seeded CSPRNG should return an Optional, and the random methods that use the global CSPRNG can trap and scream that the system is not initialized enough. If this is a likely error for you, you can check if the CSPRNG exists or not before jumping.

Also note that there is only one system for which Swift is officially distributed (Ubuntu 14.04) on which the only way to get entropy from the OS is to open a random device and read from it.

Again, I'm not only talking about urandom. As far as I'm aware, every API to retrieve cryptographically secure sequences of random bits on every platform for which Swift is distributed can potentially return an error instead of random bits. The question is, what design for our API is the most sensible way to deal with this contingency? On rethinking, I do believe that consistently returning an Optional is the best way to go about it, allowing the user to either (a) supply a deterministic fallback; (b) raise an error of their own choosing; or (c) trap--all with a minimum of fuss. This seems very Swifty to me.

* What should the default CSPRNG be? There are good arguments for using a cryptographically secure device random. (In my proposed implementation, for device random, I use Security.framework on Apple platforms (because /dev/urandom is not guaranteed to be available due to the sandbox, IIUC). On Linux platforms, I would prefer to use getrandom() and avoid using file system APIs, but getrandom() is new and unsupported on some versions of Ubuntu that Swift supports. This is an issue in and of itself.) Now, a number of these facilities strictly limit or do not guarantee availability of more than a small number of random bytes at a time; they are recommended for seeding other PRNGs but *not* as a routine source of random numbers. Therefore, although device random should be available to users, it probably shouldn’t be the default for the Swift standard library as it could have negative consequences for the system as a whole. There follows the significant task of implementing a CSPRNG correctly and securely for the default PRNG.

Theo give a talk a few years ago<Hackfest 2014: Theo de Raadt presented "arc4random - randomization for all occasions" - YouTube; on randomness and how these problems are approached in LibreSSL.

Certainly, we can learn a lot from those like Theo who've dealt with the issue. I'm not in a position to watch the talk at the moment; can you summarize what the tl;dr version of it is?

I saw it three years ago, so I don't remember all the details. The gist is that:

Ā Ā * OpenBSD's random is available from extremely early in the boot process with reasonable entropy

Ā Ā * LibreSSL includes OpenBSD's arc4random, and it's a "good" PRNG (which doesn't actually use ARC4)
Ā Ā * That implementation of arc4random is good because it is fool-proof and it has basically no failure mode
Ā Ā * Stirring is good, having multiple components take random numbers from the same source probably makes results harder to guess too
Ā Ā * Getrandom/getentropy is in all ways better than reading from random devices

Vigorously agree on all points. Thanks for the summary.

That's perfectly fine, I should think. Your 'random' and 'random(using:)'
are distinct too.

Ā·Ā·Ā·

On Sun, Nov 5, 2017 at 21:32 Alejandro Alonso <aalonso128@outlook.com> wrote:

An error that came up in development, default arguments cannot use
internal types, `shuffle(using:)` and `shuffled(using:)`. To remedy this
issue I would have to add another method requirement with no arguments that
gives a default implementation of using `_Random`. This would work, but
would appear in stdlib documentation as two separate functions.

- Alejandro

On Nov 5, 2017, 9:08 PM -0600, Xiaodi Wu <xiaodi.wu@gmail.com>, wrote:

On Sun, Nov 5, 2017 at 8:21 PM, Alejandro Alonso <aalonso128@outlook.com> > wrote:

I can agree with mostly everything in here. I think `.random` on
`RandomAccessCollection` should mimic the current design with `.first` and
`.last` by returning an optional. In terms of the naming of this, we have
to look at how python structures the call site. `random.choice([1, 2, 3,
4])` To me this reads, random choice within this array. This works because
of how it’s called. With the proposed solution, we are calling to get a
random element directly from the array. So I stick by with naming this
random.

On the subject of bike shedding the names, I can agree to use
`RandomNumberGenerator` whole heartily. As for `Randomizable`, I agree
there might be a better name for this, but the question is what?

That is a good question. I don't like `RandomlySamplable`; it might be
that `Randomizable` could be the least bad option.

One more comment that I neglected to make. You don't need to use an `enum`
in order to make a type non-initializable. A `struct` can do the same with
a private init. There are established singleton patterns for Swift and an
`enum` with a single case is...atypical. I'm also not sure why this should
be a public type at all. Why expose `Random.default.next()`? Shouldn't the
user always use `T.random`? Afaict, `Random` can be an internal type
`_Random` that the user never sees at all.

On Nov 5, 2017, 7:56 PM -0600, Xiaodi Wu <xiaodi.wu@gmail.com>, wrote:

I like this particular version. In particular, the choice of algorithms
is, afaict, correct and that is incredibly important. I had overlooked that
`arc4random` is cryptographically secure past a certain version of macOS,
but you are absolutely right. I am also on board with the fatal error
suggestion if random entropy is unavailable; I think it must be amply
documented, though.

I do think, however, that you're overloading too many things into the
word "random" when they're not the same. Take a look at Python, which is
pretty widely used for numerics. There's `rand` and `random` for getting a
random integer or floating-point value, and there's `choice` and `sample`
for choosing one or more values out of a collection without replacement.
These are sufficiently different tasks and don't all need to be called
"random" or satisfy the same requirement of the same protocol. Put another
way, it's absolutely *not* inconsistent for numeric types to have
`random()` while collection types have a differently named method.

By contrast, I think the great length of text trying to justify naming
all of these facilities `random` in order to parallel `first` and `last`
shows how the proposed design is comparatively weaker. You have to argue
that (a) `Int.random` shouldn't return an optional value because it'd be
unwieldy, and therefore `(0..<5).random` shouldn't either because it would
then be inconsistent; but (b) that `(0..<5).random` should be spelled and
behave like `(0..<5).first` and `(0..<5).last` even though the user must
handle empty collections totally differently because the return types are
not the same. Either `(0..<5).random` should behave analogously to `first`
and `last` or it should not. If it should, it only makes sense to return a
result of type `T?`. After all, if a collection doesn't have a `first`
item, then it can't have a `random` item. Put another way, having a `first`
item is a prerequisite to having a randomly selectable item. The behavior
of the Swift APIs would be very consistent if `first` returns `T?` but
`random` returns `T`. However, I agree that unwrapping `Int.random` every
time would be burdensome, and it would not make sense to have a type
support `random` but not have any instantiable values; therefore, returning
an optional value doesn't make sense, and it follows that `Int.random`
*shouldn't* behave like `first` or `last`.

Once you stop trying to make what Python calls `rand/randint` and
`choice/sample` have the same names, then finding a Swifty design for the
distinct facilities becomes much easier, and it suggests a pretty elegant
result (IMO):

[1, 2, 3, 4].choice // like `first` or `last`, this gets you a value of
type Int?
[1, 2, 3, 4].sampling(2) // like `prefix(2)` or `suffix(2)`, this gets
you a subsequence with at most two elements

Int.random // this gets you a random Int; or it may trap
Float.random // this gets you a random Float; or it may trap

With that, it also becomes clear why--and I agree with you--an
independent `Int.random(in: 0..<5)` is not necessary. `(0..<5).choice` is
fine, and it can now appropriately return a value of type `T?` because it
no longer needs to parallel `Int.random`.

* * *

More in the bikeshedding arena, I take issue with some of the names:

- I reiterate my comment that `Randomizable` is not the best name. There
are multiple dictionary definitions of "randomize" and one is "make
unpredictable, unsystematic, or random in order or arrangement." Wikipedia
gives at least five different contextual meanings for the word. What you're
doing here is specifically **random sampling** and we can do better to
clarify that, I think.

- While I agree that `RNG` can be cryptic, the alternative should be
`RandomNumberGenerator` (as it's called in other languages);
`RandomGenerator` is not quite accurate. Again, we're _consuming_
randomness to _generate_ numbers (or values of other type, based on the
result of a generated number). We're not _generating_ randomness.

On Sun, Nov 5, 2017 at 6:33 PM, Alejandro Alonso <aalonso128@outlook.com> >> wrote:

[Proposal] Random Unification by Azoy Ā· Pull Request #760 Ā· apple/swift-evolution Ā· GitHub is the current API
and proposed solution.

- Alejandro

On Nov 5, 2017, 6:18 PM -0600, Xiaodi Wu <xiaodi.wu@gmail.com>, wrote:

My comments are directed to the "more up-to-date" document that you just
linked to in your reply to Jon. Is that one outdated? If so, can you send a
link to the updated proposal and implementation for which you're soliciting
feedback?

On Sun, Nov 5, 2017 at 6:12 PM, Alejandro Alonso <aalonso128@outlook.com >>> > wrote:

The proposal and implementation have the current updated API. The link
I sent Jon was the one I brought up a few weeks ago which is outdated now.
The proposal answers all of your questions. As for `.random` being a
function, some would argue that it behaves in the same way as `.first` and
`.last` which are properties.

- Alejandro

On Nov 5, 2017, 6:07 PM -0600, Xiaodi Wu <xiaodi.wu@gmail.com>, wrote:

A few quick thoughts:

I know that there's been some discussion that `(1...10).random` is the
best spelling, but I'd like to push back on that suggestion. When I want a
random number, I tend to think of the type I want first ("I want a random
integer") and then a range ("I want a random integer between a and b"), not
the other way around. My intuition is that `Int.random(in:)` will be more
discoverable, both on that basis and because it is more similar to other
languages' syntax (`Math.random` in JavaScript and `randint` in NumPy, for
example). It also has the advantage that the type is explicit, which I
think is particularly useful in this case because the value itself is,
well, random.

I would also argue that, `random` is most appropriately a method and
not a property; there's no hard and fast rule for this, but the fact that
the result is stochastic suggests (to me) that it's not a "property" of the
range (or, for that matter, of the type).

I would reiterate here my qualms about `Source` being the term used for
a generator. These types are not a _source_ of entropy but rather a
_consumer_ of entropy.

`UnsafeRandomSource` needs to be renamed; "unsafe" has a specific
meaning in Swift--that is, memory safety, and this is not it. Moreover,
it's questionable whether this protocol is useful in any sense. What useful
generic algorithms can one write with such a protocol?

`XoroshiroRandom` cannot be seeded by any `Numeric` value; depending on
the specific algorithm it needs a seed of a specific bit width. If you
default the shared instance to being seeded with an `Int` then you will
have to have distinct implementations for 32-bit and 64-bit platforms. This
is unadvisable. On that note, your `UnsafeRandomSource` needs to have an
associated type and not a generic `<T : Numeric>` for the seed.

The default random number generator should be cryptographically secure;
however, it's not clear to me that it should be device random.

I agree with others that alternative random number generators other
than the default RNG (and, if not default, possibly also the device RNG)
should be accommodated by the protocol hierarchy but not necessarily
supplied in the stdlib.

The term `Randomizable` means something specific which is not how it's
used in your proposed protocol.

There's still the open question, not answered, about how requesting an
instance of the hardware RNG behaves when there's insufficient or no
entropy. Does it return nil, throw, trap, or wait? The proposed API does
not clarify this point, although based on the method signature it cannot
return nil or throw. Trapping might be acceptable but I'd be interested to
hear your take as to why it is preferable.

On Sun, Nov 5, 2017 at 4:43 PM, Alejandro Alonso via swift-evolution < >>>> swift-evolution@swift.org> wrote:

For the proof of concept, I had accidentally deleted that one. I have
a more up to date one which was discussed a few weeks later.
Swift Random Unification Design Ā· GitHub

- Alejandro

On Nov 5, 2017, 4:37 PM -0600, Jonathan Hull <jhull@gbis.com>, wrote:

Is there a link to the writeup? The one in the quote 404s.

Thanks,
Jon

On Nov 5, 2017, at 2:10 PM, Alejandro Alonso via swift-evolution < >>>>> swift-evolution@swift.org> wrote:

Hello once again Swift evolution community. I have taken the time to
write up the proposal for this thread, and have provided an implementation
for it as well. I hope to once again get good feedback on the overall
proposal.

- Alejandro

On Sep 8, 2017, 11:52 AM -0500, Alejandro Alonso via swift-evolution < >>>>> swift-evolution@swift.org>, wrote:

Hello swift evolution, I would like to propose a unified approach to
`random()` in Swift. I have a simple implementation here
https://gist.github.com/Azoy/5d294148c8b97d20b96ee64f434bb4f5\. This
implementation is a simple wrapper over existing random functions so
existing code bases will not be affected. Also, this approach introduces a
new random feature for Linux users that give them access to upper bounds,
as well as a lower bound for both Glibc and Darwin users. This change would
be implemented within Foundation.

I believe this simple change could have a very positive impact on new
developers learning Swift and experienced developers being able to write
single random declarations.

I’d like to hear about your ideas on this proposal, or any
implementation changes if need be.

- Alejando

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

If a generator should return an associated type, what if the associated type for Random? I assume people will want this to be UInt32.

Yea I think I’m going to keep returning an optional on all collections/ranges and getting rid of Int.random(in:). I still agree it’s less discoverable, but I think there will be plenty of discussion about it that it will be known.

- Alejandro

Ā·Ā·Ā·

On Nov 12, 2017, 9:46 PM -0600, Xiaodi Wu <xiaodi.wu@gmail.com>, wrote:
On Sun, Nov 12, 2017 at 19:47 Alejandro Alonso <aalonso128@outlook.com<mailto:aalonso128@outlook.com>> wrote:
Sorry I’ve been gone for a while, I had to do a lot of traveling.

1. Initially I made this thinking that developers had the power to determine their own lower bound. The current implementation uses the integer’s min value as a lower bound. If it makes sense to only allow unsigned integers from an RNG, then I’m perfectly fine with. I do disagree when you say that it should only generate UInt32s. The current approach allows, lets say mt19337 and mt19337-64, to be used within one generator. So if you wanted a UInt32, mt19337 would be used, and if you asked for a UInt64, mt19337-64 would be used.

I agree with Nate that this doesn't need to be--and is better off not being--generic. The two different Mersenne Twisters are properly two distinct PRNGs. A user sufficiently sophisticated to ask for their own PRNG would expect the same algorithm to be used to generate a value of any bitwidth, by concatenation of successively generated random bits if necessary.

Instead, the return type should be an associated type. 32-bit PRNG algorithms would always return values of type UInt32 and 64-bit algorithms would always return values of type UInt64.

2. The Randomizable protocol isn’t always used with integers. Think Date.random or Color.random. These types of values are difficult to express with ranges. Randomizable solves this issue.

3. I’ve made the adjustment necessary for this.

4. So while I can see your point for this, it would break the consistency with Randomizable’s random property. You could argue that we could make this property a function itself, but I think most will agree that Int.random is a cleaner api than Int.random().

5. I’ve made the adjustment necessary for this.

6. I actually forgot to implement the random api for the ranges where Bound: BinaryFloatingPoint. While implementing this, I realized these would never fail and would always return a non-optional.

Sure it can. 0.0..<0.0 is an empty range, just like 0..<0. Any collection type really should return an Optional for 'random' because collections can be empty.

So, I decided making the other Countable ranges non-optional. (0 ..< 10).random would return a non-optional, (0.0 ..< 10.0).random would return a non-optional, and Array(0 ..< 10).random would return an optional. I can agree that something like (0 ..< 10).random is hard to discover, so I added Int.random(in: 0 ..< 10) (along with BinaryFloatingPoint).

This is unsatisfying (to me, at least). The proposed design should offer one very good way to accomplish something, not two spellings for the same thing.

However, these are not requirements of Randomizable. I think these methods would benefit more if they were extension methods:

extension Randomizable where Self: FixedWidthInteger, Self.Stride: SignedInteger {
public static func random(
in range: Countable{Closed}Range,
using generator: RandomNumberGenerator
) -> Self {
return range.random(using: generator)
}
}

extension Randomizable where Self: BinaryFloatingPointer {
public static func random(
in range: {Closed}Range,
using generator: RandomNumberGenerator
) -> Self {
return range.random
}
}

I think external types that wish to do something similar, like Data.random(bytes: 128), could extend Randomizable with their own custom needs. The stdlib would at this point provide all the features needed to make this happen very simply for something like Data.random(bytes: 128).

- Alejandro

On Nov 5, 2017, 10:44 PM -0600, Nate Cook <natecook@apple.com<mailto:natecook@apple.com>>, wrote:
Thanks for continuing to push this forward, Alejandro! I’m excited about the potential of having access to these APIs as part of the standard library. Here are a few comments on some different parts of the proposal:

1) For your RandomGenerator protocol, I’m not totally clear on the semantics of the next(_:) and next(_:upperBound:) methods. Do they both have zero as their lower bound, for example? I’m not sure it makes sense to have signed integers generated directly by an RNG—perhaps T: FixedWidthInteger & UnsignedInteger would be a more useful constraint. (Does it even need to be generic? What if RNGs just generate UInt32s?)

2) Can you say more about the purpose of the Randomizable protocol? How would we use that protocol in useful ways that we wouldn’t get from being able to select random values from ranges (half-open and closed) of FixedWidthInteger / BinaryFloatingPoint? My experience has been that a full-width random value is rarely what a user needs.

3) I agree with Xiaodi that Random should probably be a struct with a single shared instance, but I don’t think it should be internal. Hiding that shared RNG would make it hard for non-stdlib additions to have the same usage, as they would need to have completely separate implementations for the ā€œdefaultā€ and custom RNG versions.

4) I would also still suggest that the simplest version of random (that you use to get a value from a range or an element from a collection) should be a function, not a property. Collection properties like first, last, and count all represent facts that already exist about a collection, and don’t change unless the collection itself changes. Choosing a random element, on the other hand, is clearly going to be freshly performed on each call. In addition, with the notable exception of count, we try to ensure O(1) performance for properties, while random will be O(n) except in random-access collections. Finally, if it is a method, we can unify the two versions by providing a single method with the shared RNG as the default parameter.

5) To match the sorted() method, shuffled() should be on Sequence instead of Collection. I don’t think either shuffled() or shuffle() needs to be a protocol requirement, since there isn’t really any kind of customization necessary for different kinds of collections. Like the sorting algorithms, both could be regular extension methods.

6) I don’t know whether or not a consensus has formed around the correct spelling of the APIs for generating random values. From the proposal it looks like the preferred ways of getting a random value in a range would be to use the random property (or method) on a range or closed range:

Ā Ā Ā Ā (0..<10).random // 7
Ā Ā Ā Ā (0.0 ... 5.0).random // 4.112312

If that’s the goal, and we don’t want those values to be optional, we’ll need an implementation of random for floating-point ranges and an overload for fixed-width integer ranges. That said, I don’t think that style is as discoverable as having static methods or initializers available on the different types:

Ā Ā Ā Ā Int.random(in: 0..<10)
Ā Ā Ā Ā Double.random(in: 0.0 ... 5.0)
Ā Ā Ā Ā // or maybe
Ā Ā Ā Ā Int(randomIn: 0..<10)
Ā Ā Ā Ā Double(randomIn: 0.0 ... 5.0)

(My only quibble with the initializer approach is that Bool would be awkward.)

In addition, this alternative approach could make creating random values more consistent with types that don’t work well in ranges:

Ā Ā Ā Ā Data.random(bytes: 128)
Ā Ā Ā Ā Color.random(r: 0...0, g: 0...1, b: 0...1, a: 1...1)

————

Thanks again!
Nate

On Nov 5, 2017, at 6:33 PM, Alejandro Alonso via swift-evolution <swift-evolution@swift.org<mailto:swift-evolution@swift.org>> wrote:

[Proposal] Random Unification by Azoy Ā· Pull Request #760 Ā· apple/swift-evolution Ā· GitHub is the current API and proposed solution.

- Alejandro

On Nov 5, 2017, 6:18 PM -0600, Xiaodi Wu <xiaodi.wu@gmail.com<mailto:xiaodi.wu@gmail.com>>, wrote:
My comments are directed to the "more up-to-date" document that you just linked to in your reply to Jon. Is that one outdated? If so, can you send a link to the updated proposal and implementation for which you're soliciting feedback?

On Sun, Nov 5, 2017 at 6:12 PM, Alejandro Alonso <aalonso128@outlook.com<mailto:aalonso128@outlook.com>> wrote:
The proposal and implementation have the current updated API. The link I sent Jon was the one I brought up a few weeks ago which is outdated now. The proposal answers all of your questions. As for `.random` being a function, some would argue that it behaves in the same way as `.first` and `.last` which are properties.

- Alejandro

On Nov 5, 2017, 6:07 PM -0600, Xiaodi Wu <xiaodi.wu@gmail.com<mailto:xiaodi.wu@gmail.com>>, wrote:
A few quick thoughts:

I know that there's been some discussion that `(1...10).random` is the best spelling, but I'd like to push back on that suggestion. When I want a random number, I tend to think of the type I want first ("I want a random integer") and then a range ("I want a random integer between a and b"), not the other way around. My intuition is that `Int.random(in:)` will be more discoverable, both on that basis and because it is more similar to other languages' syntax (`Math.random` in JavaScript and `randint` in NumPy,
for example). It also has the advantage that the type is explicit, which I think is particularly useful in this case because the value itself is, well, random.

I would also argue that, `random` is most appropriately a method and not a property; there's no hard and fast rule for this, but the fact that the result is stochastic suggests (to me) that it's not a "property" of the range (or, for that matter, of the type).

I would reiterate here my qualms about `Source` being the term used for a generator. These types are not a _source_ of entropy but rather a _consumer_ of entropy.

`UnsafeRandomSource` needs to be renamed; "unsafe" has a specific meaning in Swift--that is, memory safety, and this is not it. Moreover, it's questionable whether this protocol is useful in any sense. What useful generic algorithms can one write with such a protocol?

`XoroshiroRandom` cannot be seeded by any `Numeric` value; depending on the specific algorithm it needs a seed of a specific bit width. If you default the shared instance to being seeded with an `Int` then you will have to have distinct implementations for 32-bit and 64-bit platforms. This is unadvisable. On that note, your `UnsafeRandomSource` needs to have an associated type and not a generic `<T : Numeric>` for the seed.

The default random number generator should be cryptographically secure; however, it's not clear to me that it should be device random.

I agree with others that alternative random number generators other than the default RNG (and, if not default, possibly also the device RNG) should be accommodated by the protocol hierarchy but not necessarily supplied in the stdlib.

The term `Randomizable` means something specific which is not how it's used in your proposed protocol.

There's still the open question, not answered, about how requesting an instance of the hardware RNG behaves when there's insufficient or no entropy. Does it return nil, throw, trap, or wait? The proposed API does not clarify this point, although based on the method signature it cannot return nil or throw. Trapping might be acceptable but I'd be interested to hear your take as to why it is preferable.

On Sun, Nov 5, 2017 at 4:43 PM, Alejandro Alonso via swift-evolution <swift-evolution@swift.org<mailto:swift-evolution@swift.org>> wrote:
For the proof of concept, I had accidentally deleted that one. I have a more up to date one which was discussed a few weeks later. Swift Random Unification Design Ā· GitHub

- Alejandro

On Nov 5, 2017, 4:37 PM -0600, Jonathan Hull <jhull@gbis.com<mailto:jhull@gbis.com>>, wrote:
Is there a link to the writeup? The one in the quote 404s.

Thanks,
Jon

On Nov 5, 2017, at 2:10 PM, Alejandro Alonso via swift-evolution <swift-evolution@swift.org<mailto:swift-evolution@swift.org>> wrote:

Hello once again Swift evolution community. I have taken the time to write up the proposal for this thread, and have provided an implementation for it as well. I hope to once again get good feedback on the overall proposal.

- Alejandro

On Sep 8, 2017, 11:52 AM -0500, Alejandro Alonso via swift-evolution <swift-evolution@swift.org<mailto:swift-evolution@swift.org>>, wrote:
Hello swift evolution, I would like to propose a unified approach to `random()` in Swift. I have a simple implementation here https://gist.github.com/Azoy/5d294148c8b97d20b96ee64f434bb4f5\. This implementation is a simple wrapper over existing random functions so existing code bases will not be affected. Also, this approach introduces a new random feature for Linux users that give them access to upper bounds, as well as a lower bound for both Glibc and Darwin users. This change would be implemented within Foundation.

I believe this simple change could have a very positive impact on new developers learning Swift and experienced developers being able to write single random declarations.

I’d like to hear about your ideas on this proposal, or any implementation changes if need be.

- Alejando

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org<mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org<mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org<mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

After thinking about this for a while, I don’t agree with with an associated type on RandomNumberGenerator. I think a generic FixedWidthInteger & UnsignedInteger should be sufficient. If there were an associated type, and the default for Random was UInt32, then there might be some arguments about allowing Double to utilize the full 64 bit precision. We could make Random32 and Random64, but I think people will ask why there isn’t a Random8 or Random16 for those bit widths. The same could also be said that any experienced developer would know that his PRNG would be switched if he asked for 32 bit or 64 bit.

- Alejandro

Ā·Ā·Ā·

On Nov 12, 2017, 9:46 PM -0600, Xiaodi Wu <xiaodi.wu@gmail.com>, wrote:
On Sun, Nov 12, 2017 at 19:47 Alejandro Alonso <aalonso128@outlook.com<mailto:aalonso128@outlook.com>> wrote:
Sorry I’ve been gone for a while, I had to do a lot of traveling.

1. Initially I made this thinking that developers had the power to determine their own lower bound. The current implementation uses the integer’s min value as a lower bound. If it makes sense to only allow unsigned integers from an RNG, then I’m perfectly fine with. I do disagree when you say that it should only generate UInt32s. The current approach allows, lets say mt19337 and mt19337-64, to be used within one generator. So if you wanted a UInt32, mt19337 would be used, and if you asked for a UInt64, mt19337-64 would be used.

I agree with Nate that this doesn't need to be--and is better off not being--generic. The two different Mersenne Twisters are properly two distinct PRNGs. A user sufficiently sophisticated to ask for their own PRNG would expect the same algorithm to be used to generate a value of any bitwidth, by concatenation of successively generated random bits if necessary.

Instead, the return type should be an associated type. 32-bit PRNG algorithms would always return values of type UInt32 and 64-bit algorithms would always return values of type UInt64.

2. The Randomizable protocol isn’t always used with integers. Think Date.random or Color.random. These types of values are difficult to express with ranges. Randomizable solves this issue.

3. I’ve made the adjustment necessary for this.

4. So while I can see your point for this, it would break the consistency with Randomizable’s random property. You could argue that we could make this property a function itself, but I think most will agree that Int.random is a cleaner api than Int.random().

5. I’ve made the adjustment necessary for this.

6. I actually forgot to implement the random api for the ranges where Bound: BinaryFloatingPoint. While implementing this, I realized these would never fail and would always return a non-optional.

Sure it can. 0.0..<0.0 is an empty range, just like 0..<0. Any collection type really should return an Optional for 'random' because collections can be empty.

So, I decided making the other Countable ranges non-optional. (0 ..< 10).random would return a non-optional, (0.0 ..< 10.0).random would return a non-optional, and Array(0 ..< 10).random would return an optional. I can agree that something like (0 ..< 10).random is hard to discover, so I added Int.random(in: 0 ..< 10) (along with BinaryFloatingPoint).

This is unsatisfying (to me, at least). The proposed design should offer one very good way to accomplish something, not two spellings for the same thing.

However, these are not requirements of Randomizable. I think these methods would benefit more if they were extension methods:

extension Randomizable where Self: FixedWidthInteger, Self.Stride: SignedInteger {
public static func random(
in range: Countable{Closed}Range,
using generator: RandomNumberGenerator
) -> Self {
return range.random(using: generator)
}
}

extension Randomizable where Self: BinaryFloatingPointer {
public static func random(
in range: {Closed}Range,
using generator: RandomNumberGenerator
) -> Self {
return range.random
}
}

I think external types that wish to do something similar, like Data.random(bytes: 128), could extend Randomizable with their own custom needs. The stdlib would at this point provide all the features needed to make this happen very simply for something like Data.random(bytes: 128).

- Alejandro

On Nov 5, 2017, 10:44 PM -0600, Nate Cook <natecook@apple.com<mailto:natecook@apple.com>>, wrote:
Thanks for continuing to push this forward, Alejandro! I’m excited about the potential of having access to these APIs as part of the standard library. Here are a few comments on some different parts of the proposal:

1) For your RandomGenerator protocol, I’m not totally clear on the semantics of the next(_:) and next(_:upperBound:) methods. Do they both have zero as their lower bound, for example? I’m not sure it makes sense to have signed integers generated directly by an RNG—perhaps T: FixedWidthInteger & UnsignedInteger would be a more useful constraint. (Does it even need to be generic? What if RNGs just generate UInt32s?)

2) Can you say more about the purpose of the Randomizable protocol? How would we use that protocol in useful ways that we wouldn’t get from being able to select random values from ranges (half-open and closed) of FixedWidthInteger / BinaryFloatingPoint? My experience has been that a full-width random value is rarely what a user needs.

3) I agree with Xiaodi that Random should probably be a struct with a single shared instance, but I don’t think it should be internal. Hiding that shared RNG would make it hard for non-stdlib additions to have the same usage, as they would need to have completely separate implementations for the ā€œdefaultā€ and custom RNG versions.

4) I would also still suggest that the simplest version of random (that you use to get a value from a range or an element from a collection) should be a function, not a property. Collection properties like first, last, and count all represent facts that already exist about a collection, and don’t change unless the collection itself changes. Choosing a random element, on the other hand, is clearly going to be freshly performed on each call. In addition, with the notable exception of count, we try to ensure O(1) performance for properties, while random will be O(n) except in random-access collections. Finally, if it is a method, we can unify the two versions by providing a single method with the shared RNG as the default parameter.

5) To match the sorted() method, shuffled() should be on Sequence instead of Collection. I don’t think either shuffled() or shuffle() needs to be a protocol requirement, since there isn’t really any kind of customization necessary for different kinds of collections. Like the sorting algorithms, both could be regular extension methods.

6) I don’t know whether or not a consensus has formed around the correct spelling of the APIs for generating random values. From the proposal it looks like the preferred ways of getting a random value in a range would be to use the random property (or method) on a range or closed range:

Ā Ā Ā Ā (0..<10).random // 7
Ā Ā Ā Ā (0.0 ... 5.0).random // 4.112312

If that’s the goal, and we don’t want those values to be optional, we’ll need an implementation of random for floating-point ranges and an overload for fixed-width integer ranges. That said, I don’t think that style is as discoverable as having static methods or initializers available on the different types:

Ā Ā Ā Ā Int.random(in: 0..<10)
Ā Ā Ā Ā Double.random(in: 0.0 ... 5.0)
Ā Ā Ā Ā // or maybe
Ā Ā Ā Ā Int(randomIn: 0..<10)
Ā Ā Ā Ā Double(randomIn: 0.0 ... 5.0)

(My only quibble with the initializer approach is that Bool would be awkward.)

In addition, this alternative approach could make creating random values more consistent with types that don’t work well in ranges:

Ā Ā Ā Ā Data.random(bytes: 128)
Ā Ā Ā Ā Color.random(r: 0...0, g: 0...1, b: 0...1, a: 1...1)

————

Thanks again!
Nate

On Nov 5, 2017, at 6:33 PM, Alejandro Alonso via swift-evolution <swift-evolution@swift.org<mailto:swift-evolution@swift.org>> wrote:

[Proposal] Random Unification by Azoy Ā· Pull Request #760 Ā· apple/swift-evolution Ā· GitHub is the current API and proposed solution.

- Alejandro

On Nov 5, 2017, 6:18 PM -0600, Xiaodi Wu <xiaodi.wu@gmail.com<mailto:xiaodi.wu@gmail.com>>, wrote:
My comments are directed to the "more up-to-date" document that you just linked to in your reply to Jon. Is that one outdated? If so, can you send a link to the updated proposal and implementation for which you're soliciting feedback?

On Sun, Nov 5, 2017 at 6:12 PM, Alejandro Alonso <aalonso128@outlook.com<mailto:aalonso128@outlook.com>> wrote:
The proposal and implementation have the current updated API. The link I sent Jon was the one I brought up a few weeks ago which is outdated now. The proposal answers all of your questions. As for `.random` being a function, some would argue that it behaves in the same way as `.first` and `.last` which are properties.

- Alejandro

On Nov 5, 2017, 6:07 PM -0600, Xiaodi Wu <xiaodi.wu@gmail.com<mailto:xiaodi.wu@gmail.com>>, wrote:
A few quick thoughts:

I know that there's been some discussion that `(1...10).random` is the best spelling, but I'd like to push back on that suggestion. When I want a random number, I tend to think of the type I want first ("I want a random integer") and then a range ("I want a random integer between a and b"), not the other way around. My intuition is that `Int.random(in:)` will be more discoverable, both on that basis and because it is more similar to other languages' syntax (`Math.random` in JavaScript and `randint` in NumPy,
for example). It also has the advantage that the type is explicit, which I think is particularly useful in this case because the value itself is, well, random.

I would also argue that, `random` is most appropriately a method and not a property; there's no hard and fast rule for this, but the fact that the result is stochastic suggests (to me) that it's not a "property" of the range (or, for that matter, of the type).

I would reiterate here my qualms about `Source` being the term used for a generator. These types are not a _source_ of entropy but rather a _consumer_ of entropy.

`UnsafeRandomSource` needs to be renamed; "unsafe" has a specific meaning in Swift--that is, memory safety, and this is not it. Moreover, it's questionable whether this protocol is useful in any sense. What useful generic algorithms can one write with such a protocol?

`XoroshiroRandom` cannot be seeded by any `Numeric` value; depending on the specific algorithm it needs a seed of a specific bit width. If you default the shared instance to being seeded with an `Int` then you will have to have distinct implementations for 32-bit and 64-bit platforms. This is unadvisable. On that note, your `UnsafeRandomSource` needs to have an associated type and not a generic `<T : Numeric>` for the seed.

The default random number generator should be cryptographically secure; however, it's not clear to me that it should be device random.

I agree with others that alternative random number generators other than the default RNG (and, if not default, possibly also the device RNG) should be accommodated by the protocol hierarchy but not necessarily supplied in the stdlib.

The term `Randomizable` means something specific which is not how it's used in your proposed protocol.

There's still the open question, not answered, about how requesting an instance of the hardware RNG behaves when there's insufficient or no entropy. Does it return nil, throw, trap, or wait? The proposed API does not clarify this point, although based on the method signature it cannot return nil or throw. Trapping might be acceptable but I'd be interested to hear your take as to why it is preferable.

On Sun, Nov 5, 2017 at 4:43 PM, Alejandro Alonso via swift-evolution <swift-evolution@swift.org<mailto:swift-evolution@swift.org>> wrote:
For the proof of concept, I had accidentally deleted that one. I have a more up to date one which was discussed a few weeks later. Swift Random Unification Design Ā· GitHub

- Alejandro

On Nov 5, 2017, 4:37 PM -0600, Jonathan Hull <jhull@gbis.com<mailto:jhull@gbis.com>>, wrote:
Is there a link to the writeup? The one in the quote 404s.

Thanks,
Jon

On Nov 5, 2017, at 2:10 PM, Alejandro Alonso via swift-evolution <swift-evolution@swift.org<mailto:swift-evolution@swift.org>> wrote:

Hello once again Swift evolution community. I have taken the time to write up the proposal for this thread, and have provided an implementation for it as well. I hope to once again get good feedback on the overall proposal.

- Alejandro

On Sep 8, 2017, 11:52 AM -0500, Alejandro Alonso via swift-evolution <swift-evolution@swift.org<mailto:swift-evolution@swift.org>>, wrote:
Hello swift evolution, I would like to propose a unified approach to `random()` in Swift. I have a simple implementation here https://gist.github.com/Azoy/5d294148c8b97d20b96ee64f434bb4f5\. This implementation is a simple wrapper over existing random functions so existing code bases will not be affected. Also, this approach introduces a new random feature for Linux users that give them access to upper bounds, as well as a lower bound for both Glibc and Darwin users. This change would be implemented within Foundation.

I believe this simple change could have a very positive impact on new developers learning Swift and experienced developers being able to write single random declarations.

I’d like to hear about your ideas on this proposal, or any implementation changes if need be.

- Alejando

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org<mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org<mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org<mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

I wrote up a quick and dirty example displaying Randomizable in action working alongside Strideable Randomizable Potential Ā· GitHub . This example uses the 32 bit Unix timestamp to determine a Date’s range (0 … UInt32.max). Color’s range would obviously be 0 … 0xFFFFFF. As you can see from this example, Randomizable has the potential to provide massive functionality to types that are able to conform to it. Rust also implements a trait called Rand that achieves the same here https://doc.rust-lang.org/rand/rand/trait.Rand.html .

- Alejandro

Ā·Ā·Ā·

On Nov 15, 2017, 11:43 AM -0600, Nate Cook <natecook@apple.com>, wrote:
On Nov 12, 2017, at 7:47 PM, Alejandro Alonso <aalonso128@outlook.com<mailto:aalonso128@outlook.com>> wrote:

Sorry I’ve been gone for a while, I had to do a lot of traveling.

1. Initially I made this thinking that developers had the power to determine their own lower bound. The current implementation uses the integer’s min value as a lower bound. If it makes sense to only allow unsigned integers from an RNG, then I’m perfectly fine with. I do disagree when you say that it should only generate UInt32s. The current approach allows, lets say mt19337 and mt19337-64, to be used within one generator. So if you wanted a UInt32, mt19337 would be used, and if you asked for a UInt64, mt19337-64 would be used.

2. The Randomizable protocol isn’t always used with integers. Think Date.random or Color.random. These types of values are difficult to express with ranges. Randomizable solves this issue.

I don’t think these examples explain how the Randomizable protocol is useful. As currently proposed, types that are Randomizable can provide a random value, but the details of that are left up to the type, and vary widely. When you use .random on an integer type, you get any value between .min and .max, but on a floating-point type, it’s a value between 0.0 and 1.0. What would the range be when you use Date.random? What about Color.random?

Here’s an implementation using the Randomizable protocol as a constraint:

Ā Ā Ā Ā extension Array where Element: Randomizable {
Ā Ā Ā Ā Ā Ā Ā Ā init(randomElements: Int) {
Ā Ā Ā Ā Ā Ā Ā Ā Ā Ā Ā Ā self = (0..<randomElements).map({ _ in Element.random })
Ā Ā Ā Ā Ā Ā Ā Ā }
Ā Ā Ā Ā }

Is this useful? Note that there’s no way to constrain these random values to be in a particular range or any other criteria—they’re only going to be random values in whatever the range is that a specific types supplies.

The point I’m trying to make is that although all these types can have random values generated, their similarity is mostly coincidental. As long as we provide good ways of generating random values at the appropriate level, we don’t need the Randomizable protocol. (To me, the appropriate extension points are FixedWidthInteger, BinaryFloatingPoint, and Bool.) Without Randomizable, we can still write things like this, which are the kinds of functionality we want to add anyway:

Ā Ā Ā Ā extension Array where Element: FixedWidthInteger {
Ā Ā Ā Ā Ā Ā Ā Ā init(randomElements: Int, in bounds: Range<Element>) {
Ā Ā Ā Ā Ā Ā Ā Ā Ā Ā Ā Ā self = (0..<randomElements).map({ _ in bounds.random })
Ā Ā Ā Ā Ā Ā Ā Ā }
Ā Ā Ā Ā }

-Nate

3. I’ve made the adjustment necessary for this.

4. So while I can see your point for this, it would break the consistency with Randomizable’s random property. You could argue that we could make this property a function itself, but I think most will agree that Int.random is a cleaner api than Int.random().

5. I’ve made the adjustment necessary for this.

6. I actually forgot to implement the random api for the ranges where Bound: BinaryFloatingPoint. While implementing this, I realized these would never fail and would always return a non-optional. So, I decided making the other Countable ranges non-optional. (0 ..< 10).random would return a non-optional, (0.0 ..< 10.0).random would return a non-optional, and Array(0 ..< 10).random would return an optional. I can agree that something like (0 ..< 10).random is hard to discover, so I added Int.random(in: 0 ..< 10) (along with BinaryFloatingPoint). However, these are not requirements of Randomizable. I think these methods would benefit more if they were extension methods:

extension Randomizable where Self: FixedWidthInteger, Self.Stride: SignedInteger {
public static func random(
in range: Countable{Closed}Range,
using generator: RandomNumberGenerator
) -> Self {
return range.random(using: generator)
}
}

extension Randomizable where Self: BinaryFloatingPointer {
public static func random(
in range: {Closed}Range,
using generator: RandomNumberGenerator
) -> Self {
return range.random
}
}

I think external types that wish to do something similar, like Data.random(bytes: 128), could extend Randomizable with their own custom needs. The stdlib would at this point provide all the features needed to make this happen very simply for something like Data.random(bytes: 128).

- Alejandro

On Nov 5, 2017, 10:44 PM -0600, Nate Cook <natecook@apple.com<mailto:natecook@apple.com>>, wrote:
Thanks for continuing to push this forward, Alejandro! I’m excited about the potential of having access to these APIs as part of the standard library. Here are a few comments on some different parts of the proposal:

1) For your RandomGenerator protocol, I’m not totally clear on the semantics of the next(_:) and next(_:upperBound:) methods. Do they both have zero as their lower bound, for example? I’m not sure it makes sense to have signed integers generated directly by an RNG—perhaps T: FixedWidthInteger & UnsignedInteger would be a more useful constraint. (Does it even need to be generic? What if RNGs just generate UInt32s?)

2) Can you say more about the purpose of the Randomizable protocol? How would we use that protocol in useful ways that we wouldn’t get from being able to select random values from ranges (half-open and closed) of FixedWidthInteger / BinaryFloatingPoint? My experience has been that a full-width random value is rarely what a user needs.

3) I agree with Xiaodi that Random should probably be a struct with a single shared instance, but I don’t think it should be internal. Hiding that shared RNG would make it hard for non-stdlib additions to have the same usage, as they would need to have completely separate implementations for the ā€œdefaultā€ and custom RNG versions.

4) I would also still suggest that the simplest version of random (that you use to get a value from a range or an element from a collection) should be a function, not a property. Collection properties like first, last, and count all represent facts that already exist about a collection, and don’t change unless the collection itself changes. Choosing a random element, on the other hand, is clearly going to be freshly performed on each call. In addition, with the notable exception of count, we try to ensure O(1) performance for properties, while random will be O(n) except in random-access collections. Finally, if it is a method, we can unify the two versions by providing a single method with the shared RNG as the default parameter.

5) To match the sorted() method, shuffled() should be on Sequence instead of Collection. I don’t think either shuffled() or shuffle() needs to be a protocol requirement, since there isn’t really any kind of customization necessary for different kinds of collections. Like the sorting algorithms, both could be regular extension methods.

6) I don’t know whether or not a consensus has formed around the correct spelling of the APIs for generating random values. From the proposal it looks like the preferred ways of getting a random value in a range would be to use the random property (or method) on a range or closed range:

Ā Ā Ā Ā (0..<10).random // 7
Ā Ā Ā Ā (0.0 ... 5.0).random // 4.112312

If that’s the goal, and we don’t want those values to be optional, we’ll need an implementation of random for floating-point ranges and an overload for fixed-width integer ranges. That said, I don’t think that style is as discoverable as having static methods or initializers available on the different types:

Ā Ā Ā Ā Int.random(in: 0..<10)
Ā Ā Ā Ā Double.random(in: 0.0 ... 5.0)
Ā Ā Ā Ā // or maybe
Ā Ā Ā Ā Int(randomIn: 0..<10)
Ā Ā Ā Ā Double(randomIn: 0.0 ... 5.0)

(My only quibble with the initializer approach is that Bool would be awkward.)

In addition, this alternative approach could make creating random values more consistent with types that don’t work well in ranges:

Ā Ā Ā Ā Data.random(bytes: 128)
Ā Ā Ā Ā Color.random(r: 0...0, g: 0...1, b: 0...1, a: 1...1)

————

Thanks again!
Nate

On Nov 5, 2017, at 6:33 PM, Alejandro Alonso via swift-evolution <swift-evolution@swift.org<mailto:swift-evolution@swift.org>> wrote:

[Proposal] Random Unification by Azoy Ā· Pull Request #760 Ā· apple/swift-evolution Ā· GitHub is the current API and proposed solution.

- Alejandro

On Nov 5, 2017, 6:18 PM -0600, Xiaodi Wu <xiaodi.wu@gmail.com<mailto:xiaodi.wu@gmail.com>>, wrote:
My comments are directed to the "more up-to-date" document that you just linked to in your reply to Jon. Is that one outdated? If so, can you send a link to the updated proposal and implementation for which you're soliciting feedback?

On Sun, Nov 5, 2017 at 6:12 PM, Alejandro Alonso <aalonso128@outlook.com<mailto:aalonso128@outlook.com>> wrote:
The proposal and implementation have the current updated API. The link I sent Jon was the one I brought up a few weeks ago which is outdated now. The proposal answers all of your questions. As for `.random` being a function, some would argue that it behaves in the same way as `.first` and `.last` which are properties.

- Alejandro

On Nov 5, 2017, 6:07 PM -0600, Xiaodi Wu <xiaodi.wu@gmail.com<mailto:xiaodi.wu@gmail.com>>, wrote:
A few quick thoughts:

I know that there's been some discussion that `(1...10).random` is the best spelling, but I'd like to push back on that suggestion. When I want a random number, I tend to think of the type I want first ("I want a random integer") and then a range ("I want a random integer between a and b"), not the other way around. My intuition is that `Int.random(in:)` will be more discoverable, both on that basis and because it is more similar to other languages' syntax (`Math.random` in JavaScript and `randint` in NumPy,
for example). It also has the advantage that the type is explicit, which I think is particularly useful in this case because the value itself is, well, random.

I would also argue that, `random` is most appropriately a method and not a property; there's no hard and fast rule for this, but the fact that the result is stochastic suggests (to me) that it's not a "property" of the range (or, for that matter, of the type).

I would reiterate here my qualms about `Source` being the term used for a generator. These types are not a _source_ of entropy but rather a _consumer_ of entropy.

`UnsafeRandomSource` needs to be renamed; "unsafe" has a specific meaning in Swift--that is, memory safety, and this is not it. Moreover, it's questionable whether this protocol is useful in any sense. What useful generic algorithms can one write with such a protocol?

`XoroshiroRandom` cannot be seeded by any `Numeric` value; depending on the specific algorithm it needs a seed of a specific bit width. If you default the shared instance to being seeded with an `Int` then you will have to have distinct implementations for 32-bit and 64-bit platforms. This is unadvisable. On that note, your `UnsafeRandomSource` needs to have an associated type and not a generic `<T : Numeric>` for the seed.

The default random number generator should be cryptographically secure; however, it's not clear to me that it should be device random.

I agree with others that alternative random number generators other than the default RNG (and, if not default, possibly also the device RNG) should be accommodated by the protocol hierarchy but not necessarily supplied in the stdlib.

The term `Randomizable` means something specific which is not how it's used in your proposed protocol.

There's still the open question, not answered, about how requesting an instance of the hardware RNG behaves when there's insufficient or no entropy. Does it return nil, throw, trap, or wait? The proposed API does not clarify this point, although based on the method signature it cannot return nil or throw. Trapping might be acceptable but I'd be interested to hear your take as to why it is preferable.

On Sun, Nov 5, 2017 at 4:43 PM, Alejandro Alonso via swift-evolution <swift-evolution@swift.org<mailto:swift-evolution@swift.org>> wrote:
For the proof of concept, I had accidentally deleted that one. I have a more up to date one which was discussed a few weeks later. Swift Random Unification Design Ā· GitHub

- Alejandro

On Nov 5, 2017, 4:37 PM -0600, Jonathan Hull <jhull@gbis.com<mailto:jhull@gbis.com>>, wrote:
Is there a link to the writeup? The one in the quote 404s.

Thanks,
Jon

On Nov 5, 2017, at 2:10 PM, Alejandro Alonso via swift-evolution <swift-evolution@swift.org<mailto:swift-evolution@swift.org>> wrote:

Hello once again Swift evolution community. I have taken the time to write up the proposal for this thread, and have provided an implementation for it as well. I hope to once again get good feedback on the overall proposal.

- Alejandro

On Sep 8, 2017, 11:52 AM -0500, Alejandro Alonso via swift-evolution <swift-evolution@swift.org<mailto:swift-evolution@swift.org>>, wrote:
Hello swift evolution, I would like to propose a unified approach to `random()` in Swift. I have a simple implementation here https://gist.github.com/Azoy/5d294148c8b97d20b96ee64f434bb4f5\. This implementation is a simple wrapper over existing random functions so existing code bases will not be affected. Also, this approach introduces a new random feature for Linux users that give them access to upper bounds, as well as a lower bound for both Glibc and Darwin users. This change would be implemented within Foundation.

I believe this simple change could have a very positive impact on new developers learning Swift and experienced developers being able to write single random declarations.

I’d like to hear about your ideas on this proposal, or any implementation changes if need be.

- Alejando

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org<mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org<mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org<mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

What I was trying to respond to, by contrast, is the design of a hierarchy
of protocols CSPRNG : PRNG (or, in Alejandro's proposal, UnsafeRandomSource
: RandomSource) and the appropriate APIs to expose on each. This is
entirely inapplicable to your examples. It stands to reason that a
non-instantiable source of random numbers does not require a protocol of
its own (a hypothetical RNG : CSPRNG), since there is no reason to
implement (if done correctly) more than a single publicly non-instantiable
singleton type that could conform to it. For that matter, the concrete type
itself probably doesn't need *any* public API at all. Instead, extensions
to standard library types such as Int that implement conformance to the
protocol that Alejandro names "Randomizable" could call internal APIs to
provide all the necessary functionality, and third-party types that need to
conform to "Randomizable" could then in turn use `Int.random()` or
`Double.random()` to implement their own conformance. In fact, the concrete
random number generator type doesn't need to be public at all. All public
interaction could be through APIs such as `Int.random()`.

If there is globally-available CSPRNG that people are encouraged to use,
what is the benefit of a CSPRNG : PRNG hierarchy?

There are plenty of use cases that do not require cryptographically secure
pseudorandom sequences but can benefit from the speed of an algorithm like
xoroshiro128+. For instance, suppose I want to simulate Brownian motion in
an animation; I do not care about the cryptographic properties of my random
number source. However, the underlying function to generate a normally
distributed random number can rely on either a source of cyptographically
secure or cryptographically insecure uniformly distributed random numbers.
Put another way, a protocol hierarchy is justified because useful functions
that produce random values in various desired distributions can be written
that work with any PRNG, while there are (obviously) uses that are suitable
for CSPRNGs only.

What is the benefit of clearly identifying an algorithm as crypto-secure
if the recommendation is that you use *the* crypto-secure object/functions?

The default source of random numbers is unseedable, but the user may
instead require repeatable generation of a sequence of random numbers. The
user may have a use case that requires the use of a particular specified
CSPRNG.

Again, I'm not only talking about urandom. As far as I'm aware, every API
to retrieve cryptographically secure sequences of random bits on every
platform for which Swift is distributed can potentially return an error
instead of random bits. The question is, what design for our API is the
most sensible way to deal with this contingency? On rethinking, I do
believe that consistently returning an Optional is the best way to go about
it, allowing the user to either (a) supply a deterministic fallback; (b)
raise an error of their own choosing; or (c) trap--all with a minimum of
fuss. This seems very Swifty to me.

With Linux's getrandom, if you read from urandom (the default) and ask for
as much or less than 256 bytes, the only possible error is that urandom
hasn't been seeded <http://man7.org/linux/man-pages/man2/getrandom.2.html&gt;\.
(With more than 256 bytes, it also becomes possible for the system call to
be interrupted by a signal.) OpenBSD's getentropy is literally just
arc4random running in the kernel
<https://github.com/openbsd/src/blob/master/sys/dev/rnd.c#L889&gt;, and will
only fail if you ask for more than 256 bytes because it is a hard-coded
limit.

Yes, but again, what do you think of the possible Swift API design choices
to accommodate these errors? We have to pick one, but none of them are very
appealing.

Meanwhile, getentropy() has the problem that, if there is insufficient
randomness to initialize the entropy pool, it will block; on systems where
there is never going to be sufficient randomness (i.e. a VM), it will block
forever. By contrast, getrandom() permits a flag that will make this
scenario non-blocking.

Ironically, I don't know how you get entropy on Darwin. If it's more

Ā·Ā·Ā·

On Thu, Sep 28, 2017 at 12:16 PM, FƩlix Cloutier <felixcloutier@icloud.com> wrote:

Le 27 sept. 2017 Ơ 17:29, Xiaodi Wu <xiaodi.wu@gmail.com> a Ʃcrit :
failable that this, I'd argue that it's time to improve a bit...

FƩlix

I like the idea of splitting it into 2 separate ā€œRandomā€ proposals.

The first would have Xiaodi’s built-in CSPRNG which only has the interface:

On FixedWidthInteger:
  static func random()throws -> Self
  static func random(in range: ClosedRange<Self>)throws -> Self

On Double:
  static func random()throws -> Double
  static func random(in range: ClosedRange<Double>)throws -> Double

(Everything else we want, like shuffled(), could be built in later proposals by calling those functions)

The other option would be to remove the ā€˜throws’ from the above functions (perhaps fatalError-ing), and provide an additional function which can be used to check that there is enough entropy (so as to avoid the crash or fall back to a worse source when the CSPRNG is unavailable).

Then a second proposal would bring in the concept of RandomSources (whatever we call them), which can return however many random bytes you ask for… and a protocol for types which know how to initialize themselves from those bytes. That might be spelled like 'static func random(using: RandomSource)->Self'. As a convenience, the source would also be able to create FixedWidthIntegers and Doubles (both with and without a range), and would also have the coinFlip() and oneIn(UInt)->Bool functions. Most types should be able to build themselves off of that. There would be a default source which is built from the first protocol.

I also really think we should have a concept of Repeatably-Random as a subprotocol for the second proposal. I see far too many shipping apps which have bugs due to using arc4Random when they really needed a repeatable source (e.g. patterns and lines jump around when you resize things). If it was an easy option, people would use it when appropriate. This would just mean a sub-protocol which has an initializer which takes a seed, and the ability to save/restore state (similar to CGContexts).

The second proposal would also include things like shuffled() and shuffled(using:).

Thanks,
Jon

Ā·Ā·Ā·

On Oct 3, 2017, at 9:31 PM, Alejandro Alonso <aalonso128@outlook.com> wrote:

I really like the schedule here. After reading for a while, I do agree with Brent that stdlib should very primitive in functionality that it provides. I also agree that the most important part right now is designing the internal crypto on which the numeric types use to return their respected random number. On the discussion of how we should handle not enough entropy with the device random, from a users perspective it makes sense that calling .random should just give me a random number, but from a developers perspective I see Optional being the best choice here. While I think blocking could, in most cases, provide the user an easier API, we have to do this right and be safe here by providing a value that indicates that there is room for error here. As for the generator abstraction, I believe there should be a bare basic protocol that sets a layout for new generators and should be focusing on its requirements.

Whether or not RandomAccessCollection and MutableCollection should get .random and .shuffle/.shuffled in this first proposal is completely up in the air for me. It makes sense, to me, to include the .random in this proposal and open another one .shuffle/.shuffled, but I can see arguments that should say we create something separate for these two, or include all of it in this proposal.

- Alejandro

On Sep 27, 2017, 7:29 PM -0500, Xiaodi Wu <xiaodi.wu@gmail.com>, wrote:

On Wed, Sep 27, 2017 at 00:18 FƩlix Cloutier <felixcloutier@icloud.com <mailto:felixcloutier@icloud.com>> wrote:

Le 26 sept. 2017 Ơ 16:14, Xiaodi Wu <xiaodi.wu@gmail.com <mailto:xiaodi.wu@gmail.com>> a Ʃcrit :

On Tue, Sep 26, 2017 at 11:26 AM, FƩlix Cloutier <felixcloutier@icloud.com <mailto:felixcloutier@icloud.com>> wrote:

It's possible to use a CSPRNG-grade algorithm and seed it once to get a reproducible sequence, but when you use it as a CSPRNG, you typically feed entropy back into it at nondeterministic points to ensure that even if you started with a bad seed, you'll eventually get to an alright state. Unless you keep track of when entropy was mixed in and what the values were, you'll never get a reproducible CSPRNG.

We would give developers a false sense of security if we provided them with CSPRNG-grade algorithms that we called CSPRNGs and that they could seed themselves. Just because it says "crypto-secure" in the name doesn't mean that it'll be crypto-secure if it's seeded with time(). Therefore, "reproducible" vs "non-reproducible" looks like a good distinction to me.

I disagree here, in two respects:

First, whether or not a particular PRNG is cryptographically secure is an intrinsic property of the algorithm; whether it's "reproducible" or not is determined by the published API. In other words, the distinction between CSPRNG vs. non-CSPRNG is important to document because it's semantics that cannot be deduced by the user otherwise, and it is an important one for writing secure code because it tells you whether an attacker can predict future outputs based only on observing past outputs. "Reproducible" in the sense of seedable or not is trivially noted by inspection of the published API, and it is rather immaterial to writing secure code.

Cryptographically secure is not a property that I'm comfortable applying to an algorithm. You cannot say that you've made a cryptographically secure thing just because you've used all the right algorithms: you also have to use them right, and one of the most critical components of a cryptographically secure PRNG is its seed.

A cryptographically secure algorithm isn’t sufficient, but it is necessary. That’s why it’s important to mark them as such. If I'm a careful developer, then it is absolutely important to me to know that I’m using a PRNG with a cryptographically secure algorithm, and that the particular implementation of that algorithm is correct and secure.

It is a *feature* of a lot of modern CSPRNGs that you can't seed them:

You cannot seed or add entropy to std::random_device

Although std::random_device may in practice be backed by a software CSPRNG, IIUC, the intention is that it can provide access to a hardware non-deterministic source when available.

You cannot seed or add entropy to CryptGenRandom
You can only add entropy to /dev/(u)random
You can only add entropy to BSD's arc4random

Ah, I see. I think we mean different things when we say PRNG. A PRNG is an entirely deterministic algorithm; the output is non-random and the algorithm itself requires no entropy. If a PRNG is seeded with a random sequence of bits, its output can "appear" to be random. A CSPRNG is a PRNG that fulfills certain criteria such that its output can be appropriate for use in cryptographic applications in place of a truly random sequence *if* the input to the CSPRNG is itself random.

The examples you give above *incorporate* a CSPRNG, environment entropy, and a set of rules about when to mix in additional entropy in order to produce output indistinguishable from a random sequence, but they are *not* themselves really *pseudorandom* generators because they are not deterministic. Not only do such sources of random numbers not require an interface to allow seeding, they do not even have to be publicly instantiable: Swift need only expose a single thread-safe instance (or an instance per thread) of a single type that provides access to CryptGenRandom/urandom/arc4random, since after all the output of multiple instances of that type should be statistically indistinguishable from the output of only one.

What I was trying to respond to, by contrast, is the design of a hierarchy of protocols CSPRNG : PRNG (or, in Alejandro's proposal, UnsafeRandomSource : RandomSource) and the appropriate APIs to expose on each. This is entirely inapplicable to your examples. It stands to reason that a non-instantiable source of random numbers does not require a protocol of its own (a hypothetical RNG : CSPRNG), since there is no reason to implement (if done correctly) more than a single publicly non-instantiable singleton type that could conform to it. For that matter, the concrete type itself probably doesn't need *any* public API at all. Instead, extensions to standard library types such as Int that implement conformance to the protocol that Alejandro names "Randomizable" could call internal APIs to provide all the necessary functionality, and third-party types that need to conform to "Randomizable" could then in turn use `Int.random()` or `Double.random()` to implement their own conformance. In fact, the concrete random number generator type doesn't need to be public at all. All public interaction could be through APIs such as `Int.random()`.

Just because we can expose a seed interface doesn't mean we should, and in this case I believe that it would go against the prime objective of providing secure random numbers.

If we're talking about a Swift interface to a non-deterministic source of random numbers like urandom or arc4random, then, as I write above, not only do I agree that it doesn't need to be seedable, it also does not need to be instantiable at all, does not need to conform to a protocol that specifically requires the semantics of a non-deterministic source, does not need to expose any public interface whatsoever, and doesn't itself even need to be public. (Does it even need to be a type, as opposed to simply a free function?)

In fact, having reasoned through all of this, we can split the design task into two. The most essential part, which definitely should be part of the stdlib, would be an internal interface to a cryptographically secure platform-specific entropy source, a public protocol named something like Randomizable (to be bikeshedded), and the appropriate implementations on Boolean, binary integer, and floating point types to conform them to Randomizable so that users can write `Bool.random()` or `Int.random()`. The second part, which can be a separate proposal or even a standalone core library or third-party library, would be the protocols and concrete types that implement pseudorandom number generators, allowing for reproducible pseudorandom sequences. In other words, instead of PRNGs and CSPRNGs being the primitives on which `Int.random()` is implemented; `Int.random()` should be the standard library primitive which allows PRNGs and CSPRNGs to be seeded.

If your attacker can observe your seeding once, chances are that they can observe your reseeding too; then, they can use their own implementation of the PRNG (whether CSPRNG or non-CSPRNG) and reproduce your pseudorandom sequence whether or not Swift exposes any particular API.

On Linux, the random devices are initially seeded with machine-specific but rather invariant data that makes /dev/urandom spit out predictable numbers. It is considered "seeded" after a root process writes POOL_SIZE bytes to it. On most implementations, this initial seed is stored on disk: when the computer shuts down, it reads POOL_SIZE bytes from /dev/urandom and saves it in a file, and the contents of that file is loaded back into /dev/urandom when the computer starts. A scenario where someone can read that file is certainly not less likely than a scenario where /dev/urandom was deleted. That doesn't mean that they have kernel code execution or that they can pry into your process, but they have a good shot at guessing your seed and subsequent RNG results if no stirring happens.

Sorry, I don't understand what you're getting at here. Again, I'm talking about deterministic algorithms, not non-deterministic sources of random numbers.

Secondly, I see no reason to justify the notion that, simply because a PRNG is cryptographically secure, we ought to hide the seeding initializer (because one has to exist internally anyway) from the public. Obviously, one use case for a deterministic PRNG is to get reproducible sequences of random-appearing values; this can be useful whether the underlying algorithm is cryptographically secure or not. There are innumerably many ways to use data generated from a CSPRNG in non-cryptographically secure ways and omitting or including a public seeding initializer does not change that; in other words, using a deterministic seed for a CSPRNG would be a bad idea in certain applications, but it's a deliberate act, and someone who would mistakenly do that is clearly incapable of *using* the output from the PRNG in a secure way either; put a third way, you would be hard pressed to find a situation where it's true that "if only Swift had not made the seeding initializer public, this author would have written secure code, but instead the only security hole that existed in the code was caused by the availability of a public seeding initializer mistakenly used." The point of having both explicitly instantiable PRNGs and a layer of simpler APIs like "Int.random()" is so that the less experienced user can get the "right thing" by default, and the experienced user can customize the behavior; any user that instantiates his or her own ChaCha20Random instance is already calling for the power user interface; it is reasonable to expose the underlying primitive operations (such as seeding) so long as there are legitimate uses for it.

Nothing prevents us from using the same algorithm for a CSPRNG that is safely pre-seeded and a PRNG that people seed themselves, mind you. However, especially when it comes to security, there is a strong responsibility to drive developers into a pit of success: the most obvious thing to do has to be the right one, and suggesting to cryptographically-unaware developers that they have everything they need to manage their own seed is not a step in that direction.

I'm not opposed to a ChaCha20Random type; I'm opposed to explicitly calling it cryptographically-secure, because it is not unless you know what to do with it. It is emphatically not far-fetched to imagine a developer who thinks that they can outdo the standard library by using their own ChaCha20Random instance after it's been seeded with time() if we let them know that it's "cryptographically secure". If you're a power user and you don't like the default, known-good CSPRNG, then you're hopefully good enough to know that ChaCha20 is considered a cryptographically-secure algorithm without help labels from the language, and you know how to operate it.

I'm fully aware of the myths surrounding /dev/urandom and /dev/random. /dev/urandom might never run out, but it is also possible for it not to be initialized at all, as in the case of some VM setups. In some older versions of iOS, /dev/[u]random is reportedly sandboxed out. On systems where it is available, it can also be deleted, since it is a file. The point is, all of these scenarios cause an error during seeding of a CSPRNG. The question is, how to proceed in the face of inability to access entropy. We must do something, because we cannot therefore return a cryptographically secure answer. Rare trapping on invocation of Int.random() or permanently waiting for a never-to-be-initialized /dev/urandom would be terrible to debug, but returning an optional or throwing all the time would be verbose. How to design this API?

If the only concern is that the system might not be initialized enough, I'd say that whatever returns an instance of a global, framework-seeded CSPRNG should return an Optional, and the random methods that use the global CSPRNG can trap and scream that the system is not initialized enough. If this is a likely error for you, you can check if the CSPRNG exists or not before jumping.

Also note that there is only one system for which Swift is officially distributed (Ubuntu 14.04) on which the only way to get entropy from the OS is to open a random device and read from it.

Again, I'm not only talking about urandom. As far as I'm aware, every API to retrieve cryptographically secure sequences of random bits on every platform for which Swift is distributed can potentially return an error instead of random bits. The question is, what design for our API is the most sensible way to deal with this contingency? On rethinking, I do believe that consistently returning an Optional is the best way to go about it, allowing the user to either (a) supply a deterministic fallback; (b) raise an error of their own choosing; or (c) trap--all with a minimum of fuss. This seems very Swifty to me.

* What should the default CSPRNG be? There are good arguments for using a cryptographically secure device random. (In my proposed implementation, for device random, I use Security.framework on Apple platforms (because /dev/urandom is not guaranteed to be available due to the sandbox, IIUC). On Linux platforms, I would prefer to use getrandom() and avoid using file system APIs, but getrandom() is new and unsupported on some versions of Ubuntu that Swift supports. This is an issue in and of itself.) Now, a number of these facilities strictly limit or do not guarantee availability of more than a small number of random bytes at a time; they are recommended for seeding other PRNGs but *not* as a routine source of random numbers. Therefore, although device random should be available to users, it probably shouldn’t be the default for the Swift standard library as it could have negative consequences for the system as a whole. There follows the significant task of implementing a CSPRNG correctly and securely for the default PRNG.

Theo give a talk a few years ago <https://www.youtube.com/watch?v=aWmLWx8ut20&gt; on randomness and how these problems are approached in LibreSSL.

Certainly, we can learn a lot from those like Theo who've dealt with the issue. I'm not in a position to watch the talk at the moment; can you summarize what the tl;dr version of it is?

I saw it three years ago, so I don't remember all the details. The gist is that:

OpenBSD's random is available from extremely early in the boot process with reasonable entropy
LibreSSL includes OpenBSD's arc4random, and it's a "good" PRNG (which doesn't actually use ARC4)
That implementation of arc4random is good because it is fool-proof and it has basically no failure mode
Stirring is good, having multiple components take random numbers from the same source probably makes results harder to guess too
Getrandom/getentropy is in all ways better than reading from random devices

Vigorously agree on all points. Thanks for the summary.

I'm really not enthusiastic about `random() -> Self?` or `random() throws -> Self` when the only possible error is that some global object hasn't been initialized.

The idea of having `random` straight on integers and floats and collections was to provide a simple interface, but using a global CSPRNG for those operations comes at a significant usability cost. I think that something has to go:

Drop the random methods on FixedWidthInteger, FloatingPoint
...or drop the CSPRNG as a default
Drop the optional/throws, and trap on error

I know I wouldn't use the `Int.random()` method if I had to unwrap every single result, when getting one non-nil result guarantees that the program won't see any other nil result again until it restarts.

FƩlix

Ā·Ā·Ā·

Le 3 oct. 2017 Ơ 23:44, Jonathan Hull <jhull@gbis.com> a Ʃcrit :

I like the idea of splitting it into 2 separate ā€œRandomā€ proposals.

The first would have Xiaodi’s built-in CSPRNG which only has the interface:

On FixedWidthInteger:
  static func random()throws -> Self
  static func random(in range: ClosedRange<Self>)throws -> Self

On Double:
  static func random()throws -> Double
  static func random(in range: ClosedRange<Double>)throws -> Double

(Everything else we want, like shuffled(), could be built in later proposals by calling those functions)

The other option would be to remove the ā€˜throws’ from the above functions (perhaps fatalError-ing), and provide an additional function which can be used to check that there is enough entropy (so as to avoid the crash or fall back to a worse source when the CSPRNG is unavailable).

Then a second proposal would bring in the concept of RandomSources (whatever we call them), which can return however many random bytes you ask for… and a protocol for types which know how to initialize themselves from those bytes. That might be spelled like 'static func random(using: RandomSource)->Self'. As a convenience, the source would also be able to create FixedWidthIntegers and Doubles (both with and without a range), and would also have the coinFlip() and oneIn(UInt)->Bool functions. Most types should be able to build themselves off of that. There would be a default source which is built from the first protocol.

I also really think we should have a concept of Repeatably-Random as a subprotocol for the second proposal. I see far too many shipping apps which have bugs due to using arc4Random when they really needed a repeatable source (e.g. patterns and lines jump around when you resize things). If it was an easy option, people would use it when appropriate. This would just mean a sub-protocol which has an initializer which takes a seed, and the ability to save/restore state (similar to CGContexts).

The second proposal would also include things like shuffled() and shuffled(using:).

Thanks,
Jon

On Oct 3, 2017, at 9:31 PM, Alejandro Alonso <aalonso128@outlook.com <mailto:aalonso128@outlook.com>> wrote:

I really like the schedule here. After reading for a while, I do agree with Brent that stdlib should very primitive in functionality that it provides. I also agree that the most important part right now is designing the internal crypto on which the numeric types use to return their respected random number. On the discussion of how we should handle not enough entropy with the device random, from a users perspective it makes sense that calling .random should just give me a random number, but from a developers perspective I see Optional being the best choice here. While I think blocking could, in most cases, provide the user an easier API, we have to do this right and be safe here by providing a value that indicates that there is room for error here. As for the generator abstraction, I believe there should be a bare basic protocol that sets a layout for new generators and should be focusing on its requirements.

Whether or not RandomAccessCollection and MutableCollection should get .random and .shuffle/.shuffled in this first proposal is completely up in the air for me. It makes sense, to me, to include the .random in this proposal and open another one .shuffle/.shuffled, but I can see arguments that should say we create something separate for these two, or include all of it in this proposal.

- Alejandro

On Sep 27, 2017, 7:29 PM -0500, Xiaodi Wu <xiaodi.wu@gmail.com <mailto:xiaodi.wu@gmail.com>>, wrote:

On Wed, Sep 27, 2017 at 00:18 FƩlix Cloutier <felixcloutier@icloud.com <mailto:felixcloutier@icloud.com>> wrote:

Le 26 sept. 2017 Ơ 16:14, Xiaodi Wu <xiaodi.wu@gmail.com <mailto:xiaodi.wu@gmail.com>> a Ʃcrit :

On Tue, Sep 26, 2017 at 11:26 AM, FƩlix Cloutier <felixcloutier@icloud.com <mailto:felixcloutier@icloud.com>> wrote:

It's possible to use a CSPRNG-grade algorithm and seed it once to get a reproducible sequence, but when you use it as a CSPRNG, you typically feed entropy back into it at nondeterministic points to ensure that even if you started with a bad seed, you'll eventually get to an alright state. Unless you keep track of when entropy was mixed in and what the values were, you'll never get a reproducible CSPRNG.

We would give developers a false sense of security if we provided them with CSPRNG-grade algorithms that we called CSPRNGs and that they could seed themselves. Just because it says "crypto-secure" in the name doesn't mean that it'll be crypto-secure if it's seeded with time(). Therefore, "reproducible" vs "non-reproducible" looks like a good distinction to me.

I disagree here, in two respects:

First, whether or not a particular PRNG is cryptographically secure is an intrinsic property of the algorithm; whether it's "reproducible" or not is determined by the published API. In other words, the distinction between CSPRNG vs. non-CSPRNG is important to document because it's semantics that cannot be deduced by the user otherwise, and it is an important one for writing secure code because it tells you whether an attacker can predict future outputs based only on observing past outputs. "Reproducible" in the sense of seedable or not is trivially noted by inspection of the published API, and it is rather immaterial to writing secure code.

Cryptographically secure is not a property that I'm comfortable applying to an algorithm. You cannot say that you've made a cryptographically secure thing just because you've used all the right algorithms: you also have to use them right, and one of the most critical components of a cryptographically secure PRNG is its seed.

A cryptographically secure algorithm isn’t sufficient, but it is necessary. That’s why it’s important to mark them as such. If I'm a careful developer, then it is absolutely important to me to know that I’m using a PRNG with a cryptographically secure algorithm, and that the particular implementation of that algorithm is correct and secure.

It is a *feature* of a lot of modern CSPRNGs that you can't seed them:

You cannot seed or add entropy to std::random_device

Although std::random_device may in practice be backed by a software CSPRNG, IIUC, the intention is that it can provide access to a hardware non-deterministic source when available.

You cannot seed or add entropy to CryptGenRandom
You can only add entropy to /dev/(u)random
You can only add entropy to BSD's arc4random

Ah, I see. I think we mean different things when we say PRNG. A PRNG is an entirely deterministic algorithm; the output is non-random and the algorithm itself requires no entropy. If a PRNG is seeded with a random sequence of bits, its output can "appear" to be random. A CSPRNG is a PRNG that fulfills certain criteria such that its output can be appropriate for use in cryptographic applications in place of a truly random sequence *if* the input to the CSPRNG is itself random.

The examples you give above *incorporate* a CSPRNG, environment entropy, and a set of rules about when to mix in additional entropy in order to produce output indistinguishable from a random sequence, but they are *not* themselves really *pseudorandom* generators because they are not deterministic. Not only do such sources of random numbers not require an interface to allow seeding, they do not even have to be publicly instantiable: Swift need only expose a single thread-safe instance (or an instance per thread) of a single type that provides access to CryptGenRandom/urandom/arc4random, since after all the output of multiple instances of that type should be statistically indistinguishable from the output of only one.

What I was trying to respond to, by contrast, is the design of a hierarchy of protocols CSPRNG : PRNG (or, in Alejandro's proposal, UnsafeRandomSource : RandomSource) and the appropriate APIs to expose on each. This is entirely inapplicable to your examples. It stands to reason that a non-instantiable source of random numbers does not require a protocol of its own (a hypothetical RNG : CSPRNG), since there is no reason to implement (if done correctly) more than a single publicly non-instantiable singleton type that could conform to it. For that matter, the concrete type itself probably doesn't need *any* public API at all. Instead, extensions to standard library types such as Int that implement conformance to the protocol that Alejandro names "Randomizable" could call internal APIs to provide all the necessary functionality, and third-party types that need to conform to "Randomizable" could then in turn use `Int.random()` or `Double.random()` to implement their own conformance. In fact, the concrete random number generator type doesn't need to be public at all. All public interaction could be through APIs such as `Int.random()`.

Just because we can expose a seed interface doesn't mean we should, and in this case I believe that it would go against the prime objective of providing secure random numbers.

If we're talking about a Swift interface to a non-deterministic source of random numbers like urandom or arc4random, then, as I write above, not only do I agree that it doesn't need to be seedable, it also does not need to be instantiable at all, does not need to conform to a protocol that specifically requires the semantics of a non-deterministic source, does not need to expose any public interface whatsoever, and doesn't itself even need to be public. (Does it even need to be a type, as opposed to simply a free function?)

In fact, having reasoned through all of this, we can split the design task into two. The most essential part, which definitely should be part of the stdlib, would be an internal interface to a cryptographically secure platform-specific entropy source, a public protocol named something like Randomizable (to be bikeshedded), and the appropriate implementations on Boolean, binary integer, and floating point types to conform them to Randomizable so that users can write `Bool.random()` or `Int.random()`. The second part, which can be a separate proposal or even a standalone core library or third-party library, would be the protocols and concrete types that implement pseudorandom number generators, allowing for reproducible pseudorandom sequences. In other words, instead of PRNGs and CSPRNGs being the primitives on which `Int.random()` is implemented; `Int.random()` should be the standard library primitive which allows PRNGs and CSPRNGs to be seeded.

If your attacker can observe your seeding once, chances are that they can observe your reseeding too; then, they can use their own implementation of the PRNG (whether CSPRNG or non-CSPRNG) and reproduce your pseudorandom sequence whether or not Swift exposes any particular API.

On Linux, the random devices are initially seeded with machine-specific but rather invariant data that makes /dev/urandom spit out predictable numbers. It is considered "seeded" after a root process writes POOL_SIZE bytes to it. On most implementations, this initial seed is stored on disk: when the computer shuts down, it reads POOL_SIZE bytes from /dev/urandom and saves it in a file, and the contents of that file is loaded back into /dev/urandom when the computer starts. A scenario where someone can read that file is certainly not less likely than a scenario where /dev/urandom was deleted. That doesn't mean that they have kernel code execution or that they can pry into your process, but they have a good shot at guessing your seed and subsequent RNG results if no stirring happens.

Sorry, I don't understand what you're getting at here. Again, I'm talking about deterministic algorithms, not non-deterministic sources of random numbers.

Secondly, I see no reason to justify the notion that, simply because a PRNG is cryptographically secure, we ought to hide the seeding initializer (because one has to exist internally anyway) from the public. Obviously, one use case for a deterministic PRNG is to get reproducible sequences of random-appearing values; this can be useful whether the underlying algorithm is cryptographically secure or not. There are innumerably many ways to use data generated from a CSPRNG in non-cryptographically secure ways and omitting or including a public seeding initializer does not change that; in other words, using a deterministic seed for a CSPRNG would be a bad idea in certain applications, but it's a deliberate act, and someone who would mistakenly do that is clearly incapable of *using* the output from the PRNG in a secure way either; put a third way, you would be hard pressed to find a situation where it's true that "if only Swift had not made the seeding initializer public, this author would have written secure code, but instead the only security hole that existed in the code was caused by the availability of a public seeding initializer mistakenly used." The point of having both explicitly instantiable PRNGs and a layer of simpler APIs like "Int.random()" is so that the less experienced user can get the "right thing" by default, and the experienced user can customize the behavior; any user that instantiates his or her own ChaCha20Random instance is already calling for the power user interface; it is reasonable to expose the underlying primitive operations (such as seeding) so long as there are legitimate uses for it.

Nothing prevents us from using the same algorithm for a CSPRNG that is safely pre-seeded and a PRNG that people seed themselves, mind you. However, especially when it comes to security, there is a strong responsibility to drive developers into a pit of success: the most obvious thing to do has to be the right one, and suggesting to cryptographically-unaware developers that they have everything they need to manage their own seed is not a step in that direction.

I'm not opposed to a ChaCha20Random type; I'm opposed to explicitly calling it cryptographically-secure, because it is not unless you know what to do with it. It is emphatically not far-fetched to imagine a developer who thinks that they can outdo the standard library by using their own ChaCha20Random instance after it's been seeded with time() if we let them know that it's "cryptographically secure". If you're a power user and you don't like the default, known-good CSPRNG, then you're hopefully good enough to know that ChaCha20 is considered a cryptographically-secure algorithm without help labels from the language, and you know how to operate it.

I'm fully aware of the myths surrounding /dev/urandom and /dev/random. /dev/urandom might never run out, but it is also possible for it not to be initialized at all, as in the case of some VM setups. In some older versions of iOS, /dev/[u]random is reportedly sandboxed out. On systems where it is available, it can also be deleted, since it is a file. The point is, all of these scenarios cause an error during seeding of a CSPRNG. The question is, how to proceed in the face of inability to access entropy. We must do something, because we cannot therefore return a cryptographically secure answer. Rare trapping on invocation of Int.random() or permanently waiting for a never-to-be-initialized /dev/urandom would be terrible to debug, but returning an optional or throwing all the time would be verbose. How to design this API?

If the only concern is that the system might not be initialized enough, I'd say that whatever returns an instance of a global, framework-seeded CSPRNG should return an Optional, and the random methods that use the global CSPRNG can trap and scream that the system is not initialized enough. If this is a likely error for you, you can check if the CSPRNG exists or not before jumping.

Also note that there is only one system for which Swift is officially distributed (Ubuntu 14.04) on which the only way to get entropy from the OS is to open a random device and read from it.

Again, I'm not only talking about urandom. As far as I'm aware, every API to retrieve cryptographically secure sequences of random bits on every platform for which Swift is distributed can potentially return an error instead of random bits. The question is, what design for our API is the most sensible way to deal with this contingency? On rethinking, I do believe that consistently returning an Optional is the best way to go about it, allowing the user to either (a) supply a deterministic fallback; (b) raise an error of their own choosing; or (c) trap--all with a minimum of fuss. This seems very Swifty to me.

* What should the default CSPRNG be? There are good arguments for using a cryptographically secure device random. (In my proposed implementation, for device random, I use Security.framework on Apple platforms (because /dev/urandom is not guaranteed to be available due to the sandbox, IIUC). On Linux platforms, I would prefer to use getrandom() and avoid using file system APIs, but getrandom() is new and unsupported on some versions of Ubuntu that Swift supports. This is an issue in and of itself.) Now, a number of these facilities strictly limit or do not guarantee availability of more than a small number of random bytes at a time; they are recommended for seeding other PRNGs but *not* as a routine source of random numbers. Therefore, although device random should be available to users, it probably shouldn’t be the default for the Swift standard library as it could have negative consequences for the system as a whole. There follows the significant task of implementing a CSPRNG correctly and securely for the default PRNG.

Theo give a talk a few years ago <https://www.youtube.com/watch?v=aWmLWx8ut20&gt; on randomness and how these problems are approached in LibreSSL.

Certainly, we can learn a lot from those like Theo who've dealt with the issue. I'm not in a position to watch the talk at the moment; can you summarize what the tl;dr version of it is?

I saw it three years ago, so I don't remember all the details. The gist is that:

OpenBSD's random is available from extremely early in the boot process with reasonable entropy
LibreSSL includes OpenBSD's arc4random, and it's a "good" PRNG (which doesn't actually use ARC4)
That implementation of arc4random is good because it is fool-proof and it has basically no failure mode
Stirring is good, having multiple components take random numbers from the same source probably makes results harder to guess too
Getrandom/getentropy is in all ways better than reading from random devices

Vigorously agree on all points. Thanks for the summary.

I'm really not enthusiastic about `random() -> Self?` or `random() throws
-> Self` when the only possible error is that some global object hasn't
been initialized.

The idea of having `random` straight on integers and floats and
collections was to provide a simple interface, but using a global CSPRNG
for those operations comes at a significant usability cost. I think that
something has to go:

   1. Drop the random methods on FixedWidthInteger, FloatingPoint
      - ...or drop the CSPRNG as a default
   2. Drop the optional/throws, and trap on error

I know I wouldn't use the `Int.random()` method if I had to unwrap every
single result, when getting one non-nil result guarantees that the program
won't see any other nil result again until it restarts.

From the perspective of an app that can be suspended and resumed at any

time, ā€œuntil it restartsā€ could be as soon as the next invocation of
`Int.random()`, could it not?

Ā·Ā·Ā·

On Wed, Oct 4, 2017 at 02:39 FƩlix Cloutier <felixcloutier@icloud.com> wrote:

FƩlix

Le 3 oct. 2017 Ơ 23:44, Jonathan Hull <jhull@gbis.com> a Ʃcrit :

I like the idea of splitting it into 2 separate ā€œRandomā€ proposals.

The first would have Xiaodi’s built-in CSPRNG which only has the interface:

On FixedWidthInteger:
static func random()throws -> Self
static func random(in range: ClosedRange<Self>)throws -> Self

On Double:
static func random()throws -> Double
static func random(in range: ClosedRange<Double>)throws -> Double

(Everything else we want, like shuffled(), could be built in later
proposals by calling those functions)

The other option would be to remove the ā€˜throws’ from the above functions
(perhaps fatalError-ing), and provide an additional function which can be
used to check that there is enough entropy (so as to avoid the crash or
fall back to a worse source when the CSPRNG is unavailable).

Then a second proposal would bring in the concept of RandomSources
(whatever we call them), which can return however many random bytes you ask
for… and a protocol for types which know how to initialize themselves from
those bytes. That might be spelled like 'static func random(using:
RandomSource)->Self'. As a convenience, the source would also be able to
create FixedWidthIntegers and Doubles (both with and without a range), and
would also have the coinFlip() and oneIn(UInt)->Bool functions. Most types
should be able to build themselves off of that. There would be a default
source which is built from the first protocol.

I also really think we should have a concept of Repeatably-Random as a
subprotocol for the second proposal. I see far too many shipping apps
which have bugs due to using arc4Random when they really needed a
repeatable source (e.g. patterns and lines jump around when you resize
things). If it was an easy option, people would use it when appropriate.
This would just mean a sub-protocol which has an initializer which takes a
seed, and the ability to save/restore state (similar to CGContexts).

The second proposal would also include things like shuffled() and
shuffled(using:).

Thanks,
Jon

On Oct 3, 2017, at 9:31 PM, Alejandro Alonso <aalonso128@outlook.com> > wrote:

I really like the schedule here. After reading for a while, I do agree
with Brent that stdlib should very primitive in functionality that it
provides. I also agree that the most important part right now is designing
the internal crypto on which the numeric types use to return their
respected random number. On the discussion of how we should handle not
enough entropy with the device random, from a users perspective it makes
sense that calling .random should just give me a random number, but from a
developers perspective I see Optional being the best choice here. While I
think blocking could, in most cases, provide the user an easier API, we
have to do this right and be safe here by providing a value that indicates
that there is room for error here. As for the generator abstraction, I
believe there should be a bare basic protocol that sets a layout for new
generators and should be focusing on its requirements.

Whether or not RandomAccessCollection and MutableCollection should get
.random and .shuffle/.shuffled in this first proposal is completely up in
the air for me. It makes sense, to me, to include the .random in this
proposal and open another one .shuffle/.shuffled, but I can see arguments
that should say we create something separate for these two, or include all
of it in this proposal.

- Alejandro

On Sep 27, 2017, 7:29 PM -0500, Xiaodi Wu <xiaodi.wu@gmail.com>, wrote:

On Wed, Sep 27, 2017 at 00:18 FƩlix Cloutier <felixcloutier@icloud.com> > wrote:

Le 26 sept. 2017 Ơ 16:14, Xiaodi Wu <xiaodi.wu@gmail.com> a Ʃcrit :

On Tue, Sep 26, 2017 at 11:26 AM, FƩlix Cloutier < >> felixcloutier@icloud.com> wrote:

It's possible to use a CSPRNG-grade algorithm and seed it once to get a
reproducible sequence, but when you use it as a CSPRNG, you typically feed
entropy back into it at nondeterministic points to ensure that even if you
started with a bad seed, you'll eventually get to an alright state. Unless
you keep track of when entropy was mixed in and what the values were,
you'll never get a reproducible CSPRNG.

We would give developers a false sense of security if we provided them
with CSPRNG-grade algorithms that we called CSPRNGs and that they could
seed themselves. Just because it says "crypto-secure" in the name doesn't
mean that it'll be crypto-secure if it's seeded with time(). Therefore,
"reproducible" vs "non-reproducible" looks like a good distinction to me.

I disagree here, in two respects:

First, whether or not a particular PRNG is cryptographically secure is an
intrinsic property of the algorithm; whether it's "reproducible" or not is
determined by the published API. In other words, the distinction between
CSPRNG vs. non-CSPRNG is important to document because it's semantics that
cannot be deduced by the user otherwise, and it is an important one for
writing secure code because it tells you whether an attacker can predict
future outputs based only on observing past outputs. "Reproducible" in the
sense of seedable or not is trivially noted by inspection of the published
API, and it is rather immaterial to writing secure code.

Cryptographically secure is not a property that I'm comfortable applying
to an algorithm. You cannot say that you've made a cryptographically secure
thing just because you've used all the right algorithms: you also have to
use them right, and one of the most critical components of a
cryptographically secure PRNG is its seed.

A cryptographically secure algorithm isn’t sufficient, but it is
necessary. That’s why it’s important to mark them as such. If I'm a careful
developer, then it is absolutely important to me to know that I’m using a
PRNG with a cryptographically secure algorithm, and that the particular
implementation of that algorithm is correct and secure.

It is a *feature* of a lot of modern CSPRNGs that you can't seed them:

   - You cannot seed or add entropy to std::random_device

Although std::random_device may in practice be backed by a software
CSPRNG, IIUC, the intention is that it can provide access to a hardware
non-deterministic source when available.

   - You cannot seed or add entropy to CryptGenRandom
   - You can only add entropy to /dev/(u)random
   - You can only add entropy to BSD's arc4random

Ah, I see. I think we mean different things when we say PRNG. A PRNG is an
entirely deterministic algorithm; the output is non-random and the
algorithm itself requires no entropy. If a PRNG is seeded with a random
sequence of bits, its output can "appear" to be random. A CSPRNG is a PRNG
that fulfills certain criteria such that its output can be appropriate for
use in cryptographic applications in place of a truly random sequence *if*
the input to the CSPRNG is itself random.

The examples you give above *incorporate* a CSPRNG, environment entropy,
and a set of rules about when to mix in additional entropy in order to
produce output indistinguishable from a random sequence, but they are *not*
themselves really *pseudorandom* generators because they are not
deterministic. Not only do such sources of random numbers not require an
interface to allow seeding, they do not even have to be publicly
instantiable: Swift need only expose a single thread-safe instance (or an
instance per thread) of a single type that provides access to
CryptGenRandom/urandom/arc4random, since after all the output of multiple
instances of that type should be statistically indistinguishable from the
output of only one.

What I was trying to respond to, by contrast, is the design of a hierarchy
of protocols CSPRNG : PRNG (or, in Alejandro's proposal, UnsafeRandomSource
: RandomSource) and the appropriate APIs to expose on each. This is
entirely inapplicable to your examples. It stands to reason that a
non-instantiable source of random numbers does not require a protocol of
its own (a hypothetical RNG : CSPRNG), since there is no reason to
implement (if done correctly) more than a single publicly non-instantiable
singleton type that could conform to it. For that matter, the concrete type
itself probably doesn't need *any* public API at all. Instead, extensions
to standard library types such as Int that implement conformance to the
protocol that Alejandro names "Randomizable" could call internal APIs to
provide all the necessary functionality, and third-party types that need to
conform to "Randomizable" could then in turn use `Int.random()` or
`Double.random()` to implement their own conformance. In fact, the concrete
random number generator type doesn't need to be public at all. All public
interaction could be through APIs such as `Int.random()`.

Just because we can expose a seed interface doesn't mean we should, and
in this case I believe that it would go against the prime objective of
providing secure random numbers.

If we're talking about a Swift interface to a non-deterministic source of
random numbers like urandom or arc4random, then, as I write above, not only
do I agree that it doesn't need to be seedable, it also does not need to be
instantiable at all, does not need to conform to a protocol that
specifically requires the semantics of a non-deterministic source, does not
need to expose any public interface whatsoever, and doesn't itself even
need to be public. (Does it even need to be a type, as opposed to simply a
free function?)

In fact, having reasoned through all of this, we can split the design task
into two. The most essential part, which definitely should be part of the
stdlib, would be an internal interface to a cryptographically secure
platform-specific entropy source, a public protocol named something like
Randomizable (to be bikeshedded), and the appropriate implementations on
Boolean, binary integer, and floating point types to conform them to
Randomizable so that users can write `Bool.random()` or `Int.random()`. The
second part, which can be a separate proposal or even a standalone core
library or third-party library, would be the protocols and concrete types
that implement pseudorandom number generators, allowing for reproducible
pseudorandom sequences. In other words, instead of PRNGs and CSPRNGs being
the primitives on which `Int.random()` is implemented; `Int.random()`
should be the standard library primitive which allows PRNGs and CSPRNGs to
be seeded.

If your attacker can observe your seeding once, chances are that they can
observe your reseeding too; then, they can use their own implementation of
the PRNG (whether CSPRNG or non-CSPRNG) and reproduce your pseudorandom
sequence whether or not Swift exposes any particular API.

On Linux, the random devices are initially seeded with machine-specific
but rather invariant data that makes /dev/urandom spit out predictable
numbers. It is considered "seeded" after a root process writes POOL_SIZE
bytes to it. On most implementations, this initial seed is stored on disk:
when the computer shuts down, it reads POOL_SIZE bytes from /dev/urandom
and saves it in a file, and the contents of that file is loaded back into
/dev/urandom when the computer starts. A scenario where someone can read
that file is certainly not less likely than a scenario where /dev/urandom
was deleted. That doesn't mean that they have kernel code execution or that
they can pry into your process, but they have a good shot at guessing your
seed and subsequent RNG results if no stirring happens.

Sorry, I don't understand what you're getting at here. Again, I'm talking
about deterministic algorithms, not non-deterministic sources of random
numbers.

Secondly, I see no reason to justify the notion that, simply because a

PRNG is cryptographically secure, we ought to hide the seeding initializer
(because one has to exist internally anyway) from the public. Obviously,
one use case for a deterministic PRNG is to get reproducible sequences of
random-appearing values; this can be useful whether the underlying
algorithm is cryptographically secure or not. There are innumerably many
ways to use data generated from a CSPRNG in non-cryptographically secure
ways and omitting or including a public seeding initializer does not change
that; in other words, using a deterministic seed for a CSPRNG would be a
bad idea in certain applications, but it's a deliberate act, and someone
who would mistakenly do that is clearly incapable of *using* the output
from the PRNG in a secure way either; put a third way, you would be hard
pressed to find a situation where it's true that "if only Swift had not
made the seeding initializer public, this author would have written secure
code, but instead the only security hole that existed in the code was
caused by the availability of a public seeding initializer mistakenly
used." The point of having both explicitly instantiable PRNGs and a layer
of simpler APIs like "Int.random()" is so that the less experienced user
can get the "right thing" by default, and the experienced user can
customize the behavior; any user that instantiates his or her own
ChaCha20Random instance is already calling for the power user interface; it
is reasonable to expose the underlying primitive operations (such as
seeding) so long as there are legitimate uses for it.

Nothing prevents us from using the same algorithm for a CSPRNG that is
safely pre-seeded and a PRNG that people seed themselves, mind you.
However, especially when it comes to security, there is a strong
responsibility to drive developers into a pit of success: the most obvious
thing to do has to be the right one, and suggesting to
cryptographically-unaware developers that they have everything they need to
manage their own seed is not a step in that direction.

I'm not opposed to a ChaCha20Random type; I'm opposed to explicitly
calling it cryptographically-secure, because it is not unless you know what
to do with it. It is emphatically not far-fetched to imagine a developer
who thinks that they can outdo the standard library by using their own
ChaCha20Random instance after it's been seeded with time() if we let them
know that it's "cryptographically secure". If you're a power user and you
don't like the default, known-good CSPRNG, then you're hopefully good
enough to know that ChaCha20 is considered a cryptographically-secure
algorithm without help labels from the language, and you know how to
operate it.

I'm fully aware of the myths surrounding /dev/urandom and /dev/random.
/dev/urandom might never run out, but it is also possible for it not to be
initialized at all, as in the case of some VM setups. In some older
versions of iOS, /dev/[u]random is reportedly sandboxed out. On systems
where it is available, it can also be deleted, since it is a file. The
point is, all of these scenarios cause an error during seeding of a CSPRNG.
The question is, how to proceed in the face of inability to access entropy.
We must do something, because we cannot therefore return a
cryptographically secure answer. Rare trapping on invocation of
Int.random() or permanently waiting for a never-to-be-initialized
/dev/urandom would be terrible to debug, but returning an optional or
throwing all the time would be verbose. How to design this API?

If the only concern is that the system might not be initialized enough,
I'd say that whatever returns an instance of a global, framework-seeded
CSPRNG should return an Optional, and the random methods that use the
global CSPRNG can trap and scream that the system is not initialized
enough. If this is a likely error for you, you can check if the CSPRNG
exists or not before jumping.

Also note that there is only one system for which Swift is officially
distributed (Ubuntu 14.04) on which the only way to get entropy from the OS
is to open a random device and read from it.

Again, I'm not only talking about urandom. As far as I'm aware, every API
to retrieve cryptographically secure sequences of random bits on every
platform for which Swift is distributed can potentially return an error
instead of random bits. The question is, what design for our API is the
most sensible way to deal with this contingency? On rethinking, I do
believe that consistently returning an Optional is the best way to go about
it, allowing the user to either (a) supply a deterministic fallback; (b)
raise an error of their own choosing; or (c) trap--all with a minimum of
fuss. This seems very Swifty to me.

* What should the default CSPRNG be? There are good arguments for using a

cryptographically secure device random. (In my proposed implementation, for
device random, I use Security.framework on Apple platforms (because
/dev/urandom is not guaranteed to be available due to the sandbox, IIUC).
On Linux platforms, I would prefer to use getrandom() and avoid using file
system APIs, but getrandom() is new and unsupported on some versions of
Ubuntu that Swift supports. This is an issue in and of itself.) Now, a
number of these facilities strictly limit or do not guarantee availability
of more than a small number of random bytes at a time; they are recommended
for seeding other PRNGs but *not* as a routine source of random numbers.
Therefore, although device random should be available to users, it probably
shouldn’t be the default for the Swift standard library as it could have
negative consequences for the system as a whole. There follows the
significant task of implementing a CSPRNG correctly and securely for the
default PRNG.

Theo give a talk a few years ago
<https://www.youtube.com/watch?v=aWmLWx8ut20&gt; on randomness and how
these problems are approached in LibreSSL.

Certainly, we can learn a lot from those like Theo who've dealt with the
issue. I'm not in a position to watch the talk at the moment; can you
summarize what the tl;dr version of it is?

I saw it three years ago, so I don't remember all the details. The gist
is that:

   - OpenBSD's random is available from extremely early in the boot
   process with reasonable entropy

   - LibreSSL includes OpenBSD's arc4random, and it's a "good" PRNG
   (which doesn't actually use ARC4)
   - That implementation of arc4random is good because it is fool-proof
   and it has basically no failure mode
   - Stirring is good, having multiple components take random numbers
   from the same source probably makes results harder to guess too
   - Getrandom/getentropy is in all ways better than reading from random
   devices

Vigorously agree on all points. Thanks for the summary.

Riffing off this a bit…

I’d like to see minimal Random support in the stdlib, and then all this specialization stuff in a ā€œnon-standardā€ library. Ie, a library that ships with Swift, but is not imported by default.

As I’m developing apps, I don’t need the massive autocompletion overload and cognitive overhead that comes from trying to understand all these proposed protocols and use-cases unless I am actually going to be needing randomization. If I need randomization, I should be explicitly opting-in to it by doing ā€œimport Randomā€.

Dave

Ā·Ā·Ā·

On Oct 3, 2017, at 10:31 PM, Alejandro Alonso via swift-evolution <swift-evolution@swift.org> wrote:

I really like the schedule here. After reading for a while, I do agree with Brent that stdlib should very primitive in functionality that it provides. I also agree that the most important part right now is designing the internal crypto on which the numeric types use to return their respected random number. On the discussion of how we should handle not enough entropy with the device random, from a users perspective it makes sense that calling .random should just give me a random number, but from a developers perspective I see Optional being the best choice here. While I think blocking could, in most cases, provide the user an easier API, we have to do this right and be safe here by providing a value that indicates that there is room for error here. As for the generator abstraction, I believe there should be a bare basic protocol that sets a layout for new generators and should be focusing on its requirements.

Whether or not RandomAccessCollection and MutableCollection should get .random and .shuffle/.shuffled in this first proposal is completely up in the air for me. It makes sense, to me, to include the .random in this proposal and open another one .shuffle/.shuffled, but I can see arguments that should say we create something separate for these two, or include all of it in this proposal.

- Alejandro

On Sep 27, 2017, 7:29 PM -0500, Xiaodi Wu <xiaodi.wu@gmail.com>, wrote:

On Wed, Sep 27, 2017 at 00:18 FƩlix Cloutier <felixcloutier@icloud.com <mailto:felixcloutier@icloud.com>> wrote:

Le 26 sept. 2017 Ơ 16:14, Xiaodi Wu <xiaodi.wu@gmail.com <mailto:xiaodi.wu@gmail.com>> a Ʃcrit :

On Tue, Sep 26, 2017 at 11:26 AM, FƩlix Cloutier <felixcloutier@icloud.com <mailto:felixcloutier@icloud.com>> wrote:

It's possible to use a CSPRNG-grade algorithm and seed it once to get a reproducible sequence, but when you use it as a CSPRNG, you typically feed entropy back into it at nondeterministic points to ensure that even if you started with a bad seed, you'll eventually get to an alright state. Unless you keep track of when entropy was mixed in and what the values were, you'll never get a reproducible CSPRNG.

We would give developers a false sense of security if we provided them with CSPRNG-grade algorithms that we called CSPRNGs and that they could seed themselves. Just because it says "crypto-secure" in the name doesn't mean that it'll be crypto-secure if it's seeded with time(). Therefore, "reproducible" vs "non-reproducible" looks like a good distinction to me.

I disagree here, in two respects:

First, whether or not a particular PRNG is cryptographically secure is an intrinsic property of the algorithm; whether it's "reproducible" or not is determined by the published API. In other words, the distinction between CSPRNG vs. non-CSPRNG is important to document because it's semantics that cannot be deduced by the user otherwise, and it is an important one for writing secure code because it tells you whether an attacker can predict future outputs based only on observing past outputs. "Reproducible" in the sense of seedable or not is trivially noted by inspection of the published API, and it is rather immaterial to writing secure code.

Cryptographically secure is not a property that I'm comfortable applying to an algorithm. You cannot say that you've made a cryptographically secure thing just because you've used all the right algorithms: you also have to use them right, and one of the most critical components of a cryptographically secure PRNG is its seed.

A cryptographically secure algorithm isn’t sufficient, but it is necessary. That’s why it’s important to mark them as such. If I'm a careful developer, then it is absolutely important to me to know that I’m using a PRNG with a cryptographically secure algorithm, and that the particular implementation of that algorithm is correct and secure.

It is a *feature* of a lot of modern CSPRNGs that you can't seed them:

You cannot seed or add entropy to std::random_device

Although std::random_device may in practice be backed by a software CSPRNG, IIUC, the intention is that it can provide access to a hardware non-deterministic source when available.

You cannot seed or add entropy to CryptGenRandom
You can only add entropy to /dev/(u)random
You can only add entropy to BSD's arc4random

Ah, I see. I think we mean different things when we say PRNG. A PRNG is an entirely deterministic algorithm; the output is non-random and the algorithm itself requires no entropy. If a PRNG is seeded with a random sequence of bits, its output can "appear" to be random. A CSPRNG is a PRNG that fulfills certain criteria such that its output can be appropriate for use in cryptographic applications in place of a truly random sequence *if* the input to the CSPRNG is itself random.

The examples you give above *incorporate* a CSPRNG, environment entropy, and a set of rules about when to mix in additional entropy in order to produce output indistinguishable from a random sequence, but they are *not* themselves really *pseudorandom* generators because they are not deterministic. Not only do such sources of random numbers not require an interface to allow seeding, they do not even have to be publicly instantiable: Swift need only expose a single thread-safe instance (or an instance per thread) of a single type that provides access to CryptGenRandom/urandom/arc4random, since after all the output of multiple instances of that type should be statistically indistinguishable from the output of only one.

What I was trying to respond to, by contrast, is the design of a hierarchy of protocols CSPRNG : PRNG (or, in Alejandro's proposal, UnsafeRandomSource : RandomSource) and the appropriate APIs to expose on each. This is entirely inapplicable to your examples. It stands to reason that a non-instantiable source of random numbers does not require a protocol of its own (a hypothetical RNG : CSPRNG), since there is no reason to implement (if done correctly) more than a single publicly non-instantiable singleton type that could conform to it. For that matter, the concrete type itself probably doesn't need *any* public API at all. Instead, extensions to standard library types such as Int that implement conformance to the protocol that Alejandro names "Randomizable" could call internal APIs to provide all the necessary functionality, and third-party types that need to conform to "Randomizable" could then in turn use `Int.random()` or `Double.random()` to implement their own conformance. In fact, the concrete random number generator type doesn't need to be public at all. All public interaction could be through APIs such as `Int.random()`.

Just because we can expose a seed interface doesn't mean we should, and in this case I believe that it would go against the prime objective of providing secure random numbers.

If we're talking about a Swift interface to a non-deterministic source of random numbers like urandom or arc4random, then, as I write above, not only do I agree that it doesn't need to be seedable, it also does not need to be instantiable at all, does not need to conform to a protocol that specifically requires the semantics of a non-deterministic source, does not need to expose any public interface whatsoever, and doesn't itself even need to be public. (Does it even need to be a type, as opposed to simply a free function?)

In fact, having reasoned through all of this, we can split the design task into two. The most essential part, which definitely should be part of the stdlib, would be an internal interface to a cryptographically secure platform-specific entropy source, a public protocol named something like Randomizable (to be bikeshedded), and the appropriate implementations on Boolean, binary integer, and floating point types to conform them to Randomizable so that users can write `Bool.random()` or `Int.random()`. The second part, which can be a separate proposal or even a standalone core library or third-party library, would be the protocols and concrete types that implement pseudorandom number generators, allowing for reproducible pseudorandom sequences. In other words, instead of PRNGs and CSPRNGs being the primitives on which `Int.random()` is implemented; `Int.random()` should be the standard library primitive which allows PRNGs and CSPRNGs to be seeded.

If your attacker can observe your seeding once, chances are that they can observe your reseeding too; then, they can use their own implementation of the PRNG (whether CSPRNG or non-CSPRNG) and reproduce your pseudorandom sequence whether or not Swift exposes any particular API.

On Linux, the random devices are initially seeded with machine-specific but rather invariant data that makes /dev/urandom spit out predictable numbers. It is considered "seeded" after a root process writes POOL_SIZE bytes to it. On most implementations, this initial seed is stored on disk: when the computer shuts down, it reads POOL_SIZE bytes from /dev/urandom and saves it in a file, and the contents of that file is loaded back into /dev/urandom when the computer starts. A scenario where someone can read that file is certainly not less likely than a scenario where /dev/urandom was deleted. That doesn't mean that they have kernel code execution or that they can pry into your process, but they have a good shot at guessing your seed and subsequent RNG results if no stirring happens.

Sorry, I don't understand what you're getting at here. Again, I'm talking about deterministic algorithms, not non-deterministic sources of random numbers.

Secondly, I see no reason to justify the notion that, simply because a PRNG is cryptographically secure, we ought to hide the seeding initializer (because one has to exist internally anyway) from the public. Obviously, one use case for a deterministic PRNG is to get reproducible sequences of random-appearing values; this can be useful whether the underlying algorithm is cryptographically secure or not. There are innumerably many ways to use data generated from a CSPRNG in non-cryptographically secure ways and omitting or including a public seeding initializer does not change that; in other words, using a deterministic seed for a CSPRNG would be a bad idea in certain applications, but it's a deliberate act, and someone who would mistakenly do that is clearly incapable of *using* the output from the PRNG in a secure way either; put a third way, you would be hard pressed to find a situation where it's true that "if only Swift had not made the seeding initializer public, this author would have written secure code, but instead the only security hole that existed in the code was caused by the availability of a public seeding initializer mistakenly used." The point of having both explicitly instantiable PRNGs and a layer of simpler APIs like "Int.random()" is so that the less experienced user can get the "right thing" by default, and the experienced user can customize the behavior; any user that instantiates his or her own ChaCha20Random instance is already calling for the power user interface; it is reasonable to expose the underlying primitive operations (such as seeding) so long as there are legitimate uses for it.

Nothing prevents us from using the same algorithm for a CSPRNG that is safely pre-seeded and a PRNG that people seed themselves, mind you. However, especially when it comes to security, there is a strong responsibility to drive developers into a pit of success: the most obvious thing to do has to be the right one, and suggesting to cryptographically-unaware developers that they have everything they need to manage their own seed is not a step in that direction.

I'm not opposed to a ChaCha20Random type; I'm opposed to explicitly calling it cryptographically-secure, because it is not unless you know what to do with it. It is emphatically not far-fetched to imagine a developer who thinks that they can outdo the standard library by using their own ChaCha20Random instance after it's been seeded with time() if we let them know that it's "cryptographically secure". If you're a power user and you don't like the default, known-good CSPRNG, then you're hopefully good enough to know that ChaCha20 is considered a cryptographically-secure algorithm without help labels from the language, and you know how to operate it.

I'm fully aware of the myths surrounding /dev/urandom and /dev/random. /dev/urandom might never run out, but it is also possible for it not to be initialized at all, as in the case of some VM setups. In some older versions of iOS, /dev/[u]random is reportedly sandboxed out. On systems where it is available, it can also be deleted, since it is a file. The point is, all of these scenarios cause an error during seeding of a CSPRNG. The question is, how to proceed in the face of inability to access entropy. We must do something, because we cannot therefore return a cryptographically secure answer. Rare trapping on invocation of Int.random() or permanently waiting for a never-to-be-initialized /dev/urandom would be terrible to debug, but returning an optional or throwing all the time would be verbose. How to design this API?

If the only concern is that the system might not be initialized enough, I'd say that whatever returns an instance of a global, framework-seeded CSPRNG should return an Optional, and the random methods that use the global CSPRNG can trap and scream that the system is not initialized enough. If this is a likely error for you, you can check if the CSPRNG exists or not before jumping.

Also note that there is only one system for which Swift is officially distributed (Ubuntu 14.04) on which the only way to get entropy from the OS is to open a random device and read from it.

Again, I'm not only talking about urandom. As far as I'm aware, every API to retrieve cryptographically secure sequences of random bits on every platform for which Swift is distributed can potentially return an error instead of random bits. The question is, what design for our API is the most sensible way to deal with this contingency? On rethinking, I do believe that consistently returning an Optional is the best way to go about it, allowing the user to either (a) supply a deterministic fallback; (b) raise an error of their own choosing; or (c) trap--all with a minimum of fuss. This seems very Swifty to me.

* What should the default CSPRNG be? There are good arguments for using a cryptographically secure device random. (In my proposed implementation, for device random, I use Security.framework on Apple platforms (because /dev/urandom is not guaranteed to be available due to the sandbox, IIUC). On Linux platforms, I would prefer to use getrandom() and avoid using file system APIs, but getrandom() is new and unsupported on some versions of Ubuntu that Swift supports. This is an issue in and of itself.) Now, a number of these facilities strictly limit or do not guarantee availability of more than a small number of random bytes at a time; they are recommended for seeding other PRNGs but *not* as a routine source of random numbers. Therefore, although device random should be available to users, it probably shouldn’t be the default for the Swift standard library as it could have negative consequences for the system as a whole. There follows the significant task of implementing a CSPRNG correctly and securely for the default PRNG.

Theo give a talk a few years ago <https://www.youtube.com/watch?v=aWmLWx8ut20&gt; on randomness and how these problems are approached in LibreSSL.

Certainly, we can learn a lot from those like Theo who've dealt with the issue. I'm not in a position to watch the talk at the moment; can you summarize what the tl;dr version of it is?

I saw it three years ago, so I don't remember all the details. The gist is that:

OpenBSD's random is available from extremely early in the boot process with reasonable entropy
LibreSSL includes OpenBSD's arc4random, and it's a "good" PRNG (which doesn't actually use ARC4)
That implementation of arc4random is good because it is fool-proof and it has basically no failure mode
Stirring is good, having multiple components take random numbers from the same source probably makes results harder to guess too
Getrandom/getentropy is in all ways better than reading from random devices

Vigorously agree on all points. Thanks for the summary.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

I like this kind of layering of functionality and proposals. I would additionally separate the fundamental CSPRNG interface from the fool-proof easy functions that naive users will find on Stack Overflow.

The "easy" functions should:

* Trap on any error without throwing. Sophisticated users may be able to do something about entropy failure, so the fundamental CSPRNG interface needs to provide errors, but the easy function should either degrade somewhat (if entropy is present but insufficient) or just die (if entropy is wholly absent or nearly so).

* Remove the range-less function. Naive users often write things like `Int.random() % 100`, which unbeknownst to them is biased. Providing only the ranged interface nudges naive users toward correct usage.

* Provide an "easy" way to get some random bytes instead of a random number. Perhaps a Data initializer that returns random-filled bytes of the requested length. This helps make up for the lack of a range-less function on FixedWidthInteger.

The "easy" functions should get the best names: Int.random(in:), Data.random(length:), etc. The fundamental CSPRNG interface should have an interface that is less friendly and less discoverable.

Ā·Ā·Ā·

On Oct 3, 2017, at 11:44 PM, Jonathan Hull via swift-evolution <swift-evolution@swift.org> wrote:

I like the idea of splitting it into 2 separate ā€œRandomā€ proposals.

The first would have Xiaodi’s built-in CSPRNG which only has the interface:

On FixedWidthInteger:
  static func random()throws -> Self
  static func random(in range: ClosedRange<Self>)throws -> Self

On Double:
  static func random()throws -> Double
  static func random(in range: ClosedRange<Double>)throws -> Double

(Everything else we want, like shuffled(), could be built in later proposals by calling those functions)

The other option would be to remove the ā€˜throws’ from the above functions (perhaps fatalError-ing), and provide an additional function which can be used to check that there is enough entropy (so as to avoid the crash or fall back to a worse source when the CSPRNG is unavailable).

Then a second proposal would bring in the concept of RandomSources (whatever we call them), which can return however many random bytes you ask for… and a protocol for types which know how to initialize themselves from those bytes. That might be spelled like 'static func random(using: RandomSource)->Self'. As a convenience, the source would also be able to create FixedWidthIntegers and Doubles (both with and without a range), and would also have the coinFlip() and oneIn(UInt)->Bool functions. Most types should be able to build themselves off of that. There would be a default source which is built from the first protocol.

I also really think we should have a concept of Repeatably-Random as a subprotocol for the second proposal. I see far too many shipping apps which have bugs due to using arc4Random when they really needed a repeatable source (e.g. patterns and lines jump around when you resize things). If it was an easy option, people would use it when appropriate. This would just mean a sub-protocol which has an initializer which takes a seed, and the ability to save/restore state (similar to CGContexts).

--
Greg Parker gparker@apple.com <mailto:gparker@apple.com> Runtime Wrangler

After thinking about this for a while, I don’t agree with with an
associated type on RandomNumberGenerator. I think a generic
FixedWidthInteger & UnsignedInteger should be sufficient. If there were an
associated type, and the default for Random was UInt32, then there might be
some arguments about allowing Double to utilize the full 64 bit precision.
We could make Random32 and Random64, but I think people will ask why there
isn’t a Random8 or Random16 for those bit widths. The same could also be
said that any experienced developer would know that his PRNG would be
switched if he asked for 32 bit or 64 bit.

I don't understand. Of course, Double would require 64 bits of randomness.
It would obtain this by calling `next()` as many times as necessary to
obtain the requisite number of bits.

At base, any PRNG algorithm yields some fixed number of bits on each
iteration. You can certainly have a function that returns an arbitrary
number of random bits (in fact, I would recommend that such an algorithm be
a protocol extension method on RandomNumberGenerator), but it must be built
on top of a function that returns a fixed number of bits, where that number
is determined on a per-algorithm basis. Moreover--and this is
important--generating a random unsigned integer of arbitrary bit width in a
sound way is actually subtly _different_ from generating a floating-point
value of a certain bit width, and I'm not sure that one can be built on top
of the other. Compare, for example:

(These are essentially Swift versions of C++ algorithms.)

Basically, what I'm saying is that RandomNumberGenerator needs a `next()`
method that returns a fixed number of bits, and extension methods that
build on that to return T : FixedWidthInteger & UnsignedInteger of
arbitrary bit width or U : BinaryFloatingPoint of an arbitrary number of
bits of precision. Each individual RNG does not need to reimplement the
latter methods, just a method to return a `next()` value of a fixed number
of bits. You are welcome to use my implementation.

Ā·Ā·Ā·

On Mon, Nov 13, 2017 at 7:12 PM, Alejandro Alonso <aalonso128@outlook.com> wrote:

- Alejandro

On Nov 12, 2017, 9:46 PM -0600, Xiaodi Wu <xiaodi.wu@gmail.com>, wrote:

On Sun, Nov 12, 2017 at 19:47 Alejandro Alonso <aalonso128@outlook.com> > wrote:

Sorry I’ve been gone for a while, I had to do a lot of traveling.

1. Initially I made this thinking that developers had the power to
determine their own lower bound. The current implementation uses the
integer’s min value as a lower bound. If it makes sense to only allow
unsigned integers from an RNG, then I’m perfectly fine with. I do disagree
when you say that it should only generate UInt32s. The current approach
allows, lets say mt19337 and mt19337-64, to be used within one generator.
So if you wanted a UInt32, mt19337 would be used, and if you asked for a
UInt64, mt19337-64 would be used.

I agree with Nate that this doesn't need to be--and is better off not
being--generic. The two different Mersenne Twisters are properly two
distinct PRNGs. A user sufficiently sophisticated to ask for their own PRNG
would expect the same algorithm to be used to generate a value of any
bitwidth, by concatenation of successively generated random bits if
necessary.

Instead, the return type should be an associated type. 32-bit PRNG
algorithms would always return values of type UInt32 and 64-bit algorithms
would always return values of type UInt64.

2. The Randomizable protocol isn’t always used with integers. Think

Date.random or Color.random. These types of values are difficult to express
with ranges. Randomizable solves this issue.

3. I’ve made the adjustment necessary for this.

4. So while I can see your point for this, it would break the consistency
with Randomizable’s random property. You could argue that we could make
this property a function itself, but I think most will agree that
Int.random is a cleaner api than Int.random().

5. I’ve made the adjustment necessary for this.

6. I actually forgot to implement the random api for the ranges where
Bound: BinaryFloatingPoint. While implementing this, I realized these would
never fail and would always return a non-optional.

Sure it can. 0.0..<0.0 is an empty range, just like 0..<0. Any collection
type really should return an Optional for 'random' because collections can
be empty.

So, I decided making the other Countable ranges non-optional. (0 ..<

10).random would return a non-optional, (0.0 ..< 10.0).random would return
a non-optional, and Array(0 ..< 10).random would return an optional. I can
agree that something like (0 ..< 10).random is hard to discover, so I added
Int.random(in: 0 ..< 10) (along with BinaryFloatingPoint).

This is unsatisfying (to me, at least). The proposed design should offer
one very good way to accomplish something, not two spellings for the same
thing.

However, these are not requirements of Randomizable. I think these methods

would benefit more if they were extension methods:

extension Randomizable where Self: FixedWidthInteger, Self.Stride:
SignedInteger {
public static func random(
in range: Countable{Closed}Range,
using generator: RandomNumberGenerator
) -> Self {
return range.random(using: generator)
}
}

extension Randomizable where Self: BinaryFloatingPointer {
public static func random(
in range: {Closed}Range,
using generator: RandomNumberGenerator
) -> Self {
return range.random
}
}

I think external types that wish to do something similar, like
Data.random(bytes: 128), could extend Randomizable with their own custom
needs. The stdlib would at this point provide all the features needed to
make this happen very simply for something like Data.random(bytes: 128).

- Alejandro

On Nov 5, 2017, 10:44 PM -0600, Nate Cook <natecook@apple.com>, wrote:

Thanks for continuing to push this forward, Alejandro! I’m excited about
the potential of having access to these APIs as part of the standard
library. Here are a few comments on some different parts of the proposal:

1) For your RandomGenerator protocol, I’m not totally clear on the
semantics of the next(_:) and next(_:upperBound:) methods. Do they both
have zero as their lower bound, for example? I’m not sure it makes sense to
have signed integers generated directly by an RNG—perhaps T:
FixedWidthInteger & UnsignedInteger would be a more useful constraint.
(Does it even need to be generic? What if RNGs just generate UInt32s?)

2) Can you say more about the purpose of the Randomizable protocol? How
would we use that protocol in useful ways that we wouldn’t get from being
able to select random values from ranges (half-open and closed) of
FixedWidthInteger / BinaryFloatingPoint? My experience has been that a
full-width random value is rarely what a user needs.

3) I agree with Xiaodi that Random should probably be a struct with a
single shared instance, but I don’t think it should be internal. Hiding
that shared RNG would make it hard for non-stdlib additions to have the
same usage, as they would need to have completely separate implementations
for the ā€œdefaultā€ and custom RNG versions.

4) I would also still suggest that the simplest version of random (that
you use to get a value from a range or an element from a collection) should
be a function, not a property. Collection properties like first, last,
and count all represent facts that already exist about a collection, and
don’t change unless the collection itself changes. Choosing a random
element, on the other hand, is clearly going to be freshly performed on
each call. In addition, with the notable exception of count, we try to
ensure O(1) performance for properties, while random will be O(n) except
in random-access collections. Finally, if it is a method, we can unify the
two versions by providing a single method with the shared RNG as the
default parameter.

5) To match the sorted() method, shuffled() should be on Sequence
instead of Collection. I don’t think either shuffled() or shuffle()
needs to be a protocol requirement, since there isn’t really any kind of
customization necessary for different kinds of collections. Like the
sorting algorithms, both could be regular extension methods.

6) I don’t know whether or not a consensus has formed around the correct
spelling of the APIs for generating random values. From the proposal it
looks like the preferred ways of getting a random value in a range would be
to use the random property (or method) on a range or closed range:

    (0..<10).random // 7
    (0.0 ... 5.0).random // 4.112312

If that’s the goal, and we don’t want those values to be optional, we’ll
need an implementation of random for floating-point ranges and an overload
for fixed-width integer ranges. That said, I don’t think that style is as
discoverable as having static methods or initializers available on the
different types:

    Int.random(in: 0..<10)
    Double.random(in: 0.0 ... 5.0)
    // or maybe
    Int(randomIn: 0..<10)
    Double(randomIn: 0.0 ... 5.0)

(My only quibble with the initializer approach is that Bool would be
awkward.)

In addition, this alternative approach could make creating random values
more consistent with types that don’t work well in ranges:

    Data.random(bytes: 128)
    Color.random(r: 0...0, g: 0...1, b: 0...1, a: 1...1)

————

Thanks again!
Nate

On Nov 5, 2017, at 6:33 PM, Alejandro Alonso via swift-evolution < >> swift-evolution@swift.org> wrote:

[Proposal] Random Unification by Azoy Ā· Pull Request #760 Ā· apple/swift-evolution Ā· GitHub is the current API and
proposed solution.

- Alejandro

On Nov 5, 2017, 6:18 PM -0600, Xiaodi Wu <xiaodi.wu@gmail.com>, wrote:

My comments are directed to the "more up-to-date" document that you just
linked to in your reply to Jon. Is that one outdated? If so, can you send a
link to the updated proposal and implementation for which you're soliciting
feedback?

On Sun, Nov 5, 2017 at 6:12 PM, Alejandro Alonso <aalonso128@outlook.com> >> wrote:

The proposal and implementation have the current updated API. The link I
sent Jon was the one I brought up a few weeks ago which is outdated now.
The proposal answers all of your questions. As for `.random` being a
function, some would argue that it behaves in the same way as `.first` and
`.last` which are properties.

- Alejandro

On Nov 5, 2017, 6:07 PM -0600, Xiaodi Wu <xiaodi.wu@gmail.com>, wrote:

A few quick thoughts:

I know that there's been some discussion that `(1...10).random` is the
best spelling, but I'd like to push back on that suggestion. When I want a
random number, I tend to think of the type I want first ("I want a random
integer") and then a range ("I want a random integer between a and b"), not
the other way around. My intuition is that `Int.random(in:)` will be more
discoverable, both on that basis and because it is more similar to other
languages' syntax (`Math.random` in JavaScript and `randint` in NumPy,
for example). It also has the advantage that the type is explicit, which
I think is particularly useful in this case because the value itself is,
well, random.

I would also argue that, `random` is most appropriately a method and not
a property; there's no hard and fast rule for this, but the fact that the
result is stochastic suggests (to me) that it's not a "property" of the
range (or, for that matter, of the type).

I would reiterate here my qualms about `Source` being the term used for
a generator. These types are not a _source_ of entropy but rather a
_consumer_ of entropy.

`UnsafeRandomSource` needs to be renamed; "unsafe" has a specific
meaning in Swift--that is, memory safety, and this is not it. Moreover,
it's questionable whether this protocol is useful in any sense. What useful
generic algorithms can one write with such a protocol?

`XoroshiroRandom` cannot be seeded by any `Numeric` value; depending on
the specific algorithm it needs a seed of a specific bit width. If you
default the shared instance to being seeded with an `Int` then you will
have to have distinct implementations for 32-bit and 64-bit platforms. This
is unadvisable. On that note, your `UnsafeRandomSource` needs to have an
associated type and not a generic `<T : Numeric>` for the seed.

The default random number generator should be cryptographically secure;
however, it's not clear to me that it should be device random.

I agree with others that alternative random number generators other than
the default RNG (and, if not default, possibly also the device RNG) should
be accommodated by the protocol hierarchy but not necessarily supplied in
the stdlib.

The term `Randomizable` means something specific which is not how it's
used in your proposed protocol.

There's still the open question, not answered, about how requesting an
instance of the hardware RNG behaves when there's insufficient or no
entropy. Does it return nil, throw, trap, or wait? The proposed API does
not clarify this point, although based on the method signature it cannot
return nil or throw. Trapping might be acceptable but I'd be interested to
hear your take as to why it is preferable.

On Sun, Nov 5, 2017 at 4:43 PM, Alejandro Alonso via swift-evolution < >>> swift-evolution@swift.org> wrote:

For the proof of concept, I had accidentally deleted that one. I have a
more up to date one which was discussed a few weeks later.
Swift Random Unification Design Ā· GitHub

- Alejandro

On Nov 5, 2017, 4:37 PM -0600, Jonathan Hull <jhull@gbis.com>, wrote:

Is there a link to the writeup? The one in the quote 404s.

Thanks,
Jon

On Nov 5, 2017, at 2:10 PM, Alejandro Alonso via swift-evolution < >>>> swift-evolution@swift.org> wrote:

Hello once again Swift evolution community. I have taken the time to
write up the proposal for this thread, and have provided an implementation
for it as well. I hope to once again get good feedback on the overall
proposal.

- Alejandro

On Sep 8, 2017, 11:52 AM -0500, Alejandro Alonso via swift-evolution < >>>> swift-evolution@swift.org>, wrote:

Hello swift evolution, I would like to propose a unified approach to
`random()` in Swift. I have a simple implementation here
https://gist.github.com/Azoy/5d294148c8b97d20b96ee64f434bb4f5\. This
implementation is a simple wrapper over existing random functions so
existing code bases will not be affected. Also, this approach introduces a
new random feature for Linux users that give them access to upper bounds,
as well as a lower bound for both Glibc and Darwin users. This change would
be implemented within Foundation.

I believe this simple change could have a very positive impact on new
developers learning Swift and experienced developers being able to write
single random declarations.

I’d like to hear about your ideas on this proposal, or any
implementation changes if need be.

- Alejando

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

An alternative to this is to have the random generator write a specified number of bytes to a pointer’s memory, as David Waite and others have suggested. This is same way arc4random_buf and SecRandomCopyBytes are implemented. Each random number generator could then choose the most efficient way to provide the requested number of bytes. The protocol could look something like this:

protocol RandomNumberGenerator {
    /// Writes the specified number of bytes to the given pointer’s memory.
    func read(into p: UnsafeMutableRawPointer, bytes: Int)
}

This is less user-friendly than having a next() method, but I think that’s a good thing—we very much want people who need a random value to use higher-level APIs and just pass the RNG as a parameter when necessary.

Nate

Ā·Ā·Ā·

On Nov 13, 2017, at 7:38 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

On Mon, Nov 13, 2017 at 7:12 PM, Alejandro Alonso <aalonso128@outlook.com <mailto:aalonso128@outlook.com>> wrote:
After thinking about this for a while, I don’t agree with with an associated type on RandomNumberGenerator. I think a generic FixedWidthInteger & UnsignedInteger should be sufficient. If there were an associated type, and the default for Random was UInt32, then there might be some arguments about allowing Double to utilize the full 64 bit precision. We could make Random32 and Random64, but I think people will ask why there isn’t a Random8 or Random16 for those bit widths. The same could also be said that any experienced developer would know that his PRNG would be switched if he asked for 32 bit or 64 bit.

I don't understand. Of course, Double would require 64 bits of randomness. It would obtain this by calling `next()` as many times as necessary to obtain the requisite number of bits.

At base, any PRNG algorithm yields some fixed number of bits on each iteration. You can certainly have a function that returns an arbitrary number of random bits (in fact, I would recommend that such an algorithm be a protocol extension method on RandomNumberGenerator), but it must be built on top of a function that returns a fixed number of bits, where that number is determined on a per-algorithm basis. Moreover--and this is important--generating a random unsigned integer of arbitrary bit width in a sound way is actually subtly _different_ from generating a floating-point value of a certain bit width, and I'm not sure that one can be built on top of the other. Compare, for example:

https://github.com/xwu/NumericAnnex/blob/c962760bf974a84ec57d8c5e94c91f06584e2453/Sources/PRNG.swift#L157
https://github.com/xwu/NumericAnnex/blob/c962760bf974a84ec57d8c5e94c91f06584e2453/Sources/PRNG.swift#L316

(These are essentially Swift versions of C++ algorithms.)

Basically, what I'm saying is that RandomNumberGenerator needs a `next()` method that returns a fixed number of bits, and extension methods that build on that to return T : FixedWidthInteger & UnsignedInteger of arbitrary bit width or U : BinaryFloatingPoint of an arbitrary number of bits of precision. Each individual RNG does not need to reimplement the latter methods, just a method to return a `next()` value of a fixed number of bits. You are welcome to use my implementation.