[Proposal] Random Unification

Hello,

The collection is a subject which has elements, and we are asking for one of them at random.

it has elements, sure.
And because of its structure, it has a first and a last element and whatever.
But that random element is not an inherent property of the collection.

I find it much more natural to use the random number generator to draw random elements from collections than the other way round.
That approach also completely side-steps the problem with having to define default arguments. The user can just use any random number generator she has. Obviously, it makes sense to provide a default one named `random` to make it easily accessible.

var list = [1,2,3,4]
let a:Int? = list.randomElement //List is still [1,2,3,4] and ‘a’ contains one of the elements

Instead I would prefer to have something like:

let a = random.draw(from: list)

But now the RNG has to understand the concept of collections. I would argue it is much cleaner to write an extension on Collection.

func randomElement(using source: RandomSource = .default) -> Element? {
  guard !isEmpty else {return nil}
  let idx = Int.random(in: 0
(count - 1), using: source)
  return self[idx]
}

But then the Collection has to understand the concept of random numbers. ;-)

Not really. Collection itself doesn’t have to change it’s structure at all. We can just define a convenience function in an extension.

Well both approaches are equally clean from this point of view:

With a protocol defining random() and random(in:), you could write generic algorithms on things which know how to create themselves from a RNG. With your approach the RNG has to provide a way to get a random value for each type you want to support.

For example, without random(in:) or random(), how would you get a CGFloat between 0 and 1? Ranges are only collections if they are countable


extension RandomFoo {
   func draw<T: Collection>(from urn: T) -> T.Element? {
       guard !urn.isEmpty else { return nil }
       let idx = draw(from: urn.indices)
       return urn[idx]
   }
}

This will call itself repeatedly and hang...

We just have to define one base protocol for such extensions. Every random number generator then automatically knows how to draw elements from ranges and collections.

It isn’t automatic, thought. How would I get a random color?

···

On Nov 30, 2017, at 11:11 PM, Martin Waitz <tali@admingilde.org> wrote:

GameplayKit randomization has a number of API deficiencies that we can’t easily/efficiently solve by wrapping it in Swift.

Most significantly, it is not designed to support any cryptographic operations (which I believe is why they never added a nextBytes() or equivalent method).

For GameplayKit this is fine - the design focuses on being able to get the distribution appropriate for your game, and being able to have identical pseudorandom number sequences for each device in a multiplayer scenario. They went as far as to require all random sources to support NSSecureCoding so that they may be persisted and/or serialized over the wire, which precludes a GKRandomSource from being based on the system random sources.

-DW

···

On Dec 1, 2017, at 11:05 AM, Erica Sadun via swift-evolution <swift-evolution@swift.org> wrote:

I'd like to put forth that Gameplay Kit offers a perfectly cromulent model of random number generation and API. Why not take that as a starting point, Swiftify it so it becomes a cross platform solution, and then apply the specific questions of sequences, collections, and indices as a second step?

I don't support intermingling random functionality with existing core types.

I would personally go with:

Int.random //Returns a random Int

“Type.random” is so rarely used as to not be worth the addition, IMO. If
you really need a random element from the *entire* domain, then I think you
should have to manually create the ClosedRange<T> yourself.

Int.random(in: ClosedRange<Int>) //Works for Comparable types. Gives a
result from the closed range. Closed Range is never empty.

This is redundant. In order to pick a random element, you’re saying I
should have to do “Int.random(0 ..< 10)”? The redundancy here is that I
have to specify Int twice: once for the “.random” call, and again for the
type of the range. We can do better than that.

I don’t see how this is redundant—do you mean that you’d need to write
out the type name instead of writing `(0..<10).random()!`? I continue to be
of the opinion that picking a random integer/floating-point value/Boolean
and picking a random element from a collection are two different
operations. Yes, there’s overlap between the two, but they’re different
enough that having two ways to choose an integer between 0 and 10 would be
fine.

As I wrote elsewhere, if they're sufficiently different enough, then they
ought to have distinct names. If they are not, then there ought to be only
one of them. I'm no longer sure myself of which one I'd prefer, but in no
case should there be two things named "random."

If the proposal to make `IndexDistance == Int` is accepted, then ranges
cannot conform to Collection and we encounter some interesting difficulties
attempting to index into such a collection to pick a random value.

[0,2,3].randomElement //Returns a random element from the collection

I strongly believe this should be a method, not a property. Properties,
like .first and .last, are expected to return the same value each time you
access them. “.random” inherently breaks that.

FWIW--and this isn't a vote, I know--I largely agree with Dave DeLong's
conclusions above, and for substantially the same reasons.

Then a version of each with a ‘using:’ parameter which takes a
generator/source:

Int.random(using: RandomSource) //Returns a random Int using the given
source of randomness
Int.random(in: ClosedRange<Int>, using: RandomSource)
[0,2,3].randomElement(using: RandomSource)

In my own RandomSource & RandomSourceCreatable protocols, I frequently
use random colors and sizes as well. The issue there is that you really
want a closed range for each dimension. I wish Swift had a better notion of
dimensionality baked into the language.

What I ended up doing was having a “constraints” parameter which took an
array of constraints which corresponded to various dimensions. It works
for me, but it might be a bit complex for something in the standard library.

Honestly, given the current capabilities of Swift what this really calls
for is custom initializers/functions for dimensional types:

UIColor.random //This comes from the protocol
UIColor.random(hue: ClosedRange<CGFloat> = 0
1, saturation:
ClosedRange<CGFloat> = 0
1, brightness: ClosedRange<CGFloat> = 0
1, alpha:
ClosedRange<CGFloat> = 1
1)
//
and of course the same as above, but with ‘using:'

Then you can easily get random colors which look like they belong
together:
let myColor = UIColor.random(saturation: 0.2
0.2, brightness: 0.6
0.6)

There would probably also be a convenience version taking CGFloats and
passing them to the real function as ranges:

let myColor = UIColor.random(saturation: 0.2, brightness: 0.6)

This means that our default RandomSource needs to be publicly available,
so that the custom functions can use it as the default


It does not. Having actually implemented some version of these APIs, it's
readily apparent now to me that all custom types can simply call
Int.random(in:) (or UnsafeRawBufferPointer<T>.random(byteCount:), or
whatever else we want to have in the standard library) to get random values
from the default RNG for any built-in type and size. The actual default
random need never be exposed publicly, and since its functions are strictly
redundant to these other APIs (which, of course, are the "currency" APIs
that our purpose here is to design and make public), the default random is
required only for internal implementation of the "currency" APIs and (a) is
better off *not* exposed; (b) doesn't need to be of the same type as other
RNGs, conform to the same protocols, or for that matter, does not even need
to be a type or be written in Swift.

Assuming we have two versions of whatever new random operations we add,
where one takes an RNG as a parameter and the other uses the default, how
would I add functionality that matches this pattern in my own code? The
standard library would look have code like this:

    extension MutableCollection {
        func shuffle(using rng: RNG) {
            // shuffle the elements
        }

        func shuffle() {
            shuffle(using: _privateDefaultRNG)
        }
    }

Let’s say I’m writing something for my own app or library, and want to
add two related methods with the same pattern:

    extension Collection {
        func randomRange(length: Int, using rng: RNG) -> [SubSequence] {
            // select a starting point in 0..<count - length
            // verify that florps are still sideways
            // etc
        }

        func randomRange(length: Int) {
            return randomRange(length: Int, using: ?????)
        }
    }

What do I put in the second method so I don’t have to implement this
twice?

If this is to be used, as others say, to permit obtaining a random value
of a certain distribution passed in as the parameter `using`, then to
implement it correctly you *must* implement it twice. Actually, you may
have to implement it not just twice but differently for each distribution,
so I'm not sure that it's a viable design for the stated use case.

I don't think I've seen anyone propose that you would haven't different
distributions by passing different RNGs.

I do believe that Dave DeLong did say something along those lines earlier
in the thread. The design of the C++ standard library
RandomNumberDistribution function objects does have some echoes here,
although the actual design is somewhat different.

Is there any way any of this works for an RNG that isn't providing uniform

random bits? I don't understand how any of these primitive methods would
work with an RNG that was doing something different, regardless of how the
methods are spelled.

It may well not; that said, nothing in the proposal's RNG protocol
specifies a semantic requirement regarding the distribution of the random
bits produced, and it is not out of the question that someone may attempt
to design a suite of RNGs that provide varying distributions.

The purpose of the (using:) versions is to allow for RNGs that meet

different requirements from the default: seedable/repeatable, faster, more
secure, etc.

Yes, I understand. Even then: is a struct that has two stored properties
more secure when it is initialized with two more secure values? That may
depend on what the struct does with those values, no?

···

On Thu, Nov 30, 2017 at 6:55 PM, Nate Cook <natecook@apple.com> wrote:

On Nov 30, 2017, at 6:20 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:
On Thu, Nov 30, 2017 at 5:24 PM, Nate Cook <natecook@apple.com> wrote:

On Nov 30, 2017, at 4:30 PM, Xiaodi Wu via swift-evolution < >> swift-evolution@swift.org> wrote:
On Thu, Nov 30, 2017 at 3:58 PM, Dave DeLong via swift-evolution < >> swift-evolution@swift.org> wrote:

On Nov 30, 2017, at 2:48 PM, Jonathan Hull via swift-evolution < >>> swift-evolution@swift.org> wrote:

-Nate

:+1:

Thanks,
Jon

On Nov 27, 2017, at 10:14 AM, TellowKrinkle via swift-evolution < >>> swift-evolution@swift.org> wrote:

You say that all the `.random`s have different semantics, but to me (at
least), they are all very similar. All the methods can be summarized as
selecting a single random element from a collection
`[0, 2, 3].random` selects a single element from the given collection
`Int.random(in: 0
8)` selects a single element from the given range
`Int.random` has no range, but selects a single element from the
collection of all ints (equivalent to if the above method had a default
value for its range)
So to me these are all doing the same operation, just with different
types of inputs

2017/11/24 20:07、Alejandro Alonso <aalonso128@outlook.com>ăźăƒĄăƒŒăƒ«:

- Alejandro

---------- Forwarded message ----------
*From:* Xiaodi Wu <xiaodi.wu@gmail.com>
*Date:* Nov 24, 2017, 3:05 PM -0600
*To:* Alejandro Alonso <aalonso128@outlook.com>
*Cc:* Brent Royal-Gordon <brent@architechies.com>, Steve Canon via
swift-evolution <swift-evolution@swift.org>
*Subject:* Re: [swift-evolution] [Proposal] Random Unification

On Fri, Nov 24, 2017 at 2:55 PM, Alejandro Alonso < >>> aalonso128@outlook.com> wrote:

Regarding naming too many things “random”, I’ve talked to many
developers on my end and they all don’t find it confusing. This proposal is
aimed to make it obvious what the operation is doing when regarding random.
I still agree that the proposed solution does just that and in practice
feels good to write.

I must disagree quite strongly here. The various facilities you name
"random" have different semantics, and differences in semantics should be
reflected in differences in names. It doesn't matter that some people don't
find it confusing; it is objectively the case that you have named multiple
distinct facilities with the same name, which leads to confusion. I, for
one, get confused, and you can see on this list that people are using
arguments about one property named "random" to discuss another property
named "random". This is quite an intolerable situation.

I disagree that sample is the correct naming to use here. Getting a

sample is a verb in this context which would make it break API guidelines
just as well as `pick()`. To sample is to “take a sample or samples of
(something) for analysis.” I can agree to use `sampling()` which follows
API guidelines. This would result in the following grammar for `[“hi”,
“hello”, “hey”].sampling(2)`, “From array, get a sampling of 2"

"Sampling" is fine.

On Nov 23, 2017, 12:54 AM -0600, Xiaodi Wu , wrote:

On Wed, Nov 22, 2017 at 23:01 Alejandro Alonso <aalonso128@outlook.com> >>>> wrote:

Like I’ve said, python has different syntax grammar. We have to read
each call site and form a sentence from it. `random.choice([1, 2, 3])` to
me this reads, “Get a random choice from array”. This makes sense. Slapping
the word choice as an instance property like `[1, 2, 3].choice` reads,
“From array, get choice”. What is choice? This doesn’t make sense at all to
me. To me, the only good solution is `[1, 2, 3].random` which reads, “From
array, get random”. I actually think most users will be able to understand
this at first glance rather than choice (or any or some).

Again, my concern here is that you are proposing to name multiple
things "random". If this property should be called "random"--which I'm fine
with--then the static method "random(in:)" should be named something else,
and the static property "random" should be dropped altogether (as I
advocate for reasons we just discussed) or renamed as well. It is simply
too confusing that there are so many different "random" methods or
properties. Meanwhile, isn't your default RNG also going to be called
something like "DefaultRandom"?

In regards to the sample() function on collections, I have added this

as I do believe this is something users need. The name I gave it was pick()
as this reads, “From array, pick 2”.

The name "sample" has been used to good effect in other languages, has
a well understood meaning in statistics, and is consistent with Swift
language guidelines. The operation here is a sampling, and per Swift
guidelines the name must be a noun: therefore, 'sample' is fitting. "Pick"
does not intrinsically suggest randomness, whereas sample does, and your
proposed reading uses it as a verb, whereas Swift guidelines tell us it
must be a noun. I would advocate strongly for using well-established
terminology and sticking with "sample."

On Nov 17, 2017, 8:32 PM -0600, Xiaodi Wu via swift-evolution < >>>>> swift-evolution@swift.org>, wrote:

On Fri, Nov 17, 2017 at 7:11 PM, Brent Royal-Gordon < >>>>> brent@architechies.com> wrote:

On Nov 17, 2017, at 3:09 PM, Xiaodi Wu via swift-evolution < >>>>>> swift-evolution@swift.org> wrote:

But actually, Int.random followed by % is the much bigger issue and a
very good cautionary tale for why T.random is not a good idea. Swift should
help users do the correct thing, and getting a random value across the full
domain and computing an integer modulus is never the correct thing to do
because of modulo bias, yet it's a very common error to make. We are much
better off eliminating this API and encouraging use of the correct API,
thereby reducing the likelihood of users making this category of error.

Amen.

If (and I agree with this) the range-based notation is less intuitive
(0..<10.random is certainly less discoverable than Int.random), then we
ought to offer an API in the form of `Int.random(in:)` but not
`Int.random`. This does not preclude a `Collection.random` API as Alejandro
proposes, of course, and that has independent value as Gwendal says.

If we're not happy with the range syntax, maybe we should put
`random(in:)`-style methods on the RNG protocol as extension methods
instead. Then there's a nice, uniform style:

let diceRoll = rng.random(in: 1...6)
let card = rng.random(in: deck)
let isHeads = rng.random(in: [true, false])
let probability = rng.random(in: 0.0...1.0) // Special FloatingPoint
overload

The only issue is that this makes the default RNG's name really
important. Something like:

DefaultRandom.shared.random(in: 1...6)

Will be a bit of a pain for users.

I did in fact implement this style of RNG in NumericAnnex, but I'm not
satisfied with the design myself. Not only is it a bit of an ergonomic
thorn, there's also another drawback that actually has weighty implications:

Users aren't conditioned to reuse RNG instances. Perhaps, it is
because it can "feel" wrong that multiple random instances should come from
the *same* RNG. Instead, it "feels" more right to initialize a new RNG for
every random number. After all, if one RNG is random, two must be randomer!
This error is seen with some frequency in other languages that adopt this
design, and they sometimes resort to educating users through documentation
that isn't consistently heeded.

Of course, you and I both know that this is not ideal for performance.
Moreover, for a number of PRNG algorithms, the first few hundred or
thousand iterations can be more predictable than later iterations. (Some
algorithms discard the first n iterations, but whether that's adequate
depends on the quality of the seed, IIUC.) Both of these issues don't apply
specifically to a default RNG type that cannot be initialized and always
uses entropy from the global pool, but that's not enough to vindicate the
design, IMO. By emphasizing *which* RNG instance is being used for random
number generation, the design encourages non-reuse of non-default RNGs,
which is precisely where this common error matters for performance (and
maybe security).

Maybe we call the default RNG instance `random`, and then give the

`random(in:)` methods another name, like `choose(in:)`?

let diceRoll = random.choose(in: 1...6)
let card = random.choose(in: deck)
let isHeads = random.choose(in: [true, false])
let probability = random.choose(in: 0.0...1.0)
let diceRoll = rng.choose(in: 1...6)
let card = rng.choose(in: deck)
let isHeads = rng.choose(in: [true, false])
let probability = rng.choose(in: 0.0...1.0)

This would allow us to keep the default RNG's type private and expose
it only as an existential—which means more code will treat RNGs as black
boxes, and people will extend the RNG protocol instead of the default RNG
struct—while also putting our default random number generator under the
name `random`, which is probably where people will look for such a thing.

I've said this already in my feedback, but it can get lost in the long
chain of replies, so I'll repeat myself here because it's relevant to the
discussion. I think one of the major difficulties of discussing the
proposed design is that Alejandro has chosen to use a property called
"random" to name multiple distinct functions which have distinct names in
other languages. In fact, almost every method or function is being named
"random." We are tripping over ourselves and muddling our thinking (or at
least, I find myself doing so) because different things have the exact same
name, and if I'm having this trouble after deep study of the design, I
think it's a good sign that this is going to be greatly confusing to users
generally.

First, there's Alejandro's _static random_, which he proposes to
return an instance of type T given a type T. In Python, this is named
`randint(a, b)` for integers, and `random` (between 0 and 1) or `uniform(a,
b)` for floating-type types. The distinct names reflect the fact that
`randint` and `uniform` are mathematically quite different (one samples a
*discrete* uniform distribution and the other a *continuous* uniform
distribution), and I'm not aware of non-numeric types offering a similar
API in Python. These distinct names accurately reflect critiques from
others on this list that the proposed protocol `Randomizable` lumps
together types that don't share any common semantics for their _static
random_ method, and that the protocol is of questionable utility because
types in general do not share sufficient semantics such that one can do
interesting work in generic code with such a protocol.

Then there's Alejandro's _instance random_, which he proposes to
return an element of type T given a instance of a collection of type T. In
Python, this is named "choice(seq)" (for one element, or else throws an
error) and "sample(seq, k)" (for up to k elements). As I noted, Alejandro
was right to draw an analogy between _instance random_ and other instance
properties of a Collection such as `first` and `last`. In fact, the
behavior of Python's "choice" (if modified to return an Optional) and
"sample", as a pair, would fit in very well next to Swift's existing pairs
of `first` and `prefix(k)` and `last` and `suffix(k)`. We could trivially
Swiftify the names here; for example:

[1, 2, 3].first
[1, 2, 3].any // or `choice`, or `some`, or...
[1, 2, 3].last

[1, 2, 3].prefix(2)
[1, 2, 3].sample(2)
[1, 2, 3].suffix(2)

I'm going to advocate again for _not_ naming all of these distinct
things "random". Even in conducting this discussion, it's so hard to keep
track of what particular function a person is giving feedback about.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

On Nov 17, 2017, 8:32 PM -0600, Xiaodi Wu via swift-evolution < >>>> swift-evolution@swift.org>, wrote:

On Fri, Nov 17, 2017 at 7:11 PM, Brent Royal-Gordon < >>>> brent@architechies.com> wrote:

On Nov 17, 2017, at 3:09 PM, Xiaodi Wu via swift-evolution < >>>>> swift-evolution@swift.org> wrote:

But actually, Int.random followed by % is the much bigger issue and a
very good cautionary tale for why T.random is not a good idea. Swift should
help users do the correct thing, and getting a random value across the full
domain and computing an integer modulus is never the correct thing to do
because of modulo bias, yet it's a very common error to make. We are much
better off eliminating this API and encouraging use of the correct API,
thereby reducing the likelihood of users making this category of error.

Amen.

If (and I agree with this) the range-based notation is less intuitive
(0..<10.random is certainly less discoverable than Int.random), then we
ought to offer an API in the form of `Int.random(in:)` but not
`Int.random`. This does not preclude a `Collection.random` API as Alejandro
proposes, of course, and that has independent value as Gwendal says.

If we're not happy with the range syntax, maybe we should put
`random(in:)`-style methods on the RNG protocol as extension methods
instead. Then there's a nice, uniform style:

let diceRoll = rng.random(in: 1...6)
let card = rng.random(in: deck)
let isHeads = rng.random(in: [true, false])
let probability = rng.random(in: 0.0...1.0) // Special FloatingPoint
overload

The only issue is that this makes the default RNG's name really
important. Something like:

DefaultRandom.shared.random(in: 1...6)

Will be a bit of a pain for users.

I did in fact implement this style of RNG in NumericAnnex, but I'm not
satisfied with the design myself. Not only is it a bit of an ergonomic
thorn, there's also another drawback that actually has weighty implications:

Users aren't conditioned to reuse RNG instances. Perhaps, it is because
it can "feel" wrong that multiple random instances should come from the
*same* RNG. Instead, it "feels" more right to initialize a new RNG for
every random number. After all, if one RNG is random, two must be randomer!
This error is seen with some frequency in other languages that adopt this
design, and they sometimes resort to educating users through documentation
that isn't consistently heeded.

Of course, you and I both know that this is not ideal for performance.
Moreover, for a number of PRNG algorithms, the first few hundred or
thousand iterations can be more predictable than later iterations. (Some
algorithms discard the first n iterations, but whether that's adequate
depends on the quality of the seed, IIUC.) Both of these issues don't apply
specifically to a default RNG type that cannot be initialized and always
uses entropy from the global pool, but that's not enough to vindicate the
design, IMO. By emphasizing *which* RNG instance is being used for random
number generation, the design encourages non-reuse of non-default RNGs,
which is precisely where this common error matters for performance (and
maybe security).

Maybe we call the default RNG instance `random`, and then give the

`random(in:)` methods another name, like `choose(in:)`?

let diceRoll = random.choose(in: 1...6)
let card = random.choose(in: deck)
let isHeads = random.choose(in: [true, false])
let probability = random.choose(in: 0.0...1.0)
let diceRoll = rng.choose(in: 1...6)
let card = rng.choose(in: deck)
let isHeads = rng.choose(in: [true, false])
let probability = rng.choose(in: 0.0...1.0)

This would allow us to keep the default RNG's type private and expose
it only as an existential—which means more code will treat RNGs as black
boxes, and people will extend the RNG protocol instead of the default RNG
struct—while also putting our default random number generator under the
name `random`, which is probably where people will look for such a thing.

I've said this already in my feedback, but it can get lost in the long
chain of replies, so I'll repeat myself here because it's relevant to the
discussion. I think one of the major difficulties of discussing the
proposed design is that Alejandro has chosen to use a property called
"random" to name multiple distinct functions which have distinct names in
other languages. In fact, almost every method or function is being named
"random." We are tripping over ourselves and muddling our thinking (or at
least, I find myself doing so) because different things have the exact same
name, and if I'm having this trouble after deep study of the design, I
think it's a good sign that this is going to be greatly confusing to users
generally.

First, there's Alejandro's _static random_, which he proposes to return
an instance of type T given a type T. In Python, this is named `randint(a,
b)` for integers, and `random` (between 0 and 1) or `uniform(a, b)` for
floating-type types. The distinct names reflect the fact that `randint` and
`uniform` are mathematically quite different (one samples a *discrete*
uniform distribution and the other a *continuous* uniform distribution),
and I'm not aware of non-numeric types offering a similar API in Python.
These distinct names accurately reflect critiques from others on this list
that the proposed protocol `Randomizable` lumps together types that don't
share any common semantics for their _static random_ method, and that the
protocol is of questionable utility because types in general do not share
sufficient semantics such that one can do interesting work in generic code
with such a protocol.

Then there's Alejandro's _instance random_, which he proposes to return
an element of type T given a instance of a collection of type T. In Python,
this is named "choice(seq)" (for one element, or else throws an error) and
"sample(seq, k)" (for up to k elements). As I noted, Alejandro was right to
draw an analogy between _instance random_ and other instance properties of
a Collection such as `first` and `last`. In fact, the behavior of Python's
"choice" (if modified to return an Optional) and "sample", as a pair, would
fit in very well next to Swift's existing pairs of `first` and `prefix(k)`
and `last` and `suffix(k)`. We could trivially Swiftify the names here; for
example:

[1, 2, 3].first
[1, 2, 3].any // or `choice`, or `some`, or...
[1, 2, 3].last

[1, 2, 3].prefix(2)
[1, 2, 3].sample(2)
[1, 2, 3].suffix(2)

I'm going to advocate again for _not_ naming all of these distinct
things "random". Even in conducting this discussion, it's so hard to keep
track of what particular function a person is giving feedback about.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

What is the useful distinction between generating a random value, and choosing a random element from a collection of all possible values?

I don’t have to generate (or keep in memory) that collection.

I gave an example before of an easy to add API for random colors which allow their saturation, lightness, and alpha to be fixed. I use something very similar all the time in some graphics code I have (varying only the hue or saturation). I also do the same with Sizes and CGVectors. How would you represent that as a collection to choose an element from?

You’re not picking a random color. You’re picking a random hue or a random saturation, and then constructing a color based off that. In other words, you’re choosing a value from 0
1:

let color = UIColor(hue: (0
1.0).random(), saturation: knownSaturationValue, brightness: knownBrightnessValue, alpha: knownAlphaValue)

Dave

Also, for most of my use cases, I *need* to be able to plug in a repeatably random source
 otherwise pixels will start jumping around the screen when people resize things.

That’s cool; I’m totally in favor of the RandomSource API as a way to model different distributions or algorithms or whatever. I just don’t thing “Foo.random” is a good idea.

Cheers

Dave

···

On Nov 30, 2017, at 5:02 PM, Jonathan Hull <jhull@gbis.com> wrote:

On Nov 30, 2017, at 3:52 PM, Dave DeLong <swift@davedelong.com <mailto:swift@davedelong.com>> wrote:

Hi,

With a protocol defining random() and random(in:), you could write generic algorithms on things which know how to create themselves from a RNG. With your approach the RNG has to provide a way to get a random value for each type you want to support.

This is right, if we want a generic prototype for construction of random objects, well then we need such a protocol and a new method or initializer.
However, I‘m not yet convinced that such a protocol is necessary.
Most objects will be constructed using randomized values, according to the use case at hand. A generic randomizable protocol would not help here.

For example, without random(in:) or random(), how would you get a CGFloat between 0 and 1? Ranges are only collections if they are countable


I was assuming that there would be a random.draw(from:) for ranges.

extension RandomFoo {
  func draw<T: Collection>(from urn: T) -> T.Element? {
      guard !urn.isEmpty else { return nil }
      let idx = draw(from: urn.indices)
      return urn[idx]
  }
}

This will call itself repeatedly and hang...

You are right, Collection.indices is also a Collection. We have to explicitly use the range here.

We just have to define one base protocol for such extensions. Every random number generator then automatically knows how to draw elements from ranges and collections.

It isn’t automatic, thought. How would I get a random color?

This has already been discussed: you either want some random color from a fixed set, or you want to produce a new color using some random values. A completely random color it totally useless.

— Martin

That said, I am not sure that this proposal should give any pretense of
being suitable for cryptographic use. On implementation, the code will not
have been audited for that purpose, although the author very rightly
attempts to use the "best" possible sources of entropy available for each
platform. Perhaps explicitly _not_ supporting cryptographic operations is
the more Swifty way to go (in the sense that, where possible, Swift API
design aims to help users avoid footguns).

···

On Sat, Dec 2, 2017 at 12:23 AM, David Waite <david@alkaline-solutions.com> wrote:

On Dec 1, 2017, at 11:05 AM, Erica Sadun via swift-evolution < > swift-evolution@swift.org> wrote:

I'd like to put forth that Gameplay Kit offers a perfectly cromulent model
of random number generation and API. Why not take that as a starting point,
Swiftify it so it becomes a cross platform solution, and then apply the
specific questions of sequences, collections, and indices as a second step?

I don't support intermingling random functionality with existing core
types.

GameplayKit randomization has a number of API deficiencies that we can’t
easily/efficiently solve by wrapping it in Swift.

Most significantly, it is not designed to support any cryptographic
operations (which I believe is why they never added a nextBytes() or
equivalent method).

For GameplayKit this is fine - the design focuses on being able to get the
distribution appropriate for your game, and being able to have identical
pseudorandom number sequences for each device in a multiplayer scenario.
They went as far as to require all random sources to support NSSecureCoding
so that they may be persisted and/or serialized over the wire, which
precludes a GKRandomSource from being based on the system random sources.

I wasn't suggesting wrapping as that would not provide a cross platform solution. Instead of re-inventing the wheel, we may profit from the approaches already explored and real-world tested in prior art and existing standards.

-- E

···

On Dec 1, 2017, at 11:23 PM, David Waite <david@alkaline-solutions.com> wrote:

On Dec 1, 2017, at 11:05 AM, Erica Sadun via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

I'd like to put forth that Gameplay Kit offers a perfectly cromulent model of random number generation and API. Why not take that as a starting point, Swiftify it so it becomes a cross platform solution, and then apply the specific questions of sequences, collections, and indices as a second step?

I don't support intermingling random functionality with existing core types.

GameplayKit randomization has a number of API deficiencies that we can’t easily/efficiently solve by wrapping it in Swift.

Most significantly, it is not designed to support any cryptographic operations (which I believe is why they never added a nextBytes() or equivalent method).

For GameplayKit this is fine - the design focuses on being able to get the distribution appropriate for your game, and being able to have identical pseudorandom number sequences for each device in a multiplayer scenario. They went as far as to require all random sources to support NSSecureCoding so that they may be persisted and/or serialized over the wire, which precludes a GKRandomSource from being based on the system random sources.

-DW

Hi,

With a protocol defining random() and random(in:), you could write generic algorithms on things which know how to create themselves from a RNG. With your approach the RNG has to provide a way to get a random value for each type you want to support.

This is right, if we want a generic prototype for construction of random objects, well then we need such a protocol and a new method or initializer.
However, I‘m not yet convinced that such a protocol is necessary.
Most objects will be constructed using randomized values, according to the use case at hand. A generic randomizable protocol would not help here.

I’m not sure how you intend to get a random value from a Range without something like this.

But even ignoring that for the moment, I have a version of this in my own code and find it quite useful. What I do is setup a series of constraints on the values I want (e.g. the range I want them in) and then I generically generate a lazy collection of random values of that type. Thus, I can ask for colors with certain properties, and then I receive a collection of random colors with those properties which I use to create patterns and graphic effects.

For example, without random(in:) or random(), how would you get a CGFloat between 0 and 1? Ranges are only collections if they are countable


I was assuming that there would be a random.draw(from:) for ranges.

Sure, but Ranges of what? Ranges of CGFloats are very different than Ranges of Ints, you need some cooperation from the type itself. This is my main point.

extension RandomFoo {
func draw<T: Collection>(from urn: T) -> T.Element? {
     guard !urn.isEmpty else { return nil }
     let idx = draw(from: urn.indices)
     return urn[idx]
}
}

This will call itself repeatedly and hang...

You are right, Collection.indices is also a Collection. We have to explicitly use the range here.

We just have to define one base protocol for such extensions. Every random number generator then automatically knows how to draw elements from ranges and collections.

It isn’t automatic, thought. How would I get a random color?

This has already been discussed: you either want some random color from a fixed set, or you want to produce a new color using some random values. A completely random color it totally useless.

I wouldn’t say completely useless. In the example above, I usually constrain my random colors to have a specific effect, but completely random has its own look (which I mainly use for debugging purposes). As I mentioned earlier in the thread, the protocol can be augmented by UIColor to provide a way to only get random values with certain characteristics.

Yes, colors can be built from smaller primitives, but it is nice to have the type do that itself so you can continue to build on it generically.

In the examples you have given, everything has to be special cased (even getting a random Bool requires me to list out the options [true, false]). That isn’t useful for building large scale structures generically.

I am not arguing that we shouldn’t be able to get a random element from a collection or range
 just that we should build this in a way which is easy to build on and modify.

···

On Dec 1, 2017, at 9:39 AM, Martin Waitz <tali@admingilde.org> wrote:

People will use it for crypto whether we want them to or not.

None of the proposals involve actually writing cryptographic primitives, just calling out to well-known functions like `arc4random()` or reading from special devices like `/dev/urandom`. I would hope that Apple's security team can spare the time to review the security-critical parts and sign off on them, and I'd be perfectly happy to see this proposal be approved but the final merge into the language be deferred until they've taken a look at it.

···

On Dec 1, 2017, at 10:37 PM, Xiaodi Wu via swift-evolution <swift-evolution@swift.org> wrote:

That said, I am not sure that this proposal should give any pretense of being suitable for cryptographic use. On implementation, the code will not have been audited for that purpose, although the author very rightly attempts to use the "best" possible sources of entropy available for each platform. Perhaps explicitly _not_ supporting cryptographic operations is the more Swifty way to go (in the sense that, where possible, Swift API design aims to help users avoid footguns).

--
Brent Royal-Gordon
Architechies

That said, I am not sure that this proposal should give any pretense of
being suitable for cryptographic use. On implementation, the code will not
have been audited for that purpose, although the author very rightly
attempts to use the "best" possible sources of entropy available for each
platform. Perhaps explicitly _not_ supporting cryptographic operations is
the more Swifty way to go (in the sense that, where possible, Swift API
design aims to help users avoid footguns).

People will use it for crypto whether we want them to or not.

People are going to do all sorts of unanticipated things, sure. But this
doesn't mean that we shouldn't consider how we may best encourage users to
avoid _unintentional_ common pitfalls.

There are options to explore here. Consider, for example--and I'm not
suggesting that we be this verbose, but it is illustrative of the
trade-offs which are possible to keep in mind--if certain methods were very
clear that the result is *pseudorandom* and potentially *insecure*:
`(0..<9).insecurePseudorandomElement`. Clearly, fewer people would use this
method for crypto.

None of the proposals involve actually writing cryptographic primitives,

···

On Sat, Dec 2, 2017 at 6:00 AM, Brent Royal-Gordon <brent@architechies.com> wrote:

On Dec 1, 2017, at 10:37 PM, Xiaodi Wu via swift-evolution < > swift-evolution@swift.org> wrote:
just calling out to well-known functions like `arc4random()` or reading
from special devices like `/dev/urandom`. I would hope that Apple's
security team can spare the time to review the security-critical parts and
sign off on them, and I'd be perfectly happy to see this proposal be
approved but the final merge into the language be deferred until they've
taken a look at it.

But what *should* they use instead of our API? The OS-provided CSPRNG is almost certainly going to be the most secure thing available in the absence of specialized hardware. We should not deliberately scare users away from our API if there is nothing better on offer.

···

On Dec 2, 2017, at 1:09 PM, Xiaodi Wu via swift-evolution <swift-evolution@swift.org> wrote:

On Sat, Dec 2, 2017 at 6:00 AM, Brent Royal-Gordon <brent@architechies.com <mailto:brent@architechies.com>> wrote:

On Dec 1, 2017, at 10:37 PM, Xiaodi Wu via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

That said, I am not sure that this proposal should give any pretense of being suitable for cryptographic use. On implementation, the code will not have been audited for that purpose, although the author very rightly attempts to use the "best" possible sources of entropy available for each platform. Perhaps explicitly _not_ supporting cryptographic operations is the more Swifty way to go (in the sense that, where possible, Swift API design aims to help users avoid footguns).

People will use it for crypto whether we want them to or not.

People are going to do all sorts of unanticipated things, sure. But this doesn't mean that we shouldn't consider how we may best encourage users to avoid _unintentional_ common pitfalls.

There are options to explore here. Consider, for example--and I'm not suggesting that we be this verbose, but it is illustrative of the trade-offs which are possible to keep in mind--if certain methods were very clear that the result is *pseudorandom* and potentially *insecure*: `(0..<9).insecurePseudorandomElement`. Clearly, fewer people would use this method for crypto.

--
Greg Parker gparker@apple.com Runtime Wrangler

It’s not about OS-provided CSPRNG. It’s about the design and implementation
of _this proposal_ on top of the CSPRNG.

Earlier, we discussed how this API should minimize the number of optional
return values to improve user ergonomics. Instead, the returned value
should be a reasonable best-effort at randomness. This is sensible for a
general-use API, but it is unsuitable for a crypto-oriented API.

David Waite criticizes GameplayKit as lacking in crypto-oriented
functions—the implication being that we ought to provide them here. I
disagree. My point is that our entire design process has been geared
towards a reasonable, best-effort general-use API: It is being designed by
community members who are not specialized in this particular subfield. It
is explicitly being design for common, general use cases in mind. And the
implementation will come with no guarantee as to suitability for crypto,
nor should it have to.

Therefore, I reason, we ought not to extend the design with functions that
are explicitly geared towards crypto using primitive operations that we
haven’t audited for such suitability. Instead, we ought to make clear to
users both the features and the limitations of this API, to encourage use
where suitable and to discourage use where unsuitable.

···

On Sat, Dec 2, 2017 at 19:02 Greg Parker <gparker@apple.com> wrote:

On Dec 2, 2017, at 1:09 PM, Xiaodi Wu via swift-evolution < > swift-evolution@swift.org> wrote:

On Sat, Dec 2, 2017 at 6:00 AM, Brent Royal-Gordon <brent@architechies.com > > wrote:

On Dec 1, 2017, at 10:37 PM, Xiaodi Wu via swift-evolution < >> swift-evolution@swift.org> wrote:

That said, I am not sure that this proposal should give any pretense of
being suitable for cryptographic use. On implementation, the code will not
have been audited for that purpose, although the author very rightly
attempts to use the "best" possible sources of entropy available for each
platform. Perhaps explicitly _not_ supporting cryptographic operations is
the more Swifty way to go (in the sense that, where possible, Swift API
design aims to help users avoid footguns).

People will use it for crypto whether we want them to or not.

People are going to do all sorts of unanticipated things, sure. But this
doesn't mean that we shouldn't consider how we may best encourage users to
avoid _unintentional_ common pitfalls.

There are options to explore here. Consider, for example--and I'm not
suggesting that we be this verbose, but it is illustrative of the
trade-offs which are possible to keep in mind--if certain methods were very
clear that the result is *pseudorandom* and potentially *insecure*:
`(0..<9).insecurePseudorandomElement`. Clearly, fewer people would use this
method for crypto.

But what *should* they use instead of our API? The OS-provided CSPRNG is
almost certainly going to be the most secure thing available in the absence
of specialized hardware. We should not deliberately scare users away from
our API if there is nothing better on offer.

+1.

Even if we tried to make an RNG for crypto use it would not be widely used for crypto because it would not be good enough for someone who really really cares. At the same time, general users would suffer due to the design decisions aligned at making it suitable for crypto.

In the end, it would not serve either audience well. It is better to acknowledge up front that the goal of the stdlib is “general use” and optimize to ensure that use case is well served.

-Chris

···

On Dec 2, 2017, at 6:02 PM, Xiaodi Wu via swift-evolution <swift-evolution@swift.org> wrote:

My point is that our entire design process has been geared towards a reasonable, best-effort general-use API: It is being designed by community members who are not specialized in this particular subfield. It is explicitly being design for common, general use cases in mind. And the implementation will come with no guarantee as to suitability for crypto, nor should it have to.

I like that you're considering the balance here. I've been lightly following this thread and want to add my thoughts on keeping crypto and pseudorandomness out of the name of at least one random API intended for general use.

For someone who doesn't know or care about the subtleties of insecure or pseudorandom numbers, I'm not sure that the name insecureRandom is effectively much different than badRandom, at least in terms of the information it conveys to non-experts. To Greg's point, that's the opposite of the signal that the API name should suggest because it's what most people should use most of the time. As you say, this API is being designed for general use.

There's a cost to adding extra complexity to names, too. I don't think it's far-fetched to suspect that people who find insecureRandom in an autocomplete listing or search will think "Where's the plain random function?"... and then go looking for a community extension that will inevitably provide a trivial alias: func random() { return insecureRandom() }. That's the sort of adoption I'd expect from something for new programmers, like Swift Playgrounds. Someone's introduction to randomness in programming should probably involve no more than a straightforward mapping from the elementary definition, rather than forcing a teaching moment from more advanced math.

I think there are better places for caveat information than in the API names themselves; documentation being one clear destination. This is in contrast with Unsafe*Pointer, where the safety element is critical enough to be elevated to be more than caveat-level information. You can go really far and create really cool things before these caveats start to apply. Using randomness as a black box in an intro programming environment seems like a much more common scenario than someone attempting to roll their first crypto by only reading API names and hoping for the best.

-Kyle

···

On Dec 2, 2017, at 6:02 PM, Xiaodi Wu via swift-evolution <swift-evolution@swift.org> wrote:

Instead, we ought to make clear to users both the features and the limitations of this API, to encourage use where suitable and to discourage use where unsuitable.

I don’t have much to say about this other than that I think the discussion seems way too narrow, focusing on spelling rather than on functionality and composability. I consider the “generic random number library” design to be a mostly-solved problem, in the C++ standard library (http://en.cppreference.com/w/cpp/numeric/random\). Whatever goes into the Swift standard library does not need to have all those features right away, but should support being extended into something having the same general shape. IMO the right design strategy is to implement and use a Swift version of C++’s facilities and only then consider proposing [perhaps a subset of] that design for standardization in Swift.

···

Sent from my iPad

On Dec 2, 2017, at 5:12 PM, Kyle Murray via swift-evolution <swift-evolution@swift.org> wrote:

On Dec 2, 2017, at 6:02 PM, Xiaodi Wu via swift-evolution <swift-evolution@swift.org> wrote:

Instead, we ought to make clear to users both the features and the limitations of this API, to encourage use where suitable and to discourage use where unsuitable.

I like that you're considering the balance here. I've been lightly following this thread and want to add my thoughts on keeping crypto and pseudorandomness out of the name of at least one random API intended for general use.

For someone who doesn't know or care about the subtleties of insecure or pseudorandom numbers, I'm not sure that the name insecureRandom is effectively much different than badRandom, at least in terms of the information it conveys to non-experts. To Greg's point, that's the opposite of the signal that the API name should suggest because it's what most people should use most of the time. As you say, this API is being designed for general use.

There's a cost to adding extra complexity to names, too. I don't think it's far-fetched to suspect that people who find insecureRandom in an autocomplete listing or search will think "Where's the plain random function?"... and then go looking for a community extension that will inevitably provide a trivial alias: func random() { return insecureRandom() }. That's the sort of adoption I'd expect from something for new programmers, like Swift Playgrounds. Someone's introduction to randomness in programming should probably involve no more than a straightforward mapping from the elementary definition, rather than forcing a teaching moment from more advanced math.

I think there are better places for caveat information than in the API names themselves; documentation being one clear destination. This is in contrast with Unsafe*Pointer, where the safety element is critical enough to be elevated to be more than caveat-level information. You can go really far and create really cool things before these caveats start to apply. Using randomness as a black box in an intro programming environment seems like a much more common scenario than someone attempting to roll their first crypto by only reading API names and hoping for the best.

-Kyle
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

1 Like

I'd like to add a pointer to the information here:
http://xoroshiro.di.unimi.it

since AFAICS, the xoroshiro128+ generator and the method of "Generating
uniform doubles in the unit interval" should probably be implemented in any
modern general purpose Random API.

Please correct me if there are more up to date (higher quality and faster)
general purpose generators and ways of converting UInt64 bit patterns to
floating point [0, 1).

/Jens

···

On Sun, Dec 3, 2017 at 4:50 AM, Dave Abrahams via swift-evolution < swift-evolution@swift.org> wrote:

I don’t have much to say about this other than that I think the discussion
seems way too narrow, focusing on spelling rather than on functionality and
composability. I consider the “generic random number library” design to be
a mostly-solved problem, in the C++ standard library (
http://en.cppreference.com/w/cpp/numeric/random\). Whatever goes into the
Swift standard library does not need to have all those features right away,
but should support being extended into something having the same general
shape. IMO the right design strategy is to *implement and use* a Swift
version of C++’s facilities and only then consider proposing [perhaps a
subset of] that design for standardization in Swift.

Sent from my iPad

On Dec 2, 2017, at 5:12 PM, Kyle Murray via swift-evolution < > swift-evolution@swift.org> wrote:

On Dec 2, 2017, at 6:02 PM, Xiaodi Wu via swift-evolution < > swift-evolution@swift.org> wrote:

Instead, we ought to make clear to users both the features and the
limitations of this API, to encourage use where suitable and to discourage
use where unsuitable.

I like that you're considering the balance here. I've been lightly
following this thread and want to add my thoughts on keeping crypto and
pseudorandomness out of the name of at least one random API intended for
general use.

For someone who doesn't know or care about the subtleties of insecure or
pseudorandom numbers, I'm not sure that the name insecureRandom is
effectively much different than badRandom, at least in terms of the
information it conveys to non-experts. To Greg's point, that's the opposite
of the signal that the API name should suggest because it's what most
people should use most of the time. As you say, this API is being designed
for general use.

There's a cost to adding extra complexity to names, too. I don't think
it's far-fetched to suspect that people who find insecureRandom in an
autocomplete listing or search will think "Where's the plain random
function?"... and then go looking for a community extension that will
inevitably provide a trivial alias: func random() { return
insecureRandom() }. That's the sort of adoption I'd expect from something
for new programmers, like Swift Playgrounds. Someone's introduction to
randomness in programming should probably involve no more than a
straightforward mapping from the elementary definition, rather than forcing
a teaching moment from more advanced math.

I think there are better places for caveat information than in the API
names themselves; documentation being one clear destination. This is in
contrast with Unsafe*Pointer, where the safety element is critical enough
to be elevated to be more than caveat-level information. You can go really
far and create really cool things before these caveats start to apply.
Using randomness as a black box in an intro programming environment seems
like a much more common scenario than someone attempting to roll their
first crypto by only reading API names and hoping for the best.

-Kyle

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

xoroshiro128+ is not a cryptographically secure algorithm and would not be
incorporated into the Random API, though it is trivial to implement your
own; the proposal outlines sources of randomness that are cryptographically
secure.

···

On Wed, Dec 20, 2017 at 09:46 Jens Persson via swift-evolution < swift-evolution@swift.org> wrote:

I'd like to add a pointer to the information here:
http://xoroshiro.di.unimi.it

since AFAICS, the xoroshiro128+ generator and the method of "Generating
uniform doubles in the unit interval" should probably be implemented in any
modern general purpose Random API.

Please correct me if there are more up to date (higher quality and faster)
general purpose generators and ways of converting UInt64 bit patterns to
floating point [0, 1).

/Jens

On Sun, Dec 3, 2017 at 4:50 AM, Dave Abrahams via swift-evolution < > swift-evolution@swift.org> wrote:

I don’t have much to say about this other than that I think the
discussion seems way too narrow, focusing on spelling rather than on
functionality and composability. I consider the “generic random number
library” design to be a mostly-solved problem, in the C++ standard
library (http://en.cppreference.com/w/cpp/numeric/random\). Whatever
goes into the Swift standard library does not need to have all those
features right away, but should support being extended into something
having the same general shape. IMO the right design strategy is to *implement
and use* a Swift version of C++’s facilities and only then consider
proposing [perhaps a subset of] that design for standardization in Swift.

Sent from my iPad

On Dec 2, 2017, at 5:12 PM, Kyle Murray via swift-evolution < >> swift-evolution@swift.org> wrote:

On Dec 2, 2017, at 6:02 PM, Xiaodi Wu via swift-evolution < >> swift-evolution@swift.org> wrote:

Instead, we ought to make clear to users both the features and the
limitations of this API, to encourage use where suitable and to discourage
use where unsuitable.

I like that you're considering the balance here. I've been lightly
following this thread and want to add my thoughts on keeping crypto and
pseudorandomness out of the name of at least one random API intended for
general use.

For someone who doesn't know or care about the subtleties of insecure or
pseudorandom numbers, I'm not sure that the name insecureRandom is
effectively much different than badRandom, at least in terms of the
information it conveys to non-experts. To Greg's point, that's the opposite
of the signal that the API name should suggest because it's what most
people should use most of the time. As you say, this API is being designed
for general use.

There's a cost to adding extra complexity to names, too. I don't think
it's far-fetched to suspect that people who find insecureRandom in an
autocomplete listing or search will think "Where's the plain random
function?"... and then go looking for a community extension that will
inevitably provide a trivial alias: func random() { return
insecureRandom() }. That's the sort of adoption I'd expect from
something for new programmers, like Swift Playgrounds. Someone's
introduction to randomness in programming should probably involve no more
than a straightforward mapping from the elementary definition, rather than
forcing a teaching moment from more advanced math.

I think there are better places for caveat information than in the API
names themselves; documentation being one clear destination. This is in
contrast with Unsafe*Pointer, where the safety element is critical
enough to be elevated to be more than caveat-level information. You can go
really far and create really cool things before these caveats start to
apply. Using randomness as a black box in an intro programming environment
seems like a much more common scenario than someone attempting to roll
their first crypto by only reading API names and hoping for the best.

-Kyle

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

I created a playground to explore this question, starting with a minimal subset of the proposal’s additions and building from there. The attached playground demonstrates what’s possible with this subset on the first page, then uses subsequent pages to explore how the main random facilities of the C++ STL work under this model. (In my opinion, they work pretty well!)

The subset in the playground has three main differences from the proposal:
- It doesn't include a Randomizable protocol or a random property on numeric types.
- It doesn't include the static random(in:) methods on numeric types, either.
- The RandomNumberGenerator protocol doesn't have an associated type. Instead, it requires all conforming types to produce UInt64 values.

I’ve tried to include a bit of real-world usage in the playground to demonstrate what writing code would look like with these additions. Please take a look!

Nate

Random.playground.zip (40.5 KB)

···

On Dec 2, 2017, at 9:50 PM, Dave Abrahams via swift-evolution <swift-evolution@swift.org> wrote:

I don’t have much to say about this other than that I think the discussion seems way too narrow, focusing on spelling rather than on functionality and composability. I consider the “generic random number library” design to be a mostly-solved problem, in the C++ standard library (http://en.cppreference.com/w/cpp/numeric/random\). Whatever goes into the Swift standard library does not need to have all those features right away, but should support being extended into something having the same general shape. IMO the right design strategy is to implement and use a Swift version of C++’s facilities and only then consider proposing [perhaps a subset of] that design for standardization in Swift.

Sent from my iPad

On Dec 2, 2017, at 5:12 PM, Kyle Murray via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Dec 2, 2017, at 6:02 PM, Xiaodi Wu via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Instead, we ought to make clear to users both the features and the limitations of this API, to encourage use where suitable and to discourage use where unsuitable.

I like that you're considering the balance here. I've been lightly following this thread and want to add my thoughts on keeping crypto and pseudorandomness out of the name of at least one random API intended for general use.

For someone who doesn't know or care about the subtleties of insecure or pseudorandom numbers, I'm not sure that the name insecureRandom is effectively much different than badRandom, at least in terms of the information it conveys to non-experts. To Greg's point, that's the opposite of the signal that the API name should suggest because it's what most people should use most of the time. As you say, this API is being designed for general use.

There's a cost to adding extra complexity to names, too. I don't think it's far-fetched to suspect that people who find insecureRandom in an autocomplete listing or search will think "Where's the plain random function?"... and then go looking for a community extension that will inevitably provide a trivial alias: func random() { return insecureRandom() }. That's the sort of adoption I'd expect from something for new programmers, like Swift Playgrounds. Someone's introduction to randomness in programming should probably involve no more than a straightforward mapping from the elementary definition, rather than forcing a teaching moment from more advanced math.

I think there are better places for caveat information than in the API names themselves; documentation being one clear destination. This is in contrast with Unsafe*Pointer, where the safety element is critical enough to be elevated to be more than caveat-level information. You can go really far and create really cool things before these caveats start to apply. Using randomness as a black box in an intro programming environment seems like a much more common scenario than someone attempting to roll their first crypto by only reading API names and hoping for the best.

-Kyle
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Oh OK, I must have misunderstood this thread:
https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20171204/042034.html
(
"The strong opinion of the core team is that such an API should *not* be
designed with an attempt to service people writing crypto code."
"It is general goodness if generality for crypto use cases somehow falls
out of the design. However, the design for general use shouldn’t suffer
because of that goal."
)

I assumed that the Random API would save the user from the trouble of
making a good choice and implementation (fast and good quality) of a
"standard" general purpose prng (as well as maybe a cryptographically
secure one).

Also, the most commonly recommended ways of converting from eg 64 random
bits to an int range or a floating point range are unnecessarily bad and
slow, so I figured the webpage was worth a read, in addition to C++
stdlib's implementation.

/Jens

···

On Wed, Dec 20, 2017 at 4:55 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

xoroshiro128+ is not a cryptographically secure algorithm and would not be
incorporated into the Random API, though it is trivial to implement your
own; the proposal outlines sources of randomness that are cryptographically
secure.

On Wed, Dec 20, 2017 at 09:46 Jens Persson via swift-evolution < > swift-evolution@swift.org> wrote:

I'd like to add a pointer to the information here:
http://xoroshiro.di.unimi.it

since AFAICS, the xoroshiro128+ generator and the method of "Generating
uniform doubles in the unit interval" should probably be implemented in any
modern general purpose Random API.

Please correct me if there are more up to date (higher quality and
faster) general purpose generators and ways of converting UInt64 bit
patterns to floating point [0, 1).

/Jens

On Sun, Dec 3, 2017 at 4:50 AM, Dave Abrahams via swift-evolution < >> swift-evolution@swift.org> wrote:

I don’t have much to say about this other than that I think the
discussion seems way too narrow, focusing on spelling rather than on
functionality and composability. I consider the “generic random number
library” design to be a mostly-solved problem, in the C++ standard
library (http://en.cppreference.com/w/cpp/numeric/random\). Whatever
goes into the Swift standard library does not need to have all those
features right away, but should support being extended into something
having the same general shape. IMO the right design strategy is to *implement
and use* a Swift version of C++’s facilities and only then consider
proposing [perhaps a subset of] that design for standardization in Swift.

Sent from my iPad

On Dec 2, 2017, at 5:12 PM, Kyle Murray via swift-evolution < >>> swift-evolution@swift.org> wrote:

On Dec 2, 2017, at 6:02 PM, Xiaodi Wu via swift-evolution < >>> swift-evolution@swift.org> wrote:

Instead, we ought to make clear to users both the features and the
limitations of this API, to encourage use where suitable and to discourage
use where unsuitable.

I like that you're considering the balance here. I've been lightly
following this thread and want to add my thoughts on keeping crypto and
pseudorandomness out of the name of at least one random API intended
for general use.

For someone who doesn't know or care about the subtleties of insecure or
pseudorandom numbers, I'm not sure that the name insecureRandom is
effectively much different than badRandom, at least in terms of the
information it conveys to non-experts. To Greg's point, that's the opposite
of the signal that the API name should suggest because it's what most
people should use most of the time. As you say, this API is being designed
for general use.

There's a cost to adding extra complexity to names, too. I don't think
it's far-fetched to suspect that people who find insecureRandom in an
autocomplete listing or search will think "Where's the plain random
function?"... and then go looking for a community extension that will
inevitably provide a trivial alias: func random() { return
insecureRandom() }. That's the sort of adoption I'd expect from
something for new programmers, like Swift Playgrounds. Someone's
introduction to randomness in programming should probably involve no more
than a straightforward mapping from the elementary definition, rather than
forcing a teaching moment from more advanced math.

I think there are better places for caveat information than in the API
names themselves; documentation being one clear destination. This is in
contrast with Unsafe*Pointer, where the safety element is critical
enough to be elevated to be more than caveat-level information. You can go
really far and create really cool things before these caveats start to
apply. Using randomness as a black box in an intro programming environment
seems like a much more common scenario than someone attempting to roll
their first crypto by only reading API names and hoping for the best.

-Kyle

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

I much prefer the API from Nate Cook compared to the previous proposal. Its simpler, while still very powerful, and closer to Swift conventions (method instead of property).

···

On 8 Jan 2018, at 20:02, Nate Cook via swift-evolution <swift-evolution@swift.org> wrote:

I created a playground to explore this question, starting with a minimal subset of the proposal’s additions and building from there. The attached playground demonstrates what’s possible with this subset on the first page, then uses subsequent pages to explore how the main random facilities of the C++ STL work under this model. (In my opinion, they work pretty well!)

The subset in the playground has three main differences from the proposal:
- It doesn't include a Randomizable protocol or a random property on numeric types.
- It doesn't include the static random(in:) methods on numeric types, either.
- The RandomNumberGenerator protocol doesn't have an associated type. Instead, it requires all conforming types to produce UInt64 values.

I’ve tried to include a bit of real-world usage in the playground to demonstrate what writing code would look like with these additions. Please take a look!

Nate

<Random.playground.zip>

On Dec 2, 2017, at 9:50 PM, Dave Abrahams via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

I don’t have much to say about this other than that I think the discussion seems way too narrow, focusing on spelling rather than on functionality and composability. I consider the “generic random number library” design to be a mostly-solved problem, in the C++ standard library (http://en.cppreference.com/w/cpp/numeric/random\). Whatever goes into the Swift standard library does not need to have all those features right away, but should support being extended into something having the same general shape. IMO the right design strategy is to implement and use a Swift version of C++’s facilities and only then consider proposing [perhaps a subset of] that design for standardization in Swift.

Sent from my iPad

On Dec 2, 2017, at 5:12 PM, Kyle Murray via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Dec 2, 2017, at 6:02 PM, Xiaodi Wu via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Instead, we ought to make clear to users both the features and the limitations of this API, to encourage use where suitable and to discourage use where unsuitable.

I like that you're considering the balance here. I've been lightly following this thread and want to add my thoughts on keeping crypto and pseudorandomness out of the name of at least one random API intended for general use.

For someone who doesn't know or care about the subtleties of insecure or pseudorandom numbers, I'm not sure that the name insecureRandom is effectively much different than badRandom, at least in terms of the information it conveys to non-experts. To Greg's point, that's the opposite of the signal that the API name should suggest because it's what most people should use most of the time. As you say, this API is being designed for general use.

There's a cost to adding extra complexity to names, too. I don't think it's far-fetched to suspect that people who find insecureRandom in an autocomplete listing or search will think "Where's the plain random function?"... and then go looking for a community extension that will inevitably provide a trivial alias: func random() { return insecureRandom() }. That's the sort of adoption I'd expect from something for new programmers, like Swift Playgrounds. Someone's introduction to randomness in programming should probably involve no more than a straightforward mapping from the elementary definition, rather than forcing a teaching moment from more advanced math.

I think there are better places for caveat information than in the API names themselves; documentation being one clear destination. This is in contrast with Unsafe*Pointer, where the safety element is critical enough to be elevated to be more than caveat-level information. You can go really far and create really cool things before these caveats start to apply. Using randomness as a black box in an intro programming environment seems like a much more common scenario than someone attempting to roll their first crypto by only reading API names and hoping for the best.

-Kyle
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution