SE-0329 (Second Review): Clock, Instant, and Duration

Evolution Proposal Reviews
61

Do you have a concrete example of such library so that we can work through what APIs it would need and why Foundation can’t be imported?

I can see only two possibilities here based on your description of the hypothetical:

  • If the timestamp is used only to order entries generated by the same logger relative to each other, then any Instant will do and the tool which is processing the data doesn’t need to know about the epoch from which the timestamp is measured or the way elapsed time is counted—Date is not required.

  • If the timestamp is used to order entries generated by arbitrarily many loggers relative to each other, or is to be interpreted for any other eventual use (whether by a calendrically-aware API or not) that requires reconstructing a representation of the same instant in time on the tool-processing machine that the logger intended to log, then the epoch needs to be known as well as the way elapsed time is counted—since timestamps serialized from instants of type Date relative to a wall clock don’t have the same behavior from machine to machine, it is not suitable for this purpose.

1 Like
62

There are many other use cases than military-grade ordering. Just, like, "what time/day was it on the user's machine when he did this action?" :man_shrugging: People have ideas.

63

As @ksluder alluded to, relativistic concerns are applicable to GPS and also other embedded systems in some cases. For example: a control system for a robot.

The best approach doesn't necessarily need to explicitly address relativistic concerns. However whichever approach is decided upon should still be expressive enough for Swift running on embedded systems to use or build upon in the future.

64

I think so! Duration is set to become the standard way to represent physical time intervals across API boundaries; it ought to be able to cover time intervals that we need to routinely deal with in practice.

In the proposed implementation, Duration consists of 128 bits, and it seems wasteful not to make better use of all this vast space. (seconds + nanoseconds leaves ~34 bits unused. Switching to an Int64+Int32 scheme doesn't feel like a great tradeoff, as the extra four bytes will likely be wasted on padding anyway.)

I remember that nanoseconds used to be esoteric in the exact same way. SI units are easy to learn and trivial to look up via any search engine.

Furthermore, the Duration API can still provide the proposed nanosecond property and conversion methods that take nanosecond values. All I'm suggesting is to (1) change & document the internal representation to achieve a more usable resolution, and (2) to also provide attosecond-based factories and member properties.

Whatever representation we adopt in the stdlib now will be forever frozen as the representation of durations in the two standard system clocks. As I wrote above, timer resolutions are already dangerously close to the 1ns cutoff, and it seems reasonable to expect that they would go below that in the foreseeable future. Even if we don't care to support iterative benchmarking or simulation use cases, I feel we'd be neglectful if we did not build in ample room to cover such easily foreseeable hardware improvements -- especially as I don't see any technical reason that would force us to use a coarser Duration type.

7 Likes
65

Yes, but once again, such use case doesn't requires a full implementation of the Clock API, as there is no need for Duration, sleep support, etc…

If you just want to be able to generate a Unix Timestamp, there is already a function for that.

66

I have strong doubts that adding a third property to Duration would be beneficial — the integer/fraction split is easy to understand and has prior art in struct timespec. I'd much rather see a potential attoseconds property replace nanoseconds instead of supplement it.

67

Using full 128 bits to store a timestamp is supposed to be the overkill solution where really no one expects that the the granularity will ever be actually necessary, isn't it?
So I don't expect it will be important to have exact base-10 representations for the the tasks this proposal is all about (scheduling, measuring).
Surely, for things like scientific calculations, the situation is different — but imo it would be ok if you had to define a specific type which has all the specific properties required for the actual problem.

Although it might be that in the near future, no one will use microcontrollers anymore because it's cheaper to cut development costs and spend some more money on Linux-ready hardware instead.
Still, I think it would nice if Swift would not need a dialect for platforms which have no clock on board.

68

Since this would be division by a constant, it's actually not all that bad, just shifts and multiplications.

Is there a reason to think that extracting whole seconds would be remarkably more common than extracting milliseconds or nanoseconds?

4 Likes
69

[Note: This is my position as a Standard Library engineer; I'm not speaking for the project as a whole.]

The way I see it, we have an opportunity (& responsibility) to be selective about what goes in the Standard Library. The concurrency APIs have introduced the concept of time within the universe of the core stdlib, so I believe there is a pressing need to add proper abstractions for the system clocks. I believe this proposal answers that need very nicely.

As far as I can tell, we don't have such a forcing function for time-of-day / wall clock conversions, so we have the luxury to leave that off the table for now, as far as the core stdlib is concerned. The proposal is eminently extensible -- it allows us to implement additional flavors of clocks inside and outside the stdlib that integrate just as well.

A wall clock is a complicated addition that we can therefore leave for later. Or it may even make sense to leave it out of the (core) Standard Library indefinitely -- after all, we probably wouldn't want the stdlib to become a huge monolithic library... :wink: (Executing a task 300 seconds from now, executing it on 2022-02-01T12:00:00Z and executing it at 1am next Wednesday are three very different problems, with exponentially increasing complexity -- and some of this takes much more than simply calling the right syscalls, at least on some platforms.) We may easily end up needing a standard UTC timestamp type in the stdlib -- but if so, we will have the opportunity to work that out in a separate Swift Evolution proposal.

We are very lucky that we have Foundation already covering this area, and it's great that it's gaining support for leap seconds. (Concerns about importing the whole of Foundation to gain access to Date feel reasonable to me (on some platforms at least), but it's a resolvable technical obstacle that's best discussed in a different topic than this one. It also underlines that we may not want the stdlib to go down the same path.)

I think it would make sense for folks who'd like to work this out from first principles to create the right APIs & implementation in a package that builds on this proposal.

12 Likes
70

So, the hypothetical use case here is one where we’d like to know from a remote machine what calendar time/day it was on the user’s machine at a certain prior instant, but instead of having the user’s machine transmit this information, we stipulate that the user’s machine must instead compute a timestamp and then transmit that over the wire to another machine which then—imperfectly—approximates the user’s machine’s calendrical APIs to get a rough idea of what the user regarded to be their local time, but not really because we would also not have timezone information?

71

The current implementation is attosecond storage via a double wide Int64. Given a slew of nanosecond scale values it will be able to divide appropriately to attoseconds and offer the correct value at that scale. Now adding accessors or initializers for that is a different story; because it would require a 128 bit value to be exposed. Which we currently do not have without some sort of new numeric type.

1 Like
72

What is your evaluation of the proposal?

+0. Time is hard

Does this proposal fit well with the feel and direction of Swift?

For the most part. It is unfortunate that the most useful clock (UTCClock) for interacting across the network is in Foundation; this makes it likely we will see several different implementations of system clocks, possibly referencing the same underlying system API, each used within portions of a larger application by global reference.

This AFAICT is done to enable use of the Foundation-defined Date type, rather than a new more-accurately named type which is also convertible to Date/NSDate. However, the ramifications are that other time and calendaring features will likely be excluded from foundation in the future. I'd still far prefer e.g. a Swift.UTCInstant type.

My interpretation of this proposal is that a Clock defines a unit of measurement of time of some sort - atomic time, UTC time with omitted or duplicated seconds, UTC with elided seconds, local system uptime, system non-suspended runtime, individual CPU uptime, frame count, etc.

Duration should not be a concrete type if it is not measuring a single thing. It appears that Duration is being used to represent each of continuous time, suspended time, and UTC time. Would my task sleep(for:) 100 continuous seconds, 100 non-suspended seconds, 99 seconds because or a negative leap second, or possibly just a single second because the system clock got updated at a poor time?

That Durations are unit types is illustrated by the called-out lack of multiplication. But the actual unit of the concrete Duration type is unspecified and different per clock and per platform. The Clock and Instant protocols allow for specific durations to be present for specific units, and that should be leveraged more - even if just by wrapping the internal Duration type for these cases.

(Of course, the clock Duration itself may be defined in terms of a mix of units - e.g. a nanosecond may be atomic nanoseconds while the seconds are UTC seconds, and one second could potentially contain zero or two billion nanoseconds. Time sucks.)

This proposal only defines Clocks in terms of locally measurable terms. The lack of a stable cross-platform unit of measure means that Clocks themselves need to have their own source of measurements (e.g. a local synthetic or hardware basis). I could define a TAIInstant and TAIDuration easily enough, but a TAIClock would require going significantly beyond what is supplied in this proposal, with per-platform behavior.

If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?

While there are several parts of the Swift proposal which I like above the Java JSR 310 specification, I would point to the definition of Instant in that specification as an example of what I would like to see documented for e.g. UTCClock.

https://docs.oracle.com/javase/8/docs/api/java/time/Instant.html

Note while the Java approach is simplified (it only defines the one Clock/Instant/Duration behavior, vs ), the ability to read the equivalent to UTC Clock and all other temporal operations such as calendaring are part of the always required java.base module, vs being broken out into a separate time module and excluded for e.g. compact/embedded profiles.

How much effort did you put into your review? A glance, a quick reading, or an in-depth study?

It felt like a fair bit, but time is relative and the interaction of time standards relatively insane.

2 Likes
73

In theory, sure. In practice, if Foundation adds it, it becomes part of the toolchain and suddenly there are astronomical hurdles for deviating even slightly.

Consider the discussion about including Date in the standard library. All the community asked for was a different name -- even offering compromises such as keeping the old name around. It's just a storage type; it doesn't even need to bother defining what happens to leap seconds (that's a clock problem). Now it's been ripped from the proposal entirely and we all lose from that.

Based on that and similar experiences in the past (e.g. Data, ContiguousBytes), the chance that we add any sort of wall-clock or timestamp in the standard library will become effectively zero once Foundation ships their own thing. Just being realistic, here. It's a loophole in the evolution process, yet it constrains the ways in which the standard library is reasonably able to evolve.

4 Likes
74

A fully relativistic model of time is well out of scope, and is highly unlikely to fit any sort of interface we decide upon. For one thing, as far as modern physics is able to tell us, time is just one aspect of how our brains perceive four-dimensional spacetime; by itself, it is a flawed concept that does not really describe objective reality (consider the famous example of clocks at the top of skyscrapers running at different speeds to clocks at sea level). There is not even a universal ordering of non-causally-connected events, so even concepts like "past" and "future" break down and we need to start wondering whether event A can influence event B.

All very interesting, and extremely trippy, but not necessary for most applications. It's very much a specialised use-case.

I've already mentioned some - time-ordered UUIDs contain a timestamp which is a number of 100ns intervals since an epoch. The calendrical presentation of that instant is irrelevant.

Another example is timestamps from HTTP cookies. They start off as a string of Gregorian calendar components, but once parsed and resolved as a number of seconds after the unix epoch (using system APIs such as timegm), that original specification becomes irrelevant. All that matters is whether the system considers that instant to have passed (and the cookie expired) or not. And yeah, at the same instant, another system may disagree whether that instant has passed - for all sorts of reasons (including the user just manually setting the clock).

As for the "which cannot import Foundation" line, I just reject the entire premise of that question. It's up to you to argue why these use-cases, which do not require any functionality from Foundation, must depend on it regardless, just to be able to wrap their data in a storage type named "Date". Why can't the standard library even provide the storage type?

3 Likes
75

But they do. To generate UUIDs, you need to generate timestamps, which requires a wall time clock. If you don't want to generate UUIDs you don't even need to know there is a timestamp in the ID.

Cookie handling needs to both parse the date (and the proper place for wrapping system API such as timegm is not the stdlib, but Foundation or swift-system or sth like that), as well as a wall time clock to determine (even if wildly inaccurately) whether that instant has passed.

If all you really want is type with which to store a time-difference (in some unspecified interpretation of seconds) from an unspecified fixed reference date, we need waste no more time discussing this, as there already exists a type in the standard library to fit your criteria: Double. In fact, Int applies as well, and String could probably be mishandled to serve the purpose.

But that's nonsensical of course, for the same reason that including just a "storage type" without defining the epoch, flavour of seconds (and probably a clock and calendar that share those definitions) doesn't really make sense. A type isn't just the shape of what you can stick into it, it's the semantics of what's contained in it as well.

1 Like
76

With respect, I disagree with every word.

None of this is your place to say. By attempting to look beyond the problem, you have instead entirely missed the point.

I never said that was what I was looking for.

The fact remains that, without a currency type to communicate "duration since a reference point", the Swift standard library will be uniquely inadequate compared to all of its peers. Even the Rust standard library (which includes no calendrical operations whatsoever) defines a SystemTime type. And that type? Lo and behold - it's exactly what I'm looking for.

The examples I gave are just two instances that I have personally encountered within the last ~6 weeks. As a library developer, I'm constantly finding the Swift standard library to be deficient, and I'm getting tired of wishing I was writing Rust. I'm bringing it up because this is supposed to be the place where we voice those concerns and work towards filling those gaps in the language and its standard library.

It's a real problem that library authors are facing today, and saying "just use Foundation" is not good enough. The users of my libraries tell me they don't want that, and I am far from alone in receiving that kind of feedback. It is a very common request.

But the point is not to rant about Foundation - it is to recognise that the standard library is lacking and failing developers such as myself. And yes, to recognise that many of the concerns we have are effectively made moot by the presence of Foundation in every Swift toolchain. So saying "we'll do nothing" is also not good enough.

Some of you have been persistent in demanding examples from me, but I have yet to hear an answer to my counter-demand: show me one other language/library with this same deficiency.

We are about to add a Clock API to the standard library which cannot tell the time. Only in Swift.

1 Like
77

Well then, please do elaborate. Assuming there is something like struct NewDate { var interval: Double } in the standard library, but no real time clock, no parsing mechanism and no utility for calendrical calculations, what can you do with UUIDs and Cookies that you can't do in the absence of NewDate?

That's kind of comparing apples to oranges and may shine a light on what may be the source of this debate: The definition of "standard library".

The Rust std module contains a vast amount of functionality that, in my estimation at least, would never make it into the Swift standard library. There is TCP/UDP networking, subprocess handling, API for working with threads (edit: whoops, I forgot file IO), and, as you say, time related API. As the docs say, "Besides basic data types, the standard library is largely concerned with abstracting over differences in common platforms, most notably Windows and Unix derivatives."

A closer analogy to the Swift standard library might be the core module, which aims to be the same on every platform supported by Rust, including embedded platforms (where std::time is not available) and which aims to be a "dependency-free foundation of The Rust Standard Library".

And by no means am I saying "just use Foundation". In fact a Clock implementation in swift-system would probably be appropriate and useful especially for those that wish to not import Foundation for whatever reason. I am, however, saying this: Including in the Swift standard library a "storage type" with undefined semantics and no functionality to actually use it it does not make sense to me.

4 Likes
78

Easy: pass values to clients using a common currency type.

Again, taking an example from Rust, you have the Chrono crate, which was preceded by datetime-rs and rust-datetime crates. None of them were ever part of the standard library. Taking a brief look at them, it appears they might use strftime rather than ICU, so their localisation support is more limited, but therefore comes at a much lower cost.

And that's the point: it is up to the client to decide which functionality they need, which libraries they want to source that functionality from, and they are able to depend on only those libraries without a monolithic unwanted extra library tagging along.

Rust has a different split to Swift. The core module is not really like the Swift standard library at all; it doesn't even support malloc, so it has no dynamically-sized arrays or strings.

The point is that a timestamp type, even without calendrical functionality, is still a useful building block for the rest of the ecosystem.

Anyway, I've said enough. I think I've made my point, and that it is clear what my opinion of this proposal is. It's also important to leave room for others to share their opinions.

79

This proposal contains instant types and duration type capable of representing the number of 100-ns intervals from a point in time. For what reason is Date necessary in this case?

Determining if a string of Gregorian calendar components represents an instant in the past is, to my understanding, paradigmatically a calendrical operation. That you can perform the operation in two steps, where the first invokes a system API instead of analogous Foundation APIs, doesn’t really change that.

Nope, it’s not up to me to argue anything—I’m fully content with the proposal as-is. Foundation is a core library currently available on all platforms that Swift is available. You can feel free not to import Foundation for any reason or no reason at all—and, indeed, you don’t owe anyone an explanation. That said, the extent to which your choice is without explanation is in equal measure the extent to which it’s not anyone else’s problem to solve.

The documentation you link to says that the type is “useful for talking to external entities like the file system or other processes,” which as @ahti has said is a concern that’s primarily addressed in the Swift ecosystem by libraries other than the standard library—by design. You can disagree with that overarching decision but it’s rather firmly outside the scope of this proposal review.

Agree. I suppose the crux of your argument is that useful building blocks don’t belong in Foundation—but it’s hard to see why Foundation would be a “core library” at all if its function were anything other than to provide useful building blocks.

4 Likes
80

Well, I think it's known by now some influential libraries don't want to depend on Foundation and will therefore use their own type, just like they are avoiding Data. And having different types representing the same thing in different libraries creates dialects and makes it harder to use them together.

But if this type was in the standard library, it'd be awkward to have no way to print it as a calendar date-time. Running print(timestamp) would be expected to print something more readable than a number of seconds and milliseconds. So a minimal calendaring would have to be included, or expectations will be broken.

There are valid arguments on both sides.