[Pitch] Clock, Instant, Date, and Duration

Only if WallClock.Duration and MonotonicClock.Duration are actually different underlying types. There’s still the middle ground option I mentioned above where clocks with compatible duration types could share their Duration implementations with the “trivial” nanosecond-based duration, while preventing their confusion in generic contexts without an appropriate constraint (and leaving the door open for “exotic” duration types).

Anyway, I won’t harp on this any more because I think the downside of the concrete Duration is likely pretty low, just wanted to do a bit of exploring of the trade offs we’re making. Thanks for indulging. :slight_smile:

2 Likes

Yes, that is precisely the advantage. The API can declare that it can work with eg. any monotonic clock and there can be many such implementations (eg. a “manual stepping” one as you mentioned up-thread).

Nah, this is totally a part of the exploration I did look into so your question is 100% reasonable. There is one "exotic" duration type that could be reasonable to have as non nanosecond based - particularly a "manually controlled clock" type. In that case expressing as nanoseconds is still pretty sound. It always seems suspect to me if only one type adopts a protocol and I have yet to find a clock type that would take something else (or at least one that wouldn't be wholesale incompatible w/ a nanosecond derived duration type).

1 Like

So beyond just a marker, what new functionality does it require or provide to the clock itself (not the call sites)? It is perhaps something to consider making marker protocols for those, but I am not fully sure that is 100% appropriate; I will need to ruminate on that for a bit.

This is a great pitch to read! Thanks a lot for putting so much effort into it and referring to the prior arts.

Clocks and Durations are an important concept and something that will supplement the current Concurrency concepts greatly.

The only concern I share that was already brought up is the usage of hours/days/etc. which might be misleading. I really like the suggestion from @xwu using “approximately”.

Other than that I would love to understand a bit more when it comes to two topics:

Testability of code using clocks:
In your examples, you were showing methods that accept clocks or even concrete associated durations. Oftentimes, when writing a method that invokes an API that is Async (like the timeout). You wanna be able to pass in a clock that executed immediately in your tests.

Scheduling w.r.t Concurrency:
As it was said in various of the previous Concurrency proposals, to provide custom schedulers and methods that delay AsyncStreams it would be required to have a notion of Time inside the stdlib. I would love to understand the future directions/impacts this proposal will have on Concurrency.

Probably both of my points will go into the same direction. I would just love to see the overall picture where this is going.

Great pitch!

6 Likes

I do not envision any extra exposed functionality — it would just guarantee the invariant clock.now() <= clock.now(), something that clocks that track wall time can't do.

To clarify, a marker protocol just for monotonic clocks is enough -- they're a strict subset of any other possible clock.

Basically in that case I would say that many testable scenarios would define the functionality in terms of a generic clock and then offer an overload that takes specific clock and instants or clock and duration. The test scenario then would invoke the generic one passing in the custom clock.

I don't have anything concrete yet, but this is definitely a fundamental that is required to do more advanced things with async/await and AsyncSequence. More concretely this is the first step to allow the fundamental of Task.sleep to be controlled by a deadline/duration rather than just monotonic nanoseconds. This has a distinct overlap when it comes to distributed actors. Namely of which a distributed actor can execute something with a WallClock deadline but cannot express it in terms of a UptimeClock deadline since the distributed actor may not be locally resident on the same machine (sharing the same uptime). But, furthermore, when you want to express complex algorithms like timeouts or throttling etc there are a few integral parts; namely you need a way of waking up a task after a given duration/deadline and a way of running two tasks and seeing which one returned first. We kind-of have the latter with continuations and some fancy dancing w/ atomics/locks etc but we don't have a standardized concept of time at that layer. This proposal opens the gateway for considering other things along these lines like having functions that transact w/ deadlines etc.

3 Likes

I have not read the entirety of the pitch or the comments (bad me!) but:

  1. I absolutely agree we should have top-tier time primitives in Swift; and
  2. I feel very strongly we should deprecate the existing C and Objective-C time types in Swift wherever practical. If there's an API that takes e.g. a struct timespec, we should expose it via the relevant overlay using the Swift-native time type instead. Swift has a lot of cruft it inherits automatically from C and we'd be well-served to do some housekeeping. :slight_smile:
11 Likes

There is a lot to unpack here! I'm very excited to see this proposal come up.

Overall, I love the direction this pitch is pointing. Unifying the various temporal APIs to be more expressive and easier to understand and use is awesome and I'm thrilled this is being looked at.

Getting on to specific feedback:

What is a Clock?

I'm going to push back on the definition of a clock, which is described as:

an item to provide a concept of now plus a way to wake up after a given point in time

I'd argue that a clock is only the first part: the provider of "now". "Waking up" is a task that something does after consulting a clock. The function gets a clock, sees what "now" is, looks at when "wake up" time is, computes that duration, and then sleeps for that duration.

As a real-world example, if I want to sleep for 8 hours, I look at "now" on my clock and then set an alarm for 8 hours after now, and then I sleep; not the clock.

Thus, the entire definition of ClockProtocol should be this:

public protocol ClockProtocol {
    associatedtype Instant: InstantProtocol
    func now() -> Instant
}

Sleeping is a task performed by the consumer of a clock; not the clock itself. The clock only tells you what time it is.

Clocks Provide the Concept of Now

This leads directly into the next point:

Why is static var now: Self on InstantProtocol? A clock tells you when "now" is; it is not something intrinsic to the type itself.

I see from later examples that having this enables short-hand syntax like .now.advanced(by: .seconds(3)), but fundamentally this is the relying on the same sort of global-and-hidden dependency that Date() does.

The opening definition of a clock is that "it provides a concept of now". That's good. If we want to know when "now" is, we should ask the clock. Yes it will make a couple of call-sites a little uglier or make something be two lines instead of one. But doing this would mean that we can re-use the same type of InstantProtocol between different clocks, which is useful.

Different Kinds of Clocks

I love ClockProtocol as a concept. In my own Time library, I have many different kinds of clocks, because manipulating time (especially while debugging) is really powerful. As a trivial example, if you want to test that something stays on-screen for 30 seconds and then disappears, you could do that by sitting around for 30 seconds and then seeing if it's gone, or you could use a clock that runs 30 times faster than normal and only wait around one second. Or if you're writing some code that wants to a cron-like job every 60 minutes, creating a clock that's offset from now so you can "jump ahead" to the proper time is another exceptionally handy thing.

Thus, the ability to create something like:

public struct ScaledClock<C: ClockProtocol>: ClockProtocol {
    public typealias Instant = C.Instant
    
    public let scaleFactor: Double
    public let baseClock: C
    ...
}

public struct OffsetClock<C: ClockProtocol>: ClockProtocol {
    public typealias Instant = C.Instant
    
    public let offset: Duration
    public let baseClock: C
    ...
}

... is immensely powerful and useful.

Where I run into an issue is with the associated type on Clock. I understand the rationale about wanting to have a compile-time differentiation between Monotonic and Uptime and Wall instants, but having that associated type makes injecting a Clock extremely difficult. I can no longer do...

class MyCronJobScheduler {
    var clock: ClockProtocol
}

... and then inject the proper dependency of a scaled, offset, or "normal" clock. Instead, I must make everything generic based on the Clock type, and that is painful.

Clocks like these also underscore why now needs to be on the Clock, and not the Instant type. Asking a clock that's scaling WallClock when now is would be a very very very different answer than asking the corresponding Instant type directly. It's also something that's dependent on the scaling factor (2x, 30x, 0.5x, etc) and is therefore not something that even could be implemented on a ScaledInstant type, unless you were willing to define a different type for every possible scaling factor, and down that path lies madness.

Converting Between Clocks

Having the ability to create custom clocks (which I've shown and experienced to be immensely useful) means that a clock technically has one other task: telling you how long a duration actually is.

Imagine a clock that's sped up by a factor of 2: for every 1 second that passes on the wall clock, two seconds pass on the scaled clock.

Let's also then say I want to perform something 10 seconds from now based on that clock. So, I get now() from the clock and advance the instant by 10 seconds.

I cannot Task.sleep for 10 real seconds, because that will be 20 seconds on the scaled clock. So, I need the clock itself to convert this into a "real time" Duration that includes any scaling factors.

This brings the (proposed) definition of ClockProtocol to this:

public protocol ClockProtocol {
    associatedtype Instant: InstantProtocol
    func now() -> Instant
    
    // default implementation returns the parameter
    // this would only need to be implemented by a clock that runs slower or faster than real time
    func absoluteDuration(for clockDuration: Duration) -> Duration
}

This is also a good example for why ClockProtocol can't have static methods for sleeping or computing durations. Different ClockProtocol concretions can't always have the requisite information available statically.

Human Temporal Units

I saw that Duration has these convenience bits:

extension Duration {
  public static func hours(_ hours: Int) -> Duration 
  public static func hours(_ hours: Double) -> Duration
  public static func minutes(_ minutes: Int) -> Duration
  public static func minutes(_ minutes: Double) -> Duration
}

I would recommend leaving these off. Seconds are one of the fundamental base units of measurement. Milliseconds, microseconds, and nanoseconds are easy extrapolations off that.

Minutes and hours are human constructs and are not part of the SI system. These sort of definitions should be at the Foundation level, where Calendar and Locale exist to give these names their proper meaning.


This is getting really long, so I'm going to cut it off here. Some of this stuff has been mentioned by others already, so I apologize for repeating some points.

Again, I'm really pleased that this is getting pitched and I'm excited to see this evolve.

31 Likes

First off that should be a property and not a function, but moreover the shorthand of being able to write .now.advanced(by: .seconds(5)) is an important use case. Placing now on the clock means that cannot be done. Your request for that could be trivially implemented as an extension on the clock in your own code just fine:

extension Clock {
  static var now: Instant { Instant.now }
}

That is no longer the case. Existentials can be now formed even if there are associated types. But I would guess that your API would be better served as a non-existential form and be generic upon the clock. Any API that would potentially have a differing clock. For example if I had a structure that managed debouncing events then I would be immediately inclined to make that type generic upon the clock (even if default construction of that used a specific clock).

Im not sure this is even implementable especially for the concept of monotonic versus uptime clocks since the sleep time is not known fully. Only an approximation can be made.

To be clear: decimal representation of seconds are the fundamental base unit of measurement, and NOT integer seconds. So the most granular atom of temporal scale we have is the nanosecond scale.

Except I gave multiple examples of how this fails. You can create clocks that have information that is required to compute "now" that is not available at the static level. You noted previously that "Absolutely!" people could be defining their own ClockProtocol-conforming types.

Therefore, if clocks can be custom, how can now be statically determined?

8 Likes

Lowering the now to a instance versus static scope then negates the leading type inference. Meaning that folks would need to write out: MonotonicClock().now.advanced(by: .seconds(5)) which is fairly gnarly in my book for syntax to require folks to write. Because now they must understand that the clock is constrained to a specific type (when the previous would just allow the short-hand).

I don't think we should really focus on pedantic edges when the cases of user clocks define their own temporal concepts. Offsetting a clock by some value or scaling it by some value don't seem like use cases that should drive the ergonomics of the rest of the APIs.

Non-starters:
Removing the shorthands of .now given the proper generic signature
Requiring some sort of inhabitant on the clock type (Monotonic, Uptime, and Wall clocks don't need any sort of storage currently)

We should aim for a progressive disclosure of specificity; meaning that an API could just specify a duration, a more complex version could express it as a specific concrete deadline via an instant, and then the more complex version where a generic clock and deadline is specified. I am not diametrically opposed to inverting ClockProtocol and InstantProtocol's relationship but... that must be done with a meaningful and realistic reasoning without sacrificing the goals of the proposal and without treading into those non-starters.

As it is outlined so far it can meet that goal, but I am open to suggestions that keep those parts intact.

2 Likes

This is very interesting, and it promises to be a great advance over the existing situation. :+1:

I have many questions about things I didn't get while reading the pitch.

What exactly is the purpose of ClockProtocol and InstantProtocol? Can I, as a Swift user, define my own clocks, and then use them to schedule work?

ClockProtocol in particular is somewhat weird, as it only provides static members. Do we really need it? It seems to me that the functionality of sleep(until:) and duration(from:to:) can also be provided by the instants. (Related: why wouldn't the now property be provided by the clock? The primary purpose of a clock in everyday life is to tell the current time, after all. Also, why is InstantProtocol the right place for the addition operation, while subtraction lives in ClockProtocol?)

Can these functions be implemented in practice? How would a function like this be able to correctly schedule the work if the implementation isn't aware of the exact reference frame being used?

What's the purpose of passing in a clock instance? ClockProtocol only provides static members.

I cannot overstate how happy I am about this. :clap::tada:

Nit: It might make sense to make these generic over Numeric values. (Or BinaryInteger and FloatingPoint.)

I think Duration should also provide member function(s) that multiply by a scalar factor.

If we were starting from scratch, would we choose the name Date for the wall clock instant type?

Are there potential issues with Date not being @frozen increasing the cost of passing it around? (I was forced to freeze Hasher for performance reasons, esp. unnecessary ARC traffic around destroying it. Something like this could be relevant for microbenchmarking -- although some of it I'm sure can be mitigated. On the other hand, it's no big deal if some microbenchmarking code will need to continue using lower-level APIs.)

Edit: In hindsight, this was a rather silly question -- I wouldn't expect microbenchmarks to be dealing with wall clock times. Followup: would Duration / UptimeClock.Instant / MonotonicClock.Instant be declared frozen? Why/why not?

This is a good idea!

This sounds unfortunate. It seems reasonable to expect that serializing/deserializing an entity through Codable will not change its abstract "value" (whatever that means). So, e.g., I'd expect a deserialized Date would be guaranteed to compare equal to its original value.

There is an escape route to this, which is to implement ==/hash(into:)/< in terms of the original representation. But then, what exactly is the point of storing more bits? (Also, +/- would need to use the same approximations, or we would have weirdness like a == b but a - b != 0, or a + c != b + c.)

I don't really see how read/write times are relevant here -- we don't know what the timestamps represent or how they are used. This would be a lossy serialization.

I think this is the first time I've run across this coding pattern. (How) is it different from just extension WallClock? (As noted above, I also don't understand why it would be useful to pass around clock instances.)

12 Likes

This (I believe) is to take advantage of SE-0299 which allows for the use of implicit member syntax with "configuration" type protocols (such as ClockProtocol). IMHO, passing an argument as clock: .wall as opposed to, say, clock: WallClock.self is a nice ergonomic improvement.

2 Likes

Yes that is the intent; any user defined clock of course needs to implement it's requirements which is the async function to wake up after a given point.

Because the shorthand of .now is highly desirable when passing to a function. E.g.

func perform<Clock: ClockProtocol>(deadline: Clock.Instant, clock: Clock)

is called:

perform(deadline: .now.advanced(by: .seconds(5)), clock: .wall)

Yes, I have an implementation that I am working on that does this. Including allowing Task.sleep to work with any abstract clock. The dispatch one however may be restricted to specific clocks (but I could envision in the future Dispatch allowing this type of thing eventually).

To provide type domains for the clock itself. Having a MonotonicInstant (devoid of clock) feels incorrect, mainly because an instant is just a moment whereas the clock is the thing that measures and wakes things up.

I will caveat that the measurement of the difference of two Instants could be moved into Instant itself; and that is a reasonable refinement in my book.

1 Like

According to this definition (which I totally agree with), now() should be a member of ClockProtocol, instead of InstantProtocol.

// Instead of
public protocol InstantProtocol: Comparable, Hashable {
  static var now: Self { get }
  ...
} 
// it should be
public protocol ClockProtocol {
  associatedtype Instant: InstantProtocol
  // Note: non-static function
  func now() -> Instant

Also, I think principle of "observe what you read" applies to now as well. That means that ClockProtocol should not only provide API to read the current value of time, but also provide API to be notified about change in current time.

Observing time is tricky, because usually you neither need nor can observe every single change in current time value. Instead normally you need to observe time with certain granularity.

From my experience, there are 3 use cases here:

  1. Observe boolean value of now() >= threshold
  2. Observe integer value of floor((now() - base) / interval)
  3. Observe change in DateComponents with respect to the Calendar up to Calendar.Component.

Note that the definition above is well defined when now() is non-monotonous.

I suggest to add the following API:

protocol TimerProtocol: AnyObject {
   associatedtype Instant: InstantProtocol

    var isValid: Bool { get }
    var interval: Duration? { get }
    var fireDate: Instant { get }
    var tolerance: Duration { get set }
    func invalidate()
}

public protocol ClockProtocol {
    associatedtype Instant: InstantProtocol
    associatedtype Timer: TimerProtocol where Timer.Instant == Instant

    func now() -> Instant

    // Use case 1: observes predicate `now() >= date`
    func createTimer(fire date: Date, block: @escaping (Bool) -> Void)) -> Timer
    // Use case 2: observes `floor((now() - date) / interval)`
    func createTimer(fire date: Date, interval: TimeInterval, block: @escaping (Int) -> Void)) -> Timer
}

public class CalendarTimer<Clock: ClockProtocol>: TimerProtocol {
    ...
}

public extension ClockProtocol {
    // Use case 3: Observes change in selected date components with respect to specified calendar.
    // Common implementation for all clocks.
    func createTimer(calendar: Calendar, components: Set<Calendar.Component>, block: @escaping (DateComponents) -> Void)) -> CalendarTimer<Self> {
       ...
    }
}
2 Likes

OK, these are good to know up-front. I'm going to cogitate for a bit and will get back with an alternative API that:

  • makes now a property on ClockProtocol
  • keeps the .now() ... convenience syntax
  • does not require any storage for Monotonic/Uptime/Wall clock instances
7 Likes

Being able to unit test code that depends on a clock is also an important use case, and one that I personally find much more important than being able to use the proposed shorthand.

Other time libraries I've used have provided APIs that facilitated testing such as stateful clocks that always return the same fixed point in time or that only advance when the appropriate clock instance method is invoked. Looking at the pitched API, I don't immediately see how that kind of functionality could be easily implemented, especially given the reliance on static methods and properties.

I think the pitch would be greatly strengthened if it included potential strategies and best practices for testing code that depends on the pitched API. For example:

  • Does my code calculate proper timestamps (such as "issued at" and "expires at" timestamps for certificates)?
  • Does my code wait the proper amount of time before retrying a failed operation?
  • Does my code handle DST shifts in wall clocks properly?
17 Likes

This sounds great!

Do you have a motivating use case for a user-defined time?

Spinning off one or two requirements into a ClockProtocol feels a bit forced to me.

Can two clocks reasonably share the same Instant? That seems unlikely. But then why have two protocols for one thing?

If I understand correctly, .wall is being used here to work around an ambiguity, and a big part of why ClockProtocol and its unusual extensions exist is to make this syntax work.

What exactly are the benefits of this over a less clever design, like the one below?

public protocol TimeProtocol {
  static func sleep(until deadline: Self) async throws
  static var now: Self { get }  
  func advanced(by duration: Duration) -> Self
  func duration(from start: Self) -> Duration
}

public struct Date: TimeProtocol { ... } // Assuming we like the name `Date` for `WallTime`
public struct MonotonicTime: TimeProtocol { ... }
public struct Uptime: TimeProtocol { ... }

func perform<Time: TimeProtocol>(deadline: Time)

perform(deadline: Date.now.advanced(by: .seconds(5)))
perform(deadline: MonotonicDate.now.advanced(by: .seconds(5)))
perform(deadline: Uptime.now.advanced(by: .seconds(5)))

If desired, we can even provide a default time frame by providing a concrete function:

func perform(deadline: Date)

perform(deadline: .now.advanced(by: .seconds(5))) // OK, means wall time
perform(deadline: Uptime.now.advanced(by: .seconds(5))) // still works

Does separating ClockProtocol and InstantProtocol provide any benefits that aren't easily replicated in this sort of single-protocol setup?

4 Likes

The encapsulation of .now does not preclude a manual clock; that can exist w/ the current API as it stands. I have an implementation that does that. What Dave's objection is the case of composition (which has some distinct merit outside of the contrived examples given that we are both exploring options to accommodate). So I don't think we are cutting that off.