SE-0329 (Third review): Clock, Instant, and Duration

Evolution Proposal Reviews
21

This proposal is excellent stuff!

If we must have 2 variables to represent 1 value, I'll toss Slice in as a suffix (instead of Portion) into the Bikeshedding mix for secondsSlice & nanosecondsSlice. Could we not do, a single value so that transitioning to a future Int128 type feels more natural?

public struct Duration: Sendable {
    typealias Components = (high: Int64, low: Int64)
    public var rawValue: Components { get }
}

However, it feels like this is just emblematic of the larger issue: 128-bit storage. (See Below)

TL;DR - Prototype Tests already have (and have for years had) a DoubleWidth<FixedWidthInteger> implementation!

@Philippe_Hausler is right, the answer is to expose the storage-- Nothing against quintillionths, but I wouldn't call it attoseconds I'd jump to RawRepresentable where RawValue == Int128 and expose Duration storage with rawValue.

It's exactly the right thing to use here and (so far) Swift has only continued to flirt with Arbitrary-Precision Integers or even just straight-up Int128. The Swift repo's Tests have sported both a DoubleWidth<FixedWidthInteger> (swift/DoubleWidth.swift.gyb at main Ā· apple/swift Ā· GitHub) and a _BigInt (swift/BigInt.swift at main Ā· apple/swift Ā· GitHub) implementation for years. [NOTE: Result<T> was in Tests before Swift 2 and only moved to the Standard Library in Swift 5. We should revisit this directory's goodies more often.]

Numerics also has some work on Arbitrary-Precision Integers (Arbitrary-precision bigints Ā· Issue #5 Ā· apple/swift-numerics Ā· GitHub) .

When it comes to natural Swift-ness, look-and-feel are paramount to usability. This "2 variables == 1 value" paradigm doesn't feel natural in a language where building up value types is such a focus. At least for me, this is what I want out of Duration:

public struct Duration: Sendable {
  public var rawValue: Int128 { get }
}

extension Duration: RawRepresentable { }

I would settle for var rawValue: DoubleWidth<Int64>. If we need to shim this with a new value type under the covers for a few years, let's do that rather than put some "2 variables == 1 value" stuff in the Standard Library. At least that way we could avoid or at least minimize, some potentially code-breaking update in a future Swift with Int128 or Arbitrary-Precision Integers.

I can live with 2 properties; but we didn't do this with Concurrency, nor Distributed Actors or all the various SwiftUI-related proposals-- in those cases if something came up, solve the problem or implement a work-around until LLVM could implement a fix (Evolution is still removing work-arounds on Concurrency features and will be until at least Swift 5.6).

2 properties is a work-around, but it just feels like overhead for implementations.

Anyway, great job everyone! I'm very much looking forward to this one! (assuming we encounter no temporal anomalies or time-loops.)

1 Like
22

The implementation uses this - but a honest to goodness Int128 would be preferable; however I don't think it is really in the same spirit as RawRepresentable. I find those are reserved for things that use bitwise storage like OptionSet etc. The variable name of the underlying storage should somehow reflect the time scale it is.

That all being said: I am not sure I really want to hang the hat on that type being available. Quite honestly folks are working overtime already to get stuff done and that would be yet another REALLY big proposal (and likely implementation).

There is another option (perhaps a less tasteful one). The major reasons to expose these are for use by Foundation and other interoperation layers. We do have tools for that which can be lock-step avoidant of public API. That tool specifically is @_spi. I don't like that as an option because it prevents folks from doing their own interoperation layers... but it does resolve the "problem of naming and typing these properties".... by just removing them from the public interface. That could buy us some time to have available scheduling to make the property exposed as Int128 or settle on a better interface while letting the immediate task of scheduling work and such make forward progress.

Per alternatives: if I were to evaluate the solutions so far -

The current two properties are quite easy to use (but perhaps also easy to misuse), and is pretty straightforward to test and implement.

The tuple is a bit safer and is also relatively easy to use and is about the same per testing and implementation.

@xwu's suggestion is definitely safer with some regards than the two properties but introduces potential cases where we have bad input (the case of passing in units of .nanoseconds and then .microseconds is probably nonsensical). I would worry that the permutations on this might get kinda costly to implement and complicated - when truthfully the real need here is to access the value with just a touch of correct math.

@ksluder's suggestion refines that some but still has the problem of the bad input, however I am not sure how much more it buys per safety.

@benrimmington I tend to agree with @ksluder that the quotient thing is not incredibly intuitive.

@davedelong's suggestion to take the enumeration access with three accessors seems like a lot of surface area for something that should be VERY simple. Plus I am not really convinced that fractional is really the right word here.

The primary intent for these accessors are to provide the ability to interoperate with the C APIs that take struct timespec. I agree that the inconsistency with C APIs that take two disparate time types struct timespec and struct timeval make this a bit gnarly (having those two not be the same thing has always annoyed me). If someone is needing to interface with those APIs it is fair to ask them to do just a touch of math to convert the nanoseconds to microseconds. After all they are in the business of softening sharp APIs to begin with (see the example of setsockopt provided before).

I buy the argument of the tuple being a better solution. I am not sure an investment in the enumeration approach is really worth the tradeoff. I mean wouldn't it just be better to have an extension on struct timespec and struct timeval that initializes with a Duration? That way there is no ambiguity.

1 Like
23

There are tons of APIs that might take nanoseconds, milliseconds, etc. IOKit alone has several, not to mention any third party frameworks included in a project.

@ksluder's suggestion refines that some but still has the problem of the bad input, however I am not sure how much more it buys per safety.

What bad input? That you can request milliseconds when you meant nanoseconds? Thatā€™s far less of a risk IMO than choosing the wrong conversion factor or a weird combination of scales.

24

Since Duration is explicitly designed to model a count in attoseconds, I feel like it's a little patronizing to not offer any way to construct and deconstruct it with full precision. If we have to expose that with DoubleWidth because we don't have a portable Int128 type, well, I feel like that just means that we should make sure we aren't treading on the space that we'd want a future use of Int128 to take up. For example, we could name the property something like attosecondsAsDoubleWidth.

We can then make sure that libraries (System?) give timespec etc. initializers that convert straight from Duration so that there's as few places as possible that have to count out how many zeroes they need.

I feel like removing artificial barriers to using the full precision is in many ways more important than making sure it's convenient to project out different compositions of units, especially when there are so many possible combinations that different APIs want (timespec, scalar nanos, scalar micros, scalar millis, etc., all at varying bit-widths).

9 Likes
25

How about something along these lines?

extension Duration {
  // Access to the full precision.
  public var components: (seconds: Int64, attoseconds: Int64) { get }

  // Make it easier to interoperate with C APIs.
  public var timespecComponents: (seconds: Int64, nanoseconds: Int64) { get }
  public var timevalComponents: (seconds: Int64, microseconds: Int64) { get }
}

That would also be handy.

26

Does deconstructing to a pair of Int64 seconds and Int64 attoseconds (whose value is known to be < 1E18) not retain full precision?

27

The intent was just timespec so I was more concerned about keeping the surface area limited. If it is the decision of the core team and community that attoseconds are important then I am more than happy to add the following changes:

public struct Duration: Sendable {
  public var components: (seconds: Int64, attoseconds: Int64) { get }
}

extension Duration {
  public static func attoseconds<T: BinaryInteger>(_ value: T) -> Duration
}

I agree fully, should this be added to the proposal?

extension timespec {
  public init(_ duration: Duration)
}

extension timeval {
  public init(_ duration: Duration)
}

Does that mean that that home for the extensions (wherever it may be) should also have construction of Duration from those types too?

extension Duration {
  public init(_ ts: timespec)
  public init(_ tv: timeval)
}

@ksluder deconstructing for attoseconds and seconds would retain full precision if both are Int64.

Given the previous naming the following would be true for all values of Duration

func validate(_ duration: Duration) {
  let (seconds, attoseconds) = duration.components
  let reconstructed: Duration = .seconds(seconds) + .attoseconds(attoseconds)
  assert(duration == reconstructed)
}
5 Likes
28

I think thatā€™s a great solution until we have a true Int128. But I donā€™t think anyone who wants to deal with nanoseconds/microseconds/milliseconds should be forced to convert from attoseconds. As @benrimmington has already shown, itā€™s way too easy to pick the wrong conversion factor. And if weā€™re going to provide an API to choose other scales, it feels redundant to expose a dedicated components property that always returns seconds + attoseconds.

1 Like
29

I should clarify that I'm just speaking for myself in this instance. It just seems unfortunate that attoseconds aren't usable if they're part of the design.

Hmm. I think we need an opinion from someone who would know where these belong.

1 Like
30

So there are a few places that they could live:

Platform - This seems reasonable since it is effectively the "overlay" for the c standard library per each platform and could then be available immediately to Linux and Darwin.

libsystem swift overlay - This seems reasonable for Darwin platforms, but would be subject to availability and release schedules of the Darwin releases, and would leave out Linux (which seems sub-par).

System - It makes some sense, but that package seems more tuned to the concept of replacing system like items with thin wrappers (which Duration almost belongs in that group, but is lower due to its interaction with Concurrency)

Foundation - Again makes some sense, but that framework is more about defining platform abstractions and there are no existing APIs that take or transact in timespec or timeval in public API in Foundation.

So my pitch would be the Platform layer; which leaves room for similar definitions in the Windows ecosystem and other per-platform libc-ish type stuff.

3 Likes
31

+1 Platform layer

And I'm with @John_McCall, direct access to attoseconds seems valuable for scientific calculation. components looks more future-ready and with attoseconds the orders of magnitude for time being supplied in the proposal are exactly the assortment; that I would think, would see the most use.

32

System will most likely provide thin wrapper types over timespec and timeval which provide initializers from Duration.

33

System is probably the right place for these API. @Michael_Ilseman, what do you think?

2 Likes
34

After discussing whether System should expose getcwd with @lorentey, my impression is that System is not designed for these sorts of things.

Swift System's job is to faithfully expose the underlying system's APIs, not to reimagine them.

I view System as mostly targeting use cases that are below Foundation -- I would go as far as to suggest it is usually a mistake to import System from a regular app.

Also, by this reasoning, pretty much every single API exposed by System ought to be marked unsafe -- these are extremely low-level APIs that should not be directly used by any regular program.

Unfortunately, FilePath also lives in swift-system, but my understanding is that there is a desire to move it to a different library at some point:

Despite the wonderful FilePath type (which I hope we can move to/near the stdlib at some point), I don't consider Swift System to be the right place for cross-platform abstractions or emulation layers.

IMO, it seems reasonable for these extensions to live in the platform overlay. Creating a timespec from a duration is not, in general, a system call, and shouldn't require importing all of the low-level platform APIs exposed (or to be exposed) by swift-system.

1 Like
35
Some off-topic response to the System stuff

Having the Windows file APIs share the same code paths / entry points as their Un*x counterparts seems especially counterproductive to me -- it's a maintenance burden we don't need.

As long as we donā€™t need to call system APIs with FilePath, there should be no reason to keep the current implementation by System. Instead, Path = Root + Component is a great fit for a generic path representation which we can use to express nearly everything, from a VFS endpoint to an S3 object. Thus, we had better build it directly on String or StringProtocol and bridge it to the existing FilePath, instead of replacing the SystemString based implementation. They serve mostly different purposes and I donā€™t think picking the ā€œbalance pointā€ would be better.

36

Okay, as review manager, I have to ask: given that we'll have initializers to and from timespec in some library or another, do we have any consensus on what the (portable) accessors on Duration will be? Because it sounds like there's agreement that they shouldn't be as currently proposed, at least in terms of using Component instead of Portion, and possibly in other terms.

1 Like
37

Letā€™s see, the proposal currently has the following:

  • Duration.nanoseconds(#)
  • Duration.microseconds(#)
  • Duration.milliseconds(#)
  • Duration.seconds(#)

All of these are good and should stay, because they enable expressive syntax like sleep(for: .seconds(3)). However:

  • static func .attoseconds(#) is missing.
  • I still stand by the refactoring of secondsPortion / nanosecondsPortion to wholeSeconds and fractionalSeconds(scale:). Duration is effectively a nominal alternative to a (seconds, fractionalSeconds) tuple; thereā€™s no reason to start returning tuples from its accessors.
  • It feels weird that I have to do Duration.seconds(#) + Duration.microseconds(#) to get a duration longer than one second. I believe we should also have a standard initializer: Duration.init(wholeSeconds: {T:BinaryInteger, Float} = 0, fractionalSeconds: FractionalSecondsScale = .attoseconds(0))

Edit: I also think we should add static func .minutes(#) and Duration.init?(minutes:, wholeSeconds:, fractionalSeconds:), returning nil if (minutes*60 + wholeSeconds) > Int64.max. For the same reasons as in previous reviews, I donā€™t recommend adding .hours() or larger scales.

38

From the previous review, I still have concerns about broad use of the types without defining their units. I would like to see UTCClock, Duration and Date defined in terms of the local system behavior (e.g elided, omitted/duplicated, and atomic second behavior) , with clear consequences of working with Duration/Date values across systems or reusing these types for clocks instants/clocks with their own behaviors.

That includes using types other than Duration for ContinuousClock and SuspendingClock, since these have differing behavior per platform in many cases from the UTCClock behavior. This also gives us the opportunity to document the behavior when trying to use a these different duration types together.

39

I haven't followed the entire discussion here and have been trying very hard to not hijack or derail this thread (or the URL one :sweat_smile:). I can provide some clarifications if it helps:

System hasn't tackled time, yet, but will brush up against it soon. It depends if there's a strong need for a layout-equivalent discrete time type or if we can just convert to-from Duration. In general,:

  1. System papers over "trivial" (and this is always a potential point of contention) differences such as Int32 vs UInt64 for representing a file offset, etc.
  2. System keeps aggregate structs laid out as the OS would, but provide getters/setters or API in terms of slightly Swiftier types, so long as the conversion is fairly "trivial" (bit masks, shifts, extends, etc).
  3. System namespaces everything it can, so the raw C type would be typealiased as CInterop.TimeSpec or something like that, and timespec is not reexported from the OS module.

From this, it seems like System could just use Duration for function parameters, getters/setters, etc, in most of its APIs. I don't know if there'd be value in a separate type for namespace purposes beyond this. It's more likely that getting a CInterop.TimeSpec would be a var on Duration than an initializer, since use sites generally don't spell out the CInterop types by hand.

For example, it seems reasonable that the result of stat(2) could be a strongly-typed raw-representable struct that's layout equivalent with struct stat, but with a var lastAccess: Duration { get }.

The file system is an entity which refers to a clock, and it may make sense for a formal FileSystem or other entity to conform to ClockProtocol (or provide access to its clock which does). There, the question (and I haven't thought through it enough to say for certain) is whether its instant type needs to be a discrete layout-equivalent-with-C nominal type or if it can just be Duration. I also haven't checked whether such a clock would be truly redundant with the clocks proposed here.

2 Likes
40

There is a clock that is being proposed here that fits that bill. The one in Foundation; URL has this resource value for example. Which is a Date for the access date.