[Pitch] Calendar Sequence Enumeration

Hi everyone,

I would like to pitch an addition to Foundation's Calendar API.

Full text is here, and a PR for editorial comments is here.

Introduction

In macOS 14 / iOS 17, Calendar was rewritten entirely in Swift. One of the many benefits of this change is that we can now more easily create Swift-specific Calendar API that feels more natural than the existing enumerate methods. In addition, we are taking the opportunity to add a new field to the DateComponents type to handle one case that was only exposed via the somewhat esoteric CoreFoundation API CFCalendarDecomposeAbsoluteTime.

Motivation

The existing enumerateDates method on Calendar is basically imported from an Objective-C implementation. We can provide much better integration with other Swift API by providing a Sequence-backed enumeration.

Proposed solution

We propose a new field on DateComponents and associated options / units:

extension Calendar {
    public enum Component : Sendable {
        // .. existing fields

        @available(FoundationPreview 0.4, *)
        case dayOfYear
    }
}
extension DateComponents {
    /// A day of the year.
    /// For example, in the Gregorian calendar, can go from 1 to 365 or 1 to 366 in leap years.
    /// - note: This value is interpreted in the context of the calendar in which it is used.
    @available(FoundationPreview 0.4, *)
    public var dayOfYear: Int?
}

We also propose a new API on Calendar to use enumeration in a Sequence-friendly way:

extension Calendar {
    /// Computes the dates which match (or most closely match) a given set of components, returned as a `Sequence`.
    ///
    /// If `direction` is set to `.backward`, this method finds the previous match before the given date. The intent is that the same matches as for a `.forward` search will be found. For example, if you are searching forwards or backwards for each hour with minute "27", the seconds in the date you will get in both a `.forward` and `.backward` search would be 00.  Similarly, for DST backwards jumps which repeat times, you'll get the first match by default, where "first" is defined from the point of view of searching forwards. Therefore, when searching backwards looking for a particular hour, with no minute and second specified, you don't get a minute and second of 59:59 for the matching hour but instead 00:00.
    ///
    /// If an exact match is not possible, and requested with the `strict` option, the sequence ends.
    ///
    /// Result dates have an integer number of seconds (as if 0 was specified for the nanoseconds property of the `DateComponents` matching parameter), unless a value was set in the nanoseconds property, in which case the result date will have that number of nanoseconds, or as close as possible with floating point numbers.
    /// - parameter start: The `Date` at which to start the search.
    /// - parameter components: The `DateComponents` to use as input to the search algorithm.
    /// - parameter matchingPolicy: Determines the behavior of the search algorithm when the input produces an ambiguous result.
    /// - parameter repeatedTimePolicy: Determines the behavior of the search algorithm when the input produces a time that occurs twice on a particular day.
    /// - parameter direction: Which direction in time to search. The default value is `.forward`, which means later in time.
    @available(FoundationPreview 0.4, *)
    public func dates(startingAt start: Date,
                      matching components: DateComponents,
                      matchingPolicy: MatchingPolicy = .nextTime,
                      repeatedTimePolicy: RepeatedTimePolicy = .first,
                      direction: SearchDirection = .forward) -> DateSequence
}

extension Calendar {
    /// A `Sequence` of `Date`s which match the specified search criteria.
    /// - note: This sequence will terminate after a built-in search limit to prevent infinite loops.
    @available(FoundationPreview 0.4, *)
    public struct DateSequence : Sendable, Sequence {
        public typealias Element = Date

        public var calendar: Calendar
        public var start: Date
        public var matchingComponents: DateComponents
        public var matchingPolicy: Calendar.MatchingPolicy
        public var repeatedTimePolicy: Calendar.RepeatedTimePolicy
        public var direction: Calendar.SearchDirection
        
        public init(calendar: Calendar, start: Date, matchingComponents: DateComponents, matchingPolicy: Calendar.MatchingPolicy = .nextTime, repeatedTimePolicy: Calendar.RepeatedTimePolicy = .first, direction: Calendar.SearchDirection = .forward)

        public func makeIterator() -> Iterator

        public struct Iterator: Sendable, IteratorProtocol {
            // No public initializer
            public mutating func next() -> Element?
        }
    }
}

Detailed design

The new Sequence-based API is a great fit for Swift because it composes with all the existing algorithms and functions that exist on Sequence. For example, the following code finds the next 3 minutes after August 22, 2022 at 3:02:38 PM PDT, then uses zip to combine them with some strings. The second array naturally has 3 elements. In contrast with the existing enumerate method, no additional counting of how many values we've seen and manully setting a stop argument to break out of a loop is required.

let cal = Calendar(identifier: .gregorian)
let date = Date(timeIntervalSinceReferenceDate: 682869758.712307)   // August 22, 2022 at 7:02:38 AM PDT
let dates = zip(
    cal.dates(startingAt: date, matching: DateComponents(minute: 0), matchingPolicy: .nextTime),
    ["1st period", "2nd period", "3rd period"]
)

let description = dates
        .map { "\($0.formatted(date: .omitted, time: .shortened)): \($1)" }
        .formatted()
// 8:00 AM: 1st period, 9:00 AM: 2nd period, and 10:00 AM: 3rd period

Another example is simply using the prefix function. Here, it is combined with use of the new dayOfYear field:

var matchingComps = DateComponents()
matchingComps.dayOfYear = 234
// Including a leap year, find the next 5 "day 234"s
let nextFive = cal.dates(startingAt: date, matching: matchingComps).prefix(5)
/* 
  Result:
    2022-08-22 00:00:00 +0000
    2023-08-22 00:00:00 +0000
    2024-08-21 00:00:00 +0000 // note: leap year, one additional day in Feb
    2025-08-22 00:00:00 +0000
    2026-08-22 00:00:00 +0000
*/

The new dayOfYear option composes with existing Calendar API, and can be useful for specialized calculations.

let date = Date(timeIntervalSinceReferenceDate: 682898558.712307) // 2022-08-22 22:02:38 UTC, day 234
let dayOfYear = cal.component(.dayOfYear, from: date) // 234

let range1 = cal.range(of: .dayOfYear, in: .year, for: date) // 1..<366
let range2 = cal.range(of: .dayOfYear, in: .year, for: leapYearDate // 1..<367

// What day of the week is the 100th day of the year?
let whatDay = cal.date(bySetting: .dayOfYear, value: 100, of: Date.now)!
let dayOfWeek = cal.component(.weekday, from: whatDay) // 3 (Tuesday)

Source compatibility

The proposed changes are additive and no significant impact on existing code is expected. Some Calendar API will begin to return DateComponents results with the additional field populated.

Implications on adoption

The new API has an availability of FoundationPreview 0.4 or later.

Alternatives considered

The DateSequence API is missing one parameter that enumerateDates has - a Boolean argument to indicate if the result date is an exact match or not. In research for this proposal, we surveyed many callers of the existing enumerateDates API and found only one that did not ignore this argument. Given the greater usability of having a simple Date as the element of the Sequence, we decided to omit the value from the Sequence API. The existing enumerateDates method will continue to exist in the rare case that the exact-match value is required.

We decided not to add the new fields to the DateComponents initializer. Swift might add a new "magic with" operator which will provide a better pattern for initializing immutable struct types with var fields. Even if that proposal does not end up accepted, adding a new initializer for each new field will quickly become unmanageable, and using default values makes the initializers ambiguous. Instead, the caller can simply set the desired value after initialization.

We originally considered adding a field for Julian days, but decided this would be better expressed as a conversion from Date instead of from a DateComponents. Julian days are similar to Date in that they represent a point on a fixed timeline. For Julian days, they also assume a fixed calendar and time zone. Combining this with the open API of a DateComponents, which allows setting both a Calendar and TimeZone property, provides an opportunity for confusion. In addition, ICU defines a Julian day slightly differently than other standards and our current implementation relies on ICU for the calculations. This discrepency could lead to errors if the developer was not careful to offset the result manually.

Acknowledgments

Thanks to Tina Liu for early feedback on this proposal.

18 Likes

Hi @Tony_Parker, this looks great! Glad to see this API get introduced. Just a few questions:

  1. Unless I'm missing something, it seems that the new .dayOfYear is orthogonal to the DateSequence API — are these just being incidentally pitched together, or does the sequence API rely on the .dayOfYear field in some way? (It just feels like these could be separately-successful pitches.)
  2. Because of DateComponents' inherent nature, it's already possible to construct a DateComponents value with inconsistent components such that no date could match them (e.g., specifying a calendar, year, month, and day, with a weekday that doesn't match the actual day of the week for that date in the given calendar); presumably, the new dayOfYear field will behave like all the others in that setting an inconsistent dayOfYear will find no matching results, yes?
  3. It seems unlikely, but are there any expected differences in results between calling the existing date enumeration method and the new sequence method? Or does the new DateSequence method effectively just wrap the existing enumeration as-is?
  4. The note: on DateSequence indicates that iteration may end prematurely to avoid infinite loops: this feels a bit odd for Sequences, which are allowed to be infinite. Is this a limitation of the underlying implementation? If so, could it eventually be lifted; and if not, would it be reasonable to document what the limit is?
    • This also seems to indicate that the search may be performed eagerly, with results possibly collected and cached; is this the case, or is that reading too deeply into the wording?

Regardless, both of these seem like no-brainer additions to the existing APIs!

2 Likes

As the first official public Foundation evolution proposal (no other proposals in the repo at least), can you outline how this will work and how it may differ, if at all, from the normal Swift process? Will these proposals be entirely Apple driven, or is this now an open process for new Swift-specific Foundation APIs?

1 Like

DayOfYear

I echo @itaiferber's confusion about the inclusion of dayOfYear in the proposal about a sequence of calendar values. What's the motivating factor for including it?

As proposed, I don't think dayOfYear reaches the bar for introduction:

  1. It is redundant.

    If I have a Date and want to know its corresponding day of the year, I use the existing aCalendar.ordinality(of: .day, in: .year, for: aDate) method.

    If I want to know the number of days in a year, I use aCalendar.range(of: .day, in: .year, for: aDate) which gives me the 1 ..< 366 (for example) range.

    If I want to find the 100th day of a year, I can do it in three lines instead of the proposed two:

    let startOfYear = cal.dateInterval(of: .year, for: aDate)!.start
    let hundredthDay = cal.date(byAdding: .day, value: 100, to: startOfyear)!
    let dayOfWeek = cal.component(.weekday, from: hundredthDay)
    

    All of these proposed usages have existing equivalents of nearly identical length using current API. To me, that doesn't justify the creation of a new unit.

  2. It is confusing. It leads to ambiguity about whether I should use .day or .dayOfYear, given that they appear to operate identically in some scenarios.

  3. Is it necessary? Being able to use it as a matching parameter when enumerating dates is intriguing, and doing that with current API can be tedious. However, it also doesn't strike me as something common to do. In your research of date enumeration code for this proposal, how often did this pattern come up of needing to consistently find the 234th day of the year?

Calendar.DateSequence

I love this addition. I have a very similar type in my own Swift extensions and am very pleased to see this. I concur that the omission of the exactMatch boolean is a good choice. I also agree with the discussion around expanding the initializer and working with Julian dates.

1 Like

What happens if the stored properties of DataSequence are mutated? Does that affect any existing Iterators?

I do actually enumerate dates occasionally but it's not something I've thought much about. With that context, my initial thought (of how one might do that) was something more like the stride function, e.g.:

for date in stride(from: startDate, to: .distantFuture, by: .days) {
    …
}

At a quick glance it seems like the proposed API is a superset of the above, in functionality, but it's also quite a bit more complicated. What are your thoughts on the above approach and those trade-offs?

I guess to date (haha) I've mostly used enumerateDates, but I vaguely recall it being a bit of a pain to get to do what I want, which is usually just real simple stuff like enumerating the calendar days or months or years between two dates. The proposed new API is an improvement in some respects (re. Sequence style) but still very similar in that regard.

Hey there @itaiferber -

Responding point by point:

  1. These could be two different proposals, but they are in a close-enough area that I felt we could just do them together. As we sort out how these kinds of pitches for Swift Foundation go we can adjust how much or how little we want in each proposal over time.

  2. Yes, we handle failures to match the components by returning an empty list. Here is an example test case which I will add to the suite:

// Nonsense day-of-year
var nonsenseDayOfYear = DateComponents()
nonsenseDayOfYear.dayOfYear = 500
let shouldBeEmpty = Array(cal.dates(startingAt: beforeDate, matching: nonsenseDayOfYear))
XCTAssertTrue(shouldBeEmpty.isEmpty) // succeeds
  1. These share the same implementation, which you can see in the package-internal version of this implementation here.

  2. This note is phrased poorly; I will update it. Here is the search limit, 100. The limit is actually the number of iterations we will allow to fail. So if you fail to find 100 results in a row (e.g. by asking for dayOfYear 500, above) then you get back an empty result. If for some reason you fail 99 times then find 1, we will return values until we fail one more time. This is a bit odd, maybe, but matches the preexisting behavior of these methods.

Hey @wadetregaskis,

Mutating the properties of the sequence doesn't do anything special. It's pure value semantics. The next iterator you construct from it will have those properties.

Now, for the striding question -- this is actually a different operation on Calendar:

extension Calendar {
    public func date(byAdding component: Component, value: Int, to date: Date, wrappingComponents: Bool = false) -> Date?
}

Your comment intrigued me. Thinking about it you are right that this API, too, could benefit from a Sequence.

@available(FoundationPreview 0.4, *)
public func dates(in range: Range<Date>,
                  stridingBy component: Calendar.Component,
                  stride: Int = 1,
                  wrappingComponents: Bool = false) -> DateAddingSequence

@available(FoundationPreview 0.4, *)
public func dates(in range: PartialRangeFrom<Date>,
                  stridingBy component: Calendar.Component,
                  stride: Int = 1,
                  wrappingComponents: Bool = false) -> DateAddingSequence

example:

let startDate = Date(timeIntervalSinceReferenceDate: 682898558.712307)
let endDate = startDate + (86400 * 3) + 1 // 3 days + 1 second later
var cal = Calendar(identifier: .gregorian)
let tz = TimeZone.gmt
cal.timeZone = tz

let numberOfDays = Array(cal.dates(in: startDate..<endDate, stridingBy: .day)).count
XCTAssertEqual(numberOfDays, 3)
4 Likes

Yeah, that's [more] appealing to me, as it's simpler but still suits my occasional needs.

1 Like

Hi @Jon_Shier --

You're right, this is the first one. We welcome community API proposals, and with the recent formation of our Foundation Workgroup, we are looking forward to involving the community into Foundation's evolution more than ever before.

Our process is outlined here.

5 Likes

That last part is indeed odd. One would assume it'd reset the counter and attempt another hundred times.

The limit of 100 seems oddly small, given this is only guarding against a seemingly distant edge case…? Does that mean if you try to enumerate all the February 29ths between two years it'll never work?

This feels a little bit awkward. What do you think about shortening the labels for the first two (required) parameters:

    public func dates(from start: Date, // <-
                      where components: DateComponents, // <-
                      matchingPolicy: MatchingPolicy = .nextTime,
                      repeatedTimePolicy: RepeatedTimePolicy = .first,
                      direction: SearchDirection = .forward) -> DateSequence

With some static member shorthands on DateComponents, I think the site of use could become shorter and clearer:

for date in Calendar.current.dates(from: .now(), where: .day(1, month: 6)) {
  // 01-06-2024
  // 01-06-2025
  // 01-06-2026
  // ...etc
}

It would probably also be good to include a limited sequence or collection, which operates over a Range<Date>:

    public func dates(in range: Range<Date>, // <-
                      where components: DateComponents,
                      matchingPolicy: MatchingPolicy = .nextTime,
                      repeatedTimePolicy: RepeatedTimePolicy = .first,
                      direction: SearchDirection = .forward) -> DateSequence

(It would only be able to iterate forwards in time, since Range requires that lowerBound <= upperBound, but that's still very useful)

Correction: Of course you can iterate backwards within a range :man_facepalming:

2 Likes

Keep in mind that the enumerate API doesn't have a built-in end date (unlike the thing I just wrote above), so there is no such thing as "all the February 29s between two years".

There is probably a bug here about "running out" of failed matches. So if you search for all the February 29s it will eventually terminate instead of resetting the counter each time and looping forever. We can track that as just a bug fix if we choose, and it will apply to both this new API and the existing enumerate method.

1 Like

I probably don't know enough about calendars and dates to judge. I just had a similar reaction to @itaiferber on seeing that subtle note, given that Sequences are explicitly allowed to be infinite, and as such there's plenty of existing ways to get stuck in an infinite iteration loop if you want. But perhaps there's a war story behind that limit for date enumeration.

Couldn't we also have an optional endDate argument?

1 Like

An end date is reasonable. I think perhaps we need to think more about the potential for confusion with the intersection of range API (which cannot be 'backwards', e.g., endDate..<startDate), the optionality of the end date vs partial ranges, and the forwards/backwards argument which changes the direction of the matching.

1 Like

Hi @davedelong,

While you can find the day of year via other means, including it in date components means you can calculate it alongside other values in a single call. Furthermore it is often calculated anyway, so putting it in one set of calculations means we don't have to duplicate the work to find it.

One obvious example of where this is used is ISO8601 formatting, where we need to extract a few fields (including day of year) depending on which exact ISO8601 format we need. e.g. this (untested, may not be 100% correct):

for field in formatFields {
    switch field {
    case .year: 
        // If we use week of year later, don't bother with year
        if !formatFields.contains(.weekOfYear) {
            whichComponents.insert(.year)
        }
    case .month: 
        whichComponents.insert(.month)
    case .weekOfYear:
        whichComponents.insert([.weekOfYear, .yearForWeekOfYear])
    case .day:
        if formatFields.contains(.weekOfYear) {
            whichComponents.insert(.weekday)
        } else if formatFields.contains(.month) {
            whichComponents.insert(.day)
        } else {
            whichComponents.insert(.dayOfYear)
        }

    case .time:
        whichComponents.insert([.hour, .minute, .second])
        if includingFractionalSeconds {
            whichComponents.insert(.nanosecond)
        }
    default:
        break
    }
}
1 Like

Precisely.

When using terms like "from" it would be confusing to not actually start from that date, even if the requested direction logically implies that. e.g. "from" now to tomorrow going backwards.

It's somewhat less problematic if you could instead provide a Range of dates, because then technically neither end of the range is intrinsically the starting point, since Ranges in Swift are always ordered ascendingly. But even then I think that'll confuse some folks, because they'll think "backwards" means the reverse of forwards as they'd get with for date in now..<tomorrow { … } (not that you can do that, today).

It'd be easy if Swift had a ReversedRange type. Then you wouldn't even need custom API for forwards vs backwards because that's controlled generally through Range.reversed.

I fear it's unavoidable that an order is implied when providing two dates, and therefore also providing a direction is bound to lead to mistakes. Take for example stride: you can write stride(from: 1, to: 0: by: 1), although the result (an empty sequence) might still confuse some folks.

One way to suitably couple the date pair to the enumeration order would be through the direction enum, e.g.:

enum Direction {
    case forward
    case fowardUntil(date: Date)

    case backward
    case backwardUntil(date: Date)
}

Sadly this doesn't do anything at compile time to enforce that the dates are in a sensible order, but it at least makes it conceptually clear to readers. (and arguably there's merit in supporting 'inverted' dates, so long as you only choose to utilise that quirk, not fall into it by accident)

1 Like

Hi all,

I've posted an updated version of the pitch here (rendered).

Updates:

  • Adds optional Range limits to the "matching" API
  • Adds new "adding" API, with similar Range option
  • Removes the concrete sequence types in favor of some, which avoids the need to make limited-use public types (and enables the adding API, too)
  • Adds a lot of alternatives considered, from this thread and more experimentation
  • Expands the examples

The implementation pull request has also been updated.

1 Like

The difference between date(byAdding: …) and dates(startingAt:in:byAdding: …) might cause some confusion. Perhaps it'd make more sense to reorder the arguments in the latter, to more closely match the existing API? That'll make it clearer that they're closely related, and easier to move between them. It'd also help during auto-completion, as they'll show up next to each other and for their essential common name prefix, date(byAdding: …).

Tangentially, I do wish there were a combined type for datetime component counts, so that one could write e.g. 3.days instead of having to pass two separate arguments.

The existing enumerateDates method will continue to exist in the rare case that the exact-match value is required.

Will it, though, long-term? It seems implied to me that these new APIs are the replacements for the older ones - they're almost functionally equivalent, other than the exact match parameter in question.

If that parameter is virtually never used in the existing API, then I'm okay with the pitch making the case that it should just be dropped (and otherwise making it clear that ultimately the older APIs will be removed). Presumably it could be implemented by a 3rd party in an extension or similar, for anyone that needs it?

Other than that, looks great to me.

One of the limitations we have to face with Foundation API is that, as part of the iOS and macOS operating systems, we really can't ever actually remove anything. Perhaps we could imagine a world in which we remove an API from the SDK but it continues to exist in the binary for runtime compatibility with older apps, but at that point it's still required for us to keep it working it just the same. Our only real option is deprecation.

My philosophy with deprecation is to tread carefully. If someone has an app or library which is working fine right now and they get a new deprecation message, they really only have two options:

  1. Change their existing code, taking the risk introducing bugs or compatibility issues with their own code because our new method likely does not behave exactly the same (otherwise, why did we deprecate it?)
  2. Ignore the warning.

In some cases we really, really want people to do #1. If we deprecate things because they are inconvenient or a little less than ideal, though, we then dilute our message about when it is truly important to move to something new -- usually for security or performance reasons.

In this particular case, I think the new API is compelling enough that people will just it by default when they need this functionality. We can also come back later to deprecate the older API once the new one is well-established.

4 Likes