# SE-0346 and library evolution

**URL:** <https://forums.swift.org/t/se-0346-and-library-evolution/56659>\
**Category:** Discussion\
**Created:** [April 10, 2022, 8:30pm UTC](https://forums.swift.org/t/se-0346-and-library-evolution/56659 "2022-04-10T20:30:46Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![taylorswift](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/taylorswift/32/43413_2.png) [@taylorswift](https://forums.swift.org/u/taylorswift)\
**Post date:** [April 10, 2022, 8:30pm UTC](https://forums.swift.org/t/se-0346-and-library-evolution/56659/1 "2022-04-10T20:30:46Z")

</div>

i’m currently auditing [`swift-grammar`](https://github.com/kelvin13/swift-grammar/tree/master) for PAT candidates among its [various](https://swiftinit.org/reference/swift-grammar/grammar/parsingrule) `associatedtype`s and i’m running into the “can’t `#if` the title of the `protocol`” problem:

```swift
/// A structured parsing rule.
public 
protocol ParsingRule 
{
    /// The index type of the ``ParsingInput.source`` this rule expects.
    /// 
    /// Parsing rules must be associated with a source location type because 
    /// some applications may wish to store these indices in the returned 
    /// ``Construction``s. If the source location type were not fixed, then 
    /// different calls to ``parse(_:)`` could potentially return constructions
    /// of varying types, which would require additional abstraction, which would 
    /// be inefficient.
    /// 
    /// > Tip: 
    /// Implementations can satisfy this requirement with generics, allowing 
    /// parsing rules to be reused for different input types. 
    associatedtype Location
    /// The element type of the ``ParsingInput.source`` this rule expects.
    associatedtype Terminal 
    /// The type of the constructions produced by a successful application of this 
    /// parsing rule.
    /// 
    /// Implementations should not report failure through an ``Optional`` 
    /// construction type. Instead, implementations should [`throw`]() an ``Error``, 
    /// which allows the library to perform appropriate cleanup and backtracking.
    associatedtype Construction
    
    /// Attempts to parse an instance of ``Construction`` from the given 
    /// parsing input.
    ///
    /// The implementation is not required to clean up the state of the `input`
    /// upon throwing an error; this is handled by the library.
    /// 
    /// Implementations *should* interact with the given ``ParsingInput`` by 
    /// calling its methods and subscripts. Don’t overwrite the [`inout`]() binding or its 
    /// mutable stored properties (``ParsingInput/.index`` and ``ParsingInput/.diagnostics``)
    /// unless you really know what you’re doing.
    /// 
    /// > Tip: 
    /// Mutating `input` does *not* invalidate its indices. You can always 
    /// store an ``ParsingInput/.index`` and dereference it later, as long 
    /// as you do not overwrite the [`inout`]() binding elsewhere.
    static 
    func parse<Diagnostics>(_ input:inout ParsingInput<Diagnostics>) 
        throws -> Construction
        where Diagnostics:ParsingDiagnostics, 
                Diagnostics.Source.Index == Location, 
                Diagnostics.Source.Element == Terminal
}

```

am i mistaken or is it necessary to duplicate the _entire_ body of the `protocol`, including the doccomments, since the meaning of the contents of the protocol change if they are moved to an extension.

---

<div class="post-metadata">

**Author:** ![John\_McCall](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/john_mccall/32/1351_2.png) [@John\_McCall](https://forums.swift.org/u/John_McCall)\
**Post date:** [April 11, 2022, 3:13am UTC](https://forums.swift.org/t/se-0346-and-library-evolution/56659/2 "2022-04-11T03:13:31Z")

</div>

That's correct, yes. This was raised as a drawback during the first review, and the Core Team recognized the problem but decided that it wasn't acceptable to constrain the language design around making this easier.

I started a discussion about ways that the project could better support source-compatible library developers with tooling, but there wasn't much enthusiasm about that from the people who replied; people seemed to generally feel that they'd rather deal with these issues themselves.

---

<div class="post-metadata">

**Author:** ![taylorswift](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/taylorswift/32/43413_2.png) [@taylorswift](https://forums.swift.org/u/taylorswift)\
**Post date:** [April 11, 2022, 3:49am UTC](https://forums.swift.org/t/se-0346-and-library-evolution/56659/3 "2022-04-11T03:49:40Z")

</div>

maybe tooling would help in the long run, but based on the discussion you referenced, it sounded like this tooling isn't available yet. so i was more so asking around to see how other people are adapting to the situation _now_ and if there are any useful workarounds libraries can use to adopt this feature as it stands.

---

<div class="post-metadata">

**Author:** ![lukasa](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/lukasa/32/1232_2.png) [@lukasa](https://forums.swift.org/u/lukasa)\
**Post date:** [April 11, 2022, 11:20am UTC](https://forums.swift.org/t/se-0346-and-library-evolution/56659/4 "2022-04-11T11:20:53Z")

</div>

Perhaps I'm off-base, but re-reading [that thread](https://forums.swift.org/t/language-downgrade-tool-for-source-libraries/56019) I don't get the impression that people weren't interested. I see objections of the form "I'd rather allow `#if` in more places", which you've intimated that the Core Team is not interested in doing. I think it's not necessarily reasonable to take that feedback as being disinterested in having a tool: if it's the tool or nothing, people might feel differently.

---

<div class="post-metadata">

**Author:** ![allevato](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/allevato/32/385_2.png) [@allevato](https://forums.swift.org/u/allevato)\
**Post date:** [April 11, 2022, 2:29pm UTC](https://forums.swift.org/t/se-0346-and-library-evolution/56659/5 "2022-04-11T14:29:49Z")

</div>

> [@lukasa](#):
>
> I see objections of the form "I'd rather allow `#if` in more places", which you've intimated that the Core Team is not interested in doing.

My impression reading those threads (which could be wrong, of course) isn't that the Core Team isn't interested in allowing `#if` in more places, just that they aren't interested in specifically a C-preprocessor-style model that lets arbitrary token sequences be conditionalized out. A more syntax-structured approach seems to be more welcomed, but it suffers from the problem that all the syntactic structures need to be audited to determine where `#if` blocks can be allowed, and that until it's implemented it doesn't help folks using features in language versions that have already been released anyway.

---

<div class="post-metadata">

**Author:** ![ben-cohen](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/ben-cohen/32/296_2.png) [@ben-cohen](https://forums.swift.org/u/ben-cohen)\
**Post date:** [April 11, 2022, 2:43pm UTC](https://forums.swift.org/t/se-0346-and-library-evolution/56659/6 "2022-04-11T14:43:31Z")

</div>

> [@allevato](#):
>
> until it's implemented it doesn't help folks using features in language versions that have already been released anyway.

And even once it's implemented, it doesn't help historically, only with future language changes. If you need new tools with new `#if` capabilities in order to work around not being able to adopt new tools, you're back to square one.

---

<div class="post-metadata">

**Author:** ![lukasa](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/lukasa/32/1232_2.png) [@lukasa](https://forums.swift.org/u/lukasa)\
**Post date:** [April 11, 2022, 2:46pm UTC](https://forums.swift.org/t/se-0346-and-library-evolution/56659/7 "2022-04-11T14:46:06Z")

</div>

I don't disagree with either of you: I'm only saying that it doesn't seem appropriate to say that people were disinterested in a solution. They were interested in a solution, and that solution doesn't exist yet.

---

<div class="post-metadata">

**Author:** ![ben-cohen](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/ben-cohen/32/296_2.png) [@ben-cohen](https://forums.swift.org/u/ben-cohen)\
**Post date:** [April 11, 2022, 2:49pm UTC](https://forums.swift.org/t/se-0346-and-library-evolution/56659/8 "2022-04-11T14:49:34Z")

</div>

I think the disconnect is you're talking about "a solution" (which could include future enhancements to `#if`, but those enhancements only work going forward, not backwards) whereas John was referring to "tooling" i.e. a tool that would help adapt code, which could be applied retrospectively even for those not able to update to the newer compiler today (or in the next release).

---

<div class="post-metadata">

**Author:** ![lukasa](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/lukasa/32/1232_2.png) [@lukasa](https://forums.swift.org/u/lukasa)\
**Post date:** [April 11, 2022, 2:53pm UTC](https://forums.swift.org/t/se-0346-and-library-evolution/56659/9 "2022-04-11T14:53:29Z")

</div>

Yes that's fair. I think I probably didn't read John's original post charitably enough: specifically, I misunderstood the message as "We didn't want to make `#if` easier to use, and people didn't want the tool, so here's where we are". A better reading is "We didn't want to block _this proposal_ on making `#if` better to use, and also people don't want a tool, so we'll have to accept this limitation until we make `#if` better to use". My mistake, sorry about that.

_Edit: I suspect part of the misread is that I personally actually am interested in such a tool, but I think it's fair to say I was in the minority in that thread._

---

<div class="post-metadata">

**Author:** ![ben-cohen](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/ben-cohen/32/296_2.png) [@ben-cohen](https://forums.swift.org/u/ben-cohen)\
**Post date:** [April 11, 2022, 3:02pm UTC](https://forums.swift.org/t/se-0346-and-library-evolution/56659/10 "2022-04-11T15:02:12Z")

</div>

FWIW the solution we would apply to this problem in the standard library is probably to use GYB (writing without compiling, not guaranteed correct :)...

```swift
// Foo.swift.gyb

// repeat whole protocol twice with #if swift(>=5.7) #else
% for Variant in ['Primary', 'NoPrimary']:
% if Variant == 'Primary':
#if swift(>=5.7)
% else
#else
// swift < 5.7
% end
protocol Foo
% if Variant == 'Primary':
<Bar>
% end
// regular body for both goes here
{
  associatedtype Bar
}
#endif
% end

```

---

<div class="post-metadata">

**Author:** ![taylorswift](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/taylorswift/32/43413_2.png) [@taylorswift](https://forums.swift.org/u/taylorswift)\
**Post date:** [April 11, 2022, 3:22pm UTC](https://forums.swift.org/t/se-0346-and-library-evolution/56659/11 "2022-04-11T15:22:54Z")

</div>

gyb could be a solution for the standard library and highly industrialized libraries like `swift-syntax`. i don’t know if it’s appropriate for `swift-grammar` to gybbify just for SE-0346. and i don’t know if it’s good for gyb to proliferate more through the ecosystem, we have been trying to deprecate it for years now…

for my use case specifically, the protocol contents are small, so some amount of source duplication could be tolerated. the doccomments are the bigger problem because it is harder to keep them in sync.

---

<div class="post-metadata">

**Author:** ![ben-cohen](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/ben-cohen/32/296_2.png) [@ben-cohen](https://forums.swift.org/u/ben-cohen)\
**Post date:** [April 11, 2022, 3:27pm UTC](https://forums.swift.org/t/se-0346-and-library-evolution/56659/12 "2022-04-11T15:27:27Z")

</div>

Indeed. You should always be "sigh, I guess I should use gyb" not "ooh, I could use gyb for this".

But I do have a feeling "highly industrialized libraries like" and "strong need to keep building even on older tools" may be reasonably correlated.

---

<div class="post-metadata">

**Author:** ![taylorswift](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/taylorswift/32/43413_2.png) [@taylorswift](https://forums.swift.org/u/taylorswift)\
**Post date:** [April 11, 2022, 3:31pm UTC](https://forums.swift.org/t/se-0346-and-library-evolution/56659/13 "2022-04-11T15:31:18Z")

</div>

> [@ben-cohen](#):
>
> But I do have a feeling "highly industrialized libraries like" and "strong need to keep building even on older tools" may be reasonably correlated.

i think you are right that every highly industrialized library has a requirement of building on older toolchains. but i’d argue it’s more of a subset relationship; there’s lots of reasons a non-mechanized library might want to support older toolchains as well. it makes life easier for users if they don’t have to download the newest toolchain to use your library.

---

<div class="post-metadata">

**Author:** ![lukasa](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/lukasa/32/1232_2.png) [@lukasa](https://forums.swift.org/u/lukasa)\
**Post date:** [April 11, 2022, 6:18pm UTC](https://forums.swift.org/t/se-0346-and-library-evolution/56659/14 "2022-04-11T18:18:47Z")

</div>

If someone wanted an interesting Swift package project, rewriting gyb in Swift so that it can be embedded as a SwiftPM extensible build tool from 5.6 onward would be a really useful investment. That removes the need to have a Python interpreter on installing machine, and makes gyb much more useful for the Swift package ecosystem.

---

<div class="post-metadata">

**Author:** ![taylorswift](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/taylorswift/32/43413_2.png) [@taylorswift](https://forums.swift.org/u/taylorswift)\
**Post date:** [April 11, 2022, 6:48pm UTC](https://forums.swift.org/t/se-0346-and-library-evolution/56659/15 "2022-04-11T18:48:46Z")

</div>

[`godot-swift`](https://github.com/kelvin13/godot-swift) does this (see the file [`gyb.swift`](https://github.com/kelvin13/godot-swift/blob/8ed90d1956ccb8a4104a802683bb15bba63a0a7b/sources/godot-nativescript-generator/gyb.swift)).

i used to believe that this was an viable workaround. unfortunately SPM plugins [are not yet mature enough](https://github.com/SwiftPackageIndex/SwiftPackageIndex-Server/issues/1299) to serve as a replacement for commit-time gybbing.

---

<div class="post-metadata">

**Author:** ![lukasa](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/lukasa/32/1232_2.png) [@lukasa](https://forums.swift.org/u/lukasa)\
**Post date:** [April 12, 2022, 7:38am UTC](https://forums.swift.org/t/se-0346-and-library-evolution/56659/16 "2022-04-12T07:38:17Z")

</div>

Sorry I don’t follow how that link implies that plugins can’t do this.

---

<div class="post-metadata">

**Author:** ![taylorswift](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/taylorswift/32/43413_2.png) [@taylorswift](https://forums.swift.org/u/taylorswift)\
**Post date:** [April 12, 2022, 7:21pm UTC](https://forums.swift.org/t/se-0346-and-library-evolution/56659/17 "2022-04-12T19:21:50Z")

</div>

packages with plugins don’t build consistently on non-desktop platforms. there’s no way (that i’m aware of) to conditionally enable plugins with build flags, which means that distributing packages that contain plugins is challenging.

---

<div class="post-metadata">

**Author:** ![lukasa](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/lukasa/32/1232_2.png) [@lukasa](https://forums.swift.org/u/lukasa)\
**Post date:** [April 13, 2022, 9:25am UTC](https://forums.swift.org/t/se-0346-and-library-evolution/56659/18 "2022-04-13T09:25:25Z")

</div>

Re-reading the link I found the feedback number for that, thanks.

---

<div class="post-metadata">

**Author:** ![lorentey](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/lorentey/32/1299_2.png) [@lorentey](https://forums.swift.org/u/lorentey)\
**Post date:** [April 13, 2022, 10:28pm UTC](https://forums.swift.org/t/se-0346-and-library-evolution/56659/19 "2022-04-13T22:28:23Z")

</div>

I strongly believe that package evolution must not be overly constrained by the need to support obsolete toolchain releases. The approach we're using for many packages in the Apple Package Collection is that any new feature release is allowed to increase the required minimum toolchain version, at the discretion of the release manager.

The more support the language gives us to write code that is source-compatible with previous toolchains, the less often we'll need to exercise this. However, there is a practical limit to just how far back it is feasible to provide new package features to obsolete toolchain/language versions -- at a certain point, it becomes impractical to continue working around limitations (and bugs!) of older language releases, because the effort needed to maintain support eclipses the benefits of doing so.

(Note: Requiring a newer toolchain release is very different to bumping the minimum deployment target that the compiled code can run on -- the latter can of course only be done in a new major release.)

This largely makes this a non-issue -- if any of these packages need to add primary associated types, _and duplicating protocol definitions within #if/#else conditionals turns out to be a maintenance burden,_ then we'll simply require a Swift 5.7 toolchain in the release that introduces these.

I encourage other packages to adopt similar policies.

---

<div class="post-metadata">

**Author:** ![lukasa](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/lukasa/32/1232_2.png) [@lukasa](https://forums.swift.org/u/lukasa)\
**Post date:** [April 14, 2022, 6:37am UTC](https://forums.swift.org/t/se-0346-and-library-evolution/56659/20 "2022-04-14T06:37:11Z")

</div>

> [@lorentey](#):
>
> This largely makes this a non-issue -- if any of these packages need to add primary associated types, _and duplicating protocol definitions within #if/#else conditionals turns out to be a maintenance burden,_ then we'll simply require a Swift 5.7 toolchain in the release that introduces these.

Is not having primary associated types a meaningful limitation or a bug of a language release? I understand that this feature improves expressiveness of the APIs, but presumably anything that could be done with it can also be done without it.

[Next page](https://forums.swift.org/t/se-0346-and-library-evolution/56659.md?page=2)