i’m currently auditing swift-grammar for PAT candidates among its variousassociatedtypes and i’m running into the “can’t #if the title of the protocol” problem:
/// A structured parsing rule.
public
protocol ParsingRule
{
/// The index type of the ``ParsingInput.source`` this rule expects.
///
/// Parsing rules must be associated with a source location type because
/// some applications may wish to store these indices in the returned
/// ``Construction``s. If the source location type were not fixed, then
/// different calls to ``parse(_:)`` could potentially return constructions
/// of varying types, which would require additional abstraction, which would
/// be inefficient.
///
/// > Tip:
/// Implementations can satisfy this requirement with generics, allowing
/// parsing rules to be reused for different input types.
associatedtype Location
/// The element type of the ``ParsingInput.source`` this rule expects.
associatedtype Terminal
/// The type of the constructions produced by a successful application of this
/// parsing rule.
///
/// Implementations should not report failure through an ``Optional``
/// construction type. Instead, implementations should [`throw`]() an ``Error``,
/// which allows the library to perform appropriate cleanup and backtracking.
associatedtype Construction
/// Attempts to parse an instance of ``Construction`` from the given
/// parsing input.
///
/// The implementation is not required to clean up the state of the `input`
/// upon throwing an error; this is handled by the library.
///
/// Implementations *should* interact with the given ``ParsingInput`` by
/// calling its methods and subscripts. Don’t overwrite the [`inout`]() binding or its
/// mutable stored properties (``ParsingInput/.index`` and ``ParsingInput/.diagnostics``)
/// unless you really know what you’re doing.
///
/// > Tip:
/// Mutating `input` does *not* invalidate its indices. You can always
/// store an ``ParsingInput/.index`` and dereference it later, as long
/// as you do not overwrite the [`inout`]() binding elsewhere.
static
func parse<Diagnostics>(_ input:inout ParsingInput<Diagnostics>)
throws -> Construction
where Diagnostics:ParsingDiagnostics,
Diagnostics.Source.Index == Location,
Diagnostics.Source.Element == Terminal
}
am i mistaken or is it necessary to duplicate the entire body of the protocol, including the doccomments, since the meaning of the contents of the protocol change if they are moved to an extension.
That's correct, yes. This was raised as a drawback during the first review, and the Core Team recognized the problem but decided that it wasn't acceptable to constrain the language design around making this easier.
I started a discussion about ways that the project could better support source-compatible library developers with tooling, but there wasn't much enthusiasm about that from the people who replied; people seemed to generally feel that they'd rather deal with these issues themselves.
maybe tooling would help in the long run, but based on the discussion you referenced, it sounded like this tooling isn't available yet. so i was more so asking around to see how other people are adapting to the situation now and if there are any useful workarounds libraries can use to adopt this feature as it stands.
Perhaps I'm off-base, but re-reading that thread I don't get the impression that people weren't interested. I see objections of the form "I'd rather allow #if in more places", which you've intimated that the Core Team is not interested in doing. I think it's not necessarily reasonable to take that feedback as being disinterested in having a tool: if it's the tool or nothing, people might feel differently.
My impression reading those threads (which could be wrong, of course) isn't that the Core Team isn't interested in allowing #if in more places, just that they aren't interested in specifically a C-preprocessor-style model that lets arbitrary token sequences be conditionalized out. A more syntax-structured approach seems to be more welcomed, but it suffers from the problem that all the syntactic structures need to be audited to determine where #if blocks can be allowed, and that until it's implemented it doesn't help folks using features in language versions that have already been released anyway.
And even once it's implemented, it doesn't help historically, only with future language changes. If you need new tools with new #if capabilities in order to work around not being able to adopt new tools, you're back to square one.
I don't disagree with either of you: I'm only saying that it doesn't seem appropriate to say that people were disinterested in a solution. They were interested in a solution, and that solution doesn't exist yet.
I think the disconnect is you're talking about "a solution" (which could include future enhancements to #if, but those enhancements only work going forward, not backwards) whereas John was referring to "tooling" i.e. a tool that would help adapt code, which could be applied retrospectively even for those not able to update to the newer compiler today (or in the next release).
Yes that's fair. I think I probably didn't read John's original post charitably enough: specifically, I misunderstood the message as "We didn't want to make #if easier to use, and people didn't want the tool, so here's where we are". A better reading is "We didn't want to block this proposal on making #if better to use, and also people don't want a tool, so we'll have to accept this limitation until we make #if better to use". My mistake, sorry about that.
Edit: I suspect part of the misread is that I personally actually am interested in such a tool, but I think it's fair to say I was in the minority in that thread.
FWIW the solution we would apply to this problem in the standard library is probably to use GYB (writing without compiling, not guaranteed correct :)...
// Foo.swift.gyb
// repeat whole protocol twice with #if swift(>=5.7) #else
% for Variant in ['Primary', 'NoPrimary']:
% if Variant == 'Primary':
#if swift(>=5.7)
% else
#else
// swift < 5.7
% end
protocol Foo
% if Variant == 'Primary':
<Bar>
% end
// regular body for both goes here
{
associatedtype Bar
}
#endif
% end
gyb could be a solution for the standard library and highly industrialized libraries like swift-syntax. i don’t know if it’s appropriate for swift-grammar to gybbify just for SE-0346. and i don’t know if it’s good for gyb to proliferate more through the ecosystem, we have been trying to deprecate it for years now…
for my use case specifically, the protocol contents are small, so some amount of source duplication could be tolerated. the doccomments are the bigger problem because it is harder to keep them in sync.
i think you are right that every highly industrialized library has a requirement of building on older toolchains. but i’d argue it’s more of a subset relationship; there’s lots of reasons a non-mechanized library might want to support older toolchains as well. it makes life easier for users if they don’t have to download the newest toolchain to use your library.
If someone wanted an interesting Swift package project, rewriting gyb in Swift so that it can be embedded as a SwiftPM extensible build tool from 5.6 onward would be a really useful investment. That removes the need to have a Python interpreter on installing machine, and makes gyb much more useful for the Swift package ecosystem.
i used to believe that this was an viable workaround. unfortunately SPM plugins are not yet mature enough to serve as a replacement for commit-time gybbing.
packages with plugins don’t build consistently on non-desktop platforms. there’s no way (that i’m aware of) to conditionally enable plugins with build flags, which means that distributing packages that contain plugins is challenging.
I strongly believe that package evolution must not be overly constrained by the need to support obsolete toolchain releases. The approach we're using for many packages in the Apple Package Collection is that any new feature release is allowed to increase the required minimum toolchain version, at the discretion of the release manager.
The more support the language gives us to write code that is source-compatible with previous toolchains, the less often we'll need to exercise this. However, there is a practical limit to just how far back it is feasible to provide new package features to obsolete toolchain/language versions -- at a certain point, it becomes impractical to continue working around limitations (and bugs!) of older language releases, because the effort needed to maintain support eclipses the benefits of doing so.
(Note: Requiring a newer toolchain release is very different to bumping the minimum deployment target that the compiled code can run on -- the latter can of course only be done in a new major release.)
This largely makes this a non-issue -- if any of these packages need to add primary associated types, and duplicating protocol definitions within #if/#else conditionals turns out to be a maintenance burden, then we'll simply require a Swift 5.7 toolchain in the release that introduces these.
I encourage other packages to adopt similar policies.
Is not having primary associated types a meaningful limitation or a bug of a language release? I understand that this feature improves expressiveness of the APIs, but presumably anything that could be done with it can also be done without it.