it feels like an eternity ago when the swift-collections incubator was first announced, and for many years this repo has hosted a number of absolutely fundamental data structures:
OrderedDictionary<Key, Value>
OrderedSet<Element>
Deque<Element>
because nobody should ever be writing a Deque from scratch for production.
but it seems over time the effort has lost momentum and it’s not clear to me when, if ever, these types are actually going to graduate and ship with the toolchain.
Q: how do i get a Dictionary that preserves insertion order?
A: great question! you just need to use the OrderedDictionary type instead of a regular Dictionary.
Q: perfect! it’s not working when i type OrderedDictionary in my IDE, do i need to import something like you have to import the Dispatch module?
A: no no no no, you cannot just import OrderedCollections. you have to add this third party dependency which will clone this third party github repository. and you have to make sure you check the version number every now and then because it won’t update with your swift version.
As far as I recall, no external implementation from any of Apple's libraries has ever graduated to the standard library, and there exists no formal process to do so (other than normal evolution). This includes API from already approved proposals.
I have zero knowledge of the swift-collections development process and their future plans, but is there any particular reason to expect all such efforts to “graduate” to the standard library? What’s wrong with keeping this and related projects as separate packages? There are definite downsides to merging into the stdlib, for example it potentially slows down evolution by tying everything to the release cycle of the compiler, and certainly it makes backward deployment more difficult or impossible, which is already comes up every single time a new language feature requires runtime support.
If nothing else, I have been frustrated by the fact that you can’t integrate it into projects that use CocoaPods without restructuring the whole thing to use SPM instead.
Like the Swift Algorithms and Swift Numerics packages before it, we’re releasing Swift Collections to help incubate new functionality for the Swift Standard Library.
(emphasis added)
So at least at the time of its announcement, there seemed to be some intention to eventually move the functionality of Swift Collections into the Standard Library. It may be that there are reasons to not do that and instead keep them as separate packages, but as far as I know, that change in direction has not been discussed here.
Despite saying "Standard Library", it seems that they’re going to be separate modules in the standard library collection, which require explicit imports (like RegexBuilder, Observation and Distributed). This means that the only difference of graduation is the ability to use the library without SwiftPM.
to be clear, this is a huge motivator in and of itself. having a module ship with the toolchain (which is reasonable if both codebases have a similar release frequency) means CI can run in a sandboxed environment without network access. of course, it is totally possible to work around this - the swift CI itself uses local mirrors of these repositories for performance reasons - but the issue was never that there are no workarounds, the issue is that critical functionality isn’t “included” in the toolchain, and requires additional effort to use.
As you've pointed out yourself above, Dispatch is an example of something that, as a "core library," ships as part of the toolchain on all supported platforms but is not part of the standard library. If that's what's desired, then the ask is to make some or all of Swift Collections another core library.
On Apple platforms, Xcode has been configured out of the box with a package collection for Apple's open-source Swift packages, and the experience of using these packages is (IMO) quite smooth. If what's desired is a more frictionless user experience, then the ask is for parity in open-source tooling—for example, can SwiftPM do more with installed package collections and be similarly preconfigured?
As a specific ask, "graduating" to the standard library, whether as part of the core Swift module or as another module (either explicitly or implicitly imported—which again is a distinct ask), has two major effects that come to mind: (1) what @Slava_Pestov mentioned in terms of deployment (some of which is shared in common with core libraries, as I understand); and (2) subjecting the entire user-facing API, and any changes to it going forward, to review by the language steering group via the Evolution process. It doesn't seem that either of these effects represent solutions to the concerns that you're actually bringing up as motivation.
the standard library isn’t the same thing as the Swift module, and i personally don’t think the terms “standard” and “core” libraries are helpful, given:
if there exists such a dichotomy, then there is only one “standard” library, the Swift module, and
there exist many “core” libraries (e.g. CoreData) that don’t ship with the toolchain because many people use core as a synonym for important, and everyone thinks their own library is important.
but regardless, your assumption is correct, i believe that the most appropriate home for OrderedDictionary would be in a toolchain module and not the Swift module.
as @bbrk24 mentioned, integrating third-party repositories more deeply into SPM doesn’t help people who aren’t using SPM, and the requirement to use SPM is itself an obstacle to using these libraries.
i will also say that in my SPM-centric work, a large part of the friction arises from the need to fetch, version, and cache data from a github repository over the internet, whereas a “pure”, self-contained package that uses only toolchain modules can be built and tested in an isolated environment.
there are many experimental modules that ship with the toolchain (_Differentiation? StringProcessing?) that have added voluminous API to standard library types without going through evolution.
Some of us view the standard library as just being the core Swift module while some of us also view the standard library being the family of modules that come by default (this includes string processing, concurrency, distributed, etc.) (I don't view "core" libraries like Dispatch to be part of the standard library but I digress) In any case, whether these types would go in Swift or some new Collections module doesn't really matter because either approach would induce availability.
_Differentiation is weird because it's a leftover experimental module and shouldn't be treated like a standard library module (possibly removed from the toolchain) unlike string processing who has gone through evolution.
CoreData has nothing to do with the Swift project and is not a "core library" in the way the Swift project uses that term. As with CoreAudio, CoreGraphics, CoreText, CoreFoundation, and CoreAnimation, CoreData is an Apple system framework, and the "CoreX" naming convention relates to Apple's system architecture.
There is a "Core Libraries" layer which is like the lowest-level system libraries, and the "-Kit" frameworks are built using them. For example, AppKit and UIKit are higher-level frameworks which depend on CoreAnimation and CoreGraphics, TextKit is a higher-level framework which depends on CoreText, Foundation (not FoundationKit) is a higher-level framework which depends on CoreFoundation, etc.
(I think that's right - but this is the kind of place where there may be veterans who can correct me)
It may be an unfortunate clash of terms, but they mean very different things. I think it helps to understand why these frameworks are named as they are. They each make sense in context.
Of course, this convention seems to have been superseded by one which uses a "Swift" prefix (UIKit -> SwiftUI, CoreData -> SwiftData). It is fair to question whether that in fact makes it more confusing as to which libraries are part of the Swift project and which are proprietary Apple system frameworks, but that's perhaps getting too far from the original topic so best left to another thread if somebody wants to discuss it, I think
This dichotomy sounds right to me. I'd add that "Swift" named frameworks were named to emphasise the fact that their API is inherently Swift based so that hardly anyone could complain there about the lack of Objective-C bindings for SwiftUI or SwiftData.
To me this topic question equivalent would be: could I use framework "X" (like OrderedCollections in this case) from within environment like godbolt or one of those interview question sites. Spelling prefixes like "Core" doesn't make it a clear cut, for example I could use "Foundation" (which as Karl mentioned could be treated as a misnomer for FoundationKit), CoreFoundation and Dispatch in those environments, but not CoreServices, CoreGraphics, CoreText or AppKit.
By “core library,” I am referring to the term as used in the Swift project itself to describe Foundation, Dispatch, and XCTest.
Apple proprietary frameworks named “Core*” (e.g.: CoreData) aren’t core libraries in this usage of the term, just as Apple proprietary frameworks named “Swift*” (e.g.: SwiftData) aren’t Swift standard library frameworks.
Personally, the biggest issue to solve with "graduation" is avoiding the use of SPM, or at least avoiding public use of SPM.
Even without considering SPM itself, it's pretty annoying that things as useful as collections or algorithms are locked behind a dependency users may not know anything about. Some sort of discoverability here would be great, but even solving that issue wouldn't really help the situation too much.
At a hight level, SPM's ongoing performance and reliability issues, especially within Xcode (as the most popular SPM interface), have lead to a feeling of "avoid it if you can" in much of the community. That's in addition to the existing "don't use any dependencies at all" reaction to Node's many issues which has decreased the viability of all external dependencies. Limiting ourselves to just SPM rather than what Apple could do for the ecosystem as a whole, SPM's early Xcode integration and ongoing performance and reliability issues really left a bad impression in the community, leading many to avoid using it as much as possible. "Graduating" the various swift- libraries to no longer require SPM would allow those who avoid SPM to finally start using these valuable libraries. (Supporting dependency managers other than SPM would also help here, but that seems unlikely.)
Given those attitudes in the community, a narrower issue is that SPM doesn't support any form of private dependency. So if I create a great library that needs OrderedDictionary but I don't need to expose it directly, I have no way to depend on swift-collections without forcing everyone who depends on my library to depend on it as well. Not only does this trigger some people's gut reaction ("ugh, this library has a dependency") but it can create conflicts within a consumer's dependency tree where more than one dependency requires the same library but may have slightly varying version requirements. This is especially true for something as fundamental as swift-collections. So having no way to hide these requirements, or even rename them inside my own dependency tree, is rather detrimental to the overall ecosystem.
I'd like to see support for private dependencies rather than graduating these libraries, as it prevents all of Apple's issues with back deployment and allows for faster (relatively speaking) evolution of the libraries themselves (not that they evolve much anyway). Of course, improvements to SPM's general performance and capabilities or Xcode's SPM DX would be appreciated as well, and would improve opinions of SPM in the community at large.
I wasn't aware of this, and my personal experience with SPM is probably different, but if the majority of the Swift community tries to actively avoid using SPM (which I find baffling), I'd say that improving it and its reputation should be one of the highest priorities of the relevant workgroups.
Sometimes "avoid it if you can" manifests not as avoiding SPM as a whole, which is largely impossible unless you're a real anti-dependency zealot. Instead, it may be about avoiding anything but the bare minimum of dependencies that you're required to have (paid third-party SDKs mostly). In that case, evaluating other dependencies isn't a technical matter but one of pain mitigation. "How much longer will package resolution take if I add this?" "Does this package include a playground, which breaks Xcode?" "Will adding this break my SwiftUI previews (even more than they are now)?"
SPM's problems from the perspective of people consuming packages are largely actually Xcode problems, which puts them out of scope of the Swift workgroups.
Xcode's DX issues are, of course, but many of the underlying behaviors are fundamental SPM issues, like overall performance. For instance, the issues outlined by @pyrtsa in this thread. Notably, neither of the PRs he linked at the end have been merged in any form, so there is still quite a lot of performance left on the table. Improvements to SPM in those areas would immediately benefit Xcode without needing Xcode to do anything.
There are also various functional SPM limitations which prevent it from fully replacing other package managers (or just being used at all), like the inability to manage build configurations, and the complete block on custom build flags, just to name two. And the issue of private dependencies has nothing to do with Xcode at all. So there's really quite a lot we could do here.
There's another niche use case: when taking an interview on coding platforms like codility, coderpad, etc you won't be able using SPM (even if it was flawless). So when it comes to Swift you'd will be able using an array or a dictionary or a set, but not an ordered dictionary.
Another niche use case is attempt to use a non local SPM package when no internet is available (airplane, submarine, Devonshire, etc).
Or even the most general use case, people learning the language by completing code challenges, or any simple task or tutorial at all. Do we really need to force them to learn SPM just to get an OrderedDictionary, or be able to chunk collections?
