Yeah, I'm aware. I was thinking more along the lines of passing -D CONCURRENCY to opt into certain features early, which would require a dependency if adopted.
My experience was the opposite. When learning SwiftPM I remember relying heavily on long initialisers and the type system (eg enum cases) since they helped me discover the APIs available and figure out what I needed to do.
Given the current state of Swiftâs tooling, Result Builders would make this âlearning by discoveryâ incredibly challenging, encouraging copying and pasting etc. I would not know what types are available to me that I can or should use because, while code completion is getting better, itâs still extremely unreliable as an API discovery tool for anything but trivial cases like filling in the parameters of a function call or creating an enumerated value. Last I noticed, I donât think it can yet suggest to me types I can or should use in a result builder any more than some random, exotic C function.
I love the idea of Result Builders for defining the contents of a package but the tooling isnât there yet to help anyone but the experienced SwiftPM user or the hardcore StackOverflow/Ray Wunderlich pasteboard user.
9 Likes
Negative on this since it will introduce a lot of churn in the third-party package ecosystem (documentation, tutorials, ...), for seemingly little benefit. The Apple Silicon transition is just panning out, and it feels like there needs to be a period of relative stability, in terms of tooling.
New features will be added exclusively to the new format.
It sounds like this will force, or at least encourage, packages to adopt the new format - even if they don't strictly need the new features.
Breaking down the manifest example above shed lights on several key issues:
Are any of these issues blocking implementing any new features, or blocking fixing bugs due to an ambiguity in the current package definition? Or are they purely usability issues?
Are there examples of new, planned, features that are difficult to express cleanly with the existing giant-initializer format?
By definition, any and all features that would add a parameter to said "giant-initializer"s would be difficult to add without further increasing the complexity of the existing APIs.
To your point about forcing the community to adapt: that's just not true. Any package marking its manifest with an older version should keep working, just like they do now. Until you need the new feature you don't need to change anything, as we saw with SPM's resources feature, to name one.
2 Likes
I posted something similar in an idea above, but it was marked as a spam for 3 days. It would be interesting to get some feedback on that approach.
1 Like
Right, but the current package format has been coping with this for years now. Rather than just re-format what's already there, it would be nice to see the re-design happen at the same time as a new major feature has to be incorporated into the format. That would give more input into the design process, and also be easier to sell.
Yes, but in terms of documentation and tutorials, there's going to be a lot of churn. Take for example the swift-algorithms package README section "Adding Swift Algorithms as a Dependency". Many projects have something like this. It's now going to have to list two different ways to add the package.
2 Likes
First off, I think its very nice this is being tackled - the giant initialiser approach can definitely be improved upon. I think the discussion with static formats with escape hatches quite nicely displays why a fully dynamic format like this makes sense and I don't see a strong argument for supporting a simpler static format for a subset of use cases personally.
I'm also happy to see the conceptual merging of targets and products, which always have seemed a bit weird to me personally.
I'm a bit torn on this one, but the script-like variant is probably the more pragmatic route.
I think grouping them by type might be nicer for larger projects and more readable.
As others already mentioned, something more akin to .uses would probably be more understandable, include will always point my head back to C-header land...
I also prefer schemaVersion (or manifestSchemaVersion to make it even clearer perhaps) - rather avoid magic comments.
One could envision a world where SPM (or some other tool) could generate the appropriate markdown for that as a convenience.
2 Likes
There is a big advantage to using result builders, which is that it encodes the structure of the package into the type system at compile time, rather than building it at runtime. @Xi_Ge alludes to what this might mean:
That is â for simple packages made up only of types and constant values, we would no longer need to "run" the manifest in a sandbox. Instead, we could merely compile it and then extract all the information at build time.
More complex use cases will always need the power of running aribitrary code â but chances are this is not needed by many packages, especially many packages people tend to pull down and depend on. When a package needed the ability to run, not just compile, you could then be asked whether you trust it enough to give it this privilege, something you would probably happily grant your own code but not a copy of leftpad you just downloaded.
18 Likes
I'd just like to say that combining targets and products into a single entity might not feel like a big change, but it addresses a lot of usability problems people new to packages have (e.g. what should be imported, the fact that you can't link with products from the same package).
It also addresses a major mismatch compared to frameworks on Apple platforms (which typically contain a module corresponding to the built product) which makes it difficult to build sharable binary artifacts from packages on those platforms today.
As such, I don't think I can agree with the sentiment that this proposal is just a spelling change. This is quite a substantial change to the build model and I don't think we should be aiming to do more in a single proposal.
15 Likes
Added an example unit test to demonstrate how this could be addressed with this API: https://github.com/tomerd/swift-package-manager/blob/feature/manifest2-poc/Tests/PackageManifestTests/Tests.swift#L123
let library1 = Library("library1")
.include {
External("foo", from: "dependency1")
External("bar", from: "dependency2")
}
let library2 = Library("library2")
let package = Package()
.modules {
library1
library2
Executable("executable1")
.include {
library1
(library2, public: true)
}
Executable("executable2")
.include {
library1.name
Internal(library2.name)
}
Test("library-test", for: library1)
Test("library2-test", for: library2)
}
.dependencies {
SourceControl(at: "http://localhost/dependency1", upToNextMajor: "1.0.0")
SourceControl(at: "http://localhost/dependency2", upToNextMajor: "1.0.0")
}
In general this test file can give folks some more ideas on how the API could be used, including the fully imperative API which we want to preserve for complex use case
1 Like
I think the sentiment is is more towards solely ResultBuilders DSL than other changes. These changes could be proposed separately though. Discussion on a single aspect may neglect other proposed changes that are less controversial.
2 Likes
afaict it is true that Result Builders auto complete support in Xcode is not perfect yet. That said, there is nothing inherent to such API which prevents it from becoming great, so its more of tooling catching up.
having worked with and on both APIs, the Result Builders API suggested here with it's strongly typed approach (for modules as one example) is far simpler to type and get right than the polymorphic initializers the existing API offers.
cc @rintaro
2 Likes
being able to rely on constant evaluation and the type system instead of executing the manifest to produce a JSON static representation would be amazing for reducing security surface area. looking forward to hearing more on what is possible to that end.
4 Likes
Which platforms use a security sandbox? (A few years ago it was macOS only.)
Should the add-product and add-target subcommands (from SE-0301) be revised?
It sounds like that piece should in fact be broken out into its own proposal then.
1 Like
Agreed, but you've missed my point. With gigantic initialisers, people -- especially those learning SwiftPM -- at least had a "handrail" to climb the staircase of defining their package. One simply fills in the parameters. With result builders, the handrail is removed but the staircase remains. One would expect the tooling to be more helpful for this use case but it isn't, so people will be more inclined to copy and paste from elsewhere.
Say someone opens a package definition:
Package {
// Insertion cursor blinks.
// What goes here?
// Modules? Dependencies? Targets? Libraries?
// How does SwiftPM expect me to model stuff?
// Guess I better read the headers or
// copy and paste examples from documentation đ¤...
}
To my mind, this is a usability regression because the interface and the lack of tooling make modelling a package quite mysterious. Perhaps, in the absence of tooling, the template packages could come with commented out examples, to help clear the mystery?
Module might also be a bit too technical of a concept for many people, too, despite it being completely correct. People tend to think of modules as apps, tools, plugins, frameworks, libraries, tests etc.
I wonder if import would be better, since it matches the inter-module relationships people are familiar with in the source code? Or importable, in case the compiler can optimise out unused dependencies?
I've been keeping up with the thread and overall I'm very positive with this change. I think that having a DSL for SPM while keeping the power of being Swift code is very powerful and desirable (is one of the best things about cocoapods).
I agree with Kyle here. I wouldn't like to see this as a normal @main script. I don't see the point really, it just adds verbosity for no gain. We should be able to just define a top level Package and that should be the entry point (even better if it turns out you don't even have to run it, which seems a very interesting approach).
I hear the concerns about discovery but I'm not sure how big a problem it is. Realistically I barely write a package file from scratch. Mostly I start with the generated template and move or copy paste things around. Most of what I endup doing is adding a min platform req, and it took me a while to know the correct order, and autocompletion is not very helpful there. So I don't think the difference of discovery with this change is that big tbh.
But there is one thing that has me VERY intrigued:
Is this really true? How is this a tooling issue and not a language issue. AFAIK there is no way of restricting a type construction into a scope of a result builder (for example that Internal/External types only make sense inside an include closure). So the language won't tell the tooling what types are more appropriate inside the DSL, unless something is hardcoded somehow. I'm asking because I often find this difference when comparing APIs with Kotlin, mainly because they have a small functionality with tons of potential, what on this forums has been called in the past "self rebinding". I don't want to derail the conversation to that, but I wonder what the tooling can do to solve this if the model of the language doesn't offer how to describe it.
3 Likes
Just to be clear, Iâm all for the result builder approach. I think the regression in the UX should be clear so an adequate compensation can be made. After all, weâve figured out how to build apps in SwiftUI, even if it was through some measure of copying, pasting and tweaking until an intuition about the APIs was developed.
1 Like
You are right that the current language doesn't have a way to express like "you can only use these certain types (e.g. Internal and External) in this context". So there's no way to limit types that appear in code completion depending on the context(, nor should we do so). All we can do is prioritize items that have the right type (e.g. that conforms to Dependecy protocol).
In a result builder context, the builder type can provide the contextual type to elements in its context. For example:
protocol Animal {}
struct Lion: Animal {}
struct Penguin: Animal {}
struct Zoo {
@resultBuilder
struct Builder {
static func buildBlock(_ animals: Animal...) -> [Animal] { animals }
}
init(@Builder _ body: () -> [Animal]) {}
}
let _ = Zoo {
Lion()
<HERE>
}
We should be able to know that the contextual type is Animal because Builder.buildBlock(_:) only accept Animal. So code completion should be able to prioritize types that conform to Animal protocol, that is Lion and Penguin. (But we should/can not stop suggesting other types/functions/variables because they might have properties/methods that returns an Animal type.)
7 Likes
That being said, there are possible improvements here. Like, for example, when you manually invoke code completion (with ESC, Ctrl + space, or Ctrl + .) without filtering text, it only shows items with matching types, so you can choose from the UI. As soon as you type something, it shows normal completion items filtered by the input characters.
It's just an idea, I haven't really thought it through, so I don't know if this is a good idea or not.
1 Like