Adding to my previous post, I think it’s also important to have a reasonable expectation of timeframe. The focus for Swift 5 is ABI stability. Concurrency will almost certainly be a focus before HKT would be considered. In terms of generics features, generalized existentials are definitely considered a higher priority than HKT (maybe someday the fear many have of protocols with associated types will fade).
With all of this in mind, I would expect to see it take at least a few years before HKT is added to Swift. One way to look at this is to be disappointed. Another is to take the time to make the encoding as easy as possible to work with, build libraries with it, and produce compelling examples that use the libraries. This is a lot of work, but it would also provide an excellent demonstration of how real world Swift code can benefit from HKT.
From what I've seen the topic of HKTs appears to have been in discussion since shortly after Swift's 1st release -- so my question is whether this need to "provide compelling examples" is a new requirement (i.e. recently defined by the core team?), or whether something has already been provided and rejected? I'm still playing catch up here.
No worries on the timeline and reasonable expectations; I expected as much re the effort for ABI, stability and performance.
IMO this probably should be pitched for version 7, re I imagine version 6 would probably still be heavily focused on bedding down the former priorities, and fulfilling other long standing requests.
No, core team members have asked several times for pragmatic examples of how people want to use HKT. My (possibly inaccurate) impression is that there is mild interest in HKT but also a healthy skepticism about the need for them in Swift especially given it’s syntactic sugar for Optional, the error model, plans for async / await, etc. There has also been some skepticism expressed about whether people really need abstractions that require HKT to express instead of just adopting patterns that are adopted by concrete types,
Thus far there may be a couple of good examples in the threads on HKT but there has been a lot of throwing up the usual FP abstractions without demonstration of how they would improve Swift code in practice (again, as abstractions rather than patterns). We can and must do better than this over the next couple, years. The complexity HKT add to the type system and programming model is significant enough that I think asking for this kind of strong motivation is very reasonable.
One way to reduce the perception that HKT is only useful for fancy FP techniques many people have no interest in would be to look for examples in C++ code that has templates with template parameters. I don’t personally have the time or motivation to do that but it may be a fruitful avenue to show that HKT could be useful in other styles of programming as well. It would be great if someone with a modern C++ background was able to do that.
Thanks for the clarity. Guess I also missed the discussion around async / await ... and I see actors came into that as well. Those are big ticket items with a far greater audience than HKTs or generics in general.
I'm too rusty in that area; haven't done more than dabble in C and C++ for well over a decade. So I can only offer to focus within my current stable of languages (Scala, Java, Rust, Kotlin, Javascript, C#, F#, and Haskell) for any good examples befitting the criteria.
Anyway thanks it's a lot more clear there are far more big ticket items than I anticipated waiting in the queue so its doubtful HKTs will be a priority for the foreseeable future (and I can live with that); because overall its not necessarily a bad thing, as it provides sufficient time for the greater community to exercise the limits, which will hopefully lead to a more fruitful discussion down the line.
I'll probably write a blog article or two about those topics, for they're probably too huge for a post in a forum, but I'll report the articles here as soon as they're ready.
I'd also like to observe again that the traverse example that was given in multiple shapes is a practical pattern for utilizing some abstractions requiring HKTs, so it's not like nothing was given in that area.
In my opinion, HKT is very useful in real applications. I think is more convenient then built-in possibilities. I use the implementation found here: Swift Functors, Applicatives, and Monads in Pictures | mokacoding, where the article explains, why is it worth using.
Ok, I found some I think good examples where you can see advantages when you use monads. Of course, I think it's not any problem to read the source code in any other programming languages. I don't think FP is the cure for everything but still, I'm convinced about cases that help a lot while developing code.
Here's a practical use case for formalizing functors specifically—they allow a natural extension of "newtype deriving" for protocols with Self in non-trivial structural positions. It would a useful language feature for a struct or enum with one element to be able to selectively forward functionality of the wrapped element, by forwarding protocol conformances, e.g.:
It is easy to forward operations with Self in argument, inout, or return position, since the two-way mapping from wrapper to wrappee is trivial. However, if Self appears in a structural position, like Array<Self>, we'd need to be able to map between Array<Wrapper> to Array<Wrappee>, which in turn would require that Array have a formal notion of being a functor.
Generally when you want to learn a new topic, you need:
1 motivation to keep pushing to learn more and improve
2 time to understand each new concept
3 practice/Immersion to internalize each concept
FP languages can be really dense with concepts, even in syntax, and those concepts may be genuinely new even experienced developers coming from imperative languages. Especially who don't have a background that exposed them to more advanced math concepts like lambda calculus and category theory ;-)
If I were in your shoes I wouldn't feel stupid, just remember how you had to learn at first - slowly, with a lot of motivation to grow your understanding of the concepts. I'm in the same boat BTW
This has probably been brought up already, but the guys at https://www.pointfree.co do an excellent job of introducing you to functional programming. They show you the concept and how it works instead of naming the concept. Their first episode is fantastic, but immediately after that, they accelerate to breakneck speeds. It's going to take me a few viewings, or watching it in super-slow-mo.
I say all of this as someone who knows nothing about functional programming, and has not gotten past the middle of episode 2, where they lost me.
As promised, I published a first article on the utility of HKTs, focused on monads: Why Monads?
Other than strictly talking about the higher-kinded aspect of monads, I also wanted to give my personal "introduction" to the matter, so it could probably be useful for those who are interested in functional programming in general.
Great article, thanks for putting a lot of effort it it. I enjoying reading it through, the UserSession/UserIdentity example is a nice start.
It’s a great article, for anyone interested in language theory. However I think most coders are just looking for new tools to use and this gets quite theoretical: for them it’s fine to have a bit of black box that does the thing and only know how to use the tool in own code. For example, they would prefer to have a few lines of hello world example of running a swift http server rather than having an article explaining how http protocol works in detail.
So the first question that comes to mind is that who would use HKTs? Two scenarios below
For ”anyone”: this would have to mean really easy tools, like swift has discouraged ”for i=0; i..” indexed loops and encouraged ”for element in ...” loops. And people are discovering .map(), .flatMap() and .filter() to do even nicer things. So is there something in HKTs that could be as swifty and easy as that? Maybe something new that doesn’t just copy the current HKT conventions (even if it takes advantage of them under the hood). The only thing that comes to my mind is some protocol in stdlib that people could conform to (or stdlib would already conform to)
Only for framework/library authors and other power users. In this scenario, HKTs are not directly visible to regular coders. It might be the plumbing that makes it all work in a library, but coders wouldn’t necessarily even need to know about it. Maybe like rxSwift where you learn few things about the library API and then get on using the tool. In this scenario, different libraries can compete on what is the best “plumbing” while even providing a very similar API to coders. It would be very important for Swift to remain reasonably agnostic so that the different “plumbings” can compete on equal footing. For this scenario, I think it would be important to create HKT libraries that get used in real life, and use whatever workarounds needed to overcome shortcomings of swift language. From real life usage we could get understanding on what to bring to swift stdlib if needed.
Not to @Moximillian specifically, but I just have to ask:
Is there anyone who (1) has made extensive use of HKTs in some language that has them and (2) thinks HKTs are not worth adding to Swift?
And if so, why?
I think discussions based on answers to that would be more informative than a (possibly endless) list of actual, practical, concrete, real world examples of exactly how and why Swift should have HKTs, a feature that as far as I can understand is about lifting some limits of the type system and thereby increasing the abstraction capabilities of the language.
A Thought Experiment
What if, while discussing this topic, we sometimes try and see what happens if we replace (mentally) HKTs with HOFs, ie higher order functions, or (if sloppily defined) Swift's closures.
So we'd also pretend that
No or very few of the "main stream" language like Swift, C++, Python, Java, Obj C or Javascript had HOFs (ie closures, blocks, lambdas, etc).
We happened to be among the minority of people who've used HOFs, probably in some "academic" language, and to us they felt both obvious and indispensable (just like we probably feel about Swift's closures).
Now imagine that we had to argue for exactly why we would like HOFs to be added to Swift.
People are telling us that we have to provide compelling real world examples to justify its inclusion.
When we try to explain and/or give some examples, we are told that the concept of HOFs might be too complex to be understood by anyone and would be used only by a few power users. And that our examples are too few, abstract or contrived, and that it would be great if we could present a report on how eg using these "closures" could benefit Swift's standard library.
Public discussions would go on and on about eg:
About the C-word, what does "closure" even mean? Why call it something that nobody (except computer science wizards) understands?
Swift is for ordinary people too, it should not require that users understand untyped lambda calculus!
Actually, in languages that has HOFs, it has been observed that people tend to get all excited about closures, but in the end, they often find themselves in a mess. In fact, there's one type of problem that is so common that it even has a name: "callback hell". So, it's not obvious that adding HOFs to Swift would be worth it.
I have looked at the definition of "closure" and it doesn't make sense the way you use it. A closure is a function that "closes over" or "captures" (part of) the environment that contains all of the free variables as available in the scope of that function, while HOFs are just functions that can take one or more functions as arguments and return a function as a result. Perhaps you mean FCFs (first-class functions) and AFs (anonymous functions)?
There's no need for HOFs, you can simply use C function pointers or objects to achieve the same thing, or code generation.
...
And yet, once the dust has settled, most main stream languages has some implementation of HOFs, and "ordinary" programmers find them obvious and indispensable, not remembering exactly how or when they learned how to use them.
And of course a lot of inexperienced programmers makes a mess using closures, just like they do with any other language feature, because that's part of becoming more experienced.
Was thinking along the same lines; as both @ExFalsoQuodlibet and you surmised...
...and as your thought experiment posited; this certainly never been restricted domain of HKTs, or any other feature more recently exposed in Swift.
I can buy notion that higher kinded polymorphism would have a lesser priority up against other missing mainstream features; but what I can't buy is this cyclical justification bumf.
No quite the same thing -- If there's merit for C++ templates, and Swift's current generics; then HKTs generics on generics should equally make sense -- unless of course you see no value in either of those.
Try @Jens thought experiment. It's quite easy to throw most any feature in a hamster wheel and wonder why after 2 years no progress is being made.
