[Proposal] Higher Kinded Types (Monads, Functors, etc.)

Optional.map and Array.map do different things, and unifying them seems harmful.

They actually don’t do different things. Optional can be thought of as an array of zero or one elements; from that point of view, operations like `map` and `flatMap` which are supported on both actually do the same thing.

I understand that reasoning, but it's completely contrary to how I (and presumably many other people) normally think. To me, optional is not a 0..1 array, it's just an optional.

I don't want a flatMap on optionals (which, I believe, was thankfully removed in Swift 2).

Thankfully it lives : )

// useless function: return arg if less than 10, otherwise .None
func f(x:Int) -> Int? {
  return x < 10 ? x : .None
}

Int("9").map(f) // .Some(.Some(9))
Int("9").flatMap(f) // .Some(9)

Int("10").map(f) // .Some(.None)
Int("10").flatMap(f) // .None

So I don't want to hijack this thread with useless flame, so one last thought:

Int("9").map(f) // .Some(.Some(9))
Int("9").flatMap(f) // .Some(9)

Int("10").map(f) // .Some(.None)
Int("10").flatMap(f) // .None

My bad. But doesn't it show just how unreadable it is? I guess you're used to it, but it looks like complete gibberish to me because I don't really know the vocabulary, and, more importantly, the kind of thinking that requires using a that vocabulary is actively against the kind of thinking I'm used to.

Arguably it should just be the default behavior of map. Double-optionals are unruly.

It’s my understanding that these unruly double-Optionals are required in various parts of the Swift library. In particular with generators – a generator knows it’s got to the end when it encounters nil (.None). If it were only possible to have one level of Optional it would be impossible to enumerate an Array of Optionals. For example

[1,2,nil,4,5]

Without the ability to represent .Some(nil), the generator would reach the end after ‘2’ in the Array and would never get to ‘4’, ‘5’.

Now, I fully accept that everyone wants to write a different kind of Swift. I just don't want my code to be pestered with FP concepts, which will be unavoidable if too much FP makes it into the standard library. So please, don't forget to consider us the FP haters.

A.

I wouldn’t want the FP concepts to be an immediate burden to users of Swift (for example having to be able to explain the monad laws before writing a ‘hello world’ program). However if these concepts are present I don’t see why it would be an immediate burden, life would go on happily as before and if people wish to delve into the FP aspects then they can do – they might find it quite fun, I do (and I most certainly don’t have a PHD in category theory : )

Al

···

On 17 Dec 2015, at 12:05, Andrey Tarantsov <andrey@tarantsov.com> wrote:

Oh, thanks for clarification, I misunderstood you at first. If the proposal is about improving the type system so that HKTs are expressible (or more easily expressible) than I definitely support it :)

···

On 17 December 2015 at 13:16:39, Will Fancher (willfancher38@gmail.com) wrote:

As long as the underlying theory is not widely adopted, I think HKTs should be provided by third-party libraries (like swiftz).

I wholeheartedly agree that protocols like Monad and Functor shouldn't be in the standard library. I really think they need their own external package.

But HKTs can't be provided by third party libraries! HKTs are a language level feature! HKTs aren't a collection of types we wish were implemented. HKTs are the language level construct that allows for generic types to be re-parameterized. An HKT proposal would NOT insist that Monad and co. be written into the standard library. It would simply insist that HKTs become physically possible, which they currently are not.

Big +1 for HKTs from me!

-Thorsten

···

Am 17.12.2015 um 13:10 schrieb Will Fancher via swift-evolution <swift-evolution@swift.org>:

Optional.map and Array.map do different things, and unifying them seems harmful.

In terms of category theory, they're not different. They are both arrows from a to b in the Functor category. Obviously that's horribly abstract, so I'll put it this way: If you think of these map functions not as operations on Optional and Array, and instead think of them simply as ways to compose an arbitrary data structure, they are in fact doing the same thing. On an implementation level, and on a runtime level, they perform differently. But on a type level, a theoretical level, and a categorical level, they are the same. You merely need to accept that when using this abstraction without knowledge of the concrete type, you have no reason to make any assumptions about how the implementation and runtime will behave. At that point, all you need is to be sure that the Functor being passed in abides by the Functor laws, and that's just a matter of convention.

First, I consider myself a smart person with a very broad experience with non-functional languages, but reading this makes my mind hurt, a lot:

   typeclass Functor f where
       fmap :: (a -> b) -> f a -> f b

<snip>

This makes it possible to build functions which operate not just on Maybe, but on any Functor.

   fstrlen :: Functor f => f String -> f Int
   fstrlen fstr = fmap length faster

<snip>

   protocol Functor {
       typealias A
       func fmap<FB where FB ~= Self>(f: A -> FB.A) -> FB
   }

I understand what's going on here, but I never, ever want to see this code anywhere near (my) Swift.

HKTs that come from category theory tend to have this effect. They are hard to understand by reading the protocol definition. Admittedly, this is a big flaw with Monads and with Haskell. It takes a reasonable understanding of category theory for the purpose of this code to be obvious to the reader.

(I will point out that in those code examples, I used absolutely abysmal naming conventions. It could be made marginally more readable and understandable if the naming were better)

I'll take this time to make the point that just because you don't like the way the Functor protocol looks, doesn't mean Array and Optional aren't both Functors and Monads. As a matter of mathematics, if the Monad functions can exist on a type, and they would follow the Monad laws, that type is a Monad whether the implementor likes it or not. Of course it still needs manual implementing, but regardless, the type is theoretically a Monad. (This realization is the reason that the Java team decided to implement flatMap for Java 8's Optional class.)

I believe the way to convince the rest of us (and I would love to be convinced) is to provide a few real, “end-user” app-level use cases and explain the benefit in plain language. In particular, I don't understand the example by Jens Persson, although it seems like a valuable one.

I can (again) link to just a few of the abstract Monadic functions, all of which are very useful in practical applications.

Control.Monad

But perhaps it would also help to describe the architectural decisions that abiding by Monad imposes.

Being Monadic makes you think about data composition, which cleans up the way you design your code. Going back to the Futures example, I could implement future, and subsequently use it, like this:

    public class Promise<T> {
        private var handlers: [T -> ()] =
        private var completed: T? = nil
        
        public init() {
        }
        
        private func onComplete(handler: T -> ()) {
            if let completed = completed {
                handler(completed)
            } else {
                handlers.append(handler)
            }
        }
        
        public func complete(t: T) {
            completed = t
            for handler in handlers {
                handler(t)
            }
            handlers =
        }
        
        public var future: Future<T> {
            return Future(promise: self)
        }
    }

    public struct Future<T> {
        private let promise: Promise<T>
        
        private init(promise: Promise<T>) {
            self.promise = promise
        }
        
        public func onComplete(handler: T -> ()) {
            promise.onComplete(handler)
        }
    }

    public func useFutures() {
        downloadURLInFuture().onComplete { content in
            processContentAsync(content).onComplete { processed in
                processed.calculateSomethingAsync().onComplete {
                    ...
                        ...
                            print(finalProduct)
                        ...
                    ...
                }
            }
        }
    }

You can see how this resembles the infamous Node.js issue of callback hell, and how the nesting could quickly get out of hand. If only we had some form of composition... Arrows between Futures... Luckily, Future is a Monad (whether I like it or not!)

    // Monad

    public extension Future {
        public static func point<T>(t: T) -> Future<T> {
            let promise = Promise<T>()
            promise.complete(t)
            return promise.future
        }
        
        public func flatMap<U>(f: T -> Future<U>) -> Future<U> {
            let uPromise = Promise<U>()
            
            onComplete { t in
                f(t).onComplete { u in
                    uPromise.complete(u)
                }
            }
            
            return uPromise.future
        }
    }

Not only do I now get map and apply for free, but I also get a great method of composing Futures. The example from above can now be rewritten to be much more well composed.

    public func useFutures() {
        downloadURLInFuture()
            .flatMap(processContentAsync)
            .flatMap { $0.calculateSomethingAsync() }
            ...
            .onComplete { finalProduct in
                print(finalProduct)
            }
    }

The important thing here is that thinking about Monads helped me discover a better composition model. Now my code can be more readable and well composed.

And again, I'll mention the enormous world of functions and capabilities that can be gotten for free for the sake of even more well-composed code.

I don't want a flatMap on optionals (which, I believe, was thankfully removed in Swift 2).

And why on Earth not? The concrete implementation of it on Optional is very simple and easy to understand, and the method is incredibly useful. Personally, I use it all the time (it was not removed). And again, just because you don't like using flatMap doesn't mean Optional isn't a Monad.

Finally, I'd like to point out that it doesn't hurt any end users not familiar with Monads if Array implements a higher kinded Monad protocol. As far as they have to be concerned, Array just has these useful map and flatMap functions. Meanwhile, those of us who wish to abstract over Monads in general can write our abstract code and implement Monad on our various types which mathematically have to be Monads. It's a layer of robustness and improvement that unconcerned end users don't have to bother themselves with.

While I understand that the Swift team doesn't have the resources to implement everything that everyone wants, and that HKTs aren't very high on their list of priorities, it seems unreasonable to me that one could think of HKTs and Monads as somehow detrimental to the language.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

That’s why, when I wrote a Result type, I didn’t give it `map` and `flatMap` operations; I called them `convert` and `then`, to at least try to give some indication of what they actually mean in your code.

Now we're talking.

The stupid names are my biggest beef with FP. (The second biggest one is having to deal with mundane details of something outside of the problem domain.)

This, for example:

    public func useFutures() {
        downloadURLInFuture()
            .flatMap(processContentAsync)
            .flatMap { $0.calculateSomethingAsync() }
            ...
            .onComplete { finalProduct in
                print(finalProduct)
            }
    }

should use 'then' or 'after' or whatever.

Yes, I can understand that FP is powerful because you get some operation implementations for free (and you can have entire libraries of things that compose well). At the same time, the end result is, to me, subpar, because instead of domain-specific operations you're now using generic ones.

I don't want Swift to encourage this style; I don't want the most common promises library to use flatMap instead of then, just because Swift give them tools to do it more easily. The prevalence of those stupid operations in Rx-style libraries is already bad enough.

It’s my understanding that these unruly double-Optionals are required in various parts of the Swift library. In particular with generators – a generator knows it’s got to the end when it encounters nil (.None). If it were only possible to have one level of Optional it would be impossible to enumerate an Array of Optionals. For example

[1,2,nil,4,5]

Without the ability to represent .Some(nil), the generator would reach the end after ‘2’ in the Array and would never get to ‘4’, ‘5’.

This, by the way, is the inherent conflict between the generic, metaprogramming-enabling language and an applications language. I want the distinction in generic code, but I don't really want that distinction in my normal code.

I'm concerned that the more meta stuff we add, the more profound this effect will be.

But if you guys promise to not apply the crazy FP stuff to every single computing task, I'm ready to vote neutral. (This voting is meaningless, of course, because we're likely more than a year away from considering these changes seriously.)

A.

With respect specifically to Monads, there are numerous useful abstractions. For one, Applicative and Functor can be derived for free, requiring the no effort on the Monad instance's part. This alone eliminates a lot of code repetition. Additionally, the large majority of functions described for Haskell here are very useful:

https://hackage.haskell.org/package/base-4.8.1.0/docs/Control-Monad.html\.

Those functions come as abstractions over monads which can be easily understood (if you understand monads) and which eliminate a lot of code repetition. The existence of these abstractions makes any Monad type much more robust that it would be on its own.

To make this argument, you’ll need to make it more concrete and more relevant to more Swift programmers. Haskell is an amazing language in which one can express some truly beautiful abstractions… but Swift is not Haskell, and it caters to a different audience. If we’re to bring in more advanced type system features from Haskell, they need to be justified in the context of how this helps a significant population of Swift programmers in their everyday programming.

There are plenty of typeclasses in Haskell that could be implemented in Swift and would be useful if we had HKTs. Foldables and Traversables, for example, are simple and powerful abstractions that would be well worth having. And there are various simpler things besides Haskell typeclasses that can't be done in Swift as it stands. For example, you can't currently take a collection of type C1: CollectionType, and return a value of type C2 where C2 ~= C1, which means you can't perform arbitrary transformations on arbitrary collections. You have to choose an implementation type and use that one.

*I* know what you mean here, but make it concrete. For example, our primary map operation currently returns an array:

  extension SequenceType {
    @warn_unused_result
    func map<T>(
      @noescape includeElement: (Generator.Element) throws -> T
    ) rethrows -> [T]

With higher-kinded types, one could potentially [*] produce a collection of the same kind as Self but with the element type T. So, mapping a Set produces a Set, mapping an Array produces an Array, etc. That’s a

And you’re going to hear me say this a lot, but we are severely bandwidth-limited on design of major features, especially in the generics system because of it’s complexity.

  - Doug

[*] You wouldn’t actually be able to do this on SequenceType, because you can’t necessarily construct it.

···

On Dec 16, 2015, at 1:10 PM, Will Fancher <willfancher38@gmail.com> wrote:

With respect specifically to Monads, there are numerous useful abstractions. For one, Applicative and Functor can be derived for free,

This is the “other than implementing the basic HKTs themselves” case I was specifically trying to exclude. The fact that I can build HKTs on HKTs is not relevant to the question of the usefulness of HKTs for solving the real problems of Swift programmers.

requiring the no effort on the Monad instance's part. This alone eliminates a lot of code repetition. Additionally, the large majority of functions described for Haskell here are very useful:

https://hackage.haskell.org/package/base-4.8.1.0/docs/Control-Monad.html\.

Those functions come as abstractions over monads which can be easily understood (if you understand monads) and which eliminate a lot of code repetition.

I have spent significant hours of my life studying and working with Haskell, Monads, et al., and I still don’t think these abstractions are “easily understood.” For example, mapM: "Map each element of a structure to a monadic action, evaluate these actions from left to right, and collect the results.”

It raises these questions. Understand please that I’m not asking for the answers to the questions; I know them already (or how to discover them if I want to). The answers are never either easy or crisp, in my experience.

• What does “element of a structure” mean for an arbitrary monad? Take the I/O monad for example. What structure?!
• What is a “monadic action?” In fact, how can we even talk about “action,” which is a very imperative idea, in a purely-functional context?
• What does “collect the results” mean?

In general, the meaning each of these phrases depends on the specific monad since there is no common semantic basis for the Monad abstraction (other than its laws), only a structural one. That makes the whole exercise of understanding what a Monad is, what it’s for, and what these functions like mapM mean, a highly circular one.

The existence of these abstractions makes any Monad type much more robust that it would be on its own.

There are plenty of typeclasses in Haskell that could be implemented in Swift and would be useful if we had HKTs. Foldables and Traversables, for example, are simple and powerful abstractions that would be well worth having.

What problems would these solve for our users?

And there are various simpler things besides Haskell typeclasses that can't be done in Swift as it stands. For example, you can't currently take a collection of type C1: CollectionType, and return a value of type C2 where C2 ~= C1, which means you can't perform arbitrary transformations on arbitrary collections. You have to choose an implementation type and use that one.

Yes, it would be nice to be able to represent that constraint, and that’s an HKT thing.

This isn't to say this should all be in the standard library. I think most of these protocols would be best off in a separate package, to keep the standard simple for newcomers and for those who don't need these features.

Mmm, I don’t know. Do you want the map that comes from CollectionType, or the one array will get from its conformance to Monad, which will only be in an extension you load with a separate package? Shouldn't every CollectionType be a Functor? I haven’t thought this through deeply but I’d be surprised if it these abstractions can be useful without also being woven into the foundations of the library.

So I'm not suggesting that the Swift team create Monad, Functor, etc. in the standard library and then extend all the appropriate types to implement those (although it'd be a bonus if the team wanted to). That job would be up to whoever wants to write the package for it. But, in my opinion, the ability for such a package to exist seems very important.

If it’s truly important and general-purpose, I would have no problem putting it in the standard library. My concern is that it may not be important enough, to enough people, to warrant the attendant language complication.

-Dave

···

On Dec 16, 2015, at 1:10 PM, Will Fancher <willfancher38@gmail.com> wrote:

Yeah, but Set can’t follow the functor laws, so that map won’t apply to it. That’s why, while I agree that this kind of constraint is helpful and important for completing the generics picture, IMO it is probably less helpful/important than many people think.

-Dave

···

On Dec 16, 2015, at 1:34 PM, Douglas Gregor <dgregor@apple.com> wrote:

With higher-kinded types, one could potentially [*] produce a collection of the same kind as Self but with the element type T. So, mapping a Set produces a Set, mapping an Array produces an Array, etc.

Admittedly, the majority of my reasoning for higher kinded types comes from wanting Monads to be an available abstraction. I've been writing a lot of generic code, and finding that a lot of my types are Monads, which leads me to repeat many functions for each of these types, when those functions could be abstracted.

The fact is, most generic types can be Functors, and a lot of them can be Monads. Therefore, having generics at all suggests that the language will have a lot of these kinds of types. The standard library doesn't have many. Optionals and the various collection protocols make up the majority of them. So developers who can rely mostly on the standard library using concrete types probably have everything they need in this area. But creating one's own generic types will very often lead to reimplementing the same Monad / Applicative / Functor utility functions. I can't tell you how many times I've copy-pasted my sequence function and changed only the type signature. Futures, Eithers, and Streams are three examples of types I've been implementing as Monads, which has lead to a lot of repetition.

It is my personal opinion that Monads come up often enough that having proper support for them would be worthwhile. But I suppose the average Swift programmer may not run into these things as often as I do, so it makes sense that HKTs are low on the Swift team's priorities.

With higher-kinded types, one could potentially [*] produce a collection of the same kind as Self but with the element type T. So, mapping a Set produces a Set, mapping an Array produces an Array, etc.

I think structure preserving map is a great example and one I hope will eventually make it into the standard library someday. I would really like to see HKTs in Swift but I agree that they are a much lower priority for day-to-day development than filling out the holes in the existing generics features.

That is especially true for the majority of Swift developers who haven’t spent a lot of time studying functional programming, Haskell, reading academic papers, etc. This stuff takes significant time and thought investment to understand and become fluent in. Most developers will not devote the time to do that without a compelling reason. Doug’s request for concrete examples applicable to the average Swift developer is very reasonable.

Will, if you have ideas for libraries you would like to write that would require HKTs to implement (beyond just saving some boilerplate in the implementation), are tractable to an average Swift developer (for some reasonable definition of average), and provide compelling advantages over alternatives please do share them! I’m sure I would learn something useful and you might be able to make some headway towards motivating a higher priority for HKTs.

Matthew

I do think that the use case of HKTs is limited mostly to those of us who really want to use Monads and the like. HKTs are a feature that I think round out Swift's generics' completeness, but the main application would be some niche third party packages that have various FP programming elements. If those libraries take off and become mainstream Swift, that's a sign that those libraries are a good thing to begin with. Otherwise, I doubt the horrible naming conventions of FP (and they are horrible) would be invasive to the Swift language as a whole.

Whatever names are decided upon for the general cases, ‘flatMap’, ‘bind’, ‘map’, ‘unit’, ‘pure’, or whatever, it’ll be impossible to have something that sounds appropriate for each particular instance. I suppose a Futures library could provide 'then’ and ‘after’ functions (or whatever is deemed best), these functions would themselves simply call into ‘map’ and ‘flatMap’.

···

On 17 Dec 2015, at 13:16, Will Fancher via swift-evolution <swift-evolution@swift.org> wrote:

I do think that the use case of HKTs is limited mostly to those of us who really want to use Monads and the like. HKTs are a feature that I think round out Swift's generics' completeness, but the main application would be some niche third party packages that have various FP programming elements. If those libraries take off and become mainstream Swift, that's a sign that those libraries are a good thing to begin with. Otherwise, I doubt the horrible naming conventions of FP (and they are horrible) would be invasive to the Swift language as a whole.

While I’d love to see HKTs in Swift in the future, and it’s definitely the feature I’m most excited about, I think I’m -1 on this proposal for Swift 3.

I’m not sure if I’m formally correct here, but here’s my understanding so far: Swift’s current type system is powerful enough to express all of the individual Monads that we’re interested in, but it is not capable of a Monad protocol. (or a functor, or applicative, etc.)

If that’s the case, then I can see only two advantages to HKTs:

Code reuse for monadic functions, such as mapM, sequence, etc.
The ability to reason and code about monads generally, rather than about specific monads.

I think the first advantage is the “getting functions for free” idea. However, I’m not sure how much of an advantage it really is: the minimal complete definition for Monad is map, flatMap, and pure. Is mapM and sequence really such an extra hassle? Don’t get me wrong: not having to define them is obviously preferable. I’m just not sure how preferable.

The second advantage, while it’s something I’m certainly interested in, seems much more difficult to justify as an investment of the Swift team’s time. As someone who’s been learning Haskell for 6-ish months now, I’m only just beginning to grasp those more abstract concepts, and even then it’s still just fun and interesting, rather than useful. I can’t imagine a lot of Comonadic Costate Coalgebra whatever making it into production code.

I feel like right now Swift can get 80% of the benefits from Monadic code, since we can already write Optional, Gen, Parser, State, Par, etc.

···

On 17 Dec 2015, at 16:44, Thorsten Seitz via swift-evolution <swift-evolution@swift.org> wrote:

Big +1 for HKTs from me!

-Thorsten

Am 17.12.2015 um 13:10 schrieb Will Fancher via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>>:

Optional.map and Array.map do different things, and unifying them seems harmful.

In terms of category theory, they're not different. They are both arrows from a to b in the Functor category. Obviously that's horribly abstract, so I'll put it this way: If you think of these map functions not as operations on Optional and Array, and instead think of them simply as ways to compose an arbitrary data structure, they are in fact doing the same thing. On an implementation level, and on a runtime level, they perform differently. But on a type level, a theoretical level, and a categorical level, they are the same. You merely need to accept that when using this abstraction without knowledge of the concrete type, you have no reason to make any assumptions about how the implementation and runtime will behave. At that point, all you need is to be sure that the Functor being passed in abides by the Functor laws, and that's just a matter of convention.

First, I consider myself a smart person with a very broad experience with non-functional languages, but reading this makes my mind hurt, a lot:

   typeclass Functor f where
       fmap :: (a -> b) -> f a -> f b

<snip>

This makes it possible to build functions which operate not just on Maybe, but on any Functor.

   fstrlen :: Functor f => f String -> f Int
   fstrlen fstr = fmap length faster

<snip>

   protocol Functor {
       typealias A
       func fmap<FB where FB ~= Self>(f: A -> FB.A) -> FB
   }

I understand what's going on here, but I never, ever want to see this code anywhere near (my) Swift.

HKTs that come from category theory tend to have this effect. They are hard to understand by reading the protocol definition. Admittedly, this is a big flaw with Monads and with Haskell. It takes a reasonable understanding of category theory for the purpose of this code to be obvious to the reader.

(I will point out that in those code examples, I used absolutely abysmal naming conventions. It could be made marginally more readable and understandable if the naming were better)

I'll take this time to make the point that just because you don't like the way the Functor protocol looks, doesn't mean Array and Optional aren't both Functors and Monads. As a matter of mathematics, if the Monad functions can exist on a type, and they would follow the Monad laws, that type is a Monad whether the implementor likes it or not. Of course it still needs manual implementing, but regardless, the type is theoretically a Monad. (This realization is the reason that the Java team decided to implement flatMap for Java 8's Optional class.)

I believe the way to convince the rest of us (and I would love to be convinced) is to provide a few real, “end-user” app-level use cases and explain the benefit in plain language. In particular, I don't understand the example by Jens Persson, although it seems like a valuable one.

I can (again) link to just a few of the abstract Monadic functions, all of which are very useful in practical applications.

Control.Monad

But perhaps it would also help to describe the architectural decisions that abiding by Monad imposes.

Being Monadic makes you think about data composition, which cleans up the way you design your code. Going back to the Futures example, I could implement future, and subsequently use it, like this:

    public class Promise<T> {
        private var handlers: [T -> ()] =
        private var completed: T? = nil
        
        public init() {
        }
        
        private func onComplete(handler: T -> ()) {
            if let completed = completed {
                handler(completed)
            } else {
                handlers.append(handler)
            }
        }
        
        public func complete(t: T) {
            completed = t
            for handler in handlers {
                handler(t)
            }
            handlers =
        }
        
        public var future: Future<T> {
            return Future(promise: self)
        }
    }

    public struct Future<T> {
        private let promise: Promise<T>
        
        private init(promise: Promise<T>) {
            self.promise = promise
        }
        
        public func onComplete(handler: T -> ()) {
            promise.onComplete(handler)
        }
    }

    public func useFutures() {
        downloadURLInFuture().onComplete { content in
            processContentAsync(content).onComplete { processed in
                processed.calculateSomethingAsync().onComplete {
                    ...
                        ...
                            print(finalProduct)
                        ...
                    ...
                }
            }
        }
    }

You can see how this resembles the infamous Node.js issue of callback hell, and how the nesting could quickly get out of hand. If only we had some form of composition... Arrows between Futures... Luckily, Future is a Monad (whether I like it or not!)

    // Monad

    public extension Future {
        public static func point<T>(t: T) -> Future<T> {
            let promise = Promise<T>()
            promise.complete(t)
            return promise.future
        }
        
        public func flatMap<U>(f: T -> Future<U>) -> Future<U> {
            let uPromise = Promise<U>()
            
            onComplete { t in
                f(t).onComplete { u in
                    uPromise.complete(u)
                }
            }
            
            return uPromise.future
        }
    }

Not only do I now get map and apply for free, but I also get a great method of composing Futures. The example from above can now be rewritten to be much more well composed.

    public func useFutures() {
        downloadURLInFuture()
            .flatMap(processContentAsync)
            .flatMap { $0.calculateSomethingAsync() }
            ...
            .onComplete { finalProduct in
                print(finalProduct)
            }
    }

The important thing here is that thinking about Monads helped me discover a better composition model. Now my code can be more readable and well composed.

And again, I'll mention the enormous world of functions and capabilities that can be gotten for free for the sake of even more well-composed code.

I don't want a flatMap on optionals (which, I believe, was thankfully removed in Swift 2).

And why on Earth not? The concrete implementation of it on Optional is very simple and easy to understand, and the method is incredibly useful. Personally, I use it all the time (it was not removed). And again, just because you don't like using flatMap doesn't mean Optional isn't a Monad.

Finally, I'd like to point out that it doesn't hurt any end users not familiar with Monads if Array implements a higher kinded Monad protocol. As far as they have to be concerned, Array just has these useful map and flatMap functions. Meanwhile, those of us who wish to abstract over Monads in general can write our abstract code and implement Monad on our various types which mathematically have to be Monads. It's a layer of robustness and improvement that unconcerned end users don't have to bother themselves with.

While I understand that the Swift team doesn't have the resources to implement everything that everyone wants, and that HKTs aren't very high on their list of priorities, it seems unreasonable to me that one could think of HKTs and Monads as somehow detrimental to the language.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

I'm extremely excited about HKT. Huge +1 for me as well.

Moreover, In addition to lots of code reuse, this would allow to extend the language with very expressive and useful syntax for monadic types.
Monadic comprehension are very useful and generic.
We could imagine something the for-comprehension in scala and use it with list comprehension as well as parser combinators.

I sincerely hope this feature will see the day.

···

Sent from my iPod

On 2015/12/17, at 17:44, Thorsten Seitz via swift-evolution <swift-evolution@swift.org> wrote:

Big +1 for HKTs from me!

-Thorsten

Am 17.12.2015 um 13:10 schrieb Will Fancher via swift-evolution <swift-evolution@swift.org>:

Optional.map and Array.map do different things, and unifying them seems harmful.

In terms of category theory, they're not different. They are both arrows from a to b in the Functor category. Obviously that's horribly abstract, so I'll put it this way: If you think of these map functions not as operations on Optional and Array, and instead think of them simply as ways to compose an arbitrary data structure, they are in fact doing the same thing. On an implementation level, and on a runtime level, they perform differently. But on a type level, a theoretical level, and a categorical level, they are the same. You merely need to accept that when using this abstraction without knowledge of the concrete type, you have no reason to make any assumptions about how the implementation and runtime will behave. At that point, all you need is to be sure that the Functor being passed in abides by the Functor laws, and that's just a matter of convention.

First, I consider myself a smart person with a very broad experience with non-functional languages, but reading this makes my mind hurt, a lot:

   typeclass Functor f where
       fmap :: (a -> b) -> f a -> f b

<snip>

This makes it possible to build functions which operate not just on Maybe, but on any Functor.

   fstrlen :: Functor f => f String -> f Int
   fstrlen fstr = fmap length faster

<snip>

   protocol Functor {
       typealias A
       func fmap<FB where FB ~= Self>(f: A -> FB.A) -> FB
   }

I understand what's going on here, but I never, ever want to see this code anywhere near (my) Swift.

HKTs that come from category theory tend to have this effect. They are hard to understand by reading the protocol definition. Admittedly, this is a big flaw with Monads and with Haskell. It takes a reasonable understanding of category theory for the purpose of this code to be obvious to the reader.

(I will point out that in those code examples, I used absolutely abysmal naming conventions. It could be made marginally more readable and understandable if the naming were better)

I'll take this time to make the point that just because you don't like the way the Functor protocol looks, doesn't mean Array and Optional aren't both Functors and Monads. As a matter of mathematics, if the Monad functions can exist on a type, and they would follow the Monad laws, that type is a Monad whether the implementor likes it or not. Of course it still needs manual implementing, but regardless, the type is theoretically a Monad. (This realization is the reason that the Java team decided to implement flatMap for Java 8's Optional class.)

I believe the way to convince the rest of us (and I would love to be convinced) is to provide a few real, “end-user” app-level use cases and explain the benefit in plain language. In particular, I don't understand the example by Jens Persson, although it seems like a valuable one.

I can (again) link to just a few of the abstract Monadic functions, all of which are very useful in practical applications.

Control.Monad

But perhaps it would also help to describe the architectural decisions that abiding by Monad imposes.

Being Monadic makes you think about data composition, which cleans up the way you design your code. Going back to the Futures example, I could implement future, and subsequently use it, like this:

    public class Promise<T> {
        private var handlers: [T -> ()] =
        private var completed: T? = nil
        
        public init() {
        }
        
        private func onComplete(handler: T -> ()) {
            if let completed = completed {
                handler(completed)
            } else {
                handlers.append(handler)
            }
        }
        
        public func complete(t: T) {
            completed = t
            for handler in handlers {
                handler(t)
            }
            handlers =
        }
        
        public var future: Future<T> {
            return Future(promise: self)
        }
    }

    public struct Future<T> {
        private let promise: Promise<T>
        
        private init(promise: Promise<T>) {
            self.promise = promise
        }
        
        public func onComplete(handler: T -> ()) {
            promise.onComplete(handler)
        }
    }

    public func useFutures() {
        downloadURLInFuture().onComplete { content in
            processContentAsync(content).onComplete { processed in
                processed.calculateSomethingAsync().onComplete {
                    ...
                        ...
                            print(finalProduct)
                        ...
                    ...
                }
            }
        }
    }

You can see how this resembles the infamous Node.js issue of callback hell, and how the nesting could quickly get out of hand. If only we had some form of composition... Arrows between Futures... Luckily, Future is a Monad (whether I like it or not!)

    // Monad

    public extension Future {
        public static func point<T>(t: T) -> Future<T> {
            let promise = Promise<T>()
            promise.complete(t)
            return promise.future
        }
        
        public func flatMap<U>(f: T -> Future<U>) -> Future<U> {
            let uPromise = Promise<U>()
            
            onComplete { t in
                f(t).onComplete { u in
                    uPromise.complete(u)
                }
            }
            
            return uPromise.future
        }
    }

Not only do I now get map and apply for free, but I also get a great method of composing Futures. The example from above can now be rewritten to be much more well composed.

    public func useFutures() {
        downloadURLInFuture()
            .flatMap(processContentAsync)
            .flatMap { $0.calculateSomethingAsync() }
            ...
            .onComplete { finalProduct in
                print(finalProduct)
            }
    }

The important thing here is that thinking about Monads helped me discover a better composition model. Now my code can be more readable and well composed.

And again, I'll mention the enormous world of functions and capabilities that can be gotten for free for the sake of even more well-composed code.

I don't want a flatMap on optionals (which, I believe, was thankfully removed in Swift 2).

And why on Earth not? The concrete implementation of it on Optional is very simple and easy to understand, and the method is incredibly useful. Personally, I use it all the time (it was not removed). And again, just because you don't like using flatMap doesn't mean Optional isn't a Monad.

Finally, I'd like to point out that it doesn't hurt any end users not familiar with Monads if Array implements a higher kinded Monad protocol. As far as they have to be concerned, Array just has these useful map and flatMap functions. Meanwhile, those of us who wish to abstract over Monads in general can write our abstract code and implement Monad on our various types which mathematically have to be Monads. It's a layer of robustness and improvement that unconcerned end users don't have to bother themselves with.

While I understand that the Swift team doesn't have the resources to implement everything that everyone wants, and that HKTs aren't very high on their list of priorities, it seems unreasonable to me that one could think of HKTs and Monads as somehow detrimental to the language.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

...But I really do believe that this proposal needs a good list of very specific real-world use cases where FP enables something that isn't easily enabled otherwise.

The Future example could just implement it's own flatMap with a better name, so I don't see it as being a valid case. The image processing example is still unclear. I'm sure we can find 5 cases where FP really makes the code much smaller or easier to understand?

A.

I do think that the use case of HKTs is limited mostly to those of us who really want to use Monads and the like. HKTs are a feature that I think round out Swift's generics' completeness, but the main application would be some niche third party packages that have various FP programming elements. If those libraries take off and become mainstream Swift, that's a sign that those libraries are a good thing to begin with. Otherwise, I doubt the horrible naming conventions of FP (and they are horrible) would be invasive to the Swift language as a whole.

Yes, but to consider including this into the language, surely we can imagine (at the client code level) specifics of several libraries/niches that are enabled by HKTs?

A.

How, specifically, would you use it? To which types would it apply, and how would the change benefit users?

-Dave

···

On Dec 16, 2015, at 4:04 PM, Matthew Johnson via swift-evolution <swift-evolution@swift.org> wrote:

With higher-kinded types, one could potentially [*] produce a collection of the same kind as Self but with the element type T. So, mapping a Set produces a Set, mapping an Array produces an Array, etc.

I think structure preserving map is a great example and one I hope will eventually make it into the standard library someday.

I think Haskell is lovely, and I’ve bookmarked the Causal Commutative Arrows paper to read for later, but I think… not at all?

I think the part that benefits users is the ideas behind functional programming as made explicit in the standard library. For instance, having all the structures be mappable and foldable in appropriate ways. That this may require some extra boilerplate by the people writing the library implementing that structure is too bad, but not a huge loss, because there just aren’t that many structures that we programmers will use, so the libraries to be written are fairly limited.

I don’t think in my day job as a Swift & Obj-C programmer that I’ve ever had the desire to write code that maps and folds AND is generic in kind of structure and type over which it does so. I generally actually do care and am quite specific about whether it’s an Array or a Set, much less something less generic like a B*-Tree or AST.

  - Greg

···

On Dec 16, 2015, at 4:26 PM, Dave Abrahams via swift-evolution <swift-evolution@swift.org> wrote:

On Dec 16, 2015, at 4:04 PM, Matthew Johnson via swift-evolution <swift-evolution@swift.org> wrote:

With higher-kinded types, one could potentially [*] produce a collection of the same kind as Self but with the element type T. So, mapping a Set produces a Set, mapping an Array produces an Array, etc.

I think structure preserving map is a great example and one I hope will eventually make it into the standard library someday.

How, specifically, would you use it? To which types would it apply, and how would the change benefit users?

The ideal situation would involve being able to represent monad and being
able to type alias the behavior that that would afford us 'for free' to
specific named functions methods for our specific types, I think.

The concrete example of a HKT's usefullness that I am after is captured
here with [Causal Commutative Arrows](
http://cs.yale.edu/c2/images/uploads/ICFP-CCA.pdf\). I admit that the big
win has to do with the compile time optimization afforded, but I will argue
that the syntax's similarity to audio related pseudocode is just as big of
a win. [These slides](
http://static1.squarespace.com/static/53e04d59e4b0c0da377d20b1/t/541cb6eee4b0be37af62e23f/1411167982496/nichoiceICFP2014Presentation.pdf\)
give a hint at what I am talking about. I agree that using "Monad"
everywhere is not generally descriptive but I think that that is simply an
issue of being able to document intent.

TJ

···

On Wed, Dec 16, 2015 at 5:11 PM, Will Fancher via swift-evolution < swift-evolution@swift.org> wrote:

Admittedly, the majority of my reasoning for higher kinded types comes
from wanting Monads to be an available abstraction. I've been writing a lot
of generic code, and finding that a lot of my types are Monads, which leads
me to repeat many functions for each of these types, when those functions
could be abstracted.

The fact is, most generic types can be Functors, and a lot of them can be
Monads. Therefore, having generics at all suggests that the language will
have a lot of these kinds of types. The standard library doesn't have many.
Optionals and the various collection protocols make up the majority of
them. So developers who can rely mostly on the standard library using
concrete types probably have everything they need in this area. But
creating one's own generic types will very often lead to reimplementing the
same Monad / Applicative / Functor utility functions. I can't tell you how
many times I've copy-pasted my sequence function and changed only the type
signature. Futures, Eithers, and Streams are three examples of types I've
been implementing as Monads, which has lead to a lot of repetition.

It is my personal opinion that Monads come up often enough that having
proper support for them would be worthwhile. But I suppose the average
Swift programmer may not run into these things as often as I do, so it
makes sense that HKTs are low on the Swift team's priorities.
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

while I agree that this kind of constraint is helpful and important for completing the generics picture, IMO it is probably less helpful/important than many people think.

While I would love to see HKTs in Swift as soon as possible, and I personally would really appreciate a good Monad setup, I have to say I agree with this. It'd be great to have, but it's not as important as anything else on Swift 3's todo list.

So this brings me to a question: If the community were to write HKTs into the language and submit a pull request, what is the likelihood that it would be accepted, despite the fact that the Swift team feels it is out of scope for their own Swift 3 efforts?