[Proposal] Higher Kinded Types (Monads, Functors, etc.)

Great, Thanks!

···

On 2016-01-21, at 22:52:46, Developer <devteam.codafi@gmail.com> wrote:

It never died, the proposal draft is still published for you to comment on before we submit it. We'd like to have at least a partial implementation to go along with the change because it is so massive, so it's going to take longer than your average proposal:

Higher Kinded Types Proposal · Issue #1 · typelift/swift · GitHub

~Robert Widmann

2016/01/21 10:36、Craig Cruden via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> のメッセージ:

Did anything become of this…. This seemed to die out with no proposal.

I would think having a common protocol for types that support monadic operations (regardless if they fulfill all the monadic operations) would generally be helpful — even if they were not implemented theoretically as a Functor / Monad separate protocol definitions.

Scala seems to just treat them as one set of operations (MonadOps : map, flatMap, filter, withFilter). `filter` copies the contents into a new set, `withFilter` is just an on-demand filter for any later functions such as map/flatmap (BTW, Is swift filter on-demand, copy, or a combination???). I do notice that several different implement of the Monad protocol have been done in different repositories - by third parties.

And yes, Optionals are monads - and as such can be used in comprehension clauses.

I know that for-comprehension has been stated as probably not a Swift 3 thing (Felix indicated this but I don’t know where it is stated) - but I am wondering if they are thinking it is larger than it actually is (being that it is just a more friendly way to state existing map/flatmap combinations). If the worry is `for` having dual purpose — but if that is the case maybe making a new keyword for comprehensions like `all`.

Even if the proposal were drawn up - and it was `Deferred` at least it would potentially be on the radar.

On 2015-12-18, at 1:01:21, André Videla via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

I'm extremely excited about HKT. Huge +1 for me as well.

Moreover, In addition to lots of code reuse, this would allow to extend the language with very expressive and useful syntax for monadic types.
Monadic comprehension are very useful and generic.
We could imagine something the for-comprehension in scala and use it with list comprehension as well as parser combinators.

I sincerely hope this feature will see the day.

Sent from my iPod
On 2015/12/17, at 17:44, Thorsten Seitz via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Big +1 for HKTs from me!

-Thorsten

Am 17.12.2015 um 13:10 schrieb Will Fancher via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>>:

Optional.map and Array.map do different things, and unifying them seems harmful.

In terms of category theory, they're not different. They are both arrows from a to b in the Functor category. Obviously that's horribly abstract, so I'll put it this way: If you think of these map functions not as operations on Optional and Array, and instead think of them simply as ways to compose an arbitrary data structure, they are in fact doing the same thing. On an implementation level, and on a runtime level, they perform differently. But on a type level, a theoretical level, and a categorical level, they are the same. You merely need to accept that when using this abstraction without knowledge of the concrete type, you have no reason to make any assumptions about how the implementation and runtime will behave. At that point, all you need is to be sure that the Functor being passed in abides by the Functor laws, and that's just a matter of convention.

First, I consider myself a smart person with a very broad experience with non-functional languages, but reading this makes my mind hurt, a lot:

   typeclass Functor f where
       fmap :: (a -> b) -> f a -> f b

<snip>

This makes it possible to build functions which operate not just on Maybe, but on any Functor.

   fstrlen :: Functor f => f String -> f Int
   fstrlen fstr = fmap length faster

<snip>

   protocol Functor {
       typealias A
       func fmap<FB where FB ~= Self>(f: A -> FB.A) -> FB
   }

I understand what's going on here, but I never, ever want to see this code anywhere near (my) Swift.

HKTs that come from category theory tend to have this effect. They are hard to understand by reading the protocol definition. Admittedly, this is a big flaw with Monads and with Haskell. It takes a reasonable understanding of category theory for the purpose of this code to be obvious to the reader.

(I will point out that in those code examples, I used absolutely abysmal naming conventions. It could be made marginally more readable and understandable if the naming were better)

I'll take this time to make the point that just because you don't like the way the Functor protocol looks, doesn't mean Array and Optional aren't both Functors and Monads. As a matter of mathematics, if the Monad functions can exist on a type, and they would follow the Monad laws, that type is a Monad whether the implementor likes it or not. Of course it still needs manual implementing, but regardless, the type is theoretically a Monad. (This realization is the reason that the Java team decided to implement flatMap for Java 8's Optional class.)

I believe the way to convince the rest of us (and I would love to be convinced) is to provide a few real, “end-user” app-level use cases and explain the benefit in plain language. In particular, I don't understand the example by Jens Persson, although it seems like a valuable one.

I can (again) link to just a few of the abstract Monadic functions, all of which are very useful in practical applications.

Control.Monad

But perhaps it would also help to describe the architectural decisions that abiding by Monad imposes.

Being Monadic makes you think about data composition, which cleans up the way you design your code. Going back to the Futures example, I could implement future, and subsequently use it, like this:

    public class Promise<T> {
        private var handlers: [T -> ()] =
        private var completed: T? = nil
        
        public init() {
        }
        
        private func onComplete(handler: T -> ()) {
            if let completed = completed {
                handler(completed)
            } else {
                handlers.append(handler)
            }
        }
        
        public func complete(t: T) {
            completed = t
            for handler in handlers {
                handler(t)
            }
            handlers =
        }
        
        public var future: Future<T> {
            return Future(promise: self)
        }
    }

    public struct Future<T> {
        private let promise: Promise<T>
        
        private init(promise: Promise<T>) {
            self.promise = promise
        }
        
        public func onComplete(handler: T -> ()) {
            promise.onComplete(handler)
        }
    }

    public func useFutures() {
        downloadURLInFuture().onComplete { content in
            processContentAsync(content).onComplete { processed in
                processed.calculateSomethingAsync().onComplete {
                    ...
                        ...
                            print(finalProduct)
                        ...
                    ...
                }
            }
        }
    }

You can see how this resembles the infamous Node.js issue of callback hell, and how the nesting could quickly get out of hand. If only we had some form of composition... Arrows between Futures... Luckily, Future is a Monad (whether I like it or not!)

    // Monad

    public extension Future {
        public static func point<T>(t: T) -> Future<T> {
            let promise = Promise<T>()
            promise.complete(t)
            return promise.future
        }
        
        public func flatMap<U>(f: T -> Future<U>) -> Future<U> {
            let uPromise = Promise<U>()
            
            onComplete { t in
                f(t).onComplete { u in
                    uPromise.complete(u)
                }
            }
            
            return uPromise.future
        }
    }

Not only do I now get map and apply for free, but I also get a great method of composing Futures. The example from above can now be rewritten to be much more well composed.

    public func useFutures() {
        downloadURLInFuture()
            .flatMap(processContentAsync)
            .flatMap { $0.calculateSomethingAsync() }
            ...
            .onComplete { finalProduct in
                print(finalProduct)
            }
    }

The important thing here is that thinking about Monads helped me discover a better composition model. Now my code can be more readable and well composed.

And again, I'll mention the enormous world of functions and capabilities that can be gotten for free for the sake of even more well-composed code.

I don't want a flatMap on optionals (which, I believe, was thankfully removed in Swift 2).

And why on Earth not? The concrete implementation of it on Optional is very simple and easy to understand, and the method is incredibly useful. Personally, I use it all the time (it was not removed). And again, just because you don't like using flatMap doesn't mean Optional isn't a Monad.

Finally, I'd like to point out that it doesn't hurt any end users not familiar with Monads if Array implements a higher kinded Monad protocol. As far as they have to be concerned, Array just has these useful map and flatMap functions. Meanwhile, those of us who wish to abstract over Monads in general can write our abstract code and implement Monad on our various types which mathematically have to be Monads. It's a layer of robustness and improvement that unconcerned end users don't have to bother themselves with.

While I understand that the Swift team doesn't have the resources to implement everything that everyone wants, and that HKTs aren't very high on their list of priorities, it seems unreasonable to me that one could think of HKTs and Monads as somehow detrimental to the language.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

I've been bumping into (what I've found out to be) the need for HKT a lot
while trying to write highly composable yet optimizable code for
exploratory image (and audio (and video)) processing.

Using Core Image, vImage, vDSP, Metal, etc. directly is not an option here,
since they are much too low level / specialized / non-composable. What I'm
writing is a system that will let me try out new ideas very quickly by
reusing as much code and concepts as possible. Ie I don't want to write
specialized code for every slight variant of a concept. So a separable
convolution should be _one_ generic implementation, and the compiler (not
the programmer) should take care of specializing it for any given value
type and dimensionality, same thing with positions, a position in space is
no different from a position in a color space or a position in space&time,
so they should share a common implementation. I need it to be as
optimizable as possible, because it would be unusable even for just
experimentation otherwise. If something needs to get faster still, we can
then write a Metal version of it.

With the latest improvements in the optimizer (esp. loop unrolling) I have
managed to get the basics of this working, the compiler will generate SIMD
code from my high level code constructs (when possible of course, eg when
doing separable convolution (in any number of dimensions) and the
components are 4 or 4x4 Floats etc).

But I currently have to jump through hoops in order to write simple things
like "static array types" with type-level count and specific memory
footprint. I'm using Peano-esque natural-numbers-as-types for the Count,
and a similar nested-structs-thing to get the proper storage of Count x
Element. But it is impossible to parameterize them for both Count and
Element, so I have to do eg this:

typealias MyFloat4 = StaticArrayOf<Float>.WithCount4

instead of the more straight forward:

typealias MyFloat4 = StaticArray<Float, Count4>

So I wish it was possible to implement it like this:
struct StaticArray<Element, Count> { ... }
where
strideof(StaticArray<Float, Count4>) == 4 * strideof(Float) ==
strideof(float4)

But I've given up and accepted that such parameterization is not possible
without HKT.

Perhaps it would become possible with the proposed generic typealiases, at
least if that implied generic associated types (which I guess would be the
same as HKT?).

Anyway, I'm very impressed by (open source master) Swift's ability to
optimize very high level generic code. I just wish some of my abstractions
could be written in a more straight forward way. I think HKT would make
them simpler. (Sorry for not being more specific).

/Jens

···

On Thu, Dec 17, 2015 at 6:22 AM, Matthew Johnson via swift-evolution < swift-evolution@swift.org> wrote:

> Looking beyond functional programming abstractions, higher-kinded types
are a fancy way of saying "template template parameters”.

Thanks for bringing that up. I almost mentioned it. I didn’t because I
know the Swift team wants to avoid a lot of what has been done with
templates and am not sure exactly where you draw the line. :)

> A textbook motivation for those from C++ land is Andrei Alexandrescu's
"policy pattern", stuff like this:
>
> protocol RefStoragePolicy: <*: class> {
> typealias Ref: class
> init(ref: Ref)
> var ref: Ref
> }
>
> struct Weak<T: class>: RefStoragePolicy { weak var ref: T }
> struct Unowned<T: class>: RefStoragePolicy { unowned var ref: T }
> struct Strong<T: class>: RefStoragePolicy { var ref: T }
>
> class HeterogeneousRefConsumer<Storage: RefStoragePolicy> {
> func consumeRef<T: class>(ref: T) {
> let storage = Storage<T>(ref: ref)
> doStuffWith(storage)
> }
> }
>
> -Joe
> _______________________________________________
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

--
bitCycle AB | Smedjegatan 12 | 742 32 Östhammar | Sweden

Phone: +46-73-753 24 62
E-mail: jens@bitcycle.com