Formalizing a Numerical/ML Working Group

Procedurally, the Core Team will probably be tied up for the next few weeks at least carrying out the reorganization. But one of the goals of that reorganization is to better support having working groups for diverse efforts like this, and just speaking for myself, I think having a Swift ML working group would make a lot of sense.

20 Likes

Just to add to this discussion about Tensor terminology, I think the term Matrix is a more generic term suitable for a wider range of useful numerical operations. Tensors are often represented as matrices, and therefore plenty of matrix operations have meaning as tensor operations. However, there are plenty of useful things one can do with matrices that don't make sense as tensors.

1 Like

This confuses me a bit. Tensors are actually a more generic object than matrices, and therefore suitable for a wider range of operations. Anything you do with a matrix you can do with a tensor. The reverse is not true.
Maybe I'm mixing things up however and the mathematical tensor object is completely unrelated or at least dissimilar to the tensor object being discussed here.

Tensor have a far more restrictive set of requirements than a matrix does. For example, they have to changes basis following a particular set of rules (i.e., covariant and contravariant components)---and if they don't, they're not a tensor. So it's easy to construct matrices of data that don't follow tensor transformation rules, but in contrast, any tensor can be represented as a multi-dimensional matrix given a basis choice.

Ok, I think that's sort of what I meant. From any tensor you can always "drop down to the matrix level" given a choice of basis.

1 Like

To echo the more general point brought up a few times, you don't want to restrict your self to thinking only about tensors when designing a good numerical library, because that will unnecessarily restrict the applicability. For example, I do both a bunch of spectral modeling and a bunch of stochastic modeling---the former lends itself nicely to the tensor abstraction, but the latter does not---yet both use many of the same matrix operations.

1 Like

I think that we’re crossing wires in terms of “tensor” as a mathematical object and “tensor” as a data structure that’s used in machine learning. Oftentimes, people use the word “tensor” colloquially to refer to just a generalized matrix with a rank that may be greater than 2, without any of the connotations of consistency across bases, etc.

I suspect you are correct that in the last few years people have started abusing the terminology---and that's also why it's important to get a larger diversity of perspectives when creating a working group for a numerical computing library (so as not make that same kind of mistake). If the goal really is to create a numerical computing library to be used by a broader community, then I would hope one could recruit perspectives that aren't currently represented in these forums.

Edit: Let me add that my big worry is that a numerical library gets created that is unnecessarily very niche, and that's why I want to advocate for some broad thinking about different ways folks would use such a library. E.g., a discussion around Tensor vs Matrix vs Array is something that needs to happen with a very broad group, so as not to get trapped into one way of thinking.

4 Likes

The confusion of tensors and matrices seemed to bring limitations to S4TF. The matrix decomposition and matrix band ops were specific to one domain: linear algebra. The word "matrix" was even in the name of some raw ops - matrixBandPart, matrixSetDiag, to name a few.

The linear algebra component of S4TF wasn't close to feature-complete with Python TF's linalg, yet it limited S4TF's extensibility to other backends. Everything else operated on varied-dimensional tensors (e.g. Conv2D, Conv3D, depthToSpace) that are supported by MPSGraph and DirectML. The matrix ops could be removed from S4TF and reimplemented in a more feature-complete "numerical computing library".

1 Like

It seems that there may be enough interest in Numerical/ML to get an official working group. I would be willing to join if there was a video call kick-off. Does anyone else feel the same way?

3 Likes

I would be excited to be a part of this :+1:

2 Likes

I do!

What are the responsibilities and the expected skills of a working group member? I would love to join and contribute if my skills are beneficial for the team.

Anything, I may be lacking currently, I can learn and I do have the time.

1 Like

You would have to ask a project leader like @tkremenek, but here's my expectation. I don't expect that it will require a lot of time and dedication, because 1) we're a very small subset of the Swift community. I don't even know how many people would attend; that's why I asked on this thread. If Swift Forums has an official polling mechanism, that would be great for a moderator to employ here.

And 2) I think most people on this thread are not being hired full-time to work on using Swift for ML. In contrast, the C++ interop working group has multiple full-time contributors. We will just occasionally meet on Zoom and talk about it. We don't know much about how Swift is used outside of S4TF and Quantum, and there are a lot of ideas to discuss. I already have a few questions, like whether anybody else has used Swift-Colab or thought of a Swift substitute for NumPy.

1 Like

I'm in! Should we maybe find some official place where we can communicate and share our experiences? I don't think that this thread quite does the job

I believe unless we make a concrete proposal for a following action, even for something like setting up an initial meeting date, it might be too easy to wait for someone else to take point and not move anywhere.

A solid proposal can be agreed upon, refused or re-negotiated but it can be worked with.

I’m not fit to spear this project based on my current technical knowledge but this is a field that I’m committed to stand behind and help to make it a reality.

I propose we host an open call on May 25th, at 8pm US central time on Google Meet. That way, who wishes to join, can join. We may or may not have a solid gathering but I believe it’s a step in the right direction. I’ll create the room.

Of course it’s beneficial that people with technical knowledge joins, but even starting a think tank with developers can help.

My 2 cents :v:

P.D.: This is a proposal, so it can be agreed upon, refused or re-negotiated :wink:

2 Likes

I am currently open at that time. I think that if the Swift community wants to get anything done here, we have to take initiative ourselves. The new Swift for TensorFlow isn't going to be "official" (which I learned the hard way), and our numerical working group isn't going to be "official". Other endeavors aren't going to be approved/sponsored by Google or Apple either. If anyone else shares this opinion, please leave a like on this comment.

I'm going to ping @tkremenek and @John_McCall here in case this could still be made into something official.

2 Likes

The Core Team is setting up a number of other WGs at the moment, and that’s using most of our free bandwidth. But we can revisit this in a few months and see whether official WGs would be useful. In the meantime, of course you’re welcome to coordinate here, have meetings, and so on.

I’m not sure that a WG which covers both numerics and ML is quite right, though; those seem like rather different sets of problems. Perhaps I’m misunderstanding what you expect to cover under numerics in this group, though.

1 Like

@aj_ortiz I have an even better idea. Analyzing the last S4TF open design meetings, they all happened on a Friday at 9 AM Pacific time. We could continue where they left off, and it would be very symbolic for our community regaining what we lost.

That time of day isn’t really going to work out for people with jobs and school, so it could happen later. What about 8 PM Central Time, the Friday after next (May 27)?

@John_McCall I perceive it might be easy to eagerly correlate both as some numerics features could help building ML algorithms. Most likely they would end up being two different groups, as you mentioned, they solve different sets of problems. That'll definitely be an important point to clarify on the first meeting, the objective and scope of the group.

@philipturner Thanks for pinging them :slight_smile: I can make that work for me no problem, but I would understand that Friday night might be a little taxing for some on a 9-5 job. But if we can make it work, that's cool too.

Some potential key points:

  • Establish domain scope for the UWG (what and what not falls into it)
  • Define initial milestones that provide a foundation based on the agreed scope the group, which would like to spear in terms of research and/or implementation
  • Review what has already been built on this domain (the current state of art) (what can be salvaged and what not)