[swiftpm] Add proposal for C language support


(Daniel Dunbar) #1

Happy 2016!

I am working on an initial proposal for adding support for C language targets to the Swift package manager, and am interested in feedback:
  https://github.com/ddunbar/swift-evolution/blob/master/proposals/NNNN-swiftpm-c-language-targets.md

Some TL;DR:
- The proposal defines a basic convention for pure C language targets (no Swift/C mix and match, but other Swift targets can use the C targets).
- This is just intended to be the minimal initial feature, there will be a lot of add on work which I expect should be tackled in follow on PRs/proposals.
- The proposal doesn't try and outline every single nitty detail (e.g., exactly what C++ standard we will compile with). My intention is to pick a sensible default at implementation time and refine incrementally.

Unless there are serious objections, I am hoping to hope to land this proposal soon and start work on the feature shortly after.

Cheers,
- Daniel


(Andrew Madsen) #2

Happy 2016!

>
> I am working on an initial proposal for adding support for C language targets to the Swift package manager, and am interested in feedback:
> https://github.com/ddunbar/swift-evolution/blob/master/proposals/NNNN-swiftpm-c-language-targets.md

>
> Unless there are serious objections, I am hoping to hope to land this proposal soon and start work on the feature shortly after.

I'm happy to see this, Daniel. Best I can tell, it will provide exactly the feature set I need right now, where I'm writing a Swift library package that requires a little bit of C (ie. two or three 1-line functions) due to Swift's inability to call C variadic functions.

I do think the ability to declare that a certain target's API should not be exposed to downstream packages is important, and hope that too can be addressed relatively soon.

-Andrew

···

Daniel Dunbar via swift-build-dev wrote:


(Drew Crawford) #3

Thanks for directing me to this, I missed it.

Most projects will not conform to these conventions.

Giggle. Kind of an understatement, no?

Like, okay. Here is a project I'd like to package. (Read: I do package it, with features not in mainline swiftPM.) https://github.com/jedisct1/libsodium

Let's take a look at how this package realistically builds:

* It has tests ("make check")
* It has various --enable-foo flags
* It swaps in special implementations depending on if you have AMD64 <https://github.com/jedisct1/libsodium/blob/master/src/libsodium/Makefile.am#L162> or AVX instructions <https://github.com/jedisct1/libsodium/blob/master/src/libsodium/Makefile.am#L145> or SSE2 <https://github.com/jedisct1/libsodium/blob/master/src/libsodium/Makefile.am#L229> etc.
* The optimization level is tuned on a per-architecture basis <https://github.com/jedisct1/libsodium/blob/master/dist-build/android-armv7-a.sh#L3>
* They build (also) on Windows. They're not changing how they're packaged for "SwiftPM, the Mac/Linux build system".
* Oh and this is cryptography code. Do you *really* want to touch it?

I think an important feature of any C target proposal is that there will actually exist C targets which can be built under the proposal. Until there are C people coming out of the woodwork saying "sure, I will repackage my software this way" I think the entire value is debatable.

Getting signoff from libdispatch/CoreFoundation is necessary but not sufficient to clear that hurdle. I would think getting the other C deps in our own project family to repackage would be "table stakes" for any new C build system. The real test are projects that are third-party and less friendly.

And I do not see realistically how we are ever going to support a project like libsodium, except calling out to automake. An automake solution coincidentally supports both libdispatch and CoreFoundation right now. IMO something like that is a much, much better direction in the short-term, and once we have done the first step of "packaging" those software via automake we will have "real" C projects in our package manager and we can design our C support around the concerns of real projects instead of imaginary ones.

···

On Jan 2, 2016, at 11:00 AM, Daniel Dunbar via swift-build-dev <swift-build-dev@swift.org> wrote:

Happy 2016!

I am working on an initial proposal for adding support for C language targets to the Swift package manager, and am interested in feedback:
https://github.com/ddunbar/swift-evolution/blob/master/proposals/NNNN-swiftpm-c-language-targets.md

Some TL;DR:
- The proposal defines a basic convention for pure C language targets (no Swift/C mix and match, but other Swift targets can use the C targets).
- This is just intended to be the minimal initial feature, there will be a lot of add on work which I expect should be tackled in follow on PRs/proposals.
- The proposal doesn't try and outline every single nitty detail (e.g., exactly what C++ standard we will compile with). My intention is to pick a sensible default at implementation time and refine incrementally.

Unless there are serious objections, I am hoping to hope to land this proposal soon and start work on the feature shortly after.

Cheers,
- Daniel

_______________________________________________
swift-build-dev mailing list
swift-build-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-build-dev


(Kostiantyn Koval) #4

Hi, Happy 2016.

The proposal looks great as for me. Very nice starting point to add C family support

Best Regards,
Kostiantyn

···

On 02 Jan 2016, at 18:00, Daniel Dunbar via swift-build-dev <swift-build-dev@swift.org> wrote:

Happy 2016!

I am working on an initial proposal for adding support for C language targets to the Swift package manager, and am interested in feedback:
https://github.com/ddunbar/swift-evolution/blob/master/proposals/NNNN-swiftpm-c-language-targets.md

Some TL;DR:
- The proposal defines a basic convention for pure C language targets (no Swift/C mix and match, but other Swift targets can use the C targets).
- This is just intended to be the minimal initial feature, there will be a lot of add on work which I expect should be tackled in follow on PRs/proposals.
- The proposal doesn't try and outline every single nitty detail (e.g., exactly what C++ standard we will compile with). My intention is to pick a sensible default at implementation time and refine incrementally.

Unless there are serious objections, I am hoping to hope to land this proposal soon and start work on the feature shortly after.

Cheers,
- Daniel

_______________________________________________
swift-build-dev mailing list
swift-build-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-build-dev


(David Owens II) #5

Seems like a good start for Swift developers that need to write some C code for their project.

-David

···

Sent from my iPhone

On Jan 2, 2016, at 9:00 AM, Daniel Dunbar via swift-build-dev <swift-build-dev@swift.org> wrote:

Happy 2016!

I am working on an initial proposal for adding support for C language targets to the Swift package manager, and am interested in feedback:
https://github.com/ddunbar/swift-evolution/blob/master/proposals/NNNN-swiftpm-c-language-targets.md

Some TL;DR:
- The proposal defines a basic convention for pure C language targets (no Swift/C mix and match, but other Swift targets can use the C targets).
- This is just intended to be the minimal initial feature, there will be a lot of add on work which I expect should be tackled in follow on PRs/proposals.
- The proposal doesn't try and outline every single nitty detail (e.g., exactly what C++ standard we will compile with). My intention is to pick a sensible default at implementation time and refine incrementally.

Unless there are serious objections, I am hoping to hope to land this proposal soon and start work on the feature shortly after.

Cheers,
- Daniel

_______________________________________________
swift-build-dev mailing list
swift-build-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-build-dev


(Félix Cloutier) #6

This sounds interesting. I don't have time to read it right now but I like the idea and I'll give better feedback later.

Félix

···

Le 2 janv. 2016 à 12:28:10, Kostiantyn Koval via swift-evolution <swift-evolution@swift.org> a écrit :

Hi, Happy 2016.

The proposal looks great as for me. Very nice starting point to add C family support

Best Regards,
Kostiantyn

On 02 Jan 2016, at 18:00, Daniel Dunbar via swift-build-dev <swift-build-dev@swift.org> wrote:

Happy 2016!

I am working on an initial proposal for adding support for C language targets to the Swift package manager, and am interested in feedback:
https://github.com/ddunbar/swift-evolution/blob/master/proposals/NNNN-swiftpm-c-language-targets.md

Some TL;DR:
- The proposal defines a basic convention for pure C language targets (no Swift/C mix and match, but other Swift targets can use the C targets).
- This is just intended to be the minimal initial feature, there will be a lot of add on work which I expect should be tackled in follow on PRs/proposals.
- The proposal doesn't try and outline every single nitty detail (e.g., exactly what C++ standard we will compile with). My intention is to pick a sensible default at implementation time and refine incrementally.

Unless there are serious objections, I am hoping to hope to land this proposal soon and start work on the feature shortly after.

Cheers,
- Daniel

_______________________________________________
swift-build-dev mailing list
swift-build-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-build-dev

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Zachary Waldowski) #7

*I think an important feature of any C target proposal is that there
will actually exist C targets which can be built under the proposal*.
Until there are C people coming out of the woodwork saying "sure, I
will repackage my software this way" I think the entire value is
debatable.

I almost couldn't disagree more. No Swift project could be built with
swiftpm when it was introduced without repackaging. I don't see why C
support should be held to a different standard.

And I do not see realistically how we are ever going to support a
project like libsodium, except calling out to automake.

A potential solution (one of many possible) would look a lot like how
people generate Xcode projects for C build systems today; hand-tuning
config.h headers and such. I know many people who will go to ungodly
lengths to avoid the inevitable nightmare automake causes in a source-
distributed dependency.

IMO something like that is a much, much better direction in the short-
term, and once we have done the first step of "packaging" those
software via automake we will have "real" C projects in our package
manager and we can design our C support around the concerns of real
projects instead of imaginary ones.

There are C projects that would benefit from modularization, header
auditing, and cleanups that Swift and swiftpm would bring to it. C
projects are massively disorganized because build systems are a
ridiculous hodgepodge; we didn't be subject to that long tail of good
and bad decisions.

I don't think automake support would be a silver bullet at all, and
contradict with many goals of swiftpm and llbuild to boot. Even
targeting a really small subset of automake projects what liberties
would unnecessarily complicate the project, and then there'd be the
projects it doesn't support. (Oh? Wait? What version of the tools? Oh,
from trunk? Oh, does the project take any liberties with its own
organization? God help us when we start talking about C++…)

"imaginary" is a reductive way of phrasing the problem space. You keep
referring to "'real' projects" as a proxy for the individual project you
want support for; while there's a lot to be said for real-world use
cases, I don't think this proposal's direction should be dictated by
just libsodium.

Zachary

···

On Sat, Jan 2, 2016, at 04:57 PM, Drew Crawford via swift-build-dev wrote:

Thanks for directing me to this, I missed it.

Most projects will not conform to these conventions.

Giggle. Kind of an understatement, no?

Like, okay. Here is a project I'd like to package. (Read: I do
package it, with features not in mainline swiftPM.)
https://github.com/jedisct1/libsodium

Let's take a look at how this package realistically builds:

* It has tests ("make check")
* It has various --enable-foo flags
* It swaps in special implementations depending on if you have
  AMD64[1] or AVX instructions[2] or SSE2[3] etc.
* The optimization level is tuned on a per-architecture basis[4]
* They build (also) on Windows. They're not changing how they're
  packaged for "SwiftPM, the Mac/Linux build system".
* Oh and this is cryptography code. Do you *really* want to touch it?

*I think an important feature of any C target proposal is that there
will actually exist C targets which can be built under the proposal*.
Until there are C people coming out of the woodwork saying "sure, I
will repackage my software this way" I think the entire value is
debatable.

Getting signoff from libdispatch/CoreFoundation is necessary but not
sufficient to clear that hurdle. I would think getting the other C
deps in our own project family to repackage would be "table stakes"
for any new C build system. The real test are projects that are third-
party and less friendly.

And I do not see realistically how we are ever going to support a
project like libsodium, except calling out to automake. An automake
solution coincidentally supports both libdispatch and CoreFoundation
right now. IMO something like that is a much, much better direction
in the short-term, and once we have done the first step of "packaging"
those software via automake we will have "real" C projects in our
package manager and we can design our C support around the concerns of
real projects instead of imaginary ones.

On Jan 2, 2016, at 11:00 AM, Daniel Dunbar via swift-build-dev <swift-build- >> dev@swift.org> wrote:

Happy 2016!

I am working on an initial proposal for adding support for C language
targets to the Swift package manager, and am interested in feedback:
https://github.com/ddunbar/swift-evolution/blob/master/proposals/NNNN-swiftpm-c-language-targets.md

Some TL;DR:
- The proposal defines a basic convention for pure C language
   targets (no Swift/C mix and match, but other Swift targets can use
   the C targets).
- This is just intended to be the minimal initial feature, there
   will be a lot of add on work which I expect should be tackled in
   follow on PRs/proposals.
- The proposal doesn't try and outline every single nitty detail
   (e.g., exactly what C++ standard we will compile with). My
   intention is to pick a sensible default at implementation time and
   refine incrementally.

Unless there are serious objections, I am hoping to hope to land this
proposal soon and start work on the feature shortly after.

Cheers,
- Daniel

_______________________________________________
swift-build-dev mailing list swift-build-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-build-dev

_________________________________________________
swift-build-dev mailing list swift-build-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-build-dev

Links:

  1. https://github.com/jedisct1/libsodium/blob/master/src/libsodium/Makefile.am#L162
  2. https://github.com/jedisct1/libsodium/blob/master/src/libsodium/Makefile.am#L145
  3. https://github.com/jedisct1/libsodium/blob/master/src/libsodium/Makefile.am#L229
  4. https://github.com/jedisct1/libsodium/blob/master/dist-build/android-armv7-a.sh#L3


(Maury Markowitz) #8

The idea of including C libraries/code in Swift projects *without* having to use Xcode strikes me as *very* valuable. Two questions:

a) for item (2) in "solution" I see the advantages of placing the bridging in a separate directory, but I want to point out that this may make the interactions between various devenvs and things like GitHub more annoying. Xcode's Bridging-Header solution has it's own problems, but may be easier to work with in complex projects. Generally I think more flexibility here might be valuable, even going so far as a ".bh" for the bridging files, allowing them to be placed anywhere.

b) for (3), I'm facing this problem right now porting some very old C that has a main.c that includes main() as well as other more general code that the rest of the system uses. I solved this by changing main(), but that's not ideal, I would greatly prefer to use the original code verbatim. So for the 10% of cases where this is a problem, perhaps a compiler directive would be useful?

···

On Jan 2, 2016, at 9:00 AM, Daniel Dunbar via swift-build-dev <swift-build-dev@swift.org> wrote:

Happy 2016!

I am working on an initial proposal for adding support for C language targets to the Swift package manager, and am interested in feedback:


(Drew Crawford) #9

You keep referring to "'real' projects" as a proxy for the individual project you want support for; while there's a lot to be said for real-world use cases, I don't think this proposal's direction should be dictated by just libsodium.

Nothing about this is reductive or specific to libsodium.

* libdispatch has a build manual <https://github.com/apple/swift-corelibs-libdispatch/blob/master/INSTALL> that runs over 100 lines and involves checking out *7* other repos.
* Foundation (incl CoreFoundation) has a python-based build <https://github.com/apple/swift-corelibs-foundation/tree/master/lib> system that runs to 1900 lines
* libjpeg is "the" example of a C dependency in our documentation, and its build system includes such goodies as choosing a memory manager <http://libjpeg.cvs.sourceforge.net/viewvc/libjpeg/libjpeg/configure.ac?view=markup> or configuring libpng.

And these are just the projects that *we* are associated with!

There are C projects that would benefit from modularization, header auditing, and cleanups that Swift and swiftpm would bring to it.

Swift and swiftpm don't do anything of the kind. C developers do. All we can do is try to impose new requirements on C developers. And who is volunteering to implement those requirements?

It seems to me that if our new requirements are so amazing, it should be easy to convince a few projects to sign on to repackage. libdispatch and Foundation are *our* projects; the bar is so low we're practically cheating. Are they going to switch to this as their build system? I don't know who makes this decision, but it seems like an important question to ask.

No Swift project could be built with swiftpm when it was introduced without repackaging. I don't see why C support should be held to a different standard.

Because we're designing a package manager for the Swift language, not the C language. C has had build systems for decades. We're not going to just waltz in with a new standard for a 44-year-old language and everybody switches the next day. This is https://xkcd.com/927/.

···

On Jan 2, 2016, at 8:11 PM, Zach Waldowski via swift-build-dev <swift-build-dev@swift.org> wrote:

I think an important feature of any C target proposal is that there will actually exist C targets which can be built under the proposal. Until there are C people coming out of the woodwork saying "sure, I will repackage my software this way" I think the entire value is debatable.

I almost couldn't disagree more. No Swift project could be built with swiftpm when it was introduced without repackaging. I don't see why C support should be held to a different standard.

And I do not see realistically how we are ever going to support a project like libsodium, except calling out to automake.

A potential solution (one of many possible) would look a lot like how people generate Xcode projects for C build systems today; hand-tuning config.h headers and such. I know many people who will go to ungodly lengths to avoid the inevitable nightmare automake causes in a source-distributed dependency.

IMO something like that is a much, much better direction in the short-term, and once we have done the first step of "packaging" those software via automake we will have "real" C projects in our package manager and we can design our C support around the concerns of real projects instead of imaginary ones.

There are C projects that would benefit from modularization, header auditing, and cleanups that Swift and swiftpm would bring to it. C projects are massively disorganized because build systems are a ridiculous hodgepodge; we didn't be subject to that long tail of good and bad decisions.

I don't think automake support would be a silver bullet at all, and contradict with many goals of swiftpm and llbuild to boot. Even targeting a really small subset of automake projects what liberties would unnecessarily complicate the project, and then there'd be the projects it doesn't support. (Oh? Wait? What version of the tools? Oh, from trunk? Oh, does the project take any liberties with its own organization? God help us when we start talking about C++…)

"imaginary" is a reductive way of phrasing the problem space. You keep referring to "'real' projects" as a proxy for the individual project you want support for; while there's a lot to be said for real-world use cases, I don't think this proposal's direction should be dictated by just libsodium.

Zachary

On Sat, Jan 2, 2016, at 04:57 PM, Drew Crawford via swift-build-dev wrote:

Thanks for directing me to this, I missed it.

Most projects will not conform to these conventions.

Giggle. Kind of an understatement, no?

Like, okay. Here is a project I'd like to package. (Read: I do package it, with features not in mainline swiftPM.) https://github.com/jedisct1/libsodium

Let's take a look at how this package realistically builds:

* It has tests ("make check")
* It has various --enable-foo flags
* It swaps in special implementations depending on if you have AMD64 <https://github.com/jedisct1/libsodium/blob/master/src/libsodium/Makefile.am#L162> or AVX instructions <https://github.com/jedisct1/libsodium/blob/master/src/libsodium/Makefile.am#L145> or SSE2 <https://github.com/jedisct1/libsodium/blob/master/src/libsodium/Makefile.am#L229> etc.
* The optimization level is tuned on a per-architecture basis <https://github.com/jedisct1/libsodium/blob/master/dist-build/android-armv7-a.sh#L3>
* They build (also) on Windows. They're not changing how they're packaged for "SwiftPM, the Mac/Linux build system".
* Oh and this is cryptography code. Do you *really* want to touch it?

I think an important feature of any C target proposal is that there will actually exist C targets which can be built under the proposal. Until there are C people coming out of the woodwork saying "sure, I will repackage my software this way" I think the entire value is debatable.

Getting signoff from libdispatch/CoreFoundation is necessary but not sufficient to clear that hurdle. I would think getting the other C deps in our own project family to repackage would be "table stakes" for any new C build system. The real test are projects that are third-party and less friendly.

And I do not see realistically how we are ever going to support a project like libsodium, except calling out to automake. An automake solution coincidentally supports both libdispatch and CoreFoundation right now. IMO something like that is a much, much better direction in the short-term, and once we have done the first step of "packaging" those software via automake we will have "real" C projects in our package manager and we can design our C support around the concerns of real projects instead of imaginary ones.

On Jan 2, 2016, at 11:00 AM, Daniel Dunbar via swift-build-dev <swift-build-dev@swift.org <mailto:swift-build-dev@swift.org>> wrote:

Happy 2016!

I am working on an initial proposal for adding support for C language targets to the Swift package manager, and am interested in feedback:
https://github.com/ddunbar/swift-evolution/blob/master/proposals/NNNN-swiftpm-c-language-targets.md

Some TL;DR:
- The proposal defines a basic convention for pure C language targets (no Swift/C mix and match, but other Swift targets can use the C targets).
- This is just intended to be the minimal initial feature, there will be a lot of add on work which I expect should be tackled in follow on PRs/proposals.
- The proposal doesn't try and outline every single nitty detail (e.g., exactly what C++ standard we will compile with). My intention is to pick a sensible default at implementation time and refine incrementally.

Unless there are serious objections, I am hoping to hope to land this proposal soon and start work on the feature shortly after.

Cheers,
- Daniel

_______________________________________________
swift-build-dev mailing list
swift-build-dev@swift.org <mailto:swift-build-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-build-dev

_______________________________________________
swift-build-dev mailing list
swift-build-dev@swift.org <mailto:swift-build-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-build-dev

_______________________________________________
swift-build-dev mailing list
swift-build-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-build-dev


(Daniel Dunbar) #10

Happy 2016!

I am working on an initial proposal for adding support for C language targets to the Swift package manager, and am interested in feedback:

The idea of including C libraries/code in Swift projects *without* having to use Xcode strikes me as *very* valuable. Two questions:

a) for item (2) in "solution" I see the advantages of placing the bridging in a separate directory, but I want to point out that this may make the interactions between various devenvs and things like GitHub more annoying. Xcode's Bridging-Header solution has it's own problems, but may be easier to work with in complex projects. Generally I think more flexibility here might be valuable, even going so far as a ".bh" for the bridging files, allowing them to be placed anywhere.

We anticipate directly adding support for targets with both C and Swift code, at some point (which is where the bridging header comes in), but that is more involved and something I wanted to explicitly tackle separately. This proposal is very focused at adding support for building C source code that is a part of the project, and exposing the headers that declare the interfaces to that code. It is not really intended to tackle the problem of using C headers to expose *other* code.

b) for (3), I'm facing this problem right now porting some very old C that has a main.c that includes main() as well as other more general code that the rest of the system uses. I solved this by changing main(), but that's not ideal, I would greatly prefer to use the original code verbatim. So for the 10% of cases where this is a problem, perhaps a compiler directive would be useful?

If you are integrating the code directly into your project I feel like it makes more sense to require some adaption in order to keep the conventions simple. There are likely a large number of adaptions necessary to integrate existing C projects into these conventions, over time I expect we will allow customization for common problems (and ones that are easy to define a syntax for customizing), but I'm not sure if this one would bubble up to that level of importance.

- Daniel

···

On Jan 3, 2016, at 6:34 AM, Maury Markowitz via swift-evolution <swift-evolution@swift.org> wrote:

On Jan 2, 2016, at 9:00 AM, Daniel Dunbar via swift-build-dev <swift-build-dev@swift.org> wrote:

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Daniel Dunbar) #11

Hi Drew,

First off, I believe that the right way to handle something like libsodium is to have support for building targets using an external build system. I see that as a separate and independent feature from this proposal though. If you are interested in working on that feature I would love to discuss it more on a separate thread (I already mentioned it explicitly on another thread I am unable to find right now) -- it is a feature I would really like to see us have but don't have the bandwidth to tackle at the moment.

This proposal is very specifically targeting the desire to be able to write and build new C code as part of Swift packages, it is not designed to support importing large existing projects. While I do hope that it will feature creep over time to allow more and more C projects to fit within the supported conventions, I also expect that to be a long incremental process.

At this stage of the project, I would encourage looking at new proposals and features from a perspective of "does this add a useful new capability" and "is this in line with our goals" rather than "does this solve my immediate need X".

More comments in line...

You keep referring to "'real' projects" as a proxy for the individual project you want support for; while there's a lot to be said for real-world use cases, I don't think this proposal's direction should be dictated by just libsodium.

Nothing about this is reductive or specific to libsodium.

* libdispatch has a build manual <https://github.com/apple/swift-corelibs-libdispatch/blob/master/INSTALL> that runs over 100 lines and involves checking out *7* other repos.
* Foundation (incl CoreFoundation) has a python-based build <https://github.com/apple/swift-corelibs-foundation/tree/master/lib> system that runs to 1900 lines
* libjpeg is "the" example of a C dependency in our documentation, and its build system includes such goodies as choosing a memory manager <http://libjpeg.cvs.sourceforge.net/viewvc/libjpeg/libjpeg/configure.ac?view=markup> or configuring libpng.

And these are just the projects that *we* are associated with!

These are projects I would expect to be tackled with an "external build system" feature, not this proposal.

There are C projects that would benefit from modularization, header auditing, and cleanups that Swift and swiftpm would bring to it.

Swift and swiftpm don't do anything of the kind. C developers do. All we can do is try to impose new requirements on C developers. And who is volunteering to implement those requirements?

It seems to me that if our new requirements are so amazing, it should be easy to convince a few projects to sign on to repackage. libdispatch and Foundation are *our* projects; the bar is so low we're practically cheating. Are they going to switch to this as their build system? I don't know who makes this decision, but it seems like an important question to ask.

While I think that "support libdispatch and Foundation" are good long term goals and useful reference points for what features are still needed to get there, it isn't the immediate goal here.

No Swift project could be built with swiftpm when it was introduced without repackaging. I don't see why C support should be held to a different standard.

Because we're designing a package manager for the Swift language, not the C language. C has had build systems for decades. We're not going to just waltz in with a new standard for a 44-year-old language and everybody switches the next day. This is https://xkcd.com/927/.

We can all dream, right? :slight_smile:

I think an important feature of any C target proposal is that there will actually exist C targets which can be built under the proposal. Until there are C people coming out of the woodwork saying "sure, I will repackage my software this way" I think the entire value is debatable.

I almost couldn't disagree more. No Swift project could be built with swiftpm when it was introduced without repackaging. I don't see why C support should be held to a different standard.

I agree with Zach here. The purpose of the proposal is to add support for new code written in C designed to integrate with other Swift code. The merits of the conventions should be weighed about how easy it is to write that code, and what the implications for maintenance (on both the package manager and the code itself) are.

And I do not see realistically how we are ever going to support a project like libsodium, except calling out to automake.

A potential solution (one of many possible) would look a lot like how people generate Xcode projects for C build systems today; hand-tuning config.h headers and such. I know many people who will go to ungodly lengths to avoid the inevitable nightmare automake causes in a source-distributed dependency.

IMO something like that is a much, much better direction in the short-term, and once we have done the first step of "packaging" those software via automake we will have "real" C projects in our package manager and we can design our C support around the concerns of real projects instead of imaginary ones.

There are C projects that would benefit from modularization, header auditing, and cleanups that Swift and swiftpm would bring to it. C projects are massively disorganized because build systems are a ridiculous hodgepodge; we didn't be subject to that long tail of good and bad decisions.

I don't think automake support would be a silver bullet at all, and contradict with many goals of swiftpm and llbuild to boot. Even targeting a really small subset of automake projects what liberties would unnecessarily complicate the project, and then there'd be the projects it doesn't support. (Oh? Wait? What version of the tools? Oh, from trunk? Oh, does the project take any liberties with its own organization? God help us when we start talking about C++…)

"imaginary" is a reductive way of phrasing the problem space. You keep referring to "'real' projects" as a proxy for the individual project you want support for; while there's a lot to be said for real-world use cases, I don't think this proposal's direction should be dictated by just libsodium.

Zachary

Thanks for directing me to this, I missed it.

Most projects will not conform to these conventions.

Giggle. Kind of an understatement, no?

Like, okay. Here is a project I'd like to package. (Read: I do package it, with features not in mainline swiftPM.) https://github.com/jedisct1/libsodium

Let's take a look at how this package realistically builds:

Here is how I would expect these things to be tackled:

* It has tests ("make check")

I don't have any particular plan for this one.

* It has various --enable-foo flags

I expect that eventually the package manager will grow some level of support for this kind of thing (similar to Cargo's "features" -- http://doc.crates.io/manifest.html#the-[features]-section).

* It swaps in special implementations depending on if you have AMD64 <https://github.com/jedisct1/libsodium/blob/master/src/libsodium/Makefile.am#L162> or AVX instructions <https://github.com/jedisct1/libsodium/blob/master/src/libsodium/Makefile.am#L145> or SSE2 <https://github.com/jedisct1/libsodium/blob/master/src/libsodium/Makefile.am#L229> etc.
* The optimization level is tuned on a per-architecture basis <https://github.com/jedisct1/libsodium/blob/master/dist-build/android-armv7-a.sh#L3>

I expect that both of these would become possible with small enhancements to allow customization via the manifest file. Again, this is just the initial feature.

* They build (also) on Windows. They're not changing how they're packaged for "SwiftPM, the Mac/Linux build system".

I anticipate that we will eventually support alternative source layouts via customization in the manifest file (mostly to change where the sources are found, and the way headers are located and treated).

* Oh and this is cryptography code. Do you *really* want to touch it?

It is a non-goal of this proposal to support projects which "I don't want to touch".

I think an important feature of any C target proposal is that there will actually exist C targets which can be built under the proposal. Until there are C people coming out of the woodwork saying "sure, I will repackage my software this way" I think the entire value is debatable.

I think your language is more polarizing than is due here, and I would encourage focusing on a technical argument rather than a judgement like "the entire value is debatable". This proposal will clearly allow packages to add small bits of C code which are used by other targets. Unless you believe that is incorrect (and if so, please present a technical argument for it), then to me that is a valuable capability (and if you disagree, then please present a technical argument for it).

- Daniel

···

On Jan 3, 2016, at 12:17 AM, Drew Crawford via swift-build-dev <swift-build-dev@swift.org> wrote:

On Jan 2, 2016, at 8:11 PM, Zach Waldowski via swift-build-dev <swift-build-dev@swift.org <mailto:swift-build-dev@swift.org>> wrote:
On Sat, Jan 2, 2016, at 04:57 PM, Drew Crawford via swift-build-dev wrote:

Getting signoff from libdispatch/CoreFoundation is necessary but not sufficient to clear that hurdle. I would think getting the other C deps in our own project family to repackage would be "table stakes" for any new C build system. The real test are projects that are third-party and less friendly.

And I do not see realistically how we are ever going to support a project like libsodium, except calling out to automake. An automake solution coincidentally supports both libdispatch and CoreFoundation right now. IMO something like that is a much, much better direction in the short-term, and once we have done the first step of "packaging" those software via automake we will have "real" C projects in our package manager and we can design our C support around the concerns of real projects instead of imaginary ones.

On Jan 2, 2016, at 11:00 AM, Daniel Dunbar via swift-build-dev <swift-build-dev@swift.org <mailto:swift-build-dev@swift.org>> wrote:

Happy 2016!

I am working on an initial proposal for adding support for C language targets to the Swift package manager, and am interested in feedback:
https://github.com/ddunbar/swift-evolution/blob/master/proposals/NNNN-swiftpm-c-language-targets.md

Some TL;DR:
- The proposal defines a basic convention for pure C language targets (no Swift/C mix and match, but other Swift targets can use the C targets).
- This is just intended to be the minimal initial feature, there will be a lot of add on work which I expect should be tackled in follow on PRs/proposals.
- The proposal doesn't try and outline every single nitty detail (e.g., exactly what C++ standard we will compile with). My intention is to pick a sensible default at implementation time and refine incrementally.

Unless there are serious objections, I am hoping to hope to land this proposal soon and start work on the feature shortly after.

Cheers,
- Daniel

_______________________________________________
swift-build-dev mailing list
swift-build-dev@swift.org <mailto:swift-build-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-build-dev

_______________________________________________
swift-build-dev mailing list
swift-build-dev@swift.org <mailto:swift-build-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-build-dev

_______________________________________________
swift-build-dev mailing list
swift-build-dev@swift.org <mailto:swift-build-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-build-dev

_______________________________________________
swift-build-dev mailing list
swift-build-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-build-dev


(Drew Crawford) #12

I think I understand better what you are trying to accomplish. It is not a bad way to go about it, and I think my comments were not as constructive as they should have been.

I sent a PR that may clarify your motivation so as to head off objections like mine. I think my reaction to the document was largely based around a misunderstanding about the motivation. The title is "Package Manager C Language Target Support", and so that sounds like it is a proposal about C language support as a whole, which immediately raises questions about whether Foundation et al can be someday made to fit in its box (e.g. via extending it etc.) Whereas if we are clearer that this is merely one path to C language support, and this is the first one on the whiteboard then I think it is a lot stronger. It does solve a problem, I have a better idea of what that problem is, and the fact that it doesn't map well onto my problems is no longer troubling.

I think your language is more polarizing than is due here, and I would encourage focusing on a technical argument rather than a judgement like "the entire value is debatable".

I apologize for that.

For background, when the topic of native C support vs external build systems has come up in the past, there's been an implicit value judgment that building C with swiftPM is the "preferred" way, and using an external build system is the "compromise" way, and when we set up that value hierarchy it is necessarily the case that the two ideas are in conflict.

I'm coming around to the idea that actually these are totally unrelated problems, they only looked similar at first glance, that neither is any better or worse than the other, that we need to support both, and so we should just give both of them permission to be excellent in their own way, and that we should not try to shoehorn one of them into the problems that the other one is better at.

Your proposal is clearly going to be better at the "few files of C code" problem, it was a bad idea for me to overcomplicate it with the problems of established C projects when they can be better served by handling them with an entirely separate solution.

···

On Jan 4, 2016, at 6:48 PM, Daniel Dunbar <daniel_dunbar@apple.com> wrote:

Hi Drew,

First off, I believe that the right way to handle something like libsodium is to have support for building targets using an external build system. I see that as a separate and independent feature from this proposal though. If you are interested in working on that feature I would love to discuss it more on a separate thread (I already mentioned it explicitly on another thread I am unable to find right now) -- it is a feature I would really like to see us have but don't have the bandwidth to tackle at the moment.

This proposal is very specifically targeting the desire to be able to write and build new C code as part of Swift packages, it is not designed to support importing large existing projects. While I do hope that it will feature creep over time to allow more and more C projects to fit within the supported conventions, I also expect that to be a long incremental process.

At this stage of the project, I would encourage looking at new proposals and features from a perspective of "does this add a useful new capability" and "is this in line with our goals" rather than "does this solve my immediate need X".

More comments in line...

On Jan 3, 2016, at 12:17 AM, Drew Crawford via swift-build-dev <swift-build-dev@swift.org <mailto:swift-build-dev@swift.org>> wrote:

You keep referring to "'real' projects" as a proxy for the individual project you want support for; while there's a lot to be said for real-world use cases, I don't think this proposal's direction should be dictated by just libsodium.

Nothing about this is reductive or specific to libsodium.

* libdispatch has a build manual <https://github.com/apple/swift-corelibs-libdispatch/blob/master/INSTALL> that runs over 100 lines and involves checking out *7* other repos.
* Foundation (incl CoreFoundation) has a python-based build <https://github.com/apple/swift-corelibs-foundation/tree/master/lib> system that runs to 1900 lines
* libjpeg is "the" example of a C dependency in our documentation, and its build system includes such goodies as choosing a memory manager <http://libjpeg.cvs.sourceforge.net/viewvc/libjpeg/libjpeg/configure.ac?view=markup> or configuring libpng.

And these are just the projects that *we* are associated with!

These are projects I would expect to be tackled with an "external build system" feature, not this proposal.

There are C projects that would benefit from modularization, header auditing, and cleanups that Swift and swiftpm would bring to it.

Swift and swiftpm don't do anything of the kind. C developers do. All we can do is try to impose new requirements on C developers. And who is volunteering to implement those requirements?

It seems to me that if our new requirements are so amazing, it should be easy to convince a few projects to sign on to repackage. libdispatch and Foundation are *our* projects; the bar is so low we're practically cheating. Are they going to switch to this as their build system? I don't know who makes this decision, but it seems like an important question to ask.

While I think that "support libdispatch and Foundation" are good long term goals and useful reference points for what features are still needed to get there, it isn't the immediate goal here.

No Swift project could be built with swiftpm when it was introduced without repackaging. I don't see why C support should be held to a different standard.

Because we're designing a package manager for the Swift language, not the C language. C has had build systems for decades. We're not going to just waltz in with a new standard for a 44-year-old language and everybody switches the next day. This is https://xkcd.com/927/.

We can all dream, right? :slight_smile:

On Jan 2, 2016, at 8:11 PM, Zach Waldowski via swift-build-dev <swift-build-dev@swift.org <mailto:swift-build-dev@swift.org>> wrote:

I think an important feature of any C target proposal is that there will actually exist C targets which can be built under the proposal. Until there are C people coming out of the woodwork saying "sure, I will repackage my software this way" I think the entire value is debatable.

I almost couldn't disagree more. No Swift project could be built with swiftpm when it was introduced without repackaging. I don't see why C support should be held to a different standard.

I agree with Zach here. The purpose of the proposal is to add support for new code written in C designed to integrate with other Swift code. The merits of the conventions should be weighed about how easy it is to write that code, and what the implications for maintenance (on both the package manager and the code itself) are.

And I do not see realistically how we are ever going to support a project like libsodium, except calling out to automake.

A potential solution (one of many possible) would look a lot like how people generate Xcode projects for C build systems today; hand-tuning config.h headers and such. I know many people who will go to ungodly lengths to avoid the inevitable nightmare automake causes in a source-distributed dependency.

IMO something like that is a much, much better direction in the short-term, and once we have done the first step of "packaging" those software via automake we will have "real" C projects in our package manager and we can design our C support around the concerns of real projects instead of imaginary ones.

There are C projects that would benefit from modularization, header auditing, and cleanups that Swift and swiftpm would bring to it. C projects are massively disorganized because build systems are a ridiculous hodgepodge; we didn't be subject to that long tail of good and bad decisions.

I don't think automake support would be a silver bullet at all, and contradict with many goals of swiftpm and llbuild to boot. Even targeting a really small subset of automake projects what liberties would unnecessarily complicate the project, and then there'd be the projects it doesn't support. (Oh? Wait? What version of the tools? Oh, from trunk? Oh, does the project take any liberties with its own organization? God help us when we start talking about C++…)

"imaginary" is a reductive way of phrasing the problem space. You keep referring to "'real' projects" as a proxy for the individual project you want support for; while there's a lot to be said for real-world use cases, I don't think this proposal's direction should be dictated by just libsodium.

Zachary

On Sat, Jan 2, 2016, at 04:57 PM, Drew Crawford via swift-build-dev wrote:

Thanks for directing me to this, I missed it.

Most projects will not conform to these conventions.

Giggle. Kind of an understatement, no?

Like, okay. Here is a project I'd like to package. (Read: I do package it, with features not in mainline swiftPM.) https://github.com/jedisct1/libsodium

Let's take a look at how this package realistically builds:

Here is how I would expect these things to be tackled:

* It has tests ("make check")

I don't have any particular plan for this one.

* It has various --enable-foo flags

I expect that eventually the package manager will grow some level of support for this kind of thing (similar to Cargo's "features" -- http://doc.crates.io/manifest.html#the-[features]-section <http://doc.crates.io/manifest.html#the-[features]-section>).

* It swaps in special implementations depending on if you have AMD64 <https://github.com/jedisct1/libsodium/blob/master/src/libsodium/Makefile.am#L162> or AVX instructions <https://github.com/jedisct1/libsodium/blob/master/src/libsodium/Makefile.am#L145> or SSE2 <https://github.com/jedisct1/libsodium/blob/master/src/libsodium/Makefile.am#L229> etc.
* The optimization level is tuned on a per-architecture basis <https://github.com/jedisct1/libsodium/blob/master/dist-build/android-armv7-a.sh#L3>

I expect that both of these would become possible with small enhancements to allow customization via the manifest file. Again, this is just the initial feature.

* They build (also) on Windows. They're not changing how they're packaged for "SwiftPM, the Mac/Linux build system".

I anticipate that we will eventually support alternative source layouts via customization in the manifest file (mostly to change where the sources are found, and the way headers are located and treated).

* Oh and this is cryptography code. Do you *really* want to touch it?

It is a non-goal of this proposal to support projects which "I don't want to touch".

I think an important feature of any C target proposal is that there will actually exist C targets which can be built under the proposal. Until there are C people coming out of the woodwork saying "sure, I will repackage my software this way" I think the entire value is debatable.

I think your language is more polarizing than is due here, and I would encourage focusing on a technical argument rather than a judgement like "the entire value is debatable". This proposal will clearly allow packages to add small bits of C code which are used by other targets. Unless you believe that is incorrect (and if so, please present a technical argument for it), then to me that is a valuable capability (and if you disagree, then please present a technical argument for it).

- Daniel

Getting signoff from libdispatch/CoreFoundation is necessary but not sufficient to clear that hurdle. I would think getting the other C deps in our own project family to repackage would be "table stakes" for any new C build system. The real test are projects that are third-party and less friendly.

And I do not see realistically how we are ever going to support a project like libsodium, except calling out to automake. An automake solution coincidentally supports both libdispatch and CoreFoundation right now. IMO something like that is a much, much better direction in the short-term, and once we have done the first step of "packaging" those software via automake we will have "real" C projects in our package manager and we can design our C support around the concerns of real projects instead of imaginary ones.

On Jan 2, 2016, at 11:00 AM, Daniel Dunbar via swift-build-dev <swift-build-dev@swift.org <mailto:swift-build-dev@swift.org>> wrote:

Happy 2016!

I am working on an initial proposal for adding support for C language targets to the Swift package manager, and am interested in feedback:
https://github.com/ddunbar/swift-evolution/blob/master/proposals/NNNN-swiftpm-c-language-targets.md

Some TL;DR:
- The proposal defines a basic convention for pure C language targets (no Swift/C mix and match, but other Swift targets can use the C targets).
- This is just intended to be the minimal initial feature, there will be a lot of add on work which I expect should be tackled in follow on PRs/proposals.
- The proposal doesn't try and outline every single nitty detail (e.g., exactly what C++ standard we will compile with). My intention is to pick a sensible default at implementation time and refine incrementally.

Unless there are serious objections, I am hoping to hope to land this proposal soon and start work on the feature shortly after.

Cheers,
- Daniel

_______________________________________________
swift-build-dev mailing list
swift-build-dev@swift.org <mailto:swift-build-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-build-dev

_______________________________________________
swift-build-dev mailing list
swift-build-dev@swift.org <mailto:swift-build-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-build-dev

_______________________________________________
swift-build-dev mailing list
swift-build-dev@swift.org <mailto:swift-build-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-build-dev

_______________________________________________
swift-build-dev mailing list
swift-build-dev@swift.org <mailto:swift-build-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-build-dev


(Daniel Dunbar) #13

Great, glad we could all get on the same page!

I agree these are essentially two unrelated problems and we will end up
with two features to support them. Both will have pros and cons and we
should definitely shoot for both being "excellent"!

Thanks for the PR I'll take a look when I am back at a computer.

- Daniel

···

On Tuesday, January 5, 2016, Drew Crawford via swift-build-dev < swift-build-dev@swift.org> wrote:

I think I understand better what you are trying to accomplish. It is not
a bad way to go about it, and I think my comments were not as constructive
as they should have been.

I sent a PR that may clarify your motivation so as to head off objections
like mine. I think my reaction to the document was largely based around a
misunderstanding about the motivation. The title is "Package Manager C
Language Target Support", and so that *sounds* like it is a proposal
about C language support as a whole, which immediately raises questions
about whether Foundation et al can be someday made to fit in its box (e.g.
via extending it etc.) Whereas if we are clearer that this is *merely
one path to C language support, and this is the first one on the whiteboard* then
I think it is a lot stronger. It does solve a problem, I have a better
idea of what that problem is, and the fact that it doesn't map well onto my
problems is no longer troubling.

I think your language is more polarizing than is due here, and I would
encourage focusing on a technical argument rather than a judgement like
"the entire value is debatable".

I apologize for that.

For background, when the topic of native C support vs external build
systems has come up in the past, there's been an implicit value judgment
that building C with swiftPM is the "preferred" way, and using an external
build system is the "compromise" way, and when we set up that value
hierarchy it is necessarily the case that the two ideas are in conflict.

I'm coming around to the idea that actually these are totally unrelated
problems, they only looked similar at first glance, that neither is any
better or worse than the other, that we need to support both, and so we
should just give both of them permission to be excellent in their own way,
and that we should not try to shoehorn one of them into the problems that
the other one is better at.

Your proposal is clearly going to be better at the "few files of C code"
problem, it was a bad idea for me to overcomplicate it with the problems of
established C projects when they can be better served by handling them with
an entirely separate solution.

On Jan 4, 2016, at 6:48 PM, Daniel Dunbar <daniel_dunbar@apple.com > <javascript:_e(%7B%7D,'cvml','daniel_dunbar@apple.com');>> wrote:

Hi Drew,

First off, I believe that the right way to handle something like libsodium
is to have support for building targets using an external build system. I
see that as a separate and independent feature from this proposal though.
If you are interested in working on that feature I would love to discuss it
more on a separate thread (I already mentioned it explicitly on another
thread I am unable to find right now) -- it is a feature I would really
like to see us have but don't have the bandwidth to tackle at the moment.

This proposal is very specifically targeting the desire to be able to
write and build new C code as part of Swift packages, it is not designed to
support importing large existing projects. While I do hope that it will
feature creep over time to allow more and more C projects to fit within the
supported conventions, I also expect that to be a long incremental process.

At this stage of the project, I would encourage looking at new proposals
and features from a perspective of "does this add a useful new capability"
and "is this in line with our goals" rather than "does this solve my
immediate need X".

More comments in line...

On Jan 3, 2016, at 12:17 AM, Drew Crawford via swift-build-dev < > swift-build-dev@swift.org > <javascript:_e(%7B%7D,'cvml','swift-build-dev@swift.org');>> wrote:

You keep referring to "'real' projects" as a proxy for the individual
project you want support for; while there's a lot to be said for real-world
use cases, I don't think this proposal's direction should be dictated by
just libsodium.

Nothing about this is reductive or specific to libsodium.

* libdispatch has a build manual
<https://github.com/apple/swift-corelibs-libdispatch/blob/master/INSTALL> that
runs over 100 lines and involves checking out *7* other repos.
* Foundation (incl CoreFoundation) has a python-based build
<https://github.com/apple/swift-corelibs-foundation/tree/master/lib> system
that runs to 1900 lines
* libjpeg is "the" example of a C dependency in our documentation, and its
build system includes such goodies as choosing a memory manager
<http://libjpeg.cvs.sourceforge.net/viewvc/libjpeg/libjpeg/configure.ac?view=markup> or
configuring libpng.

And these are just the projects that *we* are associated with!

These are projects I would expect to be tackled with an "external build
system" feature, not this proposal.

There are C projects that would benefit from modularization, header
auditing, and cleanups that Swift and swiftpm would bring to it.

Swift and swiftpm don't do anything of the kind. C developers do. All we
can do is try to impose new requirements on C developers. And who is
volunteering to implement those requirements?

It seems to me that if our new requirements are so amazing, it should be
easy to convince a few projects to sign on to repackage. libdispatch and
Foundation are *our* projects; the bar is so low we're practically
cheating. Are they going to switch to this as their build system? I don't
know who makes this decision, but it seems like an important question to
ask.

While I think that "support libdispatch and Foundation" are good long term
goals and useful reference points for what features are still needed to get
there, it isn't the immediate goal here.

No Swift project could be built with swiftpm when it was introduced
without repackaging. I don't see why C support should be held to a
different standard.

Because we're designing a package manager for the Swift language, not the
C language. C has had build systems for decades. We're not going to just
waltz in with a new standard for a 44-year-old language and everybody
switches the next day. This is https://xkcd.com/927/.

We can all dream, right? :slight_smile:

On Jan 2, 2016, at 8:11 PM, Zach Waldowski via swift-build-dev < > swift-build-dev@swift.org > <javascript:_e(%7B%7D,'cvml','swift-build-dev@swift.org');>> wrote:

*I think an important feature of any C target proposal is that there will
actually exist C targets which can be built under the proposal*. Until
there are C people coming out of the woodwork saying "sure, I will
repackage my software this way" I think the entire value is debatable.

I almost couldn't disagree more. No Swift project could be built with
swiftpm when it was introduced without repackaging. I don't see why C
support should be held to a different standard.

I agree with Zach here. The purpose of the proposal is to add support for
new code written in C designed to integrate with other Swift code. The
merits of the conventions should be weighed about how easy it is to write
that code, and what the implications for maintenance (on both the package
manager and the code itself) are.

And I do not see realistically how we are ever going to support a project
like libsodium, except calling out to automake.

A potential solution (one of many possible) would look a lot like how
people generate Xcode projects for C build systems today; hand-tuning
config.h headers and such. I know many people who will go to ungodly
lengths to avoid the inevitable nightmare automake causes in a
source-distributed dependency.

IMO something like that is a much, much better direction in the
short-term, and once we have done the first step of "packaging" those
software via automake we will have "real" C projects in our package manager
and we can design our C support around the concerns of real projects
instead of imaginary ones.

There are C projects that would benefit from modularization, header
auditing, and cleanups that Swift and swiftpm would bring to it. C projects
are massively disorganized because build systems are a ridiculous
hodgepodge; we didn't be subject to that long tail of good and bad
decisions.

I don't think automake support would be a silver bullet at all, and
contradict with many goals of swiftpm and llbuild to boot. Even targeting a
really small subset of automake projects what liberties would unnecessarily
complicate the project, and then there'd be the projects it doesn't
support. (Oh? Wait? What version of the tools? Oh, from trunk? Oh, does
the project take any liberties with its own organization? God help us when
we start talking about C++…)

"imaginary" is a reductive way of phrasing the problem space. You keep
referring to "'real' projects" as a proxy for the individual project you
want support for; while there's a lot to be said for real-world use cases,
I don't think this proposal's direction should be dictated by just
libsodium.

Zachary

On Sat, Jan 2, 2016, at 04:57 PM, Drew Crawford via swift-build-dev wrote:

Thanks for directing me to this, I missed it.

Most projects will not conform to these conventions.

Giggle. Kind of an understatement, no?

Like, okay. Here is a project I'd like to package. (Read: I do package
it, with features not in mainline swiftPM.)
https://github.com/jedisct1/libsodium

Let's take a look at how this package realistically builds:

Here is how I would expect these things to be tackled:

* It has tests ("make check")

I don't have any particular plan for this one.

* It has various --enable-foo flags

I expect that eventually the package manager will grow some level of
support for this kind of thing (similar to Cargo's "features" --
http://doc.crates.io/manifest.html#the-[features]-section).

* It swaps in special implementations depending on if you have AMD64
<https://github.com/jedisct1/libsodium/blob/master/src/libsodium/Makefile.am#L162>
or AVX instructions
<https://github.com/jedisct1/libsodium/blob/master/src/libsodium/Makefile.am#L145>
or SSE2
<https://github.com/jedisct1/libsodium/blob/master/src/libsodium/Makefile.am#L229>
etc.
* The optimization level is tuned on a per-architecture basis
<https://github.com/jedisct1/libsodium/blob/master/dist-build/android-armv7-a.sh#L3>

I expect that both of these would become possible with small enhancements
to allow customization via the manifest file. Again, this is just the
initial feature.

* They build (also) on Windows. They're not changing how they're packaged
for "SwiftPM, the Mac/Linux build system".

I anticipate that we will eventually support alternative source layouts
via customization in the manifest file (mostly to change where the sources
are found, and the way headers are located and treated).

* Oh and this is cryptography code. Do you *really* want to touch it?

It is a non-goal of this proposal to support projects which "I don't want
to touch".

*I think an important feature of any C target proposal is that there will
actually exist C targets which can be built under the proposal*. Until
there are C people coming out of the woodwork saying "sure, I will
repackage my software this way" I think the entire value is debatable.

I think your language is more polarizing than is due here, and I would
encourage focusing on a technical argument rather than a judgement like
"the entire value is debatable". This proposal will clearly allow packages
to add small bits of C code which are used by other targets. Unless you
believe that is incorrect (and if so, please present a technical argument
for it), then to me that is a valuable capability (and if you disagree,
then please present a technical argument for it).

- Daniel

Getting signoff from libdispatch/CoreFoundation is necessary but not
sufficient to clear that hurdle. I would think getting the other C deps in
our own project family to repackage would be "table stakes" for any new C
build system. The real test are projects that are third-party and less
friendly.

And I do not see realistically how we are ever going to support a project
like libsodium, except calling out to automake. An automake solution
coincidentally supports both libdispatch and CoreFoundation right now. IMO
something like that is a much, much better direction in the short-term, and
once we have done the first step of "packaging" those software via automake
we will have "real" C projects in our package manager and we can design our
C support around the concerns of real projects instead of imaginary ones.

On Jan 2, 2016, at 11:00 AM, Daniel Dunbar via swift-build-dev < > swift-build-dev@swift.org > <javascript:_e(%7B%7D,'cvml','swift-build-dev@swift.org');>> wrote:

Happy 2016!

I am working on an initial proposal for adding support for C language
targets to the Swift package manager, and am interested in feedback:

https://github.com/ddunbar/swift-evolution/blob/master/proposals/NNNN-swiftpm-c-language-targets.md

Some TL;DR:
- The proposal defines a basic convention for pure C language targets (no
Swift/C mix and match, but other Swift targets can use the C targets).
- This is just intended to be the minimal initial feature, there will be a
lot of add on work which I expect should be tackled in follow on
PRs/proposals.
- The proposal doesn't try and outline every single nitty detail (e.g.,
exactly what C++ standard we will compile with). My intention is to pick a
sensible default at implementation time and refine incrementally.

Unless there are serious objections, I am hoping to hope to land this
proposal soon and start work on the feature shortly after.

Cheers,
- Daniel

_______________________________________________
swift-build-dev mailing list
swift-build-dev@swift.org
<javascript:_e(%7B%7D,'cvml','swift-build-dev@swift.org');>
https://lists.swift.org/mailman/listinfo/swift-build-dev

*_______________________________________________*
swift-build-dev mailing list
swift-build-dev@swift.org
<javascript:_e(%7B%7D,'cvml','swift-build-dev@swift.org');>
https://lists.swift.org/mailman/listinfo/swift-build-dev

_______________________________________________
swift-build-dev mailing list
swift-build-dev@swift.org
<javascript:_e(%7B%7D,'cvml','swift-build-dev@swift.org');>
https://lists.swift.org/mailman/listinfo/swift-build-dev

_______________________________________________
swift-build-dev mailing list
swift-build-dev@swift.org
<javascript:_e(%7B%7D,'cvml','swift-build-dev@swift.org');>
https://lists.swift.org/mailman/listinfo/swift-build-dev