Importing C system libraries

Kelvin_Ma1 · March 29, 2017, 5:18am

This worked! Thanks! But why is having empty git repositories strewn about
the “correct” way? System libraries should be imported from within the
project, as they are in C. You have to admit it’s getting quite silly that
Swift devs keep repositories like these
<https://github.com/kelvin13/swift-zlib> on our github accounts. That zlib
repository contains exactly ten lines of code. I used to have 6 or 7 repos
like that one up there before I got rid of them and switched to local
repos.

···

On Wed, Mar 29, 2017 at 12:03 AM, Ankit Aggarwal <ankit_aggarwal@apple.com> wrote:

In this case, these are just umbrella headers. If your modulemap contains
absolute path to the header, then you don't need the header files, but
SwiftPM will probably warn about this. Note that this is a "hack" to have
system packages inside a single repository. The correct way is to have
system package as a separate published package which you only need to do
once.

On 29-Mar-2017, at 10:26 AM, Kelvin Ma <kelvinsthirteen@gmail.com> wrote:

I will try this, but why are the header files inside the Sources
directory? System headers should live in /usr/include…

On Tue, Mar 28, 2017 at 11:48 PM, Ankit Aggarwal <ankit_aggarwal@apple.com > > wrote:

Hi,

Apologies for not replying to this earlier.

You can have multiple targets in a single package. Each target can either
be Swift or C-family. The type of target is determined by the sources
contained in it (*.c/*.cpp etc means C target, *.swift means Swift target).
So if you want to create multiple C targets, this layout should work:

Package.swift
Sources/
    Bitmap
    Cubify
    Cairo/anchor.c <---- This is just an empty file to tell SwiftPM that
this is a C target.
    Cairo/include/Cairo.h
    Cairo/include/module.modulemap
    GLFW/anchor.c
    GLFW/include/GLFW.h
    GLFW/include/module.modulemap

The modulemap is automatically generated, if not provided. This is a
package which contains two targets (one C and one Swift):
GitHub - jpsim/Yams: A Sweet and Swifty YAML parser.

If you need to pass a bunch of compiler flags, you can use SwiftPM's
pkgConfig feature but that will require you to have a separate repository
for Cario and GLFW. You can experiment without creating tags using the edit
feature
<https://github.com/apple/swift-package-manager/blob/master/Documentation/Usage.md#editable-packages>
.

PS: You can join SwiftPM slack channel for quicker turn around time:
https://lists.swift.org/pipermail/swift-build-dev/Week
-of-Mon-20160530/000497.html

Thanks,
Ankit

On Wed, Mar 29, 2017 at 6:06 AM, Michael Ilseman via swift-build-dev < >> swift-build-dev@swift.org> wrote:

This is into uncharted territory for me, but it seems you’re building
with SwiftPM. You’ll probably want to configure extra compiler flags if
that’s possible. You could also bite the bullet and build your C libraries
with SwiftPM as well. Hopefully someone on swift-build-dev can help you out.

CC-ing Ankit

On Mar 28, 2017, at 5:09 PM, Kelvin Ma <kelvinsthirteen@gmail.com> >>> wrote:

How do I compile a project with many modules? My tree looks like this:

<Selection_001.png>

On Tue, Mar 28, 2017 at 12:47 PM, Michael Ilseman <milseman@apple.com> >>> wrote:

Sure! In this example, I have built libgit2. I have a directory called
Git, and inside that I have the following module map:

module Git [system] {
       header "<my path>/libgit2/include/git2.h"
       export *
}

When I run, I use:

swift -I <path-to-“Git”-directory> -L <path-to-built-libgit2> -lgit2
foo.swift

inside foo.swift I can:

import Git
// … use libGit2

Read more about how to write a more appropriate module.map file for
your purposes at https://clang.llvm.org/docs/Modules.html\. For
example, you might be able to define link flags inside the module.map, use
umbrella directories, submodules, etc.

On Mar 28, 2017, at 6:27 AM, Kelvin Ma <kelvinsthirteen@gmail.com> >>>> wrote:

Can you give an example?

On Mon, Mar 27, 2017 at 3:59 PM, Michael Ilseman <milseman@apple.com> >>>> wrote:

Sure. At a low level, you can create a module.map file and use -L/-l
flags in your invocation of Swift. If you want to do so at a higher level,
then perhaps SwiftPM can. CCing swift-build-dev for the SwiftPM part.

> On Mar 26, 2017, at 3:20 PM, Kelvin Ma via swift-users < >>>>> swift-users@swift.org> wrote:
>
> Idk if this has been asked before, but is there a way to import C
libraries into a Swift project without creating a local git repo?
Preferably something similar to C where you can just `#include` headers and
then specify the link flags (in Package.swift?)
>
> It’s getting very cumbersome to make a bunch of empty git repos just
to use libglfw or libcairo.
> _______________________________________________
> swift-users mailing list
> swift-users@swift.org
> https://lists.swift.org/mailman/listinfo/swift-users

_______________________________________________
swift-build-dev mailing list
swift-build-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-build-dev

Aciid · March 29, 2017, 5:42am

I think the idea was that there will be one such repository which other
packages can use, that too only until the system libraries start shipping
their standard modulemap. I thought we had this written down in our
documentation somewhere but couldn't find it.

Maybe Daniel can expand on this.

···

On Wed, Mar 29, 2017 at 10:48 AM, Kelvin Ma via swift-build-dev < swift-build-dev@swift.org> wrote:

This worked! Thanks! But why is having empty git repositories strewn about
the “correct” way? System libraries should be imported from within the
project, as they are in C. You have to admit it’s getting quite silly that
Swift devs keep repositories like these
<https://github.com/kelvin13/swift-zlib> on our github accounts. That
zlib repository contains exactly ten lines of code. I used to have 6 or 7
repos like that one up there before I got rid of them and switched to local
repos.

On Wed, Mar 29, 2017 at 12:03 AM, Ankit Aggarwal <ankit_aggarwal@apple.com > > wrote:

In this case, these are just umbrella headers. If your modulemap contains
absolute path to the header, then you don't need the header files, but
SwiftPM will probably warn about this. Note that this is a "hack" to have
system packages inside a single repository. The correct way is to have
system package as a separate published package which you only need to do
once.

On 29-Mar-2017, at 10:26 AM, Kelvin Ma <kelvinsthirteen@gmail.com> wrote:

I will try this, but why are the header files inside the Sources
directory? System headers should live in /usr/include…

On Tue, Mar 28, 2017 at 11:48 PM, Ankit Aggarwal < >> ankit_aggarwal@apple.com> wrote:

Hi,

Apologies for not replying to this earlier.

You can have multiple targets in a single package. Each target can
either be Swift or C-family. The type of target is determined by the
sources contained in it (*.c/*.cpp etc means C target, *.swift means Swift
target). So if you want to create multiple C targets, this layout should
work:

Package.swift
Sources/
    Bitmap
    Cubify
    Cairo/anchor.c <---- This is just an empty file to tell SwiftPM that
this is a C target.
    Cairo/include/Cairo.h
    Cairo/include/module.modulemap
    GLFW/anchor.c
    GLFW/include/GLFW.h
    GLFW/include/module.modulemap

The modulemap is automatically generated, if not provided. This is a
package which contains two targets (one C and one Swift):
GitHub - jpsim/Yams: A Sweet and Swifty YAML parser.

If you need to pass a bunch of compiler flags, you can use SwiftPM's
pkgConfig feature but that will require you to have a separate repository
for Cario and GLFW. You can experiment without creating tags using the edit
feature
<https://github.com/apple/swift-package-manager/blob/master/Documentation/Usage.md#editable-packages>
.

PS: You can join SwiftPM slack channel for quicker turn around time:
https://lists.swift.org/pipermail/swift-build-dev/Week
-of-Mon-20160530/000497.html

Thanks,
Ankit

On Wed, Mar 29, 2017 at 6:06 AM, Michael Ilseman via swift-build-dev < >>> swift-build-dev@swift.org> wrote:

This is into uncharted territory for me, but it seems you’re building
with SwiftPM. You’ll probably want to configure extra compiler flags if
that’s possible. You could also bite the bullet and build your C libraries
with SwiftPM as well. Hopefully someone on swift-build-dev can help you out.

CC-ing Ankit

On Mar 28, 2017, at 5:09 PM, Kelvin Ma <kelvinsthirteen@gmail.com> >>>> wrote:

How do I compile a project with many modules? My tree looks like this:

<Selection_001.png>

On Tue, Mar 28, 2017 at 12:47 PM, Michael Ilseman <milseman@apple.com> >>>> wrote:

Sure! In this example, I have built libgit2. I have a directory called
Git, and inside that I have the following module map:

module Git [system] {
       header "<my path>/libgit2/include/git2.h"
       export *
}

When I run, I use:

swift -I <path-to-“Git”-directory> -L <path-to-built-libgit2> -lgit2
foo.swift

inside foo.swift I can:

import Git
// … use libGit2

Read more about how to write a more appropriate module.map file for
your purposes at https://clang.llvm.org/docs/Modules.html\. For
example, you might be able to define link flags inside the module.map, use
umbrella directories, submodules, etc.

On Mar 28, 2017, at 6:27 AM, Kelvin Ma <kelvinsthirteen@gmail.com> >>>>> wrote:

Can you give an example?

On Mon, Mar 27, 2017 at 3:59 PM, Michael Ilseman <milseman@apple.com> >>>>> wrote:

Sure. At a low level, you can create a module.map file and use -L/-l
flags in your invocation of Swift. If you want to do so at a higher level,
then perhaps SwiftPM can. CCing swift-build-dev for the SwiftPM part.

> On Mar 26, 2017, at 3:20 PM, Kelvin Ma via swift-users < >>>>>> swift-users@swift.org> wrote:
>
> Idk if this has been asked before, but is there a way to import C
libraries into a Swift project without creating a local git repo?
Preferably something similar to C where you can just `#include` headers and
then specify the link flags (in Package.swift?)
>
> It’s getting very cumbersome to make a bunch of empty git repos
just to use libglfw or libcairo.
> _______________________________________________
> swift-users mailing list
> swift-users@swift.org
> https://lists.swift.org/mailman/listinfo/swift-users

_______________________________________________
swift-build-dev mailing list
swift-build-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-build-dev

_______________________________________________
swift-build-dev mailing list
swift-build-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-build-dev

Kelvin_Ma1 · March 29, 2017, 5:52am

I figured that was the intention, but we can’t be too surprised that everyone is maintaining personal modulemap repositories (and polluting search results — just try googling for a Swift PNG library!), especially when this central repo still doesn’t exist yet.

If Swift ever comes on par with C in terms of being usage and the lingua franca of the FOSS world, I can see linux distributions shipping standard modulemaps the same way they ship C headers, but you have to admit this is years (decades?) away at best.

On the flip side of it, it does wonders in terms of motivating people (me at least) to start writing pure Swift replacements for some of these C libraries (like libpng)…

···

On Mar 29, 2017, at 12:42 AM, Ankit Aggarwal <ankit_aggarwal@apple.com> wrote:

I think the idea was that there will be one such repository which other packages can use, that too only until the system libraries start shipping their standard modulemap. I thought we had this written down in our documentation somewhere but couldn't find it.

Maybe Daniel can expand on this.

On Wed, Mar 29, 2017 at 10:48 AM, Kelvin Ma via swift-build-dev <swift-build-dev@swift.org> wrote:
This worked! Thanks! But why is having empty git repositories strewn about the “correct” way? System libraries should be imported from within the project, as they are in C. You have to admit it’s getting quite silly that Swift devs keep repositories like these on our github accounts. That zlib repository contains exactly ten lines of code. I used to have 6 or 7 repos like that one up there before I got rid of them and switched to local repos.

On Wed, Mar 29, 2017 at 12:03 AM, Ankit Aggarwal <ankit_aggarwal@apple.com> wrote:
In this case, these are just umbrella headers. If your modulemap contains absolute path to the header, then you don't need the header files, but SwiftPM will probably warn about this. Note that this is a "hack" to have system packages inside a single repository. The correct way is to have system package as a separate published package which you only need to do once.

On 29-Mar-2017, at 10:26 AM, Kelvin Ma <kelvinsthirteen@gmail.com> wrote:

I will try this, but why are the header files inside the Sources directory? System headers should live in /usr/include…

On Tue, Mar 28, 2017 at 11:48 PM, Ankit Aggarwal <ankit_aggarwal@apple.com> wrote:
Hi,

Apologies for not replying to this earlier.

You can have multiple targets in a single package. Each target can either be Swift or C-family. The type of target is determined by the sources contained in it (*.c/*.cpp etc means C target, *.swift means Swift target). So if you want to create multiple C targets, this layout should work:

Package.swift
Sources/
    Bitmap
    Cubify
    Cairo/anchor.c <---- This is just an empty file to tell SwiftPM that this is a C target.
    Cairo/include/Cairo.h
    Cairo/include/module.modulemap
    GLFW/anchor.c
    GLFW/include/GLFW.h
    GLFW/include/module.modulemap

The modulemap is automatically generated, if not provided. This is a package which contains two targets (one C and one Swift): GitHub - jpsim/Yams: A Sweet and Swifty YAML parser.

If you need to pass a bunch of compiler flags, you can use SwiftPM's pkgConfig feature but that will require you to have a separate repository for Cario and GLFW. You can experiment without creating tags using the edit feature.

PS: You can join SwiftPM slack channel for quicker turn around time: [swift-build-dev] Swift Package Manager Slack Team

Thanks,
Ankit

On Wed, Mar 29, 2017 at 6:06 AM, Michael Ilseman via swift-build-dev <swift-build-dev@swift.org> wrote:
This is into uncharted territory for me, but it seems you’re building with SwiftPM. You’ll probably want to configure extra compiler flags if that’s possible. You could also bite the bullet and build your C libraries with SwiftPM as well. Hopefully someone on swift-build-dev can help you out.

CC-ing Ankit

On Mar 28, 2017, at 5:09 PM, Kelvin Ma <kelvinsthirteen@gmail.com> wrote:

How do I compile a project with many modules? My tree looks like this:

<Selection_001.png>

On Tue, Mar 28, 2017 at 12:47 PM, Michael Ilseman <milseman@apple.com> wrote:
Sure! In this example, I have built libgit2. I have a directory called Git, and inside that I have the following module map:

module Git [system] {
       header "<my path>/libgit2/include/git2.h"
       export *
}

When I run, I use:

swift -I <path-to-“Git”-directory> -L <path-to-built-libgit2> -lgit2 foo.swift

inside foo.swift I can:

import Git
// … use libGit2

Read more about how to write a more appropriate module.map file for your purposes at https://clang.llvm.org/docs/Modules.html\. For example, you might be able to define link flags inside the module.map, use umbrella directories, submodules, etc.

On Mar 28, 2017, at 6:27 AM, Kelvin Ma <kelvinsthirteen@gmail.com> wrote:

Can you give an example?

On Mon, Mar 27, 2017 at 3:59 PM, Michael Ilseman <milseman@apple.com> wrote:
Sure. At a low level, you can create a module.map file and use -L/-l flags in your invocation of Swift. If you want to do so at a higher level, then perhaps SwiftPM can. CCing swift-build-dev for the SwiftPM part.

> On Mar 26, 2017, at 3:20 PM, Kelvin Ma via swift-users <swift-users@swift.org> wrote:
>
> Idk if this has been asked before, but is there a way to import C libraries into a Swift project without creating a local git repo? Preferably something similar to C where you can just `#include` headers and then specify the link flags (in Package.swift?)
>
> It’s getting very cumbersome to make a bunch of empty git repos just to use libglfw or libcairo.
> _______________________________________________
> swift-users mailing list
> swift-users@swift.org
> https://lists.swift.org/mailman/listinfo/swift-users

_______________________________________________
swift-build-dev mailing list
swift-build-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-build-dev

_______________________________________________
swift-build-dev mailing list
swift-build-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-build-dev

Aciid · March 29, 2017, 6:03am

I figured that was the intention, but we can’t be too surprised that everyone is maintaining personal modulemap repositories (and polluting search results — just try googling for a Swift PNG library!), especially when this central repo still doesn’t exist yet.

Yeah thats unfortunate, maybe this will improve once we have an index.

If Swift ever comes on par with C in terms of being usage and the lingua franca of the FOSS world, I can see linux distributions shipping standard modulemaps the same way they ship C headers, but you have to admit this is years (decades?) away at best.

On the flip side of it, it does wonders in terms of motivating people (me at least) to start writing pure Swift replacements for some of these C libraries (like libpng)…

I think its better to reuse existing libraries than writing from scratch (unless really needed). A good approach that works is "porting" these libraries to build with SwiftPM, see: libYAML <https://github.com/jpsim/Yams>, libressl <https://github.com/vapor/clibressl>\. However, this porting can be difficult to do right but it should become much easier once we have build settings support and custom targets layout <https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20170320/034469.html>\.

···

On 29-Mar-2017, at 11:22 AM, kelvinsthirteen@gmail.com wrote:

On Mar 29, 2017, at 12:42 AM, Ankit Aggarwal <ankit_aggarwal@apple.com <mailto:ankit_aggarwal@apple.com>> wrote:

I think the idea was that there will be one such repository which other packages can use, that too only until the system libraries start shipping their standard modulemap. I thought we had this written down in our documentation somewhere but couldn't find it.

Maybe Daniel can expand on this.

On Wed, Mar 29, 2017 at 10:48 AM, Kelvin Ma via swift-build-dev <swift-build-dev@swift.org <mailto:swift-build-dev@swift.org>> wrote:
This worked! Thanks! But why is having empty git repositories strewn about the “correct” way? System libraries should be imported from within the project, as they are in C. You have to admit it’s getting quite silly that Swift devs keep repositories like these <https://github.com/kelvin13/swift-zlib> on our github accounts. That zlib repository contains exactly ten lines of code. I used to have 6 or 7 repos like that one up there before I got rid of them and switched to local repos.

On Wed, Mar 29, 2017 at 12:03 AM, Ankit Aggarwal <ankit_aggarwal@apple.com <mailto:ankit_aggarwal@apple.com>> wrote:
In this case, these are just umbrella headers. If your modulemap contains absolute path to the header, then you don't need the header files, but SwiftPM will probably warn about this. Note that this is a "hack" to have system packages inside a single repository. The correct way is to have system package as a separate published package which you only need to do once.

On 29-Mar-2017, at 10:26 AM, Kelvin Ma <kelvinsthirteen@gmail.com <mailto:kelvinsthirteen@gmail.com>> wrote:

I will try this, but why are the header files inside the Sources directory? System headers should live in /usr/include…

On Tue, Mar 28, 2017 at 11:48 PM, Ankit Aggarwal <ankit_aggarwal@apple.com <mailto:ankit_aggarwal@apple.com>> wrote:
Hi,

Apologies for not replying to this earlier.

You can have multiple targets in a single package. Each target can either be Swift or C-family. The type of target is determined by the sources contained in it (*.c/*.cpp etc means C target, *.swift means Swift target). So if you want to create multiple C targets, this layout should work:

Package.swift
Sources/
    Bitmap
    Cubify
    Cairo/anchor.c <---- This is just an empty file to tell SwiftPM that this is a C target.
    Cairo/include/Cairo.h
    Cairo/include/module.modulemap
    GLFW/anchor.c
    GLFW/include/GLFW.h
    GLFW/include/module.modulemap

The modulemap is automatically generated, if not provided. This is a package which contains two targets (one C and one Swift): GitHub - jpsim/Yams: A Sweet and Swifty YAML parser.

If you need to pass a bunch of compiler flags, you can use SwiftPM's pkgConfig feature but that will require you to have a separate repository for Cario and GLFW. You can experiment without creating tags using the edit feature <https://github.com/apple/swift-package-manager/blob/master/Documentation/Usage.md#editable-packages>\.

PS: You can join SwiftPM slack channel for quicker turn around time: [swift-build-dev] Swift Package Manager Slack Team

Thanks,
Ankit

On Wed, Mar 29, 2017 at 6:06 AM, Michael Ilseman via swift-build-dev <swift-build-dev@swift.org <mailto:swift-build-dev@swift.org>> wrote:
This is into uncharted territory for me, but it seems you’re building with SwiftPM. You’ll probably want to configure extra compiler flags if that’s possible. You could also bite the bullet and build your C libraries with SwiftPM as well. Hopefully someone on swift-build-dev can help you out.

CC-ing Ankit

On Mar 28, 2017, at 5:09 PM, Kelvin Ma <kelvinsthirteen@gmail.com <mailto:kelvinsthirteen@gmail.com>> wrote:

How do I compile a project with many modules? My tree looks like this:

<Selection_001.png>

On Tue, Mar 28, 2017 at 12:47 PM, Michael Ilseman <milseman@apple.com <mailto:milseman@apple.com>> wrote:
Sure! In this example, I have built libgit2. I have a directory called Git, and inside that I have the following module map:

module Git [system] {
       header "<my path>/libgit2/include/git2.h"
       export *
}

When I run, I use:

swift -I <path-to-“Git”-directory> -L <path-to-built-libgit2> -lgit2 foo.swift

inside foo.swift I can:

import Git
// … use libGit2

Read more about how to write a more appropriate module.map file for your purposes at https://clang.llvm.org/docs/Modules.html\. For example, you might be able to define link flags inside the module.map, use umbrella directories, submodules, etc.

On Mar 28, 2017, at 6:27 AM, Kelvin Ma <kelvinsthirteen@gmail.com <mailto:kelvinsthirteen@gmail.com>> wrote:

Can you give an example?

On Mon, Mar 27, 2017 at 3:59 PM, Michael Ilseman <milseman@apple.com <mailto:milseman@apple.com>> wrote:
Sure. At a low level, you can create a module.map file and use -L/-l flags in your invocation of Swift. If you want to do so at a higher level, then perhaps SwiftPM can. CCing swift-build-dev for the SwiftPM part.

> On Mar 26, 2017, at 3:20 PM, Kelvin Ma via swift-users <swift-users@swift.org <mailto:swift-users@swift.org>> wrote:
>
> Idk if this has been asked before, but is there a way to import C libraries into a Swift project without creating a local git repo? Preferably something similar to C where you can just `#include` headers and then specify the link flags (in Package.swift?)
>
> It’s getting very cumbersome to make a bunch of empty git repos just to use libglfw or libcairo.
> _______________________________________________
> swift-users mailing list
> swift-users@swift.org <mailto:swift-users@swift.org>
> https://lists.swift.org/mailman/listinfo/swift-users

_______________________________________________
swift-build-dev mailing list
swift-build-dev@swift.org <mailto:swift-build-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-build-dev

_______________________________________________
swift-build-dev mailing list
swift-build-dev@swift.org <mailto:swift-build-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-build-dev

Kelvin_Ma1 · March 29, 2017, 5:25pm

I don’t think this is a good approach, the libressl
<https://github.com/vapor/clibressl> repo is pretty much just the source
code of the C library copied and pasted into a Swift module. That’s not a
good thing™. The linux build paradigm is, the library maintains its own
*official* source repository, the OS package distributors build it, users
install the built dependencies with `sudo apt-get install libwhatever-dev`
and the project source just builds and links to the library managed by the
system.

Here you have to keep the forked library source up to date with the real
library source, download it from the internet and build the entire project
plus the libraries of which there could be many.

IMO the ideal way to import system libs would be for the user to install
the dependency with `apt`, just as you would for any C program, have the
`include` statements within the project source (like you would for any C
program), and then have the paths to `/usr/include` and `/usr/lib` and
`/usr/local/include` etc in a Makefile (or Package.swift). Usually it’s
only “specialty” libraries like libspiro that people download and build
manually, and even then, it’s downloaded from the Spiro project’s own
official repository, not a third party fork. That’s the “accepted” way to
do things, that linux ecosystems are designed around. Of course, this is
very similar to the modulemap system that currently works in Swift. I just
wish modulemaps could be streamlined a little, maybe combined with the top
level Package.swift.

···

On Wed, Mar 29, 2017 at 1:03 AM, Ankit Aggarwal <ankit_aggarwal@apple.com> wrote:

On 29-Mar-2017, at 11:22 AM, kelvinsthirteen@gmail.com wrote:

I figured that was the intention, but we can’t be too surprised that
everyone is maintaining personal modulemap repositories (and polluting
search results — just try googling for a Swift PNG library!), especially
when this central repo still doesn’t exist yet.

Yeah thats unfortunate, maybe this will improve once we have an index.

If Swift ever comes on par with C in terms of being usage and the lingua
franca of the FOSS world, I can see linux distributions shipping standard
modulemaps the same way they ship C headers, but you have to admit this is
years (decades?) away at best.

On the flip side of it, it does wonders in terms of motivating people (me
at least) to start writing pure Swift replacements for some of these C
libraries (like libpng)…

I think its better to reuse existing libraries than writing from scratch
(unless really needed). A good approach that works is "porting" these
libraries to build with SwiftPM, see: libYAML
<https://github.com/jpsim/Yams>, libressl
<https://github.com/vapor/clibressl>\. However, this porting can be
difficult to do right but it should become much easier once we have build
settings support and custom targets layout
<https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20170320/034469.html>
.

On Mar 29, 2017, at 12:42 AM, Ankit Aggarwal <ankit_aggarwal@apple.com> > wrote:

I think the idea was that there will be one such repository which other
packages can use, that too only until the system libraries start shipping
their standard modulemap. I thought we had this written down in our
documentation somewhere but couldn't find it.

Maybe Daniel can expand on this.

On Wed, Mar 29, 2017 at 10:48 AM, Kelvin Ma via swift-build-dev < > swift-build-dev@swift.org> wrote:

This worked! Thanks! But why is having empty git repositories strewn
about the “correct” way? System libraries should be imported from within
the project, as they are in C. You have to admit it’s getting quite silly
that Swift devs keep repositories like these
<https://github.com/kelvin13/swift-zlib> on our github accounts. That
zlib repository contains exactly ten lines of code. I used to have 6 or 7
repos like that one up there before I got rid of them and switched to local
repos.

On Wed, Mar 29, 2017 at 12:03 AM, Ankit Aggarwal < >> ankit_aggarwal@apple.com> wrote:

In this case, these are just umbrella headers. If your modulemap
contains absolute path to the header, then you don't need the header files,
but SwiftPM will probably warn about this. Note that this is a "hack" to
have system packages inside a single repository. The correct way is to have
system package as a separate published package which you only need to do
once.

On 29-Mar-2017, at 10:26 AM, Kelvin Ma <kelvinsthirteen@gmail.com> >>> wrote:

I will try this, but why are the header files inside the Sources
directory? System headers should live in /usr/include…

On Tue, Mar 28, 2017 at 11:48 PM, Ankit Aggarwal < >>> ankit_aggarwal@apple.com> wrote:

Hi,

Apologies for not replying to this earlier.

You can have multiple targets in a single package. Each target can
either be Swift or C-family. The type of target is determined by the
sources contained in it (*.c/*.cpp etc means C target, *.swift means Swift
target). So if you want to create multiple C targets, this layout should
work:

Package.swift
Sources/
    Bitmap
    Cubify
    Cairo/anchor.c <---- This is just an empty file to tell SwiftPM
that this is a C target.
    Cairo/include/Cairo.h
    Cairo/include/module.modulemap
    GLFW/anchor.c
    GLFW/include/GLFW.h
    GLFW/include/module.modulemap

The modulemap is automatically generated, if not provided. This is a
package which contains two targets (one C and one Swift):
GitHub - jpsim/Yams: A Sweet and Swifty YAML parser.

If you need to pass a bunch of compiler flags, you can use SwiftPM's
pkgConfig feature but that will require you to have a separate repository
for Cario and GLFW. You can experiment without creating tags using the edit
feature
<https://github.com/apple/swift-package-manager/blob/master/Documentation/Usage.md#editable-packages>
.

PS: You can join SwiftPM slack channel for quicker turn around time:
https://lists.swift.org/pipermail/swift-build-dev/Week
-of-Mon-20160530/000497.html

Thanks,
Ankit

On Wed, Mar 29, 2017 at 6:06 AM, Michael Ilseman via swift-build-dev < >>>> swift-build-dev@swift.org> wrote:

This is into uncharted territory for me, but it seems you’re building
with SwiftPM. You’ll probably want to configure extra compiler flags if
that’s possible. You could also bite the bullet and build your C libraries
with SwiftPM as well. Hopefully someone on swift-build-dev can help you out.

CC-ing Ankit

On Mar 28, 2017, at 5:09 PM, Kelvin Ma <kelvinsthirteen@gmail.com> >>>>> wrote:

How do I compile a project with many modules? My tree looks like this:

<Selection_001.png>

On Tue, Mar 28, 2017 at 12:47 PM, Michael Ilseman <milseman@apple.com> >>>>> wrote:

Sure! In this example, I have built libgit2. I have a directory
called Git, and inside that I have the following module map:

module Git [system] {
       header "<my path>/libgit2/include/git2.h"
       export *
}

When I run, I use:

swift -I <path-to-“Git”-directory> -L <path-to-built-libgit2> -lgit2
foo.swift

inside foo.swift I can:

import Git
// … use libGit2

Read more about how to write a more appropriate module.map file for
your purposes at https://clang.llvm.org/docs/Modules.html\. For
example, you might be able to define link flags inside the module.map, use
umbrella directories, submodules, etc.

On Mar 28, 2017, at 6:27 AM, Kelvin Ma <kelvinsthirteen@gmail.com> >>>>>> wrote:

Can you give an example?

On Mon, Mar 27, 2017 at 3:59 PM, Michael Ilseman <milseman@apple.com> >>>>>> wrote:

Sure. At a low level, you can create a module.map file and use -L/-l
flags in your invocation of Swift. If you want to do so at a higher level,
then perhaps SwiftPM can. CCing swift-build-dev for the SwiftPM part.

> On Mar 26, 2017, at 3:20 PM, Kelvin Ma via swift-users < >>>>>>> swift-users@swift.org> wrote:
>
> Idk if this has been asked before, but is there a way to import C
libraries into a Swift project without creating a local git repo?
Preferably something similar to C where you can just `#include` headers and
then specify the link flags (in Package.swift?)
>
> It’s getting very cumbersome to make a bunch of empty git repos
just to use libglfw or libcairo.
> _______________________________________________
> swift-users mailing list
> swift-users@swift.org
> https://lists.swift.org/mailman/listinfo/swift-users

_______________________________________________
swift-build-dev mailing list
swift-build-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-build-dev

_______________________________________________
swift-build-dev mailing list
swift-build-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-build-dev

Aciid · March 29, 2017, 5:42pm

I agree that libressl isn't a good example because of the security questions it raises, but libYAML is a good self contained package IMO. With custom targets layout proposal, the forks can be replaced by submodules. The problem with apt-get approach is, it takes away the "portability" from the package and it doesn't work in cases when you don't have full access to the system. That said, we do support this approach upto a certain degree. For e.g., swiftpm allows system package authors to declare hints for installing a system package dependency (via brew or apt-get). The /usr/include, /usr/lib etc are searched by default and non-standard paths are supported using pkg config. See: https://github.com/apple/swift-package-manager/blob/master/Documentation/Usage.md#require-system-libraries

···

On 29-Mar-2017, at 10:55 PM, Kelvin Ma <kelvinsthirteen@gmail.com> wrote:

I don’t think this is a good approach, the libressl <https://github.com/vapor/clibressl> repo is pretty much just the source code of the C library copied and pasted into a Swift module. That’s not a good thing™. The linux build paradigm is, the library maintains its own *official* source repository, the OS package distributors build it, users install the built dependencies with `sudo apt-get install libwhatever-dev` and the project source just builds and links to the library managed by the system.

Here you have to keep the forked library source up to date with the real library source, download it from the internet and build the entire project plus the libraries of which there could be many.

IMO the ideal way to import system libs would be for the user to install the dependency with `apt`, just as you would for any C program, have the `include` statements within the project source (like you would for any C program), and then have the paths to `/usr/include` and `/usr/lib` and `/usr/local/include` etc in a Makefile (or Package.swift). Usually it’s only “specialty” libraries like libspiro that people download and build manually, and even then, it’s downloaded from the Spiro project’s own official repository, not a third party fork. That’s the “accepted” way to do things, that linux ecosystems are designed around. Of course, this is very similar to the modulemap system that currently works in Swift. I just wish modulemaps could be streamlined a little, maybe combined with the top level Package.swift.

On Wed, Mar 29, 2017 at 1:03 AM, Ankit Aggarwal <ankit_aggarwal@apple.com <mailto:ankit_aggarwal@apple.com>> wrote:

On 29-Mar-2017, at 11:22 AM, kelvinsthirteen@gmail.com <mailto:kelvinsthirteen@gmail.com> wrote:

I figured that was the intention, but we can’t be too surprised that everyone is maintaining personal modulemap repositories (and polluting search results — just try googling for a Swift PNG library!), especially when this central repo still doesn’t exist yet.

Yeah thats unfortunate, maybe this will improve once we have an index.

If Swift ever comes on par with C in terms of being usage and the lingua franca of the FOSS world, I can see linux distributions shipping standard modulemaps the same way they ship C headers, but you have to admit this is years (decades?) away at best.

On the flip side of it, it does wonders in terms of motivating people (me at least) to start writing pure Swift replacements for some of these C libraries (like libpng)…

I think its better to reuse existing libraries than writing from scratch (unless really needed). A good approach that works is "porting" these libraries to build with SwiftPM, see: libYAML <https://github.com/jpsim/Yams>, libressl <https://github.com/vapor/clibressl>\. However, this porting can be difficult to do right but it should become much easier once we have build settings support and custom targets layout <https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20170320/034469.html>\.

On Mar 29, 2017, at 12:42 AM, Ankit Aggarwal <ankit_aggarwal@apple.com <mailto:ankit_aggarwal@apple.com>> wrote:

I think the idea was that there will be one such repository which other packages can use, that too only until the system libraries start shipping their standard modulemap. I thought we had this written down in our documentation somewhere but couldn't find it.

Maybe Daniel can expand on this.

On Wed, Mar 29, 2017 at 10:48 AM, Kelvin Ma via swift-build-dev <swift-build-dev@swift.org <mailto:swift-build-dev@swift.org>> wrote:
This worked! Thanks! But why is having empty git repositories strewn about the “correct” way? System libraries should be imported from within the project, as they are in C. You have to admit it’s getting quite silly that Swift devs keep repositories like these <https://github.com/kelvin13/swift-zlib> on our github accounts. That zlib repository contains exactly ten lines of code. I used to have 6 or 7 repos like that one up there before I got rid of them and switched to local repos.

On Wed, Mar 29, 2017 at 12:03 AM, Ankit Aggarwal <ankit_aggarwal@apple.com <mailto:ankit_aggarwal@apple.com>> wrote:
In this case, these are just umbrella headers. If your modulemap contains absolute path to the header, then you don't need the header files, but SwiftPM will probably warn about this. Note that this is a "hack" to have system packages inside a single repository. The correct way is to have system package as a separate published package which you only need to do once.

On 29-Mar-2017, at 10:26 AM, Kelvin Ma <kelvinsthirteen@gmail.com <mailto:kelvinsthirteen@gmail.com>> wrote:

I will try this, but why are the header files inside the Sources directory? System headers should live in /usr/include…

On Tue, Mar 28, 2017 at 11:48 PM, Ankit Aggarwal <ankit_aggarwal@apple.com <mailto:ankit_aggarwal@apple.com>> wrote:
Hi,

Apologies for not replying to this earlier.

You can have multiple targets in a single package. Each target can either be Swift or C-family. The type of target is determined by the sources contained in it (*.c/*.cpp etc means C target, *.swift means Swift target). So if you want to create multiple C targets, this layout should work:

Package.swift
Sources/
    Bitmap
    Cubify
    Cairo/anchor.c <---- This is just an empty file to tell SwiftPM that this is a C target.
    Cairo/include/Cairo.h
    Cairo/include/module.modulemap
    GLFW/anchor.c
    GLFW/include/GLFW.h
    GLFW/include/module.modulemap

The modulemap is automatically generated, if not provided. This is a package which contains two targets (one C and one Swift): GitHub - jpsim/Yams: A Sweet and Swifty YAML parser.

If you need to pass a bunch of compiler flags, you can use SwiftPM's pkgConfig feature but that will require you to have a separate repository for Cario and GLFW. You can experiment without creating tags using the edit feature <https://github.com/apple/swift-package-manager/blob/master/Documentation/Usage.md#editable-packages>\.

PS: You can join SwiftPM slack channel for quicker turn around time: [swift-build-dev] Swift Package Manager Slack Team

Thanks,
Ankit

On Wed, Mar 29, 2017 at 6:06 AM, Michael Ilseman via swift-build-dev <swift-build-dev@swift.org <mailto:swift-build-dev@swift.org>> wrote:
This is into uncharted territory for me, but it seems you’re building with SwiftPM. You’ll probably want to configure extra compiler flags if that’s possible. You could also bite the bullet and build your C libraries with SwiftPM as well. Hopefully someone on swift-build-dev can help you out.

CC-ing Ankit

On Mar 28, 2017, at 5:09 PM, Kelvin Ma <kelvinsthirteen@gmail.com <mailto:kelvinsthirteen@gmail.com>> wrote:

How do I compile a project with many modules? My tree looks like this:

<Selection_001.png>

On Tue, Mar 28, 2017 at 12:47 PM, Michael Ilseman <milseman@apple.com <mailto:milseman@apple.com>> wrote:
Sure! In this example, I have built libgit2. I have a directory called Git, and inside that I have the following module map:

module Git [system] {
       header "<my path>/libgit2/include/git2.h"
       export *
}

When I run, I use:

swift -I <path-to-“Git”-directory> -L <path-to-built-libgit2> -lgit2 foo.swift

inside foo.swift I can:

import Git
// … use libGit2

Read more about how to write a more appropriate module.map file for your purposes at https://clang.llvm.org/docs/Modules.html\. For example, you might be able to define link flags inside the module.map, use umbrella directories, submodules, etc.

On Mar 28, 2017, at 6:27 AM, Kelvin Ma <kelvinsthirteen@gmail.com <mailto:kelvinsthirteen@gmail.com>> wrote:

Can you give an example?

On Mon, Mar 27, 2017 at 3:59 PM, Michael Ilseman <milseman@apple.com <mailto:milseman@apple.com>> wrote:
Sure. At a low level, you can create a module.map file and use -L/-l flags in your invocation of Swift. If you want to do so at a higher level, then perhaps SwiftPM can. CCing swift-build-dev for the SwiftPM part.

> On Mar 26, 2017, at 3:20 PM, Kelvin Ma via swift-users <swift-users@swift.org <mailto:swift-users@swift.org>> wrote:
>
> Idk if this has been asked before, but is there a way to import C libraries into a Swift project without creating a local git repo? Preferably something similar to C where you can just `#include` headers and then specify the link flags (in Package.swift?)
>
> It’s getting very cumbersome to make a bunch of empty git repos just to use libglfw or libcairo.
> _______________________________________________
> swift-users mailing list
> swift-users@swift.org <mailto:swift-users@swift.org>
> https://lists.swift.org/mailman/listinfo/swift-users

_______________________________________________
swift-build-dev mailing list
swift-build-dev@swift.org <mailto:swift-build-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-build-dev

_______________________________________________
swift-build-dev mailing list
swift-build-dev@swift.org <mailto:swift-build-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-build-dev

Kelvin_Ma1 · March 29, 2017, 6:55pm

I agree that portability is valuable and that it’s something that’s lacking
in the c/makefile workflow. I just don’t think empty git repositories are
the right solution. Perhaps we can get the best of both worlds with
something like this in the SwiftPM:

.Package(include: "cairo.h", link: "cairo" version: "1.1.4", remote: "
git://anongit.freedesktop.org/git/cairo")

It would search for the cairo lib on local machine in /usr/lib and
/usr/local/lib , and if it can’t find it, or if the version doesn’t match,
it will then download the C sources from the Cairo project’s *official* git
repository and build it for you. Options could be added to specify
alternative search paths or user prompts.

···

On Wed, Mar 29, 2017 at 12:42 PM, Ankit Aggarwal <ankit_aggarwal@apple.com> wrote:

I agree that libressl isn't a good example because of the security
questions it raises, but libYAML is a good self contained package IMO. With
custom targets layout proposal, the forks can be replaced by submodules.
The problem with apt-get approach is, it takes away the "portability" from
the package and it doesn't work in cases when you don't have full access to
the system. That said, we do support this approach upto a certain degree.
For e.g., swiftpm allows system package authors to declare hints for
installing a system package dependency (via brew or apt-get). The
/usr/include, /usr/lib etc are searched by default and non-standard paths
are supported using pkg config. See: Apple · GitHub
swift-package-manager/blob/master/Documentation/Usage.md#
require-system-libraries

On 29-Mar-2017, at 10:55 PM, Kelvin Ma <kelvinsthirteen@gmail.com> wrote:

I don’t think this is a good approach, the libressl
<https://github.com/vapor/clibressl> repo is pretty much just the source
code of the C library copied and pasted into a Swift module. That’s not a
good thing™. The linux build paradigm is, the library maintains its own
*official* source repository, the OS package distributors build it, users
install the built dependencies with `sudo apt-get install libwhatever-dev`
and the project source just builds and links to the library managed by the
system.

Here you have to keep the forked library source up to date with the real
library source, download it from the internet and build the entire project
plus the libraries of which there could be many.

IMO the ideal way to import system libs would be for the user to install
the dependency with `apt`, just as you would for any C program, have the
`include` statements within the project source (like you would for any C
program), and then have the paths to `/usr/include` and `/usr/lib` and
`/usr/local/include` etc in a Makefile (or Package.swift). Usually it’s
only “specialty” libraries like libspiro that people download and build
manually, and even then, it’s downloaded from the Spiro project’s own
official repository, not a third party fork. That’s the “accepted” way to
do things, that linux ecosystems are designed around. Of course, this is
very similar to the modulemap system that currently works in Swift. I just
wish modulemaps could be streamlined a little, maybe combined with the top
level Package.swift.

On Wed, Mar 29, 2017 at 1:03 AM, Ankit Aggarwal <ankit_aggarwal@apple.com> > wrote:

On 29-Mar-2017, at 11:22 AM, kelvinsthirteen@gmail.com wrote:

I figured that was the intention, but we can’t be too surprised that
everyone is maintaining personal modulemap repositories (and polluting
search results — just try googling for a Swift PNG library!), especially
when this central repo still doesn’t exist yet.

Yeah thats unfortunate, maybe this will improve once we have an index.

If Swift ever comes on par with C in terms of being usage and the lingua
franca of the FOSS world, I can see linux distributions shipping standard
modulemaps the same way they ship C headers, but you have to admit this is
years (decades?) away at best.

On the flip side of it, it does wonders in terms of motivating people (me
at least) to start writing pure Swift replacements for some of these C
libraries (like libpng)…

I think its better to reuse existing libraries than writing from scratch
(unless really needed). A good approach that works is "porting" these
libraries to build with SwiftPM, see: libYAML
<https://github.com/jpsim/Yams>, libressl
<https://github.com/vapor/clibressl>\. However, this porting can be
difficult to do right but it should become much easier once we have build
settings support and custom targets layout
<https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20170320/034469.html>
.

On Mar 29, 2017, at 12:42 AM, Ankit Aggarwal <ankit_aggarwal@apple.com> >> wrote:

I think the idea was that there will be one such repository which other
packages can use, that too only until the system libraries start shipping
their standard modulemap. I thought we had this written down in our
documentation somewhere but couldn't find it.

Maybe Daniel can expand on this.

On Wed, Mar 29, 2017 at 10:48 AM, Kelvin Ma via swift-build-dev < >> swift-build-dev@swift.org> wrote:

This worked! Thanks! But why is having empty git repositories strewn
about the “correct” way? System libraries should be imported from within
the project, as they are in C. You have to admit it’s getting quite silly
that Swift devs keep repositories like these
<https://github.com/kelvin13/swift-zlib> on our github accounts. That
zlib repository contains exactly ten lines of code. I used to have 6 or 7
repos like that one up there before I got rid of them and switched to local
repos.

On Wed, Mar 29, 2017 at 12:03 AM, Ankit Aggarwal < >>> ankit_aggarwal@apple.com> wrote:

In this case, these are just umbrella headers. If your modulemap
contains absolute path to the header, then you don't need the header files,
but SwiftPM will probably warn about this. Note that this is a "hack" to
have system packages inside a single repository. The correct way is to have
system package as a separate published package which you only need to do
once.

On 29-Mar-2017, at 10:26 AM, Kelvin Ma <kelvinsthirteen@gmail.com> >>>> wrote:

I will try this, but why are the header files inside the Sources
directory? System headers should live in /usr/include…

On Tue, Mar 28, 2017 at 11:48 PM, Ankit Aggarwal < >>>> ankit_aggarwal@apple.com> wrote:

Hi,

Apologies for not replying to this earlier.

You can have multiple targets in a single package. Each target can
either be Swift or C-family. The type of target is determined by the
sources contained in it (*.c/*.cpp etc means C target, *.swift means Swift
target). So if you want to create multiple C targets, this layout should
work:

Package.swift
Sources/
    Bitmap
    Cubify
    Cairo/anchor.c <---- This is just an empty file to tell SwiftPM
that this is a C target.
    Cairo/include/Cairo.h
    Cairo/include/module.modulemap
    GLFW/anchor.c
    GLFW/include/GLFW.h
    GLFW/include/module.modulemap

The modulemap is automatically generated, if not provided. This is a
package which contains two targets (one C and one Swift):
GitHub - jpsim/Yams: A Sweet and Swifty YAML parser.

If you need to pass a bunch of compiler flags, you can use SwiftPM's
pkgConfig feature but that will require you to have a separate repository
for Cario and GLFW. You can experiment without creating tags using the edit
feature
<https://github.com/apple/swift-package-manager/blob/master/Documentation/Usage.md#editable-packages>
.

PS: You can join SwiftPM slack channel for quicker turn around time:
https://lists.swift.org/pipermail/swift-build-dev/Week
-of-Mon-20160530/000497.html

Thanks,
Ankit

On Wed, Mar 29, 2017 at 6:06 AM, Michael Ilseman via swift-build-dev < >>>>> swift-build-dev@swift.org> wrote:

This is into uncharted territory for me, but it seems you’re building
with SwiftPM. You’ll probably want to configure extra compiler flags if
that’s possible. You could also bite the bullet and build your C libraries
with SwiftPM as well. Hopefully someone on swift-build-dev can help you out.

CC-ing Ankit

On Mar 28, 2017, at 5:09 PM, Kelvin Ma <kelvinsthirteen@gmail.com> >>>>>> wrote:

How do I compile a project with many modules? My tree looks like this:

<Selection_001.png>

On Tue, Mar 28, 2017 at 12:47 PM, Michael Ilseman <milseman@apple.com >>>>>> > wrote:

Sure! In this example, I have built libgit2. I have a directory
called Git, and inside that I have the following module map:

module Git [system] {
       header "<my path>/libgit2/include/git2.h"
       export *
}

When I run, I use:

swift -I <path-to-“Git”-directory> -L <path-to-built-libgit2> -lgit2
foo.swift

inside foo.swift I can:

import Git
// … use libGit2

Read more about how to write a more appropriate module.map file for
your purposes at https://clang.llvm.org/docs/Modules.html\. For
example, you might be able to define link flags inside the module.map, use
umbrella directories, submodules, etc.

On Mar 28, 2017, at 6:27 AM, Kelvin Ma <kelvinsthirteen@gmail.com> >>>>>>> wrote:

Can you give an example?

On Mon, Mar 27, 2017 at 3:59 PM, Michael Ilseman <milseman@apple.com >>>>>>> > wrote:

Sure. At a low level, you can create a module.map file and use
-L/-l flags in your invocation of Swift. If you want to do so at a higher
level, then perhaps SwiftPM can. CCing swift-build-dev for the SwiftPM part.

> On Mar 26, 2017, at 3:20 PM, Kelvin Ma via swift-users < >>>>>>>> swift-users@swift.org> wrote:
>
> Idk if this has been asked before, but is there a way to import C
libraries into a Swift project without creating a local git repo?
Preferably something similar to C where you can just `#include` headers and
then specify the link flags (in Package.swift?)
>
> It’s getting very cumbersome to make a bunch of empty git repos
just to use libglfw or libcairo.
> _______________________________________________
> swift-users mailing list
> swift-users@swift.org
> https://lists.swift.org/mailman/listinfo/swift-users

_______________________________________________
swift-build-dev mailing list
swift-build-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-build-dev

_______________________________________________
swift-build-dev mailing list
swift-build-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-build-dev

Aciid · March 29, 2017, 7:07pm

Yep, we need to have better support for system packages. Unfortunately it is not covered in the Swift 4 Package Manager roadmap <https://lists.swift.org/pipermail/swift-evolution-announce/2017-January/000307.html>, but community driven proposal are always welcomed!

···

On 30-Mar-2017, at 12:25 AM, Kelvin Ma <kelvinsthirteen@gmail.com> wrote:

I agree that portability is valuable and that it’s something that’s lacking in the c/makefile workflow. I just don’t think empty git repositories are the right solution. Perhaps we can get the best of both worlds with something like this in the SwiftPM:

    .Package(include: "cairo.h", link: "cairo" version: "1.1.4", remote: "git://anongit.freedesktop.org/git/cairo <http://anongit.freedesktop.org/git/cairo>"\)

It would search for the cairo lib on local machine in /usr/lib and /usr/local/lib , and if it can’t find it, or if the version doesn’t match, it will then download the C sources from the Cairo project’s *official* git repository and build it for you. Options could be added to specify alternative search paths or user prompts.

On Wed, Mar 29, 2017 at 12:42 PM, Ankit Aggarwal <ankit_aggarwal@apple.com <mailto:ankit_aggarwal@apple.com>> wrote:
I agree that libressl isn't a good example because of the security questions it raises, but libYAML is a good self contained package IMO. With custom targets layout proposal, the forks can be replaced by submodules. The problem with apt-get approach is, it takes away the "portability" from the package and it doesn't work in cases when you don't have full access to the system. That said, we do support this approach upto a certain degree. For e.g., swiftpm allows system package authors to declare hints for installing a system package dependency (via brew or apt-get). The /usr/include, /usr/lib etc are searched by default and non-standard paths are supported using pkg config. See: https://github.com/apple/swift-package-manager/blob/master/Documentation/Usage.md#require-system-libraries

On 29-Mar-2017, at 10:55 PM, Kelvin Ma <kelvinsthirteen@gmail.com <mailto:kelvinsthirteen@gmail.com>> wrote:

I don’t think this is a good approach, the libressl <https://github.com/vapor/clibressl> repo is pretty much just the source code of the C library copied and pasted into a Swift module. That’s not a good thing™. The linux build paradigm is, the library maintains its own *official* source repository, the OS package distributors build it, users install the built dependencies with `sudo apt-get install libwhatever-dev` and the project source just builds and links to the library managed by the system.

Here you have to keep the forked library source up to date with the real library source, download it from the internet and build the entire project plus the libraries of which there could be many.

IMO the ideal way to import system libs would be for the user to install the dependency with `apt`, just as you would for any C program, have the `include` statements within the project source (like you would for any C program), and then have the paths to `/usr/include` and `/usr/lib` and `/usr/local/include` etc in a Makefile (or Package.swift). Usually it’s only “specialty” libraries like libspiro that people download and build manually, and even then, it’s downloaded from the Spiro project’s own official repository, not a third party fork. That’s the “accepted” way to do things, that linux ecosystems are designed around. Of course, this is very similar to the modulemap system that currently works in Swift. I just wish modulemaps could be streamlined a little, maybe combined with the top level Package.swift.

On Wed, Mar 29, 2017 at 1:03 AM, Ankit Aggarwal <ankit_aggarwal@apple.com <mailto:ankit_aggarwal@apple.com>> wrote:

On 29-Mar-2017, at 11:22 AM, kelvinsthirteen@gmail.com <mailto:kelvinsthirteen@gmail.com> wrote:

I figured that was the intention, but we can’t be too surprised that everyone is maintaining personal modulemap repositories (and polluting search results — just try googling for a Swift PNG library!), especially when this central repo still doesn’t exist yet.

Yeah thats unfortunate, maybe this will improve once we have an index.

If Swift ever comes on par with C in terms of being usage and the lingua franca of the FOSS world, I can see linux distributions shipping standard modulemaps the same way they ship C headers, but you have to admit this is years (decades?) away at best.

On the flip side of it, it does wonders in terms of motivating people (me at least) to start writing pure Swift replacements for some of these C libraries (like libpng)…

I think its better to reuse existing libraries than writing from scratch (unless really needed). A good approach that works is "porting" these libraries to build with SwiftPM, see: libYAML <https://github.com/jpsim/Yams>, libressl <https://github.com/vapor/clibressl>\. However, this porting can be difficult to do right but it should become much easier once we have build settings support and custom targets layout <https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20170320/034469.html>\.

On Mar 29, 2017, at 12:42 AM, Ankit Aggarwal <ankit_aggarwal@apple.com <mailto:ankit_aggarwal@apple.com>> wrote:

I think the idea was that there will be one such repository which other packages can use, that too only until the system libraries start shipping their standard modulemap. I thought we had this written down in our documentation somewhere but couldn't find it.

Maybe Daniel can expand on this.

On Wed, Mar 29, 2017 at 10:48 AM, Kelvin Ma via swift-build-dev <swift-build-dev@swift.org <mailto:swift-build-dev@swift.org>> wrote:
This worked! Thanks! But why is having empty git repositories strewn about the “correct” way? System libraries should be imported from within the project, as they are in C. You have to admit it’s getting quite silly that Swift devs keep repositories like these <https://github.com/kelvin13/swift-zlib> on our github accounts. That zlib repository contains exactly ten lines of code. I used to have 6 or 7 repos like that one up there before I got rid of them and switched to local repos.

On Wed, Mar 29, 2017 at 12:03 AM, Ankit Aggarwal <ankit_aggarwal@apple.com <mailto:ankit_aggarwal@apple.com>> wrote:
In this case, these are just umbrella headers. If your modulemap contains absolute path to the header, then you don't need the header files, but SwiftPM will probably warn about this. Note that this is a "hack" to have system packages inside a single repository. The correct way is to have system package as a separate published package which you only need to do once.

On 29-Mar-2017, at 10:26 AM, Kelvin Ma <kelvinsthirteen@gmail.com <mailto:kelvinsthirteen@gmail.com>> wrote:

I will try this, but why are the header files inside the Sources directory? System headers should live in /usr/include…

On Tue, Mar 28, 2017 at 11:48 PM, Ankit Aggarwal <ankit_aggarwal@apple.com <mailto:ankit_aggarwal@apple.com>> wrote:
Hi,

Apologies for not replying to this earlier.

You can have multiple targets in a single package. Each target can either be Swift or C-family. The type of target is determined by the sources contained in it (*.c/*.cpp etc means C target, *.swift means Swift target). So if you want to create multiple C targets, this layout should work:

Package.swift
Sources/
    Bitmap
    Cubify
    Cairo/anchor.c <---- This is just an empty file to tell SwiftPM that this is a C target.
    Cairo/include/Cairo.h
    Cairo/include/module.modulemap
    GLFW/anchor.c
    GLFW/include/GLFW.h
    GLFW/include/module.modulemap

The modulemap is automatically generated, if not provided. This is a package which contains two targets (one C and one Swift): GitHub - jpsim/Yams: A Sweet and Swifty YAML parser.

If you need to pass a bunch of compiler flags, you can use SwiftPM's pkgConfig feature but that will require you to have a separate repository for Cario and GLFW. You can experiment without creating tags using the edit feature <https://github.com/apple/swift-package-manager/blob/master/Documentation/Usage.md#editable-packages>\.

PS: You can join SwiftPM slack channel for quicker turn around time: [swift-build-dev] Swift Package Manager Slack Team

Thanks,
Ankit

On Wed, Mar 29, 2017 at 6:06 AM, Michael Ilseman via swift-build-dev <swift-build-dev@swift.org <mailto:swift-build-dev@swift.org>> wrote:
This is into uncharted territory for me, but it seems you’re building with SwiftPM. You’ll probably want to configure extra compiler flags if that’s possible. You could also bite the bullet and build your C libraries with SwiftPM as well. Hopefully someone on swift-build-dev can help you out.

CC-ing Ankit

On Mar 28, 2017, at 5:09 PM, Kelvin Ma <kelvinsthirteen@gmail.com <mailto:kelvinsthirteen@gmail.com>> wrote:

How do I compile a project with many modules? My tree looks like this:

<Selection_001.png>

On Tue, Mar 28, 2017 at 12:47 PM, Michael Ilseman <milseman@apple.com <mailto:milseman@apple.com>> wrote:
Sure! In this example, I have built libgit2. I have a directory called Git, and inside that I have the following module map:

module Git [system] {
       header "<my path>/libgit2/include/git2.h"
       export *
}

When I run, I use:

swift -I <path-to-“Git”-directory> -L <path-to-built-libgit2> -lgit2 foo.swift

inside foo.swift I can:

import Git
// … use libGit2

Read more about how to write a more appropriate module.map file for your purposes at https://clang.llvm.org/docs/Modules.html\. For example, you might be able to define link flags inside the module.map, use umbrella directories, submodules, etc.

On Mar 28, 2017, at 6:27 AM, Kelvin Ma <kelvinsthirteen@gmail.com <mailto:kelvinsthirteen@gmail.com>> wrote:

Can you give an example?

On Mon, Mar 27, 2017 at 3:59 PM, Michael Ilseman <milseman@apple.com <mailto:milseman@apple.com>> wrote:
Sure. At a low level, you can create a module.map file and use -L/-l flags in your invocation of Swift. If you want to do so at a higher level, then perhaps SwiftPM can. CCing swift-build-dev for the SwiftPM part.

> On Mar 26, 2017, at 3:20 PM, Kelvin Ma via swift-users <swift-users@swift.org <mailto:swift-users@swift.org>> wrote:
>
> Idk if this has been asked before, but is there a way to import C libraries into a Swift project without creating a local git repo? Preferably something similar to C where you can just `#include` headers and then specify the link flags (in Package.swift?)
>
> It’s getting very cumbersome to make a bunch of empty git repos just to use libglfw or libcairo.
> _______________________________________________
> swift-users mailing list
> swift-users@swift.org <mailto:swift-users@swift.org>
> https://lists.swift.org/mailman/listinfo/swift-users

_______________________________________________
swift-build-dev mailing list
swift-build-dev@swift.org <mailto:swift-build-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-build-dev

_______________________________________________
swift-build-dev mailing list
swift-build-dev@swift.org <mailto:swift-build-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-build-dev