CLibPQ is put together in the simplest way: there's a module.modulemap
at the root and that's it. I would like to make some changes to the
module hierarchy and I'm not sure how to go about it. Here's what I'd
like to do:
* Create `CLibPQ.OSXHomebrew` and `CLibPQ.Ubuntu` that contain the
right mappings for those platforms (the header files are in different
places).
* Create `CLibPQ` that conditionally imports the right one:
* In `PGVersion` we'd be able to `import CLibPQ` as before.
What's the right project layout to make this work?
Namespaced C modules are an interesting topic and worth pursuing in
their own right; but maybe there is a better way to do what I'm trying
to accomplish?
CLibPQ is put together in the simplest way: there's a module.modulemap
at the root and that's it. I would like to make some changes to the
module hierarchy and I'm not sure how to go about it. Here's what I'd
like to do:
* Create `CLibPQ.OSXHomebrew` and `CLibPQ.Ubuntu` that contain the
right mappings for those platforms (the header files are in different
places).
* Create `CLibPQ` that conditionally imports the right one:
CLibPQ.swift: if os(Linux)
import CLibPQ.Ubuntu #else
import CLibPQ.OSXHomebrew #endif
* In `PGVersion` we'd be able to `import CLibPQ` as before.
What's the right project layout to make this work?
You can’t do what you are trying here with a module map sadly.
We need to add explicit support for this sort of thing to swiftpm. Our current ideas are:
1. Mangle the module map for obvious relocations (/usr -> /usr/local)
2. Making it possible to specify platform module maps for exceptions eg. Ubuntu.modulemap, etc.
3. Changing /usr to $ROOT and then having the user have to specify root if it is not /usr
Namespaced C modules are an interesting topic and worth pursuing in
their own right; but maybe there is a better way to do what I'm trying
to accomplish?
I think namespaces needs discussion for Swift in general.
In the case of `libpq`, even something obvious like `/usr -> /usr/local`
wouldn't work because on Ubuntu it ends up in `/usr/include/postgres`.
Since we can't have macros in the module map, I think option (2) -- platform
module maps -- would be a great combination of hygiene and flexibility (if
tedious).
As this gets fancier there will be a temptation to allow using very
specific platform names -- Kubuntu-15.10 -- and there will be a need also
for customization by the end user. I can't imagine that problems like this
have not shown up for the iPhone dev team already, which has to support 3
major versions and a bunch of slightly variant kinds of ARM.
One thought would be to expand macros in SwiftPM: allow macros in the
module map. Then add an `sh()` macro to call the system shell. Now anything
is possible! But this seems like an invitation to spurious dependencies and
unreadable build files.
Another option would seem to be modelling the daylights out of build
environments but I think this runs afoul of enumerations. Right now we have
`os(Linux)` but we'd really need `os(Ubuntu)`, `os(RedHat)` and so forth to
handle dependencies like these. And if people are working on a new distro
-- or merely want a new platform tag even though they are on a stock distro
-- this would fail unless they rebuilt the Swift compiler or package
manager with an extended enumeration.
I'm sure you've thought a lot about all this stuff already and I'd like to
know what you see as the way forward for Swift.
Best,
Jason
···
On Wed, 9 Dec 2015 at 10:31 Max Howell <max.howell@apple.com> wrote:
> Behold, a binding for libpq:
>
> GitHub - solidsnack/CLibPQ: Porting libpq to Mac
>
> And a little app that uses it:
>
> GitHub - solidsnack/PGVersion: Print the Postgres library version -- now for Mac
>
> CLibPQ is put together in the simplest way: there's a module.modulemap
> at the root and that's it. I would like to make some changes to the
> module hierarchy and I'm not sure how to go about it. Here's what I'd
> like to do:
>
> * Create `CLibPQ.OSXHomebrew` and `CLibPQ.Ubuntu` that contain the
> right mappings for those platforms (the header files are in different
> places).
>
> * Create `CLibPQ` that conditionally imports the right one:
>
> CLibPQ.swift:
> if os(Linux)
> import CLibPQ.Ubuntu
> #else
> import CLibPQ.OSXHomebrew
> #endif
>
> * In `PGVersion` we'd be able to `import CLibPQ` as before.
>
> What's the right project layout to make this work?
You can’t do what you are trying here with a module map sadly.
We need to add explicit support for this sort of thing to swiftpm. Our
current ideas are:
1. Mangle the module map for obvious relocations (/usr -> /usr/local)
2. Making it possible to specify platform module maps for exceptions eg.
Ubuntu.modulemap, etc.
3. Changing /usr to $ROOT and then having the user have to specify root if
it is not /usr
> Namespaced C modules are an interesting topic and worth pursuing in
> their own right; but maybe there is a better way to do what I'm trying
> to accomplish?
I think namespaces needs discussion for Swift in general.
In the case of `libpq`, even something obvious like `/usr -> /usr/local` wouldn't work because on Ubuntu it ends up in `/usr/include/postgres`.
Yes, I consider this quite a nasty issue that I’d like solve asap.
Since we can't have macros in the module map, I think option (2) -- platform module maps -- would be a great combination of hygiene and flexibility (if tedious).
As this gets fancier there will be a temptation to allow using very specific platform names -- Kubuntu-15.10 -- and there will be a need also for customization by the end user. I can't imagine that problems like this have not shown up for the iPhone dev team already, which has to support 3 major versions and a bunch of slightly variant kinds of ARM.
Yes, I think versioning will become necessary also. Which is problematic for defining the semantic version of these module map packages. But can be done easily as part of the filename:
KUbuntu-15.10.modulemap
We can bake in some obvious inheritance (Ubuntu is the parent “class” of XUbuntu/KUbuntu etc.)
One thought would be to expand macros in SwiftPM: allow macros in the module map. Then add an `sh()` macro to call the system shell. Now anything is possible! But this seems like an invitation to spurious dependencies and unreadable build files.
Indeed, we would like to avoid such things altogether if possible. We believe that allowing arbitrary execution makes it harder for the tool to remain reliable as people take further and further advantage. We want to avoid the kinds of programmable configuration detection exhibited by the likes of autoconf for as long as possible.
Another option would seem to be modelling the daylights out of build environments but I think this runs afoul of enumerations. Right now we have `os(Linux)` but we'd really need `os(Ubuntu)`, `os(RedHat)` and so forth to handle dependencies like these. And if people are working on a new distro -- or merely want a new platform tag even though they are on a stock distro -- this would fail unless they rebuilt the Swift compiler or package manager with an extended enumeration.
Well, the module maps aren’t Swift so it wouldn’t work. Certainly we could allow some kind of if syntax in module maps, but I think the elegance of one-file per platform will be enough and is much simpler.
I'm sure you've thought a lot about all this stuff already and I'd like to know what you see as the way forward for Swift.
I think a combination of platform specific module maps with platform versions and a Default.modulemap that details the standard layout for /usr which can then be adapted to other prefixes will do the trick.
The Default.modulemap may be problematic, since as system-libraries evolve they may change their installation layout. In my experience I have never seen this happen to a stable library, but if we don’t take it into account we could introduce a truly nasty situation in the future.
I’ll write up a full proposal to swift-build-dev@swift.org.
Maybe I wasn’t clear here. This passage is about the platform names — how
do we keep discovery flexible? I’m assuming there is an enum somewhere — OSX,
Linux, iOS — in the compiler and what I’d like to suggest is, a centralized
registry like that will create maintenance headaches for maintainers and
frustrate developers, too. So taking one file per platform as a given, as
long as platform discovery is easily extended then porting libraries is
easy — add a new platform file, or even pass the “compatible platform” as
an option.
Another option would seem to be modelling the daylights out of build
environments but I think this runs afoul of enumerations. Right now we have
`os(Linux)` but we'd really need `os(Ubuntu)`, `os(RedHat)` and so forth to
handle dependencies like these. And if people are working on a new distro
-- or merely want a new platform tag even though they are on a stock distro
-- this would fail unless they rebuilt the Swift compiler or package
manager with an extended enumeration.
Well, the module maps aren’t Swift so it wouldn’t work. Certainly we could
allow some kind of if syntax in module maps, but I think the elegance of
one-file per platform will be enough and is much simpler.
> Another option would seem to be modelling the daylights out of build environments but I think this runs afoul of enumerations. Right now we have `os(Linux)` but we'd really need `os(Ubuntu)`, `os(RedHat)` and so forth to handle dependencies like these. And if people are working on a new distro -- or merely want a new platform tag even though they are on a stock distro -- this would fail unless they rebuilt the Swift compiler or package manager with an extended enumeration.
Well, the module maps aren’t Swift so it wouldn’t work. Certainly we could allow some kind of if syntax in module maps, but I think the elegance of one-file per platform will be enough and is much simpler.
Maybe I wasn’t clear here. This passage is about the platform names — how do we keep discovery flexible? I’m assuming there is an enum somewhere — OSX, Linux, iOS — in the compiler and what I’d like to suggest is, a centralized registry like that will create maintenance headaches for maintainers and frustrate developers, too. So taking one file per platform as a given, as long as platform discovery is easily extended then porting libraries is easy — add a new platform file, or even pass the “compatible platform” as an option.
Well, I suppose it comes down to: are the OSes in os(...) strings or enums?
If they are strings then the compiler doesn’t need to be changed to add a
new OS — it just needs to have a standard way to look it up. For example it
might check lsb_release first, then try sw_vers. If the string there
presented matches something in an os(...) condition in a macro, then it
works. There is no internally maintained list of “valid OSes” so there is
no need to extend such a list to handle each new release of Ubuntu, Debian,
&c
···
On Wed, 9 Dec 2015 at 13:08, Max Howell <max.howell@apple.com> wrote:
Another option would seem to be modelling the daylights out of build
environments but I think this runs afoul of enumerations. Right now we have
`os(Linux)` but we'd really need `os(Ubuntu)`, `os(RedHat)` and so forth to
handle dependencies like these. And if people are working on a new distro
-- or merely want a new platform tag even though they are on a stock distro
-- this would fail unless they rebuilt the Swift compiler or package
manager with an extended enumeration.
Well, the module maps aren’t Swift so it wouldn’t work. Certainly we
could allow some kind of if syntax in module maps, but I think the
elegance of one-file per platform will be enough and is much simpler.
Maybe I wasn’t clear here. This passage is about the platform names — how
do we keep discovery flexible? I’m assuming there is an enum somewhere —
OSX, Linux, iOS — in the compiler and what I’d like to suggest is, a
centralized registry like that will create maintenance headaches for
maintainers and frustrate developers, too. So taking one file per platform
as a given, as long as platform discovery is easily extended then porting
libraries is easy — add a new platform file, or even pass the “compatible
platform” as an option.