memory requirements for building swift on linux?


(Luca Finzi Contini) #1

Hi,
I have tried to build the swift ecosystem on an Ubuntu 15.10 machine loaded
with 16GB of RAM and 16GB of swap space.
What happens is that if you run the "utils/build-script -t" command as
suggested, this will use all processors available to speedup the build
BUT
it will also try to perform parallel links of tools when finished
compiling.
This is when memory is filled up totally and disks start to grind.
I tried running the whole thing with just one single job, using:

utils/build-script -j 1 -t

and it took more time to finish but it finished :slight_smile:

I think there could be room for some enhancements in the way that the build
script uses processors : either you know how much RAM isneeded to build the
whole thing, or just get down to one single job for the purpose of linking.

Message: 7

···

Date: Mon, 14 Dec 2015 09:03:36 -0500
From: Kevin Lundberg <kevin@klundberg.com>
To: swift-users@swift.org
Subject: [swift-users] memory requirements for building swift on
        linux?
Message-ID: <566ECC38.9030506@klundberg.com>
Content-Type: text/plain; charset=utf-8

Hi All,

I've set up a linux VM (ubuntu 15.10) in virtualbox on my windows PC to
build and test swift on linux, but I've been unable to successfully
build it to completion. I have all the repos including the corelibs
cloned locally, and have followed all the steps as far as I know for
installing dependencies. Every time I build, it fails somewhere during
linking, the last time failing when linking "CXX shared library
lib/libLTO.so".

I don't have the full error, but I'm curious if there's some RAM
requirement for building swift. My VM had 2GB, and I just bumped it up
to 4GB to try again. Has anyone else run into this, and is there a
recommended minimum needed to successfully build swift with? Could
something else be the cause here instead of RAM if 2GB is in fact enough?

Thanks!

-Kevin

------------------------------

Message: 8
Date: Mon, 14 Dec 2015 14:30:22 +0000
From: Tobias Scholze <tobias.scholze@gmail.com>
To: Kevin Lundberg <kevin@klundberg.com>, swift-users@swift.org
Subject: Re: [swift-users] memory requirements for building swift on
        linux?
Message-ID:
        <CAHZb28bw_7M10b+j69h55ykxunfPUocD=
SatPMp-doNymNnsGg@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi,
to build Swift from the sources I used a 16 GB bare metal machine and it
was not enough.
So I think, you need a lot of memory ( > 16 GB) to have fun while building
it.

- Toby

Kevin Lundberg via swift-users <swift-users@swift.org> schrieb am Mo., 14. > Dez. 2015 um 15:03 Uhr:

> Hi All,
>
> I've set up a linux VM (ubuntu 15.10) in virtualbox on my windows PC to
> build and test swift on linux, but I've been unable to successfully
> build it to completion. I have all the repos including the corelibs
> cloned locally, and have followed all the steps as far as I know for
> installing dependencies. Every time I build, it fails somewhere during
> linking, the last time failing when linking "CXX shared library
> lib/libLTO.so".
>
> I don't have the full error, but I'm curious if there's some RAM
> requirement for building swift. My VM had 2GB, and I just bumped it up
> to 4GB to try again. Has anyone else run into this, and is there a
> recommended minimum needed to successfully build swift with? Could
> something else be the cause here instead of RAM if 2GB is in fact enough?
>
> Thanks!
>
> -Kevin
> _______________________________________________
> swift-users mailing list
> swift-users@swift.org
> https://lists.swift.org/mailman/listinfo/swift-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <
https://lists.swift.org/pipermail/swift-users/attachments/20151214/5ef837e0/attachment-0001.html
>

------------------------------

Message: 9
Date: Mon, 14 Dec 2015 09:55:19 -0500
From: Kevin Lundberg <kevin@klundberg.com>
To: Tobias Scholze <tobias.scholze@gmail.com>, swift-users@swift.org
Subject: Re: [swift-users] memory requirements for building swift on
        linux?
Message-ID: <566ED857.90601@klundberg.com>
Content-Type: text/plain; charset="utf-8"

I was able to build it on my 8GB macbook successfully, but there may be
platform differences there that caused that.

It seems that 4GB isn't enough either, fails when linking clang-3.8.
I'll have to figure something else out for linux for me. Thanks!

On 12/14/2015 9:30 AM, Tobias Scholze wrote:
> Hi,
> to build Swift from the sources I used a 16 GB bare metal machine and
> it was not enough.
> So I think, you need a lot of memory ( > 16 GB) to have fun while
> building it.
>
> - Toby
>
> Kevin Lundberg via swift-users <swift-users@swift.org
> <mailto:swift-users@swift.org>> schrieb am Mo., 14. Dez. 2015 um > > 15:03 Uhr:
>
> Hi All,
>
> I've set up a linux VM (ubuntu 15.10) in virtualbox on my windows
> PC to
> build and test swift on linux, but I've been unable to successfully
> build it to completion. I have all the repos including the corelibs
> cloned locally, and have followed all the steps as far as I know for
> installing dependencies. Every time I build, it fails somewhere
during
> linking, the last time failing when linking "CXX shared library
> lib/libLTO.so".
>
> I don't have the full error, but I'm curious if there's some RAM
> requirement for building swift. My VM had 2GB, and I just bumped it
up
> to 4GB to try again. Has anyone else run into this, and is there a
> recommended minimum needed to successfully build swift with? Could
> something else be the cause here instead of RAM if 2GB is in fact
> enough?
>
> Thanks!
>
> -Kevin
> _______________________________________________
> swift-users mailing list
> swift-users@swift.org <mailto:swift-users@swift.org>
> https://lists.swift.org/mailman/listinfo/swift-users
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <
https://lists.swift.org/pipermail/swift-users/attachments/20151214/15e62774/attachment-0001.html
>

------------------------------

Message: 10
Date: Mon, 14 Dec 2015 09:04:22 -0800
From: Daniel Dunbar <daniel_dunbar@apple.com>
To: Kevin Lundberg <kevin@klundberg.com>
Cc: swift-users@swift.org
Subject: Re: [swift-users] memory requirements for building swift on
        linux?
Message-ID: <954151C6-67C8-40A9-ADC7-5D7292F7EC04@apple.com>
Content-Type: text/plain; charset="utf-8"

For those hitting this issue, you might try building without debug info to
see if that helps (build-script -R).

- Daniel

> On Dec 14, 2015, at 6:55 AM, Kevin Lundberg via swift-users < > swift-users@swift.org> wrote:
>
> I was able to build it on my 8GB macbook successfully, but there may be
platform differences there that caused that.
>
> It seems that 4GB isn't enough either, fails when linking clang-3.8.
I'll have to figure something else out for linux for me. Thanks!
>
> On 12/14/2015 9:30 AM, Tobias Scholze wrote:
>> Hi,
>> to build Swift from the sources I used a 16 GB bare metal machine and
it was not enough.
>> So I think, you need a lot of memory ( > 16 GB) to have fun while
building it.
>>
>> - Toby
>>
>> Kevin Lundberg via swift-users < <mailto:swift-users@swift.org>
swift-users@swift.org <mailto:swift-users@swift.org>> schrieb am Mo., 14. > Dez. 2015 um 15:03 Uhr:
>> Hi All,
>>
>> I've set up a linux VM (ubuntu 15.10) in virtualbox on my windows PC to
>> build and test swift on linux, but I've been unable to successfully
>> build it to completion. I have all the repos including the corelibs
>> cloned locally, and have followed all the steps as far as I know for
>> installing dependencies. Every time I build, it fails somewhere during
>> linking, the last time failing when linking "CXX shared library
>> lib/libLTO.so".
>>
>> I don't have the full error, but I'm curious if there's some RAM
>> requirement for building swift. My VM had 2GB, and I just bumped it up
>> to 4GB to try again. Has anyone else run into this, and is there a
>> recommended minimum needed to successfully build swift with? Could
>> something else be the cause here instead of RAM if 2GB is in fact
enough?
>>
>> Thanks!
>>
>> -Kevin
>> _______________________________________________
>> swift-users mailing list
>> swift-users@swift.org <mailto:swift-users@swift.org>
>> https://lists.swift.org/mailman/listinfo/swift-users <
https://lists.swift.org/mailman/listinfo/swift-users>
>
>
> _______________________________________________
> swift-users mailing list
> swift-users@swift.org
> https://lists.swift.org/mailman/listinfo/swift-users

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <
https://lists.swift.org/pipermail/swift-users/attachments/20151214/78f22f84/attachment-0001.html
>

------------------------------

Message: 11
Date: Mon, 14 Dec 2015 09:22:58 -0800
From: Joe Groff <jgroff@apple.com>
To: David Turnbull <dturnbull@gmail.com>
Cc: swift-users <swift-users@swift.org>
Subject: Re: [swift-users] Here's a multi-platform OpenGL loader for
        Swift.
Message-ID: <42D182F2-C966-4CBD-802E-C0803DDCB97A@apple.com>
Content-Type: text/plain; charset="utf-8"

> On Dec 12, 2015, at 1:39 PM, David Turnbull via swift-users < > swift-users@swift.org> wrote:
>
> It's looking like the Epoxy requirement for SwiftGL is going away real
soon now. I was able to lookup the OpenGL symbols from pure Swift code. No
compiled C code is needed at all. What follows is roughly how I will do it.
It lazily loads the pointer the first time you use it. The Linux version is
only slightly different.
>
> Does anyone know a way to do this without the unsafeBitCast?
>
> import Darwin
>
> let handle =
dlopen("/System/Library/Frameworks/OpenGL.framework/Versions/Current/OpenGL",
RTLD_LAZY)
>
> func DLOADglGetIntegerv(a:GLenum, _ b:UnsafeMutablePointer<GLint>) ->
Void {
> glGetIntegerv = unsafeBitCast(dlsym(handle, "glGetIntegerv"),
glGetIntegerv.dynamicType)
> glGetIntegerv(a,b)
> }
>
> public var glGetIntegerv:@convention(c) (GLenum,
UnsafeMutablePointer<GLint>) -> Void = { DLOADglGetIntegerv($0,$1) }

I think the unsafeBitCast is fundamentally necessary here to coerce the
void pointer you get from dlsym to a C function pointer. I don't think it's
a problem. This is pretty awesome!

-Joe

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <
https://lists.swift.org/pipermail/swift-users/attachments/20151214/cdccd7e6/attachment-0001.html
>

------------------------------

_______________________________________________
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users

End of swift-users Digest, Vol 1, Issue 43
******************************************