How to use a C static library(libJavaScriptCore.a) with Swift on Linux?

Hi, I need to use JavaScriptCore to run some JS code with my Swift app and I need it to be multi-platform. On macOS it's easy, JavaScriptCore is available and ready to use.

On Linux, I compiled JavaScriptCore as a static library which appears to work as expected through it's C API.

However It's not very clear to me how to make it work with Swift.

The library comes in two folders, /lib where the .a files are and /Headers/JavaScriptCore where the .h files live. Like this:

├── Headers
│   └── JavaScriptCore
│       ├── JSBase.h
│       ├── JSContextRef.h
│       ├── JSObjectRef.h
│       ├── JSStringRef.h
│       ├── JSTypedArray.h
│       ├── JSValueRef.h
│       ├── JavaScript.h
│       └── WebKitAvailability.h

and

└── lib
    ├── libJavaScriptCore.a
    ├── libLowLevelInterpreterLib.a
    ├── libWTF.a
    ├── libbmalloc.a
    ├── libgtest.so
    ├── libmbmalloc.so
    └── system
        └── libmbmalloc.so

How can I make this library available for use in Swift? I've looked into C-Interop docs, tried to link the library with -Xlinker but did bir have a success so far.

I would like to use it with Vapor.

Thanks!

1 Like

Hi. Can you share with me the header files and binaries, ideally with the SPM example? I would like to try this myself :)

Hi @stuchlej, I have the library compiled on Ubuntu ARM64 VM so maybe it can be better if you compile the JavaScriptCore library yourself.

It's pretty easy, just follow the instructions here: How to Build JavaScriptCore on Your Machine - Constellation Scorpius

The files will be under WebKit/WebKitBuild/Release folder (headers are in ./JavaScriptCore/Headers/JavaScriptCore and the .a files are in ./lib).

Then create a new Vapor project that will have the Package.swift

Also very easy to follow: Vapor: Install → Linux

Now you have your Swift project with SPM setup ready.

There is this repo that allows you to use JavaScriptCore with the WebkitGTK.

This works fine but as I said, I want to use the static JavaScriptCore library.

If you succeed, please let me know!

1 Like

Hi, I have looked into it and I think I have found something useful. I think I'm on to something, but I wasnt able to get it fully working.

TL;DR The swiftc should be able to link the static libraries. It just need to know what libraries are required. The issue is therefore not with Swift, but WebKit. Either provide Swift the full list of dependencies, or build the JSC differencly.

Before I start, a little discalimer. I looked into this as a challange. I'm really not that competent when it comes to "compiling and linking stuff," but I trying to improve myself in that regard. Therefore this post will be long, because I want to present my thought process so you're able to validate my conclusions.

I'll describe my today's story.

Setting things up

I have built the JSC as you have instruced me, and done all the following steps involving Vaport and the JavaScriptKit repo. Then I realized, I didn't need to install Vapor at all, so if I'm missing something, let me know.

I have created a new swift package via swift package init --type executable and created new target cjsc with the same structure as the CJavaScriptCore target but I didn't include the modulemap. It confuses me why the modulemap is there in the first place. It looks to me like some weird combination of .systemLibrary target and .target, but I've decided to let it go.

In the main.swift file in my executable target, I have added following statements:

import cjsc
JSGarbageCollect(nil)

so I am sure, that symbols are being resolved.

Getting on track

First I have tried to use Package.swift cSettings, linkerSettings etc. but it didn't seem to work, so I have went back to good 'ol swift package cli.

I have tried to specify header path using -Xcc -I/home/mikolas/Developer/WebKit/WebKitBuild/Release/JavaScriptCore/Headers \ which worked. Then I have tried to provide library search path via -Xcc -L/home/mikolas/Developer/WebKit/WebKitBuild/Release/lib/ which did not work. The console output stated, that the -L was ignored. Using -v I have found out, that all of the commands are passed to the swiftc. It makes sence, that the -L was ignored, I assume, that the swiftc runs clang and finally the linker and passing some linking options to clang won't have any effect.

I have therefore replaced -Xcc with -Xlinker which was a step in the right direction ... but. No libraries were linked. I have passed the --help command to the linker via -Xlinker --help and then -Xlinker --verbose. The verbose output of the linker contained all of the files the linker attempted to open - including ing the /home/mikolas/Developer/WebKit/WebKitBuild/Release/lib/ - but it tried to open libraries like libgcc.a or libc.a. I would expect, that it would try libJavaScriptCore.a...

Then I realized, that the linker searches all of the library search paths and tries to open all of the libraries it's looking for. And it wasn't looking for "libJavaScriptCore.a" anywhere - because nobody told to the linker to do so.

Reducing the number of errors

It was clear to me, that I need to add a -Xlinker -lJavaScriptCore so the linker knows what it should look for. But this is where the fun starts. Turns out (to nobody's surprise), that the libJavaScriptCore.a contains a lot of symbols that need to be linked. The linker returned about 1610 errors.

I have then started to add various other libraries. I will now list the libraries I have added and the number of errors:

library                  ; err  ; note
-lJavaScriptCore         ; 1610 ; the base library
-lWTF                    ;  828 ; same directory as jsc
-lstdc++                 ;  719 ; I have tried it, since a lot of std:: errors were present
-lbmalloc                ;  522 ; same directory as jsc
-lLowLevelInterpreterLib ;  487 ; same directory as jsc
-lm                      ;  381 ; I have tried bunch of libraries from  $ pkg-config --libs --static javascriptcoregtk-4.0
-licuuc                  ;  230 ; Some symbols I have googled were from this library, which should be something related to international component for unicde...

230 missing symbols down from 1610 looks quite promising, but at that point I gave up.

My last build command:

swift build \
-Xcc -I/home/mikolas/Developer/WebKit/WebKitBuild/Release/JavaScriptCore/Headers \
-Xlinker -L/home/mikolas/Developer/WebKit/WebKitBuild/Release/lib/ \
-Xlinker -lJavaScriptCore -Xlinker -lWTF  -Xlinker -lstdc++ \
-Xlinker -lbmalloc  -Xlinker -lLowLevelInterpreterLib -Xlinker -lm \
-Xlinker -licuuc

My conclusion is, that the problem is not in the Swift compiler. We just don't know how to link JavaScriptCore statically properly. I think this might be a better question for some "Web Kit forums" or someone with general knowledge of "where to look for the list of dependencies".

(At this point I've briefly meditated on my renewed appreciation for pkg-config and all the problems it solves for me.)
(Thinking about it, .pc files for pkg-config can be generated ... it should be somehow possible to dump the full list of dependencies using make, right?...)

2 Likes

I got exactly at the same point with a bit different path.

In the modulemap file I pointed to an include.h file that includes the headers of the library and I modified the headers to include from local paths(#include "JSValue.h" instead of #include <JavaScriptCore/JSValue.h>). Then I linked the .a files using the -L./path/to/lib flags and that's the point where I got the exact same errors as you.

Do you know a place where people with Swift and C knowledge discuss stuff? Previously there was a bug in the JavaScriptCore library and failed to compile but the developers of the library were super helpful and fixed the bug almost the same day once I asked for help.

Unfortunately they were not very interested in the linking the library question.

2 Likes

Yes, my local university. :smile: Anyway, questions like this don't get much attention imho, because you (and me) should really be able to figure it out on our own. Asking people to help you build something is like asking people to solve your homework, if you know what I mean :slight_smile: I'm not saying such questions should not be asked, but don't get frustrated if you don't find anyone who is willing to answer it. :slight_smile: I don't want to dive back to my lifestory again, so lets focus on:

Now, as I understand the issue at hand, we need to figure out what libraries we need to link. I have previously mentioned, that cmake should be able to tell us what libraries are being used in the project.

I have found this post on google cmake - How do I list the defined make targets from the command line? - Stack Overflow which describes how to make cmake print the structure of the project, following worked for me:

$ Tools/Scripts/build-webkit --jsc-only --cmakeargs="-DENABLE_STATIC_JSC=ON -DUSE_THIN_ARCHIVES=OFF --graphviz=test.graph "
$ cd WebKitBuild/Release/
$ dot -Tjpg test.graph > out.jpg

The result is this image which contains all of the tagets, libraries and dependencies:

The option is documented here CMakeGraphVizOptions — CMake 3.24.0-rc4 Documentation .

Now, I can immediately recognize some libraries like ICU. Ultimately, this solves nothing. But I think it is interesteng, so I've felt the need to post it here (for my own future reference :slight_smile:).

So at this point, we're where we beign: we know about some of the libraries we need to link, but some are still missing.

Next I was about to try another trick: CMAKE_EXPORT_COMPILE_COMMANDS. CMAKE_EXPORT_COMPILE_COMMANDS — CMake 3.24.0-rc4 Documentation Setting this variable to ON will result in cmake creating a file called compile_commands.json which contains all of the compile commands ... but that's not what we're looking for, right? We're looking for link commands. Turns out, such a feature is not implemented ... yet. Add Link Database Generation (#22067) · Issues · CMake / CMake · GitLab

But then I've realized, that cmake does not build anything on it's own. It is used to generate build commands for other build systems - like make, Xcode or ... Ninja in this case.

So I have opened /home/mikolas/Developer/WebKit/WebKitBuild/Release and found file called build.ninja. This file has 237K lines (and some of the lines have 9K characters) so you'll need to use some text editor, that's not going to freeze on you (MATE's Pluma in my case). I think it's safe to say, that this file contains everything there is :smiley:
I have spent next 1/2 trying to understand the contents. If you're interested in how "things get linked", simply search for # Link the .

The section # Link the executable bin/jsc [*] links the lib/libJavaScriptCore.a, so it should contain all the link statements we need. And I assume it does in LINK_LIBRARIES where we can observe which libraries are linked statically, and which dynamically.

In conclusion, I think that we're near the end. It should be sufficient to rewrite the link commands for bin/jsc for our Swift compiler.

Anyway, I'm not going to try this out today, other things require my attention (chores around the house and fun stuff like that.) I realize my posts are really - really long, but I like to describe my thought process in detail. Not only so people are able to validate my results, but for my own future reference.

[*] bin/jsc is command line interpret of JavaScript, so I fell confident it should ... "contain" all of the things required.

Edit: I have also look at this post: Linux static-executable linking errors

1 Like

During the exercise I definitely learned quite a lot about C and its tooling without having any intention to do so :slight_smile:

The sheer complexity of using a library is quite amusing and teaches me to appreciate Swift and Apple's IDE. On the other hand it's also fascinating to see the lower level workings of the machinery but I'm not there yet to be able to solve it I guess but it kind of feels like you are almost there.

2 Likes

I’m on the same page. The most important thing for me was “learning the questions I should ask.”

The process of building a software is not that complicated on the high level, there definitely should be some mooc courses (for example od Edx.org, sadly iTunes U is no longer with us), for example a quick search: Linux Tools for Software Development.

Regarding the software that is already built, there is a great book Advanced Apple Debugging & Reverse Engineering | raywenderlich.com and I’ve heard, that there is going to be new edition for Swift 5. You can find a lot of information about Mach-O in section III (which is to my understanding Darwin equivalent of ELF (executable and linkable format) on Linux.

2 Likes

OK, So I have been able to complete the build.

The final version of my build script looks like this:

#!/bin/bash

swift build \
-Xcc -I/home/mikolas/Developer/WebKit/WebKitBuild/Release/JavaScriptCore/Headers \
-Xlinker -L/home/mikolas/Developer/WebKit/WebKitBuild/Release/lib/ \
-Xlinker -lstdc++ \
-Xlinker -lm \
-Xlinker -ldl \
-Xlinker -lJavaScriptCore \
-Xlinker -lWTF \
-Xlinker -licudata \
-Xlinker -licui18n \
-Xlinker -licuuc \
-Xlinker -lbmalloc \
-Xlinker -ldl \
-Xlinker -latomic \
-Xlinker -lLowLevelInterpreterLib \

Following stdout:

Building for debugging...
[1/3] Compiling cjsc blank.c
[2/5] Emitting module wkusage
[3/5] Compiling wkusage main.swift
[6/7] Wrapping AST for wkusage for debugging
[7/7] Linking wkusage
Build complete! (1.28s)

The conclusion of my previous post was not sufficient, I've had to link additionaly -lm, the math library and -lstdc++, the C++ STL.

However, I have no idea whether this works at runtime or not. Just passing build does not mean you won't get segfault :smiley:

Let me know whether this works or not :)

4 Likes

WoW! It did work!

[1847/1847] Linking Run
Build complete! (79.92s)
[ NOTICE ] Server starting on http://192.168.64.5:8080
[ INFO ] GET / [request-id: 9E9952F6-EEE1-41E2-8642-457761D5B993]
Optional(42)

That "42" is computed purely in JavaScript and served upon HTTP request to Vapor, which is made in Swift.

Thank you so much, as someone with no serious experience in C I wouldn't dare to investigate those errors and find out the missing links and link them.

I can't thank you enough.

5 Likes

Super cool! Thanks for sharing. Now you can build sth like https://bun.sh/ but in swift :nerd_face:

2 Likes