AWS Lambda Runtime API

@mrmikemon
@tachyonics

I’d like to know this too. Would really like to see some information on getting started so we can take it for a spin and help move it forward.

@mrmikemon
@GalCohen

Thanks guys, appreciate your interest!

We actually have a full example showing a service with a REST-like API with some operations using DynamoDB as a persistence layer coded up and waiting approval. I don't have a firm timeline when we will be able to release it but in the past it hasn't taken long from this point.

Thanks @tachyonics ! Looking forward to it. Has compiling for the lambda execution environment been worked out yet?

@GalCohen Not yet. My team has a build script for building the Swift Toolchain for Amazon Linux 2 (which is what we run our services on). One of the next things I have planned for us is to finish adapting the script for Amazon Linux 2017.03 which is what Lambda uses.

I forked @tonisuter 's great implementation of the Lambda runtime API and was able to add support for async tasks/operations as well.

I was having a crash related to (what I believe) DNS resolution, but adding the /lib/x86_64-linux-gnu/libnss_dns.so.2 lib to the layer fixed it.

However, there is still the HTTPS error to sort out.

Toni, good job on what you did!

1 Like

Wanted to highlight that @jasonzurita posted about this at https://www.jasonzurita.com/websites-using-swift-and-aws-lambda :tada:

2 Likes

natanrolnik had the idea of trying to build the Swift source using the AWS Lambda linux (Amazon Linux) environment to resolve the HTTPS issue. I tried this out, by using this docker image, but I run into the following error (since it looks like you are trying something similar, I am not sure if you ran into the same thing @sebsto):

bash-4.2# ./swift/utils/build-script -r
./swift/utils/build-script: note: Using toolchain default
./swift/utils/build-script: fatal error: can't find clang (please install clang-3.5 or a later version)

Here is what I did to get the above:

docker run -it --name swiftawslinux amazonlinux /bin/bash
yum install cmake ninja-build clang python uuid-dev libicu-dev icu-devtools libbsd-dev libedit-dev libxml2-dev libsqlite3-dev swig libpython-dev libncurses5-dev pkg-config libblocksruntime-dev libcurl4-openssl-dev systemtap-sdt-dev tzdata rsync
mkdir swift-source
cd !$
git clone https://github.com/apple/swift.git
./swift/utils/update-checkout --clone
./swift/utils/build-script -r

If anyone has any suggestions of what to try, that would be great!

1 Like

IMHO, there are two possibilities to address the GNUTLS/HTTPS issue

  • a/ compile GNUTls on Ubuntu with compile-time options to point to Amazon Linux's cacerts
  • b/ compile SWIFT on Amazon Linux.

I naively thought that a/ was easiest as I would deal with less dependencies issues. I am struggling on this since several days. GNUTls has so many dependencies and I am still blocked by a lib or other not being installed or correctly identified.

I tried b/ yesterday. The issue reported above by Jason is easy to work around. You need clang, cmake3 installed. I softlinked cmake -> cmake3 etc ... I am blocked further
The problem with the approcah above is that the yum install command can not find most of these dependencies :

No package uuid-dev available.
No package libicu-dev available.
No package icu-devtools available.
No package libbsd-dev available.
No package libedit-dev available.
No package libxml2-dev available.
No package libsqlite3-dev available.
Package swig-3.0.12-11.amzn2.0.3.x86_64 already installed and latest version
No package libpython-dev available.
No package libncurses5-dev available.
No package pkg-config available.
No package libblocksruntime-dev available.
No package libcurl4-openssl-dev available.
No package systemtap-sdt-dev available.

At this stage, I would love to receive help by someone use to build native Linux projects to solve teh dependency issues on a/ or b/

3 Likes

Wondering if it is a dead end on the Swift on AWS Lambda? I would like to help out on getting this to work. I have no idea where to start with this. Like all of you I can write the AWS Lambda Runtime API but getting it to work with API is not there

1 Like

Regarding the SSL path issue discussed above by @natanrolnik and @sebsto. One potential solution might be to make a small change in swift-corelibs-foundation to allow for setting the path with an environment variable rather than relying on the default location that the Ubuntu based docker image uses. There already is support for this feature but it is limited to Android.

#if os(Android) || os(Linux)
...
  if let caInfo = getenv("URLSessionCertificateAuthorityInfoFile")  {
...
    try! CFURLSession_easy_setopt_ptr(rawHandle, CFURLSessionOptionCAINFO, caInfo).asError()
  }
#endif

We could change this line to also allow for Linux to configure the environment variable.

5 Likes

This would be even more convenient in a Lambda context than on Android- there it’s quite hard to provide the correct file path, in practice.

I've found a way to solve the HTTPS issue by using swift-nio 2.
I prepared an open-source framework https://github.com/swift-sprinter to support the development of the solution I've found.

Solving the HTTPS issue allows interacting with the AWS ecosystem.
I prepared documentation and three examples you can find here LambdaSwiftSprinter.

S3Test shows how to interact with an S3 bucket by using an early version of an open-source version of aws-sdk based on nio 2.

The project is in an early stage and it's open to collaboration and I hope someone will join me.

10 Likes

Wow, that sounds great! CC @georgebarnett, @tomerd, @kevints

2 Likes

Good news! Can't wait to use it myself :slight_smile:

It could be interesting once it's stable to do some benchmarks to see if it's a competitive option for production use.

Update:
Well we already have some elements there about general Swift performance.

@Damien_Petrilli , I like your idea.
I've opened an issue on the library, I would be very grateful if you can help me on this.


Feel free to open a new issue if this doesn't describe your idea.

You can consider the core LambdaSwiftSprinter package quite stable as it was ~100% tested (HelloWorld Example).

The unstable part is the one related to the nio 2.0 plugin and the aws-sdk as the related libraries are under development.

1 Like

I’ve been using your runtime for a couple of months, and absolutely loving it! I’m getting 100ms round-trip times for AWS API calls but I don’t know if this is an issue with the way I’m using the AWS SDK (the open source one on Github that you reference in your examples) or something else. Any notes you could add to the Readme on performance tips would be amazing! Again, really great project!!!

2 Likes

Hi @mr_j_tree - do you have any code to share? Have you tried out any logging or instrumentation around performance? What’s your code doing? Which services are you using? We’ll need more info to help diagnose.

Also if anyone here is at Re:invent this week and wants to connect and/or help move Swift on lambda forward let’s chat

Hi, I've been doing some investigation into this in the last week. You can see the results in this thread https://github.com/swift-aws/aws-sdk-swift/issues/202#issuecomment-560266370. Main things to extract from this are

  1. Most of the time is spent in the HTTP request part of the AWS API calls. This can be improved upon. The connection pool work that is happening on the async-http-client should improve things considerably if you are doing more than one api call in the lambda.
  2. cold starts hurt, really bad. The process of creating and signing a request takes over 100ms on the first call, subsequent calls it takes approx 1.5ms. If the lambda is warm the first call takes 1.5ms.
  1. I got same results. I'm waiting for the connection pooling in async-http-client before publishing the full comparison with other languages.
  2. The cold start issue is mitigated by the provisioned concurrency of AWS Lambda announced yesterday: https://aws.amazon.com/about-aws/whats-new/2019/12/aws-lambda-announces-provisioned-concurrency/
Terms of Service

Privacy Policy

Cookie Policy