AWS Lambda Runtime API

Server Development
41

Just for completeness and information of everyone interested in Lambda:

I started a discussion about how a Linux distribution other than Ubuntu can be officially supported by Swift: Official platform support for other Linux Distributions (and a case for Amazon Linux 2)

Maybe you are interested in this?

2 Likes
42

Hi all,

I finally completed the performance evaluation of Swift on AWS Lambda, I’ve published all the results in this article

I compared Swift performances not only among different languages but also between Ubuntu and the Amazon Linux 2 build provided by @fabianfett.

It’s well known that the overall performance of lambda is dependent on the code size and the allocated memory (and vCPU).

I compared the performance of lambda with the same amount of Memory and vCPU using a basic HelloWorld example.

Refer to this article to better understand what is Cold and Warm start and how AWS Lambda works Anatomy of AWS Lambda - DEV Community 👩‍💻👨‍💻.

AWS Swift vs other Languages

  • Code size gives an advantage to all the runtimes already supported by AWS
  • Cold Start for Swift is ~190ms, Go ~75ms, Python ~127ms, NodeJS ~128ms
  • Warm Start for Swift is ~1.19ms, Go ~0.44ms, Python 1.34ms, NodeJS ~1.06ms

With a small penalty on cold start the performance of Swift is comparable to Python and NodeJS

AWS Swift Lambda Layer Ubuntu vs Amazon Linux 2

  • The Amazon Linux 2 custom run time is around ~20MB and the Ubuntu custom runtime is ~40MB
  • The reduced size of code gives at Cold start advantage of ~140ms to Amazon Linux 2 [Amazon Linux 2 (~190 ms) vs Ubuntu (~329 ms)]
  • At warm start there is no substantial difference between Amazon Linux 2 and Ubuntu.

The small penalty paid by Ubuntu is compensated by a more stable and certified building pipeline.

Swift with libCURL vs AsyncHTTPClient (Swift NIO)

Swift-Sprinter vs Swift-NIO opinionated AWS Lambda Runtime

  • Both framework use Swift NIO
  • Swift Sprinter performs slightly better with a difference of ~0.20 ms at warm start
  • On cold start it depends on the runtime version.
  • Swift Sprinter can be built with Ubuntu and Amazon Linux, Swift-NIO opinionated can be built only with Amazon Linux 2

Conclusions

Swift using AsyncHTTPClient NIO with connection Pooling shows good performance on AWS Lambda compared to most popular languages.

The only advantage of using Amazon Linux 2 rather than Ubuntu is given by ~140ms on cold start, when there is no much difference at warm start.

In the wait to have a stable Amazon Linux 2 swift version, we should note that this advantage is not so huge and can be mitigated using provisioned concurrency.

8 Likes
43

Thanks for the interesting insight, I just read the results but it seems like a good enough spot to be in.

44

Thanks very much for this analysis @Andrea-Scuderi. I agree that this is a pretty good spot to be in. However even with the connection pool we're (1.19ms for a warm start) still considerably slower than Go (0.44ms for a warm start). Do you have an idea where that time is lost?

45

Hi @johannesweiss,
I was disappointed too... we all want to win :smiley: the language challenge.

I added an enhancement request on the Swift-Sprinter repository Evaluate internal WARM performance of LambdaSwiftSprinterNioPlugin with HelloWorld · Issue #38 · swift-sprinter/aws-lambda-swift-sprinter · GitHub .

I'll add consideration there to avoid noise on the forum. Happy if you can take a look to it and if you can give some suggestion on how to evaluate internal performances.
I added the code in the issue so it can be easy to follow.

Principal suspects we need to investigate more:

  • In the Go example, the lambda doesn't pass the Data and Context buffer
  • Some evaluation/instance of the Data/Context could be improved in Swift
  • AsyncHTTPClient performance could be improved
46

These results are on an optimized build, right? Sorry, just triple checking :sweat_smile:

1 Like
47

That sounds interesting, which particular bit shows up is slow in your benchmarking/profiling?

48

The build has been built with release flag.

1 Like
49

@johannesweiss
I added some log to print the value. (This alter for ~0.20 ms the duration).

START RequestId: 8e120434-2108-4a59-9bb4-956b6a6b5e84 Version: $LATEST
   nio-plugin: 1065.549714000099ms:httpClient.execute
   nio-plugin: 0.00033700007406878285ms:result.status.isValid()
   nio-plugin: 0.003169000137859257ms:body.getData(at:...)
core: 1065.6208529999276ms: getNextInvocation
core: 0.008160000106727239ms: lambdaRuntimeTraceId
core: 0.000874000306794187ms: lambdas[handlerName]
core: 0.008774999969318742ms: context
core: 0.005031999990023905ms: commonHandler
   nio-plugin: 0.05818800036649918ms:.data(httpBody)
   nio-plugin: 0.488504999793804ms:httpClient.execute
core: 0.5696690000149829ms: postInvocationResponse
core: 0.0049599998419580515ms: wait
END RequestId: 8e120434-2108-4a59-9bb4-956b6a6b5e84
REPORT RequestId: 8e120434-2108-4a59-9bb4-956b6a6b5e84	Duration: 1.42 ms	Billed Duration: 100 ms	Memory Size: 256 MB	Max Memory Used: 84 MB

Results:
Total time: 1.42ms
getNextInvocation = ~0.82253ms (evaluated by difference excluding other internal times)
postInvocationResponse = ~0.5696690000149829ms

the AsyncHTTPClient post run in ~ 0.488504999793804ms
(AsyncHTTPClient from current master branch)

I've added a full report in this issue: Evaluate internal WARM performance of LambdaSwiftSprinterNioPlugin with HelloWorld · Issue #38 · swift-sprinter/aws-lambda-swift-sprinter · GitHub

Note:
By adding Data/Context to the golang example the result doesn't change (~0.44ms in total)