I propose the development of three new plugins to streamline the scaffolding, building, packaging, and deployment of Lambda functions written in Swift.
The proposal is detailed in the attached pull request. Kindly provide your comments, questions, and feedback on the document.
I didnât touch AWS lambda functions in any language much but if I were to develop one at some point, Iâd like to use Swift for it and your proposed plugins all seem to make the process much easier, so +1.
As said, Iâm not very familiar with AWS lambda functions, so please see this more as a question, but is the deploy plugin a typical way for deploying lambda functions? I understand that the plugin provides a minimal setup for the lambda to actually work but arenât those normally part of a much bigger project and not much use on their own?
Hello Felix,
You're correct. In professional or large-scale projects, we will use Infrastructure as Code tools such as AWS CloudFormation, SAM, the Serverless Framework, Terraform, Pulumni to write a description of the infrastructure we want to deploy, and these tools will create it for us. These tools take care of the order of dependencies, the updates, and eventually the disposal of the infrastructure.
This allows your infrastructure to be version-controlled and repeatable. You can deploy the exact same infrastructure across multiple environments (dev, staging, test, prod) or regions.
However, the goal of these new plugins is not to address these professional development scenarios - they are already well covered by a variety of tools.
We want to simplify the life of newcomers. Think about the typical iOS developers that venture into server-side programming. Learning AWS is complex. We want to give them familiar tools (a Swift plugin) to help them get started as easily as possible, without installing and configuring other tools that they know.
Once they're familiar with the development and basic deployment of Lambda functions, it will be time to dive deeper and learn the art of deploying code in the cloud.
We just finished a (hopefully) comprehensive document about this. Check it out and share your feedback.
I really like this idea, we need to lower the barrier of entry to deploy simple, well-established types of services like Lambdas in Swift without first spending weeks learning the details of how all the AWS services work. I agree with Seb, the target audience wouldn't be the people who are already comfortable with deploying Lambdas from scratch, it'd be for those who write their first Lambda in Swift and want to deploy it in minutes. So a big +1 from me.
Would the deploy plugin have any configuration options to use API gateway and not just the function URL? Or really just some method to have authentication bundled in?
Thank you, Jeff.
I thought about that too. I think we have to draw a line between a basic deployment to help users to get started quickly with a basic configuration and a real project type of deployment.
Deploying a Lambda with an API Gateway is not easy at the API level (despite being just one line in a SAM template).
At the API level, we need at least these 6 steps (some of them include 2, sometimes 3 API calls):
A plugin can do that but will make a series of opinionated choices that are likely to be challenged by users. I mean, as soon as the user exits the basic learning zone, there is a high chance that they will want to somehow customize the API Gateway configuration, and the defaults provided by the plugin will not suffice.
I want to avoid reinventing the wheel. Users have full control when they use the AWS CLI (not recommended in this scenario), AWS SAM, or the AWS CDK (without even considering the many high-quality non-AWS options such as the Serverless Framework, Terraform, or Pulumi).
My initial thought was to keep the plugin with the minimum (raw Lambda function and Lambda exposed as a URL) and leave the rest to existing tools.
The Swift AWS Lambda project is full of more advanced examples and has a comprehensive deployment guide to explain options for real-life scenarios.
My question for the community: where is that line between easy getting started and flexibility for real-life deployment ?
That all makes sense! If thereâs documentation pointing to how to âmigrateâ from having the plugin deploy to the world of IAC, that would also go a long way for progressive disclosure here. Maybe even just a section in the docs on âwhat you canât do with thisâ and pointing to the various tools that can do that.
As you rightly mention, thereâs almost too many options on how to set things up without this!
That's a great idea.
Maybe we should add an --export option to the plugin. That option will "export" the default settings the plugin uses in a SAM template. Developers would receive a starting point to build on and start their IaC journey.
swift package lambda-deploy --export
I'm going to shamelessly throw my project into the ring but bear with me:
The proposal largely covers why you don't want to go in a direction like this, but my fear is a lambda packager simply does not do enough towards building real apps on AWS. Okay so the plugin gets a lambda deployed, but next you want to hook it up to a queue, an API gateway, a custom domain, put in a VPC, all these things. Sure you can special case something like APIG but whatever then configuration options grow because you will never support everything someone wants off the bat.
Something like my swift-cloud SDK goes way beyond what a simple plugin can do, and allows you to be highly productive across many AWS services, linking them together with best practices around permissions and roles, VPCs etc. Here's a video showing how we easily deployed Lambdas, added queues, etc. as we needed them: [Video] Deploying an Email System with SwiftCloud @ SwiftToolkit.dev
I'm not saying I've built the best thing but I think it's the direction that we need to go if we really want to make deploying Swift services productive for the end user.
Love it! I've recently been experimenting with V2 and (having only used the 0.0.x series) loving the swift package archive command (as well as the rest of the new API). This looks to be a very nice enhancement of that functionality.
A few questions:
lambda-deploy get the "function name" parameter for CreateFunction?--create flag to lambda-deploy to confirm the user actually wants to create a new lambda, rather than update an existing one to ward off typo issues?lambda-build with a Swift SDK require the user to pre-install the SDK with a specific name, or can the plugin do that?lambda-build --cross-compile defaults to docker? I would think that the goal of not having too many dependencies is better served by not requiring users to install docker by default.lambda-build / lambda-deploy support compiling for lambdas on different architectures than the current machine? E.g. if I use an x86 machine for development can I build + deploy to ARM or vice-versa?lambda-build has an --output-directory flag, does lambda-deploy need an --input-directory flag to match? Or how does lambda-deploy pick up the output of lambda-build.Hello @AndrewBarba
Thank you for your feedback. I love your Swift Cloud project. This is Infrastructure as code for Swift developers. Thank you for having created this project and for maintaining it.
I think the proposed lambda-deploy plugin and your project do not compete. Instead, they complement each other.
Following Swift's principles of progressive disclosure, the lambda-deploy plugin allows Swift developers to easily get started with a first deployment. This plugin will allow developers to have quick gratification and satisfaction to have a first function deployed.
As soon as developers become more serious about their deployment, they will discover Infrastructure as code. This is another learning step for developers. They will have multiple options to deploy their code with IaC tools, Swift-Cloud being one super interesting and appealing for Swift developers.
Furthermore, one of the design objectives of the Swift AWS Lambda runtime project is to minimize dependencies on other libraries. We currently depend on swift-nio and swift-logging and don't want to add any other dependencies to get started (except an AWS account and a pair of access key and secret access key).
Exploring Swift-Cloud is high on my to-do list for this year. I think, as you wrote, this project goes in the right direction to bring IaC to Swift developers.
Hello @willft
The current archive plugin takes the ZIP name from the target executable in Package.swift.
We were planning to use the same approach.
I love the idea of avoiding typos and at the same type I would like to keep the initial path as clean and short as possible.
What about an --update flag instead of a --create flag to keep the initial steps as clean and short as possible
That's an area I need to explore. My initial thinking is to ask the plugin to install it automatically if not present.
My plan is to start with docker only because this code already exists. Then add the Swift SDK support (musl and Amazon Linux 2) as the technology gets more robust and we clarify the installation / dependency question.
There will be always one dependency for cross compiling. Either docker, The Linux SDK (musl) or an (non existing at the moment) Amazon Linux custom SDK.
I proposed to keep docker by default to reuse the existing code of the archive plugin and keep the default behavior as similar as the existing archive plugin
Good point that I kept deliberately out-of-scope at the moment.
I was focusing on the developer experience for Swift developers new to AWS. These are likely to develop from a macOS Arm laptop.
I should at minima add a note in the proposal saying this debate is not settled yet.
What do you think would be the correct default ? And the required options ?
Good point, I will need to clarify that.
I would like the default to work out of the box, while giving user the possibility to change them.
My initial thinking was to reuse the default proposed by the archive plugin.
Thank you so much for your review and your feedback.
Ah, that makes good sense.
The reason I suggested the --create rather than --update is I believe people will update more than create as well as the fact that updating a function which doesn't exist won't work but creating it will.
However, now that I understand the function name is coming from Package.swift, I wonder if this is even necessary? Since the function name isn't something that needs re-typing with every CLI invocation it seems much less likely to be typo'd.
I love the idea of asking to install it automatically. It totally makes sense to start with docker since that's more fleshed out. I still hope we can switch the default one day because I think "make sure you install docker" is a heavier-weight dependency for a starter project than "if you don't have the SDK we'll do it for you (with prompting)."
I also would guess the SDK will be faster than starting a whole VM, but I won't hang my hat on that until it's available to test.
Yeah that makes perfect sense. I think it's useful to add eventually if we imagine teams >1 using this, or eventually using these in CD pipelines.
I haven't thought a tremendous amount about this, but I'll give it a shot:
--arch <value> flag where value is arm64 or x86_64 (matching the CreateFunction values) to lambda-build and lambda-deploy.x86_64) and just default to the current machine's architecture. If that's not feasible then I think there's good arguments either way.Yeah the current archive has great default behavior.