Reilly Wood

I’ve recently been playing around with some cloud development tooling by a startup named Pulumi, and I thought I’d write up my first impressions.

Pulumi can be summed up as “Infrastructure as Code – but really, we mean it this time.” Instead of mucking around with YAML files and proprietary syntax, you define the infrastructure you need in actual code (JavaScript or Python, with more languages coming later). Is your production environment slightly different from your test environment? No problem, that’s just an if statement (or a more complex abstraction) away.

It’s still very early days (as of this writing, Pulumi is on version 0.14.3), but the general approach shows a ton of promise.

Pulumi basics

I first heard about Pulumi from this ‘Tooling in the age of serverless computing’ talk by Donna Malayeri (of Pulumi, formerly of the Azure Functions team at Microsoft). It’s a great overview of serverless tooling options, and I learned a lot from the slides. Best of all, there are code samples for each tooling provider available on GitHub. Let’s take a look at the Pulumi ones.

You can work at a few different levels of abstraction with Pulumi. You can write a serverless function with complete control over every aspect of AWS Lambda and API Gateway. If that’s too nitty-gritty, you can use @pulumi/aws-serverless to simplify configuration of API Gateway+Swagger. If you like, you can even move your Lambda function’s code into the Pulumi script.

Then, at the highest level of abstraction, you can use @pulumi/cloud to define cloud components that aren’t tied to any particular cloud. This isn’t fully built out yet – it’s only been implemented for AWS at this time – but it’s by far the most exciting to me. Seriously, take a look at the example where Donna defines a NoSQL DB table and a serverless function that uses it in just 24 lines of code.

My motivation for using Pulumi

Most of my hobby coding in the last year and a bit has been done using AWS Lambda and Azure Functions. I’m using Serverless Framework for the most complex one, and it’s been OK. It provides a layer of abstraction around Lambda and other AWS resources, which is nice. What’s not so nice is that your infrastructure definition is moved to an ever-growing YAML file with unpleasant syntax once you go beyond very basic scenarios.

This is how to define an S3 bucket conditionally in Serverless Framework, i.e. if and only if an environment variable named S3_LOGGING_BUCKET is not empty:

  Conditions:
    CreateS3Bucket: 
      Fn::Not:
        - Fn::Equals:
          - ${file(env.${opt:stage}.yml):S3_LOGGING_BUCKET}
          - ""
  Resources:
    S3LoggingBucket:
      Type: "AWS::S3::Bucket"
      Condition: "CreateS3Bucket"
      Properties: ...

This is… unpleasant. It’s verbose, it’s not especially well documented, and I probably spent an hour of my life figuring out how to accomplish this conceptually trivial thing. When I first read about Pulumi, a lightbulb went off. “You mean I could have written that logic with an if statement instead? Sign me up!”

if(environment.S3_LOGGING_BUCKET != "") {
    // provision S3 bucket
}

My experience with Pulumi

Over a few weekends, I attempted to migrate my Serverless-managed Node function to Pulumi. I’ve run into a few barriers and so I haven’t been able to successfully finish that yet, but it is still very early days for Pulumi. Pulumi is progressing at an impressive rate and I’m fairly confident that I’ll be able to accomplish this in the not-so-far future.

I was hoping to use the highest level of abstraction, and so I started using Pulumi’s abstract API class (formerly HttpEndpoint). It all went well until I started trying to use other Node packages in my function. Turns out this is a scenario where 1) you need to do something a little nonstandard to import modules, 2) the recommended approach wasn’t documented. Pulumi team members were super responsive and helpful in explaining how to fix this.

I’m now able to import+use Node packages in my function, but then I encountered an issue with POST APIs. The short version is that standalone POST calls work, but modern browsers ‘preflight’ POST calls with an OPTIONS call and OPTIONS calls fail even when I try to handle them explicitly. I’m guessing this is a Pulumi bug/missing feature that will be implemented eventually.

Next steps

I’m considering ditching the higher-level @pulumi/cloud API abstraction and just using Pulumi to upload a Lambda deployment package. This will require some further investigation – I need to figure out the best way to put together a transpiled-from-TypeScript deployment package that only contains my production npm dependencies. I don’t think that will be especially difficult, but it needs to wait until I have another spare weekend afternoon. Perhaps the POST issue will get sorted out before that. :)