Photo

Intro to esbuild

Dec 15, 2023

If you build Node.js applications for serverless environments like AWS Lambda, bundling your code with esbuild can greatly improve performance by shrinking the size of your deployed code. This article will introduce you to esbuild and show you how to integrate it into your serverless application deployment process.

Target audience

This introductory article is intended for server-side JavaScript and TypeScript developers that are new to bundling and have never used esbuild. You should already be familiar with Node.js. This article uses Node.js v20.x, which is the LTS version at the time of writing. If you are new to Node.js, take a minute to install it using nvm or nvm-windows, then come back and you should be able to follow along with the examples.

What is bundling?

The idea behind bundling is pretty simple. A bundler translates your application’s source code into JavaScript and combines it into one or more output files. The process is very similar to what a compiler does for a compiled language like C, except the output is JavaScript instead of machine code.

Bundlers were originally designed to produce code to run in the browser. Since browsers only support JavaScript, a bundler is needed if you want to use another language like TypeScript or JSX, or if you want to use newer JavaScript features that aren’t supported by all browsers.

In addition to simply translating and combining your code, bundlers can also:

  • Shrink or “minify” code by shortening variable names and performing other optimizations that make code size smaller
  • Bundle and minify other types of files like CSS and images
  • Fingerprint output files so they can be efficiently cached in the browser and intermediate caches (like a CDN)

Only recently has bundling become popular for server-side code, with more and more developers using serverless platforms like AWS Lambda. Before serverless, Node.js servers were long-running and had plenty of memory and disk space so minimising code size wasn’t as important.

What is esbuild?

esbuild is an extremely fast bundler that is perfect for bundling serverless applications. It is fully featured and supports TypeScript, JSX/TSX, CSS, and can be extended with plugins.

It’s worth pointing out that although it supports TypeScript, it doesn’t do type checking. It just translates TypeScript to JavaScript, so you still need to use TypeScript tools to check types.

Why bundle?

So why would you want to bundle your server side code? In a word: performance. Bundling your code can greatly improve the performance of your serverless application by reducing the size of your deployed code. This can lead to faster cold starts, and lower memory usage.

Why not bundle?

There are cases where bundling doesn’t make sense. If your Lambda function is very simple and uses small dependencies, bundling will be overkill.

There are also some limitations to bundling that we’ll get into later, especially when bundling your node_modules dependencies.

Setting up

Let’s dive in and see how esbuild works. We’ll start by setting up a simple project.

terminal
mkdir esbuild-example
cd esbuild-example
nvm use 20
npm init -y
npm install -D typescript esbuild @types/aws-lambda ts-node @tsconfig/node20

We’ve installed a few dependencies here:

  • typescript - the TypeScript compiler so we can do type checking later
  • esbuild - the bundler
  • @types/aws-lambda - TypeScript types for the AWS Lambda runtime
  • ts-node - a TypeScript runtime that can run TypeScript files directly
  • @tsconfig/node20 - a TypeScript configuration for targeting the Node.js 20 runtime

We’re using the @tsconfig/node20 preset to target the Node.js 20 runtime. This is a preset that we can use to extend our tsconfig.json file. See here for more information.

For now our tsconfig.json file is very simple:

tsconfig.json
{
  "extends": "@tsconfig/node20/tsconfig.json"
}

A simple example

For our example, let’s start with an SQS message handler. I’m not going to cover the details of how to set up SQS and deploy your function, since I want to focus on bundling, but checkout the resources at the end of the article if you want to dig into how to deploy something like this.

At its most basic, your lambda function is a handler which we can define in an index.ts file like this:

index.ts
import { SQSEvent } from "aws-lambda";

export async function handler(event: SQSEvent) {
  console.log("Event: ", event.Records[0].body);
}

A few things to note:

  • We’re using TypeScript - which isn’t natively supported by Node.js - so we need esbuild to translate it into JavaScript.
  • We’re using ESM module syntax becuse we’re using an import statement.

Now we’re ready to start digging into what esbuild can do for us. We’ll start by running it directly from the command line without any options:

terminal
> npx esbuild index.ts

"use strict";
export async function handler(event) {
  console.log("Event: ", event.Records[0].body);
}

esbuild has translated our TypeScript into JavaScript, stripping out the type information including SQSEvent and the import statement. It also added a "use strict"; statement at the top of the file, to enable Strict mode.

Let’s quickly look at some minified output by adding the --minify option:

terminal
> npx esbuild index.ts --minify

"use strict";export async function handler(o){console.log("Event: ",o.Records[0].body)}

It shrunk the code down by removing extra whitespace and shortening variable names. This is a very simple example, but you can see how this could make a big difference in the size of your code bundle.

Bundling

So far all we’ve done is “transpiled” our TypeScript into JavaScript. Now let’s talk about bundling multiple files together.

We can demonstrate bundling by adding another file to our project. We’ll add a simple utility library with a log function that logs out information about an SQS event:

lib/utils.ts
import { SQSEvent } from "aws-lambda";

export async function log(event: SQSEvent) {
  console.log("Message ID:", event.Records[0].messageId);
}

Now import it into our index.ts file, and add a call to handler() with a test event so we can test the handler later by running the bundle:

index.ts
import { SQSEvent } from "aws-lambda";
import { log } from "./utils";

export async function handler(event: SQSEvent) {
  log(event);
}

// Test the handler
handler({
  Records: [
    {
      awsRegion: "us-east-1",
      md5OfBody: "123",
      eventSource: "aws:sqs",
      eventSourceARN: "arn:aws:sqs:us-east-1:123456789012:my-queue",
      messageId: "123",
      body: "Hello world",
      receiptHandle: "456",
      attributes: {
        ApproximateReceiveCount: "1",
        SentTimestamp: "123456789",
        SenderId: "123456789",
        ApproximateFirstReceiveTimestamp: "123456789",
      },
      messageAttributes: {},
    },
  ],
});

If we run esbuild on this file it bundles both files into one:

terminal
> npx esbuild index.ts --bundle --platform=node --format=esm

// lib/utils.ts
async function log(event) {
  console.log("Message ID:", event.Records[0].messageId);
}

// index.ts
async function handler(event) {
  log(event);
}
handler({
  Records: [
    {
      awsRegion: "us-east-1",
      md5OfBody: "123",
      eventSource: "aws:sqs",
      eventSourceARN: "arn:aws:sqs:us-east-1:123456789012:my-queue",
      messageId: "123",
      body: "Hello world",
      receiptHandle: "456",
      attributes: {
        ApproximateReceiveCount: "1",
        SentTimestamp: "123456789",
        SenderId: "123456789",
        ApproximateFirstReceiveTimestamp: "123456789"
      },
      messageAttributes: {}
    }
  ]
});
export {
  handler
};

I added the --platform=node and --format=esm options to tell esbuild that we’re targeting Node.js and that we want to use ESM modules. If you don’t specify a platform, esbuild assumes you want to bundle for the browser and will use iife format, and if you don’t specify a format it will use cjs format.

Try running esbuild without the --format=esm option if you’d like to see the output. It’s a bit more verbose and includes some extra code to format module.exports.

CommonJS (CJS) vs. ECMAScript modules (ESM)?

We are going to focus on the newer, ESM format for a few reasons:

  • It’s the future of JavaScript modules
  • ESM modules lead to better “tree-shaking” which refers to removing unused code
  • They support code splitting and lazy loading

To use ESM modules in Node.js you just need to set “type”: “module” in your application’s package.json file, or by using the .mjs file extension. See here for more information.

That said you may want to stick with CommonJS for your use case, if you are using an older version of Node.js, or if you are using an older CommonJS library that doesn’t support ESM and can’t be bundled. We’ll look more closely at why some packages can’t be bundled later in this article.

Outputting to a file

We’d like to run our example code using node so we’ll now output it to a file. We’ll use the --outfile option to specify the output file:

terminal
> npx esbuild index.ts --bundle --platform=node --format=esm --outfile=dist/index.mjs

  dist/index.mjs  323b

 Done in 3ms

Notice that I’m using the .mjs extension. This will tell node that the file is an ESM module.

Next we run the resulting bundle, which will run the handler with the test event:

terminal
> node dist/index.mjs
Message ID: 123

Source maps

We want to minify our bundle, but we also want to be able to debug it, and figure out where errors are thrown from. To determine where an error was thrown we need the stack trace that Node.js generates, but the stack traces for minified code are not very useful.

To demonstrate this let’s update the log() function to throw an error:

lib/utils.ts
import { SQSEvent } from "aws-lambda";

export async function log(event: SQSEvent) {
  console.log("Message ID:", event.Records[0].messageId);
  throw new Error("Something bad happened");
}

Now run esbuild with the --minify option:

terminal
> npx esbuild index.ts --bundle --platform=node --format=esm --minify --outfile=dist/index.mjs

When you run the bundle you get a very unhelpful stack trace:

terminal
> node dist/index.mjs

async function t(e){throw console.log("Message ID:",e.Records[0].messageId),new Error("Something bad happened")}async function o(e){t(e)}o({Records:[{awsRegion:"us-east-1",md5OfBody:"123",eventSource:"aws:sqs",eventSourceARN:"arn:aws:sqs:us-east-1:123456789012:my-queue",messageId:"123",body:"Hello world",receiptHandle:"456",attributes:{ApproximateReceiveCount:"1",SentTimestamp:"123456789",SenderId:"123456789",ApproximateFirstReceiveTimestamp:"123456789"},messageAttributes:{}}]});export{o as handler};
                                                                            ^

Error: Something bad happened
    at t (file:///Users/rschick/esbuild-testing/example-1/dist/index.mjs:1:77)
    at o (file:///Users/rschick/esbuild-testing/example-1/dist/index.mjs:1:133)
    at file:///Users/rschick/esbuild-testing/example-1/dist/index.mjs:1:138
    at ModuleJob.run (node:internal/modules/esm/module_job:218:25)
    at async ModuleLoader.import (node:internal/modules/esm/loader:329:24)
    at async loadESM (node:internal/process/esm_loader:34:7)
    at async handleMainPromise (node:internal/modules/run_main:113:12)

The fix for this is to use a source map. A source map is a file that maps the minified code back to the original source code.

Let’s add the --sourcemap option to tell esbuild to generate a source map for the bundle:

terminal
> npx esbuild index.ts --bundle --platform=node --format=esm --minify --sourcemap --outfile=dist/index.mjs

  dist/index.mjs      541b
  dist/index.mjs.map  1.4kb

 Done in 5ms

Node doesn’t enable source maps by default, so we also need to add the --enable-source-maps option when we run the bundle:

terminal
> node --enable-source-maps dist/index.mjs

/Users/rschick/esbuild-testing/example-1/lib/utils.ts:5
  throw new Error("Something bad happened");
        ^

Error: Something bad happened
    at log (/Users/rschick/esbuild-testing/example-1/lib/utils.ts:5:9)
    at handler (/Users/rschick/esbuild-testing/example-1/index.ts:6:3)
    at <anonymous> (/Users/rschick/esbuild-testing/example-1/index.ts:10:1)
    at ModuleJob.run (node:internal/modules/esm/module_job:218:25)
    at async ModuleLoader.import (node:internal/modules/esm/loader:329:24)
    at async loadESM (node:internal/process/esm_loader:34:7)
    at async handleMainPromise (node:internal/modules/run_main:113:12)

Node.js v20.10.0

Now we have a useful stack trace that points to the original source code.

When you’re running in an environment like Lambda, you’ll need to enable source maps using the --enable-source-maps option, which you can set using the NODE_OPTIONS environment variable. Alternatively you could use an error tracking service like Sentry, which lets you upload your source map files and will then automatically translate stack traces.

Downsides of bundling

There are a few downsides to bundling that you should be aware of:

  • Adds a build step - Running esbuild adds an extra step to your CI/CD and local development processes. Luckily all the popular serverless development and deployment tools now have built-in support for bundling with esbuild, so setting it up is pretty easy.

  • Messes with file-relative paths - Any code in your application or dependencies that relies on file-relative paths will break. This includes code that uses __dirname or import.meta.url to get the location of the current file.

  • Can’t bundle some dependencies - Some dependencies can’t be bundled because they have binary dependencies, or because they use dynamic require() statements, and a variety of other reasons. This can require a lot of trial and error to figure out which dependencies can be bundled, and can result in errors that may not reveal themselves until they are in production.

Conclusion

In this article we’ve covered the basics of esbuild and how to use it to bundle your serverless application. We’ve seen how bundling and minification can improve performance by shrinking the size of your deployed code, and how to use source maps to debug your minified code.

References and further reading