Fixing my deployment mistakes

Read the original here

I recently published a blog post showing how I use Github Actions to deploy apps to Vercel from my monorepo. That solution was very basic but it worked. However, I expected it to work for me a little longer than it did 🤦‍♂️

So in this post I’m going to work on fixing a few issues that have cropped up. I’m going to structure this blog post a little differently than normal. Instead of fixing the problems, perfecting my solution and writing about it – I’m going to write this post while I work and publish it with very little editing (probably none because I’m quite lazy).

I think this’ll be an interesting exercise. It’ll force me to think out loud and then get to share my process with everyone.

So without further ado, let’s break down the problems.

Yesterday, all my troubles seemed so far away, and I noticed that my builds were failing. Upon closer inspection the reason was that I’m hitting my Vercel limits.

Deploying vntg/jxd
Error: Resource is limited - try again in 6 hours (more than 100, code: "api-deployments-free-per-day").

Enter fullscreen mode
Exit fullscreen mode

Also, with the addition of a bunch more projects into the monorepo, the build time has now crept up towards the 10min mark. That is kinda my personal patience limit…

So those are the two main things I want to solve today.

In the last post I talked about two potential solutions to this problem:

Parallelise the builds with Github’s matrices strategy
Use ts-ignore to only build and deploy projects that have actually changed

I could probably get away with just using ts-ignore for now but think doing both would be beneficial for a few reasons:

Even greater performance
Simplify my code so that everything deployment related lives within the workflow

Before I start on those two improvements I’d like to recap the existing deploy.ts file and look for some improvements before transitioning away. I think it’ll end up being quicker if I 1) refresh the solution in my mind and 2) migrate a simpler solution.

// deploy.ts
import util from "util";
import { exec } from "child_process"; const asyncExec = util.promisify(exec); const projects = [ { path: "apps/bigjournal/web", // Is this needed? projectId: "", }, // ...
]; async function run() { const token = process.env.VERCEL_TOKEN; if (!token) { // Unnecessary in GH actions  throw new Error("missing vercel token"); } const orgId = process.env.VERCEL_ORG_ID; if (!orgId) { throw new Error("missing org id"); } for (const { path, projectId } of projects) { // Matrix console.log(`deploying ${path}`); console.log(`pulling vercel settings`); await asyncExec( `VERCEL_PROJECT_ID=${projectId} vercel pull --yes --environment=production --token=${token}` ); console.log(`running vercel build`); await asyncExec( `VERCEL_PROJECT_ID=${projectId} vercel build --prod --token=${token} ${path}` // Path is only used here! ); console.log(`deploying to vercel`); await asyncExec( `VERCEL_PROJECT_ID=${projectId} vercel deploy --prod --prebuilt --token=${token}` ); console.log(`✅ deployed ${path}`); }
} run() .then(() => console.log("✅ complete")) .catch((e) => { console.error(e); process.exit(1); });

Enter fullscreen mode
Exit fullscreen mode

My first thought is do I even need the path in the configuration? Since we have to run vercel from the root, and the path is stored in the Vercel settings that are pulled… is it needed?

So I’m going to test this locally with my Big Journal project:

export VERCEL_ORG_ID="..." VERCEL_PROJECT_ID="..."
vercel pull --yes --environment=production
vercel deploy --prod

Enter fullscreen mode
Exit fullscreen mode

✅ that has worked perfectly! So now I know that path isn’t required! I will quickly update the deploy script and push (just to be extra safe).

Something I haven’t worked with before and so I’ll need to refer to the documentation.

jobs: example_matrix: strategy: matrix: version: [10, 12, 14] os: [ubuntu-latest, windows-latest]

Enter fullscreen mode
Exit fullscreen mode

Looking at this example it seems simple enough to split out deploy into a separate job, that depends on the CI job, and have a matrix of project IDs.

After a few changes this is the workflow I am left with:

name: ci/cd on: push: branches: ["main"] env: DATABASE_URL: ${{secrets.DATABASE_URL}} TURBO_TOKEN: ${{ secrets.VERCEL_TOKEN }} TURBO_TEAM: vntg jobs: ci: name: ci timeout-minutes: 15 runs-on: ubuntu-latest steps: - name: checkout uses: actions/checkout@v4 with: fetch-depth: 2 - uses: pnpm/action-setup@v3 with: version: 8 - name: setup node uses: actions/setup-node@v4 with: node-version: 20 cache: 'pnpm' - name: deps run: pnpm install - name: build run: pnpm build cd: name: cd timeout-minutes: 15 runs-on: ubuntu-latest needs: [ci] strategy: matrix: project: - ... # big journal env: VERCEL_ORG_ID: ${{ secrets.VERCEL_ORG_ID }} VERCEL_TOKEN: ${{ secrets.VERCEL_TOKEN }} VERCEL_PROJECT_ID: ${{matrix.project}} steps: - name: checkout uses: actions/checkout@v4 with: fetch-depth: 2 - uses: pnpm/action-setup@v3 with: version: 8 - name: setup node uses: actions/setup-node@v4 with: node-version: 20 cache: 'pnpm' - name: install vercel cli run: pnpm install --global vercel@latest - name: pull run: vercel pull --yes --environment=production --token=${VERCEL_TOKEN} - name: build & deploy run: vercel deploy --prod --token=${VERCEL_TOKEN}

Enter fullscreen mode
Exit fullscreen mode

I also removed the old deploy.ts script and any dependencies it had before pushing and eagerly awaiting the results.

Already this gives a huge performance improvement of ~5x 🙌 but doesn’t help with the biggest problem of hitting our Vercel build limits. For that let’s look into turbo-ignore.

Looking at the docs, it seems fairly simple:

npx turbo-ignore workspace --task=build

Enter fullscreen mode
Exit fullscreen mode

This command will exit with 1 if the project needs to be rebuilt (based on the commit history).

I can handle this with continue-on-error and if within the workflow. First, I’ll also need to add the workspace value to the matrix using a map.

The final cd job looks like this:

cd: name: cd timeout-minutes: 15 runs-on: ubuntu-latest needs: [ci] strategy: matrix: project: - id: ... workspace: "@bigjournal/web" env: VERCEL_ORG_ID: ${{ secrets.VERCEL_ORG_ID }} VERCEL_TOKEN: ${{ secrets.VERCEL_TOKEN }} VERCEL_PROJECT_ID: ${{matrix.project.id}} steps: - name: checkout uses: actions/checkout@v4 with: fetch-depth: 2 - uses: pnpm/action-setup@v3 with: version: 8 - name: setup node uses: actions/setup-node@v4 with: node-version: 20 cache: 'pnpm' - name: install vercel cli run: pnpm install --global vercel@latest - id: check name: check run: npx turbo-ignore ${{ matrix.project.workspace }} --task=build continue-on-error: true - name: pull if: steps.check.outcome != 'success' run: vercel pull --yes --environment=production --token=${VERCEL_TOKEN} - name: build & deploy if: steps.check.outcome != 'success' run: vercel deploy --prod --token=${VERCEL_TOKEN}

Enter fullscreen mode
Exit fullscreen mode

So I’ve now implemented both fixes and I’m much happier with the overall result. Time will tell if this fixes all deployment issues but right now I’m confident (although I said that last time).

Fixing my deployment mistakes

Like this:

Related

Discover more from Coursity

Leave a Comment Cancel Reply