Visual regression testing with screenshots

A visual regression test screenshots your pages, compares each one with an approved baseline, and fails when too many pixels changed. That catches what unit tests miss: a CSS change that breaks the footer, a font that stops loading, a banner that pushes the layout down. The script below does it in about 70 lines of Node.js with pixelmatch, and gets the screenshots from an API, so there is no browser to install and every run captures with the same Chrome, fonts and viewport.

How it works

1. Capture: each page in a list is screenshotted (full page, desktop and phone sizes).
2. Baseline: the first run, or a run with --update, saves the screenshots to baseline/. Commit that folder.
3. Compare: later runs diff each new screenshot against its baseline with pixelmatch, which ignores anti-aliasing noise.
4. Report: above the limit (0.1% of pixels by default), the script writes diff/<name>.png (changes in red) plus the new screenshot, and exits with code 1 so CI fails.

1. Install

Node 18 or newer (for the built-in fetch), then the two image libraries:

npm install pixelmatch pngjs

Get an API key with 25 free renders a month at /pricing (no signup) and export it as SHOTMINT_KEY. A comparison of two pages at two sizes uses 4 renders per run.

2. The script

Save as visual-check.mjs (it uses top-level await, so keep the .mjs extension). Edit PAGES to list your pages; each entry can set its own width, height, mobile and hide.

// visual-check.mjs: screenshot each page, compare it with the saved baseline,
// write a diff image, and exit 1 if any page changed more than the limit.
// Usage: node visual-check.mjs            compare against baselines
//        node visual-check.mjs --update   save the current screenshots as baselines
import { mkdir, readFile, writeFile } from "node:fs/promises";
import { existsSync } from "node:fs";
import { PNG } from "pngjs";
import pixelmatch from "pixelmatch";

const SITE = process.env.SITE_URL || "https://example.com";
const PAGES = [
  { name: "home", path: "/" },
  { name: "home-mobile", path: "/", width: 390, height: 844, mobile: true },
];
const MAX_DIFF_PERCENT = 0.1; // fail above 0.1% changed pixels
const UPDATE = process.argv.includes("--update");

async function screenshot(page) {
  const params = new URLSearchParams({
    url: SITE + page.path,
    full_page: "true",
    width: String(page.width || 1280),
    height: String(page.height || 800),
    block_ads: "true",
    // hide things that change on every load (clocks, carousels, chat widgets)
    hide_selectors: page.hide || "",
  });
  if (page.mobile) params.set("mobile", "true");
  const res = await fetch("https://shotmint.creasework.com/v1/screenshot?" + params, {
    headers: { "X-API-Key": process.env.SHOTMINT_KEY },
  });
  if (!res.ok) throw new Error(`${page.name}: ${res.status} ${await res.text()}`);
  return PNG.sync.read(Buffer.from(await res.arrayBuffer()));
}

// Full-page heights can differ; pad both images to the same size so the
// extra area counts as changed instead of crashing the comparison.
function pad(img, width, height) {
  if (img.width === width && img.height === height) return img;
  const out = new PNG({ width, height }); // transparent
  PNG.bitblt(img, out, 0, 0, img.width, img.height, 0, 0);
  return out;
}

await mkdir("baseline", { recursive: true });
await mkdir("diff", { recursive: true });
let failed = 0;

for (const page of PAGES) {
  const current = await screenshot(page);
  const file = `baseline/${page.name}.png`;
  if (UPDATE || !existsSync(file)) {
    await writeFile(file, PNG.sync.write(current));
    console.log(`${page.name}: baseline saved`);
    continue;
  }
  const base = PNG.sync.read(await readFile(file));
  const width = Math.max(base.width, current.width);
  const height = Math.max(base.height, current.height);
  const a = pad(base, width, height), b = pad(current, width, height);
  const diff = new PNG({ width, height });
  const changed = pixelmatch(a.data, b.data, diff.data, width, height, { threshold: 0.1 });
  const percent = (changed / (width * height)) * 100;
  const sizeNote = base.height !== current.height ? ` (height ${base.height} -> ${current.height})` : "";
  if (percent > MAX_DIFF_PERCENT) {
    failed++;
    await writeFile(`diff/${page.name}.png`, PNG.sync.write(diff));
    await writeFile(`diff/${page.name}-current.png`, PNG.sync.write(current));
    console.log(`FAIL ${page.name}: ${percent.toFixed(3)}% changed${sizeNote}, see diff/${page.name}.png`);
  } else {
    console.log(`ok   ${page.name}: ${percent.toFixed(3)}% changed${sizeNote}`);
  }
}
process.exit(failed ? 1 : 0);

Run it twice: the first run saves baselines, the second compares.

export SHOTMINT_KEY=your_key
node visual-check.mjs            # home: baseline saved ...
node visual-check.mjs            # ok   home: 0.000% changed ...
node visual-check.mjs --update   # accept the current look as the new baseline

What a failure looks like

To test the script, we hid every paragraph on the page between two runs. It printed:

FAIL home: 6.922% changed (height 843 -> 800), see diff/home.png
FAIL home-mobile: 15.247% changed (height 943 -> 844), see diff/home-mobile.png

and exited with code 1. The diff image shows the missing text in red over a faded copy of the baseline, and the area that no longer exists (the page got shorter) as a solid red band. Full-page heights change whenever content does, so the script pads both images to the same size rather than refusing to compare them. To look at two screenshots side by side, with a swipe slider and onion skin, drop them into the free image comparison tool.

3. Run it in CI (GitHub Actions)

Commit visual-check.mjs and the baseline/ folder, add the key as a repository secret named SHOTMINT_KEY (Settings → Secrets and variables → Actions), and save this as .github/workflows/visual.yml. Point SITE_URL at a deployment the API can reach: production, staging or a preview URL (not localhost, because the screenshots are taken from our servers).

name: Visual regression
on:
  pull_request:
  workflow_dispatch:

jobs:
  visual:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 22
      - run: npm install pixelmatch pngjs
      - name: Compare screenshots with baselines
        env:
          SHOTMINT_KEY: ${{ secrets.SHOTMINT_KEY }}
          SITE_URL: https://staging.example.com
        run: node visual-check.mjs
      - name: Upload diffs
        if: failure()
        uses: actions/upload-artifact@v4
        with:
          name: visual-diffs
          path: diff/

When a pull request changes the look of a page, the job fails and the diffs are attached to the run as an artifact. If the change was intended, run node visual-check.mjs --update locally and commit the new baselines. For preview deployments and scheduled screenshots, see screenshots in GitHub Actions.

Keeping it from being flaky

Cause of false failures	Fix
Clocks, carousels, rotating testimonials Anything that changes on every load	List their selectors in hide, e.g. hide: ".carousel, time"
Cookie banners and ads	block_ads is already on in the script
Animations caught mid-way	Animations and transitions are frozen in full-page captures; add delay: "1000" to the params for slow JS
Live data (prices, counts, feeds)	Point SITE_URL at staging with fixed seed data, or hide those elements
Font and rendering differences between machines	None needed: every capture runs on the same server Chrome
Tiny real changes you don't care about	Raise MAX_DIFF_PERCENT or pixelmatch's threshold (0 to 1, default 0.1)

Compared with Playwright or BackstopJS snapshots

Playwright's toHaveScreenshot() and BackstopJS do the same comparison with a local browser. They are the better choice when you already run a browser in CI, need to log in or click through a flow before capturing, or test a site that isn't reachable from the internet. Their main pain is that a baseline made on a Mac doesn't match a Linux CI runner (fonts and anti-aliasing differ), so teams end up generating baselines in Docker. An API capture avoids that because the browser is always the same, and it keeps a 300 MB browser out of your CI image. The trade-off: the page must be publicly reachable, and each capture is a paid render after the free 25 a month (from €5 for 2,000, credits never expire).

What we tested

We ran the script above, unchanged except for the key, against the live API on Node 22 with pixelmatch 8 and pngjs 7: first run saved both baselines, the second reported 0.000% and exited 0, a run with all paragraphs hidden failed with the output shown above and wrote the diff images, and a full-page Wikipedia article compared at 0.000% (desktop) and 0.011% (mobile), well under the limit. The GitHub Actions workflow runs the same command; its GitHub-specific steps (checkout, secrets, artifact upload) were not run here.

More guides: Use cases

• Monitor a website for visual changes (Node.js + GitHub Actions)
• Bulk website screenshots from a URL list (Python, Node.js, API)
• Website thumbnails for link directories, bookmarks & portfolios
• Dynamic OG images for static sites: Next.js, Astro, Hugo, Jekyll
• Next.js OG Image: next/og CSS limits vs a real browser
• Web page to Markdown for LLMs and RAG: measured, with code
• LangChain web page loader: WebBaseLoader vs content API, measured
• ChatGPT custom GPT Action: screenshots and page reading

All guides: guides · integrations · comparisons