Bulk website screenshots from a list of URLs

You have a spreadsheet of 500 URLs and need a screenshot of each one. Below are three complete scripts that read urls.csv (one url column), save one PNG per site, keep going when a site fails and write the failures to failed.txt so you can rerun just those. We ran all three on the same list, including a long Wikipedia article and a domain that doesn't exist.

Python + Playwright (async, 4 at a time)

import asyncio, csv, re
from pathlib import Path
from playwright.async_api import async_playwright

CONCURRENCY = 4
out = Path("shots"); out.mkdir(exist_ok=True)

def filename(url):
    return re.sub(r"[^a-zA-Z0-9]+", "-", url.split("://", 1)[-1]).strip("-")[:100] + ".png"

async def shoot(browser, sem, url, failed):
    async with sem:
        page = await browser.new_page(viewport={"width": 1280, "height": 800})
        try:
            await page.goto(url, wait_until="load", timeout=30000)
            await page.screenshot(path=out / filename(url), timeout=15000)
            print("ok  ", url)
        except Exception as e:
            print("FAIL", url, type(e).__name__)
            failed.append(url)
        finally:
            await page.close()

async def main():
    with open("urls.csv", newline="") as f:
        urls = [row["url"].strip() for row in csv.DictReader(f) if row["url"].strip()]
    sem, failed = asyncio.Semaphore(CONCURRENCY), []
    async with async_playwright() as p:
        browser = await p.chromium.launch()
        await asyncio.gather(*(shoot(browser, sem, u, failed) for u in urls))
        await browser.close()
    Path("failed.txt").write_text("\n".join(failed))
    print(f"{len(urls) - len(failed)} saved, {len(failed)} failed")

asyncio.run(main())

One browser and a fresh page per URL is the pattern that scales: launching a browser per URL is slow, and reusing a single page leaks state between sites. The Semaphore caps how many pages are open at once. Start at the number of CPU cores and lower it if pages time out, because each Chromium tab can use 100–300 MB of RAM. Install with pip install playwright and playwright install chromium. On Ubuntu 23.10+ or in Docker, launch with args=["--no-sandbox"] if Chrome refuses to start.

Node.js + Puppeteer (worker pool)

import puppeteer from "puppeteer";
import { readFileSync, mkdirSync, writeFileSync } from "node:fs";

const CONCURRENCY = 4;
const urls = readFileSync("urls.csv", "utf8").split("\n").slice(1).map(s => s.trim()).filter(Boolean);
const filename = url => url.split("://").pop().replace(/[^a-zA-Z0-9]+/g, "-").replace(/^-|-$/g, "").slice(0, 100) + ".png";
mkdirSync("shots", { recursive: true });

const browser = await puppeteer.launch({ args: ["--no-sandbox"] });
const queue = [...urls], failed = [];

async function worker() {
  const page = await browser.newPage();
  await page.setViewport({ width: 1280, height: 800 });
  for (let url; (url = queue.shift()); ) {
    try {
      await page.goto(url, { waitUntil: "networkidle2", timeout: 30000 });
      await page.screenshot({ path: "shots/" + filename(url) });
      console.log("ok  ", url);
    } catch (e) {
      console.log("FAIL", url, e.message.split("\n")[0]);
      failed.push(url);
    }
  }
  await page.close();
}

await Promise.all(Array.from({ length: CONCURRENCY }, worker));
await browser.close();
writeFileSync("failed.txt", failed.join("\n"));
console.log(`${urls.length - failed.length} saved, ${failed.length} failed`);

Save it as bulk.mjs (it uses top-level await). Each of the 4 workers keeps one tab and pulls the next URL from a shared queue, so a slow site only holds up one worker. This simple CSV split assumes one URL per line with no quoted commas; use a CSV parser for real exports. A failed site shows its reason, for example net::ERR_NAME_NOT_RESOLVED.

Things that go wrong at a few hundred URLs

• Cookie banners on every other screenshot. Inject CSS to hide the common ones, or block consent scripts with request interception (page.route in Playwright, page.setRequestInterception in Puppeteer).
• One site hangs the whole run. Always set a goto timeout and a screenshot timeout. With networkidle, pages that keep a connection open (chat widgets, analytics beacons) never go idle, so fall back to load plus a short delay.
• Memory grows until the box swaps. Close every page in a finally. For lists in the thousands, restart the browser every few hundred URLs.
• Duplicate or messy rows. Strip whitespace, skip empty rows, add https:// to bare domains and dedupe before you start. The filename function above turns each URL into a safe, unique-enough name.
• Sites block datacenter IPs. Some sites serve a bot check to cloud servers. Your screenshot then shows a challenge page, not an error, so look at a sample of the output.
• Resuming. Point the script at failed.txt (add a url header line) to retry only the failures.

The same batch through an API (no browser to run)

If you'd rather not run Chromium on your machine, send each URL to a screenshot API. This version uses Shotmint, keeps 3 requests in flight, retries when the server is busy (503) or returns a transient 5xx, stops cleanly when you run out of credits (402) and records URLs that can't be captured:

import csv, os, re, time
from concurrent.futures import ThreadPoolExecutor
from pathlib import Path
import requests

API = "https://shotmint.creasework.com/v1/screenshot"
KEY = os.environ["SHOTMINT_KEY"]
out = Path("shots"); out.mkdir(exist_ok=True)

def filename(url):
    return re.sub(r"[^a-zA-Z0-9]+", "-", url.split("://", 1)[-1]).strip("-")[:100] + ".png"

def shoot(url):
    for attempt in range(4):
        r = requests.get(API, params={"url": url, "width": 1280, "height": 800},
                         headers={"X-API-Key": KEY}, timeout=90)
        if r.ok:
            (out / filename(url)).write_bytes(r.content)
            return url, "ok"
        if r.status_code == 402:
            raise SystemExit("Out of credits: " + r.json()["error"])
        if r.status_code in (429, 500, 502, 503, 504):
            time.sleep(2 ** attempt)   # busy or transient: back off and retry
            continue
        return url, f"{r.status_code} {r.json().get('error', '')}"   # bad URL, blocked site...
    return url, "gave up after retries"

with open("urls.csv", newline="") as f:
    urls = [row["url"].strip() for row in csv.DictReader(f) if row["url"].strip()]

with ThreadPoolExecutor(max_workers=3) as pool:
    results = list(pool.map(shoot, urls))

for url, status in results:
    print(status.ljust(6), url)
failed = [u for u, s in results if s != "ok"]
Path("failed.txt").write_text("\n".join(failed))
print(f"{len(urls) - len(failed)} saved, {len(failed)} failed")

The four real sites came back as 1280×800 PNGs, and the dead domain returned 400 URL host is not reachable or not allowed. Failed renders are not charged, so that run used 4 credits. Add "full_page": "true" for the whole page (up to 16,000px tall), "block_ads": "true" to drop common ads and cookie banners, "format": "jpeg" for smaller files, or "cache_ttl": 86400 so rerunning the same list within a day costs nothing.

Approach	Best for
Playwright / Puppeteer script Free, but you run Chromium and handle its memory, timeouts and blocks	One-off runs on your laptop, or when you need to log in or click before capturing.
Screenshot API 1 credit per screenshot: 100 free, then €5 for 2,000 (credits never expire)	Recurring jobs, servers without Chrome, serverless functions and lists you want done without babysitting.

More on single captures: Playwright screenshots, Puppeteer screenshots and Selenium in Python. For a single page with no code, use the free full-page screenshot tool. To show the results as thumbnails on a site, see website thumbnails for link directories.