Headless browsers look like a first-time visitor from nowhere, so European sites greet every capture with a consent dialog. We captured six news and dev sites at 1280×800 in headless Chromium (load event + 1.5 s): four came out covered by a consent dialog. Here is what removes them, measured on the same six sites.
| Site | Plain headless capture |
|---|---|
| Le Monde in-house consent wall, no known CMP | Covered: full-screen wall, scrolling locked |
| Corriere della Sera in-house dialog | Covered: centered dialog over a dimmed page |
| Der Spiegel Sourcepoint, iframe from the site's own domain | Covered: "Welcome!" consent-or-subscribe dialog |
| The Guardian Sourcepoint | Covered: "It's your choice" panel over the top half |
| BBC no consent dialog outside the EU/UK | Clean, but a sticky "Subscribe now" bar |
| Stack Overflow | Clean, with a survey notice bar |
Three things decide which fix works:
[id^="sp_message_container"] for Sourcepoint) cleared Spiegel and the Guardian.position: fixed box over the page that mentions cookies and has an accept or reject button. Finding that box under the viewport center and hiding it cleared Le Monde and Corriere.Clicking "Accept" is the other common approach. It works, but it's a consent decision made on behalf of nobody, it loads every ad and tracker the dialog was holding back (heavier, slower, more layout shift), and the button text changes with the language and the site's A/B tests. Hiding the dialog leaves the page as a visitor who hasn't decided yet sees it.
import { chromium } from "playwright";
// Consent-platform hosts: if their script never loads, their banner never appears
const CMP_HOSTS = /(^|\.)(cookielaw\.org|onetrust\.com|cookiebot\.com|consensu\.org|usercentrics\.eu|privacy-mgmt\.com|trustarc\.com|didomi\.io|cookiepro\.com|iubenda\.com|osano\.com|quantcast\.com)$/;
// Banners built in-house: hide the outermost fixed/sticky box under the viewport center
// (and the bottom edge) if it talks about cookies and has an accept/reject-style button
function hideConsentOverlays() {
const topic = /cookie|consent|gdpr|privacy|datenschutz|données|partners/i;
const action = /accept|agree|allow|reject|refuse|decline|accepter|akzeptieren|zustimmen|accetta/i;
for (const [fx, fy] of [[0.5, 0.5], [0.5, 0.1], [0.5, 0.9], [0.5, 0.98]]) {
let box = null;
for (let el = document.elementFromPoint(innerWidth * fx, innerHeight * fy); el && el !== document.body; el = el.parentElement) {
if (["fixed", "sticky"].includes(getComputedStyle(el).position)) box = el;
}
if (!box || !topic.test(box.innerText)) continue;
const buttons = [...box.querySelectorAll("button, a, [role=button]")];
if (buttons.some((b) => action.test(b.innerText))) box.style.setProperty("display", "none", "important");
}
}
const url = process.argv[2] ?? "https://www.lemonde.fr/";
const browser = await chromium.launch();
const page = await browser.newPage({ viewport: { width: 1280, height: 800 } });
await page.route("**/*", (route) =>
CMP_HOSTS.test(new URL(route.request().url()).hostname) ? route.abort() : route.continue());
await page.goto(url, { waitUntil: "load" });
await page.waitForTimeout(1500); // in-house banners often slide in after load
await page.evaluate(hideConsentOverlays);
// Known consent containers (Sourcepoint serves its iframe from the site's own domain,
// so blocking hosts misses it), and undo the scroll lock many banners set
await page.addStyleTag({ content: `
[id^="sp_message_container"], #onetrust-consent-sdk, #CybotCookiebotDialog,
#usercentrics-root, #didomi-host, .qc-cmp2-container, .fc-consent-root { display: none !important; }
html, body { overflow: auto !important; }` });
await page.screenshot({ path: "no-banners.png" });
await browser.close();Save it as no-banners.mjs and run node no-banners.mjs https://www.spiegel.de/. All four covered sites came out clean, and the two notice bars stayed, because neither talks about cookies. The text check is what stops the script from hiding a sticky header: a fixed box is only hidden if it mentions cookies or consent and contains an accept/reject-style button. Add words in your target languages to both patterns.
Puppeteer works the same way: page.setRequestInterception(true) plus request.abort() for the host check, then the same page.evaluate and page.addStyleTag calls.
Shotmint's block_ads=true does all three steps: it blocks ad, tracker and consent-platform hosts, hides the known consent containers, and after the page settles hides fixed overlays that ask for cookie consent. With it, all four covered sites above came out clean.
curl -o lemonde.png -H "X-API-Key: $SHOTMINT_KEY" \
"https://shotmint.creasework.com/v1/screenshot?url=https://www.lemonde.fr/&block_ads=true&delay=1500"For anything that isn't a consent dialog, add CSS selectors in hide_selectors (Node 18+, ESM):
import { writeFile } from "node:fs/promises";
const params = new URLSearchParams({
url: "https://www.bbc.com/",
block_ads: "true", // consent banners, ad and tracker requests
hide_selectors: "#subliminal-banner", // BBC's "Subscribe now" bar is not a consent banner
delay: "1500",
});
const res = await fetch(`https://shotmint.creasework.com/v1/screenshot?${params}`, {
headers: { "X-API-Key": process.env.SHOTMINT_KEY },
});
if (!res.ok) throw new Error(`${res.status}: ${await res.text()}`);
await writeFile("bbc.png", Buffer.from(await res.arrayBuffer()));delay waits after the load event (in milliseconds, max 10000), which gives late-loading banners time to appear before they're hidden. Both requests above were run as written against the live API. Options are listed in the docs; a free key covers testing.
No signup, no card. Get a key in one click, then buy credits only if you need more (from €5 for 2,000, never expire). From a script, an AI agent, Make or Zapier, POST /v1/keys/free (no body) returns a key as JSON (openapi.json, no-code recipes).
All guides: guides · integrations · comparisons