Headless Browsers 101: What a Headless Browser Is, How It Works and How to Use One with a Proxy

Developer guide · Last reviewed 28 September 2026 · 12 min read

Headless browsers are real web browsers that run without a visible window. They load pages, run JavaScript, store cookies and render layouts exactly like the browser on your desktop, but they are driven by code instead of a person. This guide explains what a headless browser is, how headless Chrome and the major automation frameworks work, what teams use them for, their limits, and how to run a headless browser through a proxy, with working examples. For a deeper look at why most automation runs on Chromium, see our guide to web automation with Chromium.

What Is a Headless Browser?

A normal browser has two parts: the engine that fetches, parses and renders web pages, and the interface around it, meaning the window, tabs, address bar and buttons. A headless browser keeps the engine and drops the interface. It still builds the page, runs every script, applies CSS, handles cookies and storage, and calculates the layout. It simply never paints that result to a screen unless you ask for a screenshot.

Instead of a mouse and keyboard, a program controls it: “open this URL, wait for this element, click this button, read this text, save a PDF”. That makes them the standard tool for automated testing, for collecting data from JavaScript-heavy sites, and for any job that needs a real browser many times over on a server.

How Headless Browsers Work

1. The browser starts in headless mode on a server, a container or your laptop, with a flag such as --headless.
2. A control protocol opens. Chromium-based browsers expose the Chrome DevTools Protocol; the W3C WebDriver standard and its newer bidirectional version work across browsers.
3. A framework sends commands over that protocol from your code: navigate, wait, click, type, evaluate JavaScript, intercept requests.
4. The browser executes them exactly as it would for a user, including loading every script, image and API call the page makes.
5. Your code reads the results: the rendered DOM, network responses, screenshots, PDFs or performance timings.

The web standard also gives automated browsers a way to identify themselves: the navigator.webdriver property indicates whether the browser is controlled by automation, so that sites and scripts can tell. Automation is visible by design; honest automation respects that.

Headless or headed: a debugging tip

Every framework can show the window while you develop. Run the same script with the window visible, slow it down, and watch each step: you will see cookie banners, pop-ups, redirects and layout changes that explain most failures. Record a trace or video of failed runs on the server, too. Playwright’s trace viewer, for example, replays every action with the page state, network calls and console messages, which is far quicker than guessing from logs. When the script works reliably with the window visible, switch the flag back and deploy it.

Headless Chrome Today

For years, Chrome’s headless mode was a separate, simplified browser implementation that happened to ship inside the Chrome binary. Google has since unified the two: headless Chrome is now the real Chrome, creating but not displaying its windows, with all other functions available. You start it with the --headless flag.

# Linux
google-chrome --headless --screenshot=page.png https://example.com

# macOS
open -a "Google Chrome" --args --headless

# Windows
start chrome --headless

Since Chrome 132, the old headless implementation is available only as a standalone binary called chrome-headless-shell, downloadable from the Chrome for Testing dashboard. It starts faster and uses fewer resources, which some teams prefer for simple screenshot or PDF jobs, but it does not behave exactly like Chrome.

Headless Browser Frameworks: Playwright, Puppeteer and Selenium

Under the hood the choice of protocol matters less than it used to. Chromium browsers speak the Chrome DevTools Protocol, while the newer WebDriver BiDi standard gives all browsers a common, two-way channel for events such as network requests and console messages. Puppeteer, for example, can control Chrome or Firefox over either the DevTools Protocol or WebDriver BiDi.

All three can run the browser headless or with a visible window. Develop with the window visible so you can see what happens, then switch to headless on the server.

What Headless Browsers Are Used For

• End-to-end testing. Run user journeys, from sign-up to checkout, on every code change in continuous integration, across browsers.
• Collecting data from JavaScript-heavy sites. Single-page applications build their content in the browser; a scripted browser sees what a user sees. See our web scraping guide.
• Screenshots and PDFs. Generate invoices, reports, social previews or visual archives of pages at scale.
• Monitoring. Check that pages load, prices display and forms work, from several countries, every few minutes.
• Performance measurement. Record load times, layout shifts and network waterfalls in a controlled environment.
• Visual and localization QA. Compare how a page renders in different languages, currencies and regions.

Example: checking one page from three countries

A retailer wants to confirm that its product page shows the right price, currency and delivery message in Germany, France and Spain every hour. A script opens three isolated browser contexts, each through a proxy in one of those countries, loads the page, waits for the price element, reads the text and saves a screenshot. If the currency is wrong or the element is missing, it sends an alert with the screenshot attached. The same pattern works for checking ads, cookie consent banners, translations and legal notices market by market.

Headless Browser or HTTP Client?

A headless browser downloads every script, image and font and runs all of the page’s code. An HTTP client fetches one document. For the same page, the browser typically uses many times more CPU, memory and bandwidth, which matters when you pay for proxy traffic by the gigabyte.

• Use an HTTP client (Python requests, curl, Node fetch) when the data is in the HTML or in a JSON API the page calls. Look in your browser’s DevTools network tab first; the data is often there.
• Use a browser when content only appears after JavaScript runs, when you need to interact (click, scroll, log in to your own account), or when you need screenshots and PDFs.
• Combine them: use the browser once to discover the page’s API calls, then fetch those APIs directly.

How to Use a Headless Browser with a Proxy

Scripted browsers send the same kinds of requests as desktop browsers, so they meet the same limits: rate limits per IP, and content that changes by country. A proxy gives each browser, or each isolated browser context, its own IP and location. Playwright can set the proxy for the whole browser or for each context separately, with a username and password for HTTP(S) proxies.

from playwright.sync_api import sync_playwright

PROXY = {"server": "http://v2.proxyempire.io:5000",
         "username": "USERNAME", "password": "PASSWORD"}

with sync_playwright() as p:
    browser = p.chromium.launch(headless=True, proxy=PROXY)
    page = browser.new_page()
    # skip images and media to save proxy bandwidth
    page.route("**/*", lambda r: r.abort()
               if r.request.resource_type in ("image", "media", "font") else r.continue_())
    page.goto("https://example.com", wait_until="domcontentloaded")
    print(page.title())
    browser.close()

import puppeteer from "puppeteer";

const browser = await puppeteer.launch({
  headless: true,
  args: ["--proxy-server=http://v2.proxyempire.io:5000"],
});
const page = await browser.newPage();
await page.authenticate({ username: "USERNAME", password: "PASSWORD" });
await page.goto("https://example.com");
console.log(await page.title());
await browser.close();

Pick the rotation to match the job. A browser session makes dozens of requests per page, so for anything that needs continuity, like a multi-step form or a cart on your own test store, use a sticky session that keeps one IP. For many independent pages, a new IP per session spreads the load. Rotating residential proxies target country, city, ZIP, ISP and ASN at no extra charge, and mobile proxies show the mobile web as 4G and 5G users see it. Our sticky vs rotating guide explains the difference.

Limits and Good Practice

• Resources. Each browser instance needs hundreds of megabytes of memory. Reuse browsers, open new contexts instead of new browsers, and close pages you no longer need.
• Waiting. Wait for specific elements or network responses, not fixed sleeps. Frameworks with auto-waiting remove most flaky timing problems.
• Bandwidth. Block images, video and fonts when you only need text. On pay-per-GB proxies this can cut costs dramatically.
• Politeness. Keep concurrency per site low, respect robots.txt for sites you do not own, and collect only public data.
• Transparency. Automated browsers are identifiable by design. Use them for lawful testing, monitoring and public-data collection, not to deceive sites or other users.

When requests fail, our guide to proxy error codes explains what each browser error means, including ERR_TUNNEL_CONNECTION_FAILED.

Headless Browsers FAQ

References

1. Chrome for Developers: Chrome Headless modeThe –headless flag and the unified headless mode
2. Chrome for Developers: chrome-headless-shellThe old headless implementation as a standalone binary
3. Playwright: Network, HTTP proxyGlobal and per-context proxies with credentials
4. Playwright: BrowsersChromium, WebKit and Firefox support
5. Puppeteer documentationControlling Chrome and Firefox from Node.js
6. Selenium: Chrome specific functionalityChrome options, including headless
7. MDN: Navigator.webdriverHow a browser signals that automation controls it