← Back to blog

A Browser Agent Reads Your Site Through Its Labels, Not Its Design

⭐ Featured

Browser Use, the open-source browser-agent company, published an explainer on October 18 about how an AI agent operates a web browser. Most of it is for people who build agents. One section is for everyone who runs a website: how the agent actually reads your page before it decides what to click.

The short answer is that it reads your page the way a screen reader does, not the way a person does. That changes what "a good website" means when the visitor is software choosing a vendor for someone.

The loop behind every browser agent

Browser Use describes an agent as three parts: a model that decides what to do, a runner that keeps the loop going and hands the model its tools, and a browser that holds the session, cookies and tabs. The loop itself is simple. Observe the page, pick an action, execute it, check the result, repeat until the goal is reached.

The important word is "observe". The model never sees your site directly. It sees whatever representation the tools hand it, and every decision after that depends on how good that representation is.

Two ways to see a page

There are two common methods, and many agents use both. The first is structured page data: the DOM, which lists the page's elements and text, or the accessibility tree, which describes each control by name, role and state. Browser Use's example is "a button named Search". The model can find that control without guessing where it sits on screen.

The second is a screenshot. A vision model reads the rendered page and picks coordinates to click. Browser Use is candid that this is the weaker path for small controls, because detail gets lost when the image is resized. Screenshots mostly matter for things that expose no useful structure, like a canvas-based interface.

So a pricing page whose plans are labelled text, with buttons that say what they do, is easy to read in either mode. A pricing page that is a designed image with an unlabelled "Go" button is readable only by squinting at pixels, and the agent squints badly.

What people actually send agents to do

The explainer includes rounded shares from Browser Use Cloud runs, taken from a snapshot on October 18, 2026. Extracting structured data was 14% of runs. Getting past a login was 18%. Applying for jobs was 6%, finding filings and reports 5%. Researching open questions, searching across sites and combining the findings into a sourced answer, was 6%. Booking and buying was 3%. About 15% went unclassified.

Two caveats before anyone builds a strategy on those numbers. They describe one company's cloud, which skews toward developers automating their own workflows rather than buyers shopping. And Browser Use says plainly that the figures count what people run, not how often it works. Research at 6% and buying at 3% is small. It is also a category that did not exist as a traffic source a few years ago, and it is the one where a vendor gets picked or skipped.

Where agents fail, and which failures are yours

Browser Use lists the limitations without much spin: bot management that blocks automated traffic, sessions that expire, prompt injection, unpredictable costs, and results that are simply wrong. An agent can select the wrong product or report success before a form is saved.

Some of those belong to the agent builder. Some belong to you. If your bot rules treat every automated visitor as an attacker, the research agent never reaches your comparison page. If your key facts, such as who you serve, what it costs and what you integrate with, live only in images or behind a demo form, the agent that does get in comes back with nothing to quote. It then fills the gap with a competitor whose page it could read.

My view: of the failures on that list, unreadable pages are the cheapest one for a site owner to fix, and the one most teams have not looked at.

What to check this week

Open your three most important pages, usually the homepage, pricing and the main product page, and look at them as text. A browser's accessibility inspector shows the tree an agent gets. Check that every button has a name that says what it does, that plan names and prices are text rather than images, and that the sentence saying who you are for appears on the page in words.

Then check your bot settings for the user agents of the AI assistants your buyers use. Blocking all of them is a decision, and it should be made on purpose rather than inherited from a default firewall rule.

The bigger picture

Answer engines and browser agents are converging on the same job: reading the web on a buyer's behalf and coming back with a shortlist. A citation in a ChatGPT answer and a click by an agent on your pricing page both depend on the same thing, a page that a machine can read and quote. Measuring whether that is happening, question by question and engine by engine, is the work.

ClawWorld runs AI visibility work for B2B startups. Start with a free baseline report: 10 buyer questions, who ChatGPT and Claude name instead of you, and the pages they cite.

Get your free baseline report →