What Is a Browser Agent?
A browser agent is an automated system that operates a web browser to complete a task: loading pages, reading what they contain, and acting on them by clicking, typing and navigating. What separates it from a scripted scraper is that the sequence of actions is decided while the task runs rather than written in advance.
The loop, and how a page is read
The structure is a loop: observe the current page, choose one action, perform it, observe the result. Observation takes one of a few forms in practice — the accessibility tree, a serialised form of the DOM, a screenshot with the interactive elements annotated, or the network responses underneath the page. Which form is in use determines what the agent is able to notice at all.
The action set is deliberately small — click, type, scroll, navigate, wait, extract — because a small action space is what keeps the loop tractable. The difficulty is not in the action but in the correspondence between the element that was intended and the element that receives it, which is where a page's own structure, its overlays and its timing all intervene.
That correspondence is settled by how an element is addressed, and the three schemes in use go stale in three different ways. A selector — a CSS path or an XPath expression — is written against the page's markup and lasts exactly as long as that markup does. An identifier assigned by the agent's own observation is meaningful within the step that produced it and not beyond it, because the next observation renumbers whatever it finds. A coordinate lifted from a screenshot holds until something scrolls, reflows, or finishes loading beneath the pointer. What the three share is the manner of their failure: a stale reference does not raise an error, it lands the action on whichever element now occupies that address, and the loop observes the resulting page and carries on from it.
| Observation form | Consequence |
|---|---|
| Accessibility tree | Compact and structured. Carries what assistive technology is exposed to, and misses what a page renders without exposing. |
| Serialised DOM | Complete but large; the token cost grows with the page’s complexity. |
| Annotated screenshot | Matches what a person sees, including canvas and image content; positions must be mapped back to elements. |
| Network responses | The underlying data without the page, available only where the site’s own requests can be read. |
The observation form is the agent's entire world for that step. A control that appears in none of the forms in use cannot be operated, however plainly it is visible on screen.
The failure modes that belong to browsers specifically
A browser agent inherits everything the open web does to automated clients. Sessions expire mid-task. Layouts change between one run and the next. Consent dialogs, rate limits and bot checks interrupt a sequence that worked yesterday. A page that renders after an asynchronous request reads as empty to an agent that observed it a moment too early.
Two points deserve stating because they are usually left implicit. Everything read from a page is untrusted input, and text written on a page as an instruction is a live injection vector — the mechanism is set out on this registry's page for prompt injection. And an action taken in a browser is generally taken with the credentials of whichever session is loaded, which makes the blast radius of a mistaken click a question about the account rather than about the agent.
The web is also not transactional, and that decides what a retry is permitted to mean. A step that timed out may still have been received; a form resubmitted after an ambiguous response can produce a second order, a second message or a second payment, and no rollback is available to the thing that issued it. A browser offers no signal distinguishing an action that can be repeated harmlessly from one that cannot, so the distinction has to be declared by whoever specifies the task. Navigation, scrolling and reading are repeatable. Submission generally is not, and a retry policy that treats the two alike will eventually duplicate something irreversible. This is why a browser task is commonly gated at the submission and left ungated at everything above it: the checkpoint is placed where repetition stops being free, not where the task looks difficult.
How the Registry classifies a browser-operated task
Operating a browser to log into a portal, download the most recent invoice and file it is classified as information retrieval here: something that already exists is located and moved, and nothing is manufactured along the way.
The Registry's canonical brief for this entry is filed as:
Operate a web browser to log into a supplier portal, download the most recent invoice, and file it.
Submitted for assessment it is
classified as Information retrieval, and its wording is hashed once — to
d8880e1f3b982784…, the first sixteen of sixty-four hexadecimal
characters — with the wording itself never stored. The hash is what the derivation
reads.
That class's own page is /tasks/retrieval.
Classification is one of three inputs. The other two are the configuration submitted with the task, and the permanent chart derived from that configuration — fixed by the model name, the training cutoff and the temperature alone, and never reading the task at all. The same ascendant, ruling planet and harmony therefore appear on every assessment a given configuration receives, whatever it was asked to do. The derivation is published in full at /method.
What this page does not claim about browser agent
Which sites tolerate an automated client, and under what terms, is a matter between an operator and those sites. The canonical brief above describes an operation, not a permission.
The Registry does not run this task, does not inspect any system's output for it, and validates no assessment it issues against what afterwards happens. What it does is compute — from a published method, for one submitted task and one submitted configuration — a verdict and a recommended execution window. It computes neither on this page.
Questions about browser agent
- What is a browser agent?
- A browser agent is an automated system that drives a web browser to complete a task, deciding its next action from what the current page shows rather than following a script written in advance. It observes the page, acts on it, and observes the result, repeatedly.
- How is a browser agent different from web scraping?
- A scraper follows a fixed sequence written against a known page structure. A browser agent chooses each action at run time from what it observes, which lets it continue when a page changes and also means its behaviour is not fully known before it runs.
- Why is the accessibility tree used to read pages?
- Because it is a compact, structured description of the interactive and readable content, much smaller than the full DOM and easier to act on than pixels. Its limitation is the same as its strength: anything a page renders without exposing it to assistive technology is not in the tree.
- Is it safe to run a browser agent in a logged-in session?
- An action taken in that session carries the account’s authority, so the exposure is whatever the account can do. The usual containment is a separate account with the narrowest workable permissions, an approval step before consequential actions, and the assumption that any page content read during the task may be adversarial.
- How does a browser agent decide which element to act on?
- By addressing it in whatever form the observation supplied: a selector written against the markup, an identifier the agent assigned to an element while observing the page, or a coordinate taken from a screenshot. All three go stale — markup is edited, a re-observation renumbers, a reflow moves a coordinate — and a stale reference sends the action to whichever element now holds that address rather than raising an error.
Related entries
Order an assessment for information retrieval tasks
The Registry issues a permanent, numbered task risk assessment for one submitted task and one submitted configuration. EUR 1.90 Standard, EUR 4.90 Extended, EUR 14.90 Full Chart, which adds the permanent chart. Assessments from EUR 1.90; machine-readable at /pricing.json.