Autonomous AI agents browse in real browsers, on real devices, from real addresses. Here is how to tell whether a human is involved.
For most of the web's life there have been two kinds of visitor. Humans and crawlers. People clicked, scrolled, and bought things. Crawlers came to index, they said who they were in the request, and you decided whether to let them in. Both were well understood. Between robots.txt and a crawler list most sites had it covered.
There is now a third kind of visitor, and it does not fit either box.
An autonomous AI agent is an application given a goal. Book me a table. Compare these insurance quotes. Find the cheapest flight and tell you when to buy. To do this it drives a browser the same way as an individual. It loads the page, it runs the JavaScript, it clicks buttons. Often, it uses a genuine browser and device, over a genuine residential connection.
Which means the traffic is not fake, but it's not human. That is the problem.
Why the User-Agent cannot help you here
We wrote about understanding the User-Agent string, and showed what each segment tells you. It's still one of the most useful things a developer can learn, and it's still the fastest way to understand the limits of what a browser or device will volunteer about itself.
The User-Agent is a self-declaration of what it is. The industry has treated it as evidence for thirty years and mostly got away with it, because most software had no reason to lie.
An agent-driven browser does not need to lie. Its User-Agent says Chrome on Windows because it genuinely is Chrome on Windows. Nothing in that header is false. The same goes for every other check you run: the address is residential, so the proxy check passes; the request rate looks human, because one agent working through a shopping list is not significant; and there is no crawler name, because it hasn't claimed to be a crawler.
Crawlers announce themselves. Autonomous agents do not.
What it costs, depends on who you are
If you run a website, you are paying to serve pages. Origin requests, bandwidth, database queries, and a checkout with a conversion rate that is quietly sliding because a growing share of visitors were never going to buy anything. Your analytics is now aggregating people and AI agents and reporting the result as your audience.
If you are a publisher, the same traffic looks like inventory. It gets an ad call. It gets counted.
If you are buying media, this raises concern. You bid, you win, you pay, and the impression renders in a browser, and nobody is watching. Every measurement downstream inherits the problem. It is invalid traffic, but it does not look like any invalid traffic your filters were built to catch, because it came from a real browser on a real device.
Nobody in the bid stream gets to make a sensible decision, because nobody has the one fact that decides everything: is a human viewing the ad.
HumanConfidence
To help address this, we built the property that answers the question. HumanConfidence returns one of three answers for a request:
- High. There is positive evidence of a person. Serve the page, bid on the impression, and count the visit.
- Medium. The evidence is mixed. Some signals point at a person, some do not, and we are telling you that honestly instead of picking a side for you.
- Low. Almost certainly no human is directly involved in viewing the page.
Unpacking the recipe
HumanConfidence is not a score we ask you to take on faith. It's created from eight properties, and you can read how the rules are applied, in the same way we dissect a User-Agent string. Here is what is inside.
Each property does not carry the same weight.
Three signals that settle it on their own
Any one of these on its own returns Low. Each is proof of automation rather than one hint among several, so none of them gets averaged away by the others.
IsCrawler. The request says it is a crawler. The well-behaved half of the problem, and the reason this stays in the recipe: a declared crawler and a disguised agent are two different situations needing two different decisions. You don't need to guess which one you are looking at.
HasWebDriver. The browser advertises a web driver, which a browser under automation is required to do. The most direct signal there is. When present, it is strong, and when absent, it proves very little, because it reports what the browser advertises rather than what an automated client might be concealing.
IsHeadless. The browser says it is running without a visible window. Perfectly ordinary in a test suite, not on a checkout page.
Four signals that get counted
None of these settles anything alone, so the recipe counts how many of them fail rather than treating any single one as a verdict.
Was the window actually in view? IsVisible answers it. A page that loads, runs, and is never put in front of anyone tells you a lot about what is on the other end.
Is this a browser a real person could be using today? Two properties answer, BrowserReleaseYear and BrowserReleaseAge. People update their browsers at a rate that is quite well understood. The current draft counts a build over 24 months old as failing, and a browser we cannot identify fails too. A build more than three months old does not fail, but it stops the answer being High.
Does the address argue against a human? HumanProbability is our existing 0 to 10 read on the network side, where the published bands put 7 and above at high confidence. That is the threshold the recipe uses. An address nobody has rated yet is deliberately not counted against the request, because a real person should not lose a check for something they did not do.
Is this a connection people browse on? ConnectionType answers it. The check passes on broadband and cellular. This is the clearest reason the device signals cannot answer the question alone.
And then it just counts
No failed checks is High. One failed check is Medium. Two or more is Low.
That is it. Eight inputs, seven checks, one count. If you disagree with the answer on a particular request, you can see precisely which check tipped it, because every one of them is a property sitting in the same response.
Compare that with a vendor who scores between 1 and 100 and arrives with no workings shown and requires a support ticket to understand more.
Do not take our word for any of that
Everything in the section is published, and you can go and read it.
The recipe lives in a public repository, 51Degrees/derived-properties, as a plain text file called HumanConfidence.yaml.
There is also a tester you can run in your browser. Set the properties one request carries, mark any of them absent, and the page tells you which band comes out and why it got there.
We are not aware of another vendor in this space who publishes the logic behind their traffic scoring. We would be genuinely pleased to be corrected. Until then, one of us is showing you the workings.
Why nobody else can answer this alone
When you look at the properties needed, HasWebDriver, IsHeadless, IsVisible, IsCrawler, BrowserReleaseYear, and BrowserReleaseAge are device and browser questions. HumanProbability and ConnectionType are network questions. Answering "is a human involved" needs both, in the same request, at the same moment.
A device detection provider can see the browser but knows nothing about the connection. An IP intelligence provider can see the connection but knows nothing about the browser. Either one, on its own, is guessing at half the picture.
51Degrees provides both, from one pipeline, in one call. That is the reason this property exists here and not somewhere else.
What you are actually buying
If you are weighing this up against your current setup, here is the honest case.
Both halves, one call. You are not buying a bot score to bolt onto a device vendor and an address vendor, then arguing with their support desks about which one got it wrong. Our device and network evidence arrive together, already reconciled.
Evidence instead of a verdict you cannot inspect. Every ingredient is a property you can read, log, and challenge, and the rules are in a public file you can read before you buy. When something looks wrong, you can see which check caused it, which means you can address it rather than open a ticket.
It goes where your traffic already goes. One pipeline, in the cloud or on your own infrastructure, arrives in the same response, and passed into the bid stream. This is not a separate tag with its own dashboard.
We have been doing this for years. Over a million device combinations, updated every working day, ISO 9001 and 27001 certified, with crawler and AI crawler detection already in the same product. Agent detection is a new property built on strong foundations.
And we will tell you what it cannot do. Which is the next section, because you should not have to find that out yourself.
Where the answer stops
We would rather tell you the limits than have you find them.
It needs our JavaScript to have run. Two of the eight properties, whether the window was in view and whether the browser advertises a web driver, simply cannot be known from a request on its own. The first request for a session has no value, and later ones do. A pipeline that never runs our JavaScript at all never gets a value.
That is a choice rather than oversight, and it is worth understanding why. We could have computed a band from the server-side properties alone and handed you a number on every single request. It would have been a weaker answer, because it would not know whether anybody was actually looking at the page. We would rather give you nothing than give you that.
When it has nothing, it says nothing. There is no Unknown value to misinterpret and no default quietly standing in for a real answer. Either the recipe reaches a band, or the property has no value at all, and the pipeline tells you which ingredient was missing.
It answers one question. HumanConfidence tells you whether a human is involved in a request. Not which agent, not who sent it, not what it wants. It is not a fraud verdict: plenty of non-human traffic is entirely welcome, and for you to decide.
None of this is a hedge. It is an honest answer, and why we provide you with the ingredients along with the verdict.
What to do with it
Read HumanConfidence on the page view, then act on it.
- Websites. Route Low traffic away from your expensive paths. Keep your analytics about genuine humans, and report agent traffic separately, because it is about to become a number your management will ask for.
- Publishers. Decide whether an impression with no human behind it should get an ad call at all. Right now, most of them do. This is also where the signal is established, on the page, which is what makes it available to everyone downstream.
- Bidders. Treat Low as invalid traffic before you bid rather than after you have paid for it.
Crawlers and bots are yesterday's problem. The web now has visitors that browse like people, look like people based on the checks you run today, but they are not. You cannot manage traffic you cannot see, and you should not pay for it.
Your ads have an audience. Make sure it has a pulse.
Read the recipe, then talk to us
HumanConfidence is computed by the 51Degrees Pipeline from the device and network data in the same response you already read, which is why it arrives alongside everything else rather than as a separate lookup.
Read the recipe, throw your most awkward examples at the tester, and then talk to us about getting it into your own traffic.
Talk to us about HumanConfidence