How to keep your web scraper's User-Agent up to date (without maintaining your own list)

The problem: User-Agent strings rot

If you run a scraper for more than a couple of months, you have probably lived this: it worked fine in January, and by May it is getting 403s or endless CAPTCHA pages from sites that were previously happy. Nothing in your code changed. What changed is the world around it — Chrome ships a new stable version roughly every four weeks, and the User-Agent string you hardcoded now claims to be a browser version that almost nobody on the real internet runs anymore.

Anti-bot systems weight this heavily because it is cheap to check. A request presenting Chrome/109 on Windows 10 in mid-2026 sits in a tiny, suspicious slice of traffic: mostly bots copy-pasting the same stale gist. Worse, hand-rolled UA strings often get the format subtly wrong — a real modern Chrome UA on Windows always reports Windows NT 10.0; Win64; x64 and zeroes the minor version (e.g. Chrome/138.0.0.0) due to User-Agent reduction. Fake generators that invent full minor versions or old NT tokens are trivially fingerprintable.

The fix: fetch the UA instead of hardcoding it

The Latest Chrome User-Agent API keeps one authoritative, correctly-formatted set of Chrome UA strings — one per major platform (Windows, macOS, Linux, Android, iOS) — updated with each Chrome stable release. Instead of baking a string into your code, you fetch it at startup.

Step 1: Get the current UA for your target platform

Call GET /latest. The response gives you the current User-Agent string along with the platform and Chrome version it corresponds to. In Python:

ua = requests.get(API_URL + "/latest", headers=rapidapi_headers).json()

Then use the returned string on every outgoing request:

session.headers["User-Agent"] = ua["user_agent"]

Fetch it once when your scraper boots (or once per day via a cache), not on every request — the value only changes when Chrome ships a release.

Step 2: Rotate realistically with /random

If you run many concurrent workers, having every worker present the identical UA from the same IP range is itself a pattern. GET /random returns a current UA for a randomly chosen platform on each call, so your pool looks like a plausible mix of Windows, macOS, and Android users rather than a thousand clones. Assign one UA per worker or per session — do not rotate the UA between requests inside one logical session, because a "user" whose browser changes mid-session is a stronger bot signal than a stale UA.

Step 3: Cache the full set with /all

For high-volume scrapers, call GET /all once, cache the full platform-to-UA map locally, and pick from it in-process. This keeps your API usage to a handful of calls per day regardless of scrape volume. GET /platforms lists the supported platform keys if you want to build the mapping dynamically.

Edge cases worth knowing

Consistency matters more than freshness. If you claim to be Chrome on Android, your Accept-Language, sec-ch-ua client hints, and viewport (in headless browsers) should not scream desktop Linux. The UA is necessary, not sufficient.
Headless browsers need the override too. Playwright and Puppeteer ship a UA containing HeadlessChrome by default. Override it with the string from /latest for the matching platform.
iOS is different on purpose. Chrome on iOS is a WebKit wrapper and its UA uses the CriOS token instead of Chrome. The API returns the correct iOS format — don't "fix" it.
Cache with a TTL, not forever. A 24-hour cache means you pick up new Chrome releases within a day of them shipping, with essentially zero API cost.

FAQ

How quickly are new Chrome versions reflected? The dataset is updated with each Chrome stable release, so within the release cycle — no action needed on your side beyond cache expiry.

Does using a current UA guarantee I won't get blocked? No. It removes one common, cheap detection signal. Rate limiting, IP reputation, and TLS/browser fingerprinting are separate problems.

Why not just scrape the UA from a website that lists them? You'd be writing a scraper to support your scraper, and those pages change layout. A stable JSON endpoint is the boring, reliable option.

Copy-paste code

Calling latest-chrome-user-agent-api.p.rapidapi.com — replace YOUR_RAPIDAPI_KEY with your key.

cURL

curl --request GET \
  --url 'https://latest-chrome-user-agent-api.p.rapidapi.com/latest' \
  --header 'X-RapidAPI-Key: YOUR_RAPIDAPI_KEY' \
  --header 'X-RapidAPI-Host: latest-chrome-user-agent-api.p.rapidapi.com'

Python

import requests

url = "https://latest-chrome-user-agent-api.p.rapidapi.com/latest"
querystring = {}
headers = {
    "X-RapidAPI-Key": "YOUR_RAPIDAPI_KEY",
    "X-RapidAPI-Host": "latest-chrome-user-agent-api.p.rapidapi.com",
}

response = requests.get(url, headers=headers, params=querystring)
print(response.json())

JavaScript (fetch)

const url = 'https://latest-chrome-user-agent-api.p.rapidapi.com/latest';
const options = {
  method: 'GET',
  headers: {
    'X-RapidAPI-Key': 'YOUR_RAPIDAPI_KEY',
    'X-RapidAPI-Host': 'latest-chrome-user-agent-api.p.rapidapi.com',
  },
};

const response = await fetch(url, options);
console.log(await response.json());

Node.js (axios)

import axios from 'axios';

const options = {
  method: 'GET',
  url: 'https://latest-chrome-user-agent-api.p.rapidapi.com/latest',
  params: {},
  headers: {
    'X-RapidAPI-Key': 'YOUR_RAPIDAPI_KEY',
    'X-RapidAPI-Host': 'latest-chrome-user-agent-api.p.rapidapi.com',
  },
};

const { data } = await axios.request(options);
console.log(data);