AutrokBot, Autrok's crawler
AutrokBot is the crawler that Autrok, a Pomars product (pomars.com), uses to read public used-car listings in Belo Horizonte and compare the asking price with the FIPE table and with similar cars. If it showed up in your site's logs, this page explains who it is, what it does and how to ask it not to come back.
It always identifies itself with this user agent:
AutrokBot/1.0 (+https://autrok.com/bot)What it collects
Only the listing's facts: make, model, version, year, mileage, color, fuel, transmission, asking price, city and the page address, so that every car on Autrok links back to the original listing. It never copies photos or descriptions, does not read plates and keeps no name, phone, e-mail or any other data of private sellers; for dealers and rental companies, only the business name.
How it visits websites
It reads and follows each site's robots.txt: blocked pages are never visited, and it starts from sitemaps or stock lists rather than search pages. It makes one request at a time per site, at least 2 seconds apart (more if the site asks for a Crawl-delay), uses no cookies, never signs in and starts at most one pass every 20 hours per site.
When it stops
If a site answers 401, 403 or 429, or shows a bot check, AutrokBot stops at once and never tries to get around the block. After a 429 it waits at least 24 hours (or the Retry-After, if longer); after a refusal, at least 7 days. Repeated server errors also make it stop.
FIPE table
FIPE table values are not collected by AutrokBot: they come from a data service and are shown only per model, next to each listing.
How to ask it not to visit your site
Block AutrokBot in your site's robots.txt, as in the example below: it stops visiting within 24 hours. Or write to bot@autrok.com with the site's address: within 24 hours we stop collecting and take its listings off Autrok.
To block AutrokBot in robots.txt:
User-agent: AutrokBot
Disallow: /Questions or requests about AutrokBot: bot@autrok.com