GlopalBot is the crawler of Glopal, which translates e-commerce sites. It loads pages of the sites Glopal translates, or is asked to check, to capture their content for translation and translation quality checks.
Mozilla/5.0 (compatible; GlopalBot/1.0; +https://bot.glopalstore.com/docs)
It runs headless Chrome, so loading a page also fetches the page's scripts, styles and images.
Requests to the crawled site are signed with Web Bot Auth (HTTP Message Signatures) with Signature-Agent: "https://bot.glopalstore.com". The public keys are at https://bot.glopalstore.com/.well-known/http-message-signatures-directory. Requests come from AWS (us-east-1) without a fixed IP range: verify the signature, not the IP.
GlopalBot follows robots.txt, including Crawl-delay. To block it:
User-agent: GlopalBot
Disallow: /Per site it loads at most 4 pages at a time, starting at one page every 2 seconds and never more than 2 pages per second. It halves its rate when the site answers 429, 503 or 403.