Custom User Agent
Some sites block automated requests from tools like Sitepager. If your scans are returning blank screenshots, 403 errors, or CAPTCHA pages, your site is likely blocking Sitepager.
A custom user agent gives Sitepager an identifier your site can recognize and allow through.
Where to find it
Section titled “Where to find it”The Custom user agent toggle is in the Advanced Settings section when creating or editing a scan. It is disabled by default.
How it works
Section titled “How it works”By default, Sitepager uses a standard Chrome user agent. Your site cannot tell Sitepager apart from a regular browser visit. This is fine for most sites.
If your site blocks automated traffic, you need a way to identify Sitepager and let it through. When you enable a custom user agent in Browser-like mode, Sitepager appends a tag to the end of the standard browser user agent:
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/145.0.0.0 Safari/537.36 Sitepager-Core/1.0 (your-workspace-id; my-bot-label)You choose the bot label. Then you add a rule on your site that says: if the user agent contains this label, let the request through.
There is also a Raw mode where you provide the entire user agent string yourself. Use this only if you need full control over the string.
Setting it up
Section titled “Setting it up”- Open your scan settings and go to Advanced Settings
- Enable Custom user agent
- Select Browser-like (Recommended)
- Enter a short bot identifier (for example,
sitepager-scanner) - Check the preview to confirm the full string looks correct
- Save the scan
Once set up, go to your site and allowlist that identifier so Sitepager’s requests are let through. See Allow Sitepager Through Your Firewall.
Default user agents
Section titled “Default user agents”If you do not set a custom user agent, Sitepager uses standard browser strings:
| Device | Default user agent |
|---|---|
| Desktop | Chrome on macOS |
| Mobile | Safari on iPhone |
| Tablet | Safari on iPad |
These do not contain any Sitepager identifier. Your site cannot distinguish them from regular visitors.
Use Browser-like mode unless you have a specific reason not to. It keeps requests looking like real browser traffic while adding just enough to identify Sitepager in your firewall rules.
Keep the bot identifier short and descriptive. Something like sitepager-scanner or pre-publish-check is enough.
Test after adding the rule. Run a scan to confirm pages load correctly. If you still see blank screenshots or errors, the rule may not be matching. Check your server logs for the full user agent string.
Sitepager does not use static IP addresses. You cannot add rules by IP. The user agent is the recommended way to identify and allow Sitepager requests through your firewall.