Allow Sitepager Through Your Firewall

If your site is blocking Sitepager, you need to add a rule that allows requests containing your Sitepager user agent identifier. The steps depend on where your site is hosted.

Before following the steps below, set up a custom user agent in your scan settings so Sitepager has an identifier your site can recognize. See Custom User Agent.

---

Cloudflare

If your site uses Cloudflare (either directly or through a platform integration):

1. Go to your Cloudflare dashboard
2. Navigate to Security > WAF > Custom Rules
3. Create a new rule:
   • Field: User Agent
   • Operator: contains
   • Value: your bot identifier (for example, sitepager-scanner)
   • Action: Skip (skip all remaining rules)
4. Save and deploy

If your site is in Under Attack Mode, the custom rule should still let Sitepager through. If it does not, temporarily switch to a lower security level while running scans.

---

Webflow

Webflow does not have a built-in WAF or bot detection that blocks automated browsers. Sitepager should be able to scan published Webflow sites without any special configuration. Webflow does not support IP or user agent allowlisting on their end.

Password-protected Webflow sites If your Webflow site is password-protected (site-wide or per-page), Webflow uses a form-based password page, not HTTP basic auth. Use Sitepager’s login feature to enter the password before scanning. See Test Authenticated Pages.

Webflow sites routed through Cloudflare If you are routing your Webflow site through Cloudflare using Orange-to-Orange (O2O), Cloudflare’s bot protection may block Sitepager. Follow the Cloudflare steps above to allowlist.

---

Framer

Framer does not block automated browsers under normal conditions. Its built-in protections focus on traffic spikes, not on identifying or blocking headless browsers.

Password-protected Framer sites If your Framer site is password-protected, use Sitepager’s login feature to handle the password gate. See Test Authenticated Pages.

Framer Enterprise with Cloudflare integration Follow the Cloudflare steps above. Standard Framer plans do not support IP or user agent allowlisting.

---

Vercel, Netlify, or other hosting

Most hosting platforms do not block automated browsers by default. If you have added bot protection (like Vercel’s Firewall or a third-party WAF), create a rule that allowlists your bot identifier in the user agent string.

---

Custom server (Nginx, Apache, etc.)

Add a rule to your server configuration that skips bot protection when the user agent contains your Sitepager identifier.

Nginx example:

if ($http_user_agent ~* "sitepager-scanner") {
    set $skip_bot_check 1;
}

Apache example:

SetEnvIfNoCase User-Agent "sitepager-scanner" allow_bot

Adjust the variable names and logic to fit your existing bot protection setup.

---

Password-protected staging sites

If your staging site is behind a login page or password form rather than a firewall, the user agent is not the issue. Use Sitepager’s login feature to authenticate before scanning. See Test Authenticated Pages.

This applies to Webflow password-protected sites, Framer password-protected sites, and any other platform that gates access behind a form.