--- title: Static assets scraping - Web description: Options to enable static assets scraping for Zoning and Session Replay source_url: html: https://docs.contentsquare.com/en/web/static-assets-scraping/ md: https://docs.contentsquare.com/en/web/static-assets-scraping/index.md --- > ## Documentation index > Fetch the complete documentation index at: https://docs.contentsquare.com/llms.txt > Use this file to discover all available pages before exploring further. Contentsquare uses two complementary methods to collect static resources for Session Replay and Zoning: **Static Resource Scraper** and **Static Resource Manager (SRM)**. The Scraper is the primary and always-active collector, but requires configuration to access your resources. SRM can then be configured to collect any resources the Scraper can't access. How Scraper and SRM work together * **SRM enabled** (default): The Scraper collects resources, SRM collects base64 images * **SRM explicitly enabled for online resources on a specific pageview**: SRM also collects resources from the browser for that pageview, and the Scraper keeps collecting them (requires [explicit tag command](https://docs.contentsquare.com/en/web/static-assets-scraping/#remote-resources-authenticated-assets)) * **SRM enabled for online resources with the [`skipScrapingForUploadedResources` option](https://docs.contentsquare.com/en/web/static-assets-scraping/#reduce-scraper-requests)**: SRM replaces the Scraper for that pageview, except for [unsupported resources](https://docs.contentsquare.com/en/web/static-assets-scraping/#unsupported-resources) * **SRM disabled**: Only the Scraper collects resources ## Static Resource Scraper The Scraper runs automatically and asynchronously downloads publicly accessible resources (CSS, images, fonts) from your website: * It operates every 6 hours for frequently accessed pages * Falls back to live scraping when resources aren't cached * Requires [IP allowlisting](https://docs.contentsquare.com/en/web/static-assets-scraping/#allowing-ip-addresses) **or** [custom header configuration](https://docs.contentsquare.com/en/web/static-assets-scraping/#using-a-custom-static-header-alternative) * Stores resources securely for 13 months **Limitations:** * Assets must be accessible from Contentsquare's IP addresses. * Assets behind authentication or VPN are not accessible. * Resources that change frequently (cache-busted URLs) may become outdated. To enable the Scraper, allow Contentsquare to access your assets using one of the following: * [Allowing IP addresses (recommended)](https://docs.contentsquare.com/en/web/static-assets-scraping/#allowing-ip-addresses) * [Using a custom static header (alternative)](https://docs.contentsquare.com/en/web/static-assets-scraping/#using-a-custom-static-header-alternative) ### Allowing IP addresses (recommended) The Scraper needs explicit permission to access your website through your proxy, firewall, or server configuration. IP allowlisting is the recommended method as it's simpler to implement and maintain. **Why this is needed:** Your firewall or security systems may block automated requests. By allowlisting Contentsquare's IP addresses, you allow our Scraper to download static resources without triggering security alerts or rate limiting. Allow ports `80` (HTTP) and `443` (HTTPS) and the following IP addresses to prevent your proxy, firewall, or server configuration from blocking the scraper. ```plaintext 52.18.162.157 20.75.90.236 100.24.76.90 34.192.98.148 20.67.250.109 54.247.44.196 52.51.9.12 35.72.153.38 35.73.99.41 34.192.240.128 ``` Performance * The scraper checks if resources were already downloaded within the last 6 hours before fetching again * Only publicly accessible resources are downloaded * The scraper identifies itself via user-agent headers ### Using a custom static header (alternative) If you cannot allowlist by IP, you can use a custom header-based approach. **Security benefit:** This method allows you to verify requests are legitimately from Contentsquare by validating the custom header value. **When to use this approach:** * IP allowlisting is restricted by security policy * You need more granular control over scraper access * You want to monitor and log scraper requests separately When selecting this option, Contentsquare adds a custom header to the project settings. ```json { "headers": { "my-new-header-key": "myKeyValue" } } ``` You can then validate that scraper requests contain the specific header and value. ```javascript receivedHeaderExample = "myKeyValue"; const CONTENTSQUARE_CUSTOM_HEADER = "myKeyValue"; if (receivedHeaderExample === CONTENTSQUARE_CUSTOM_HEADER) { // } ``` ### Validation To verify scraping is working correctly: 1. Check your Session Replays render with correct styling and images 2. Review server logs for successful requests from Contentsquare IPs Contact support if replays show missing resources after configuration. ## Static Resource Manager (SRM) For resources the Scraper can't access, the **Static Resource Manager (SRM)** collects them directly from the visitor's browser and uploads them to Contentsquare servers. SRM is enabled by default for all projects. When SRM is needed: * Resources behind authentication or login * VPN-protected or private network resources * Assets requiring specific user cookies or headers * Dynamically generated, or user-context-dependent content * Base64-encoded data URLs How SRM works: * The user's browser downloads the resource during the user session * The tag computes a unique hash and checks if Contentsquare already has it * If new, the tag uploads the resource directly to Contentsquare servers * Resources are deduplicated across sessions using hash-based identification To enable SRM, note how it handles the two types of resources: * [Data URLs (base64 images)](https://docs.contentsquare.com/en/web/static-assets-scraping/#data-urls-base64-images): Collected automatically — no implementation required * [Remote resources (authenticated assets)](https://docs.contentsquare.com/en/web/static-assets-scraping/#remote-resources-authenticated-assets): Must be explicitly enabled for each pageview using a tag command **Limitations:** * **Missing or restrictive cache policies**: If resources don't have a `Cache-Control` header that allows browser caching, SRM makes additional requests to the resource server for every session. This also occurs when resources return 404 errors (for example, when the page references assets the server can't find). * **CORS restrictions**: Resources with incompatible Cross-Origin Resource Sharing (CORS) policies can't be fetched by the browser, preventing SRM from collecting them. * **Lazy loading not respected**: SRM downloads every resource it finds as soon as it discovers it in the DOM or in the CSS, regardless of lazy loading implementation. This may defeat your lazy loading strategy and cause all resources to load immediately. If your site relies on lazy loading for performance, avoid enabling online resource collection with SRM. SRM availability When these limitations cause excessive server load or repeated failures, Contentsquare may disable SRM for your project. You can [check your project configuration](https://docs.contentsquare.com/en/web/static-assets-scraping/#how-to-check-if-srm-is-enabled-for-your-project) to confirm whether it's enabled or not. ### Data URLs (base64 images) When SRM is enabled for your project, the Contentsquare tag automatically collects base64-encoded data URLs found in the DOM. This ensures that embedded images appear correctly in Session Replay. No additional implementation is required for data URLs. ### Remote resources (authenticated assets) For remote resources that are behind authentication or not accessible to the Scraper, you must explicitly enable online asset collection **for each pageview** using a tag command. Push the following command **before** the pageview starts: ```javascript window._uxa.push(["replay:resourceManager:enableForOnlineResource:nextPageviewOnly"]); ``` Timing is important This command must be pushed **before** the pageview starts. For the first pageview after the tag loads, push the command before the tag starts. For subsequent pageviews using `trackPageview`, push the command before calling `trackPageview`. **Example with `trackPageview`:** ```javascript // Enable SRM for the next pageview window._uxa.push(["replay:resourceManager:enableForOnlineResource:nextPageviewOnly"]); // Then track the pageview window._uxa.push(["trackPageview", location.pathname + location.search + location.hash]); ``` ### Reduce Scraper requests When SRM collects online resources, the Tag still sends every resource URL to the Scraper: SRM alone doesn't reduce the requests the Scraper sends to your servers. To stop sending the URLs of the resources SRM collects, use the `skipScrapingForUploadedResources` option. **When to use this option:** Use `skipScrapingForUploadedResources` when reducing the Scraper's requests to your servers is your priority, for example when the Scraper's traffic is a concern for your infrastructure. **Don't use this option:** Keep both collectors active (don't use `skipScrapingForUploadedResources`) when replay reliability is your priority. The Scraper acts as a fallback: if SRM fails to collect a resource, the Scraper still provides it, so replays are more complete. **Prerequisites:** * SRM is enabled for your project: [`isStarted`](https://docs.contentsquare.com/en/web/static-assets-scraping/#how-to-check-if-srm-is-enabled-for-your-project) is `true`. * Resources hosted on another origin, such as a CDN or another subdomain, return an `Access-Control-Allow-Origin` header matching your website origin. This includes the fonts and images referenced in your stylesheets. Otherwise, the browser blocks the request and SRM can't collect the resource. * Resources hosted on another origin load without cookie-based authentication. SRM doesn't send the visitor's cookies to other origins. * Resources return a `Cache-Control` header allowing browser caching. Otherwise, SRM requests them from your servers again for every session. * The Scraper can still access your website, through [IP allowlisting](https://docs.contentsquare.com/en/web/static-assets-scraping/#allowing-ip-addresses) or a [custom static header](https://docs.contentsquare.com/en/web/static-assets-scraping/#using-a-custom-static-header-alternative), to collect the [unsupported resources](https://docs.contentsquare.com/en/web/static-assets-scraping/#unsupported-resources). Push the command with the option **before** the pageview starts, like the command without option: ```javascript window._uxa.push([ "replay:resourceManager:enableForOnlineResource:nextPageviewOnly", { skipScrapingForUploadedResources: true }, ]); ``` The option applies to the next pageview only. The Scraper still collects the [unsupported resources](https://docs.contentsquare.com/en/web/static-assets-scraping/#unsupported-resources). Missing resources in replays When using `skipScrapingForUploadedResources`, the Scraper doesn't collect the resources SRM supports. If SRM fails to collect one of them, the resource is missing from replays. ### Unsupported resources SRM doesn't collect the following resources. The Scraper collects them, even with the `skipScrapingForUploadedResources` option: * SVG `` elements referencing an external file, such as SVG sprites * SVG `` elements * `` elements outside a `` element, such as in `