Data Syncing

Data Connect uses an ETL (Extract, Transform, Load) process to move data from Contentsquare to your data warehouse once per day.

When you first set up Data Connect or add a new table, an initial sync copies all historical data from Contentsquare to your warehouse. This process:

• Creates the necessary tables and views
• Copies all historical data (which can take several hours or days depending on data volume)
• Runs only once per table, unless a full resync is needed

After the initial sync, Data Connect performs incremental updates during each sync window. These updates:

• Only transfer new or changed data since the last sync
• Are much faster than initial syncs
• Maintain data consistency while minimizing warehouse load
• Run daily

As a project or account admin, use Analysis setup > Data Connect > [Select your warehouse destination] > Details to define which event tables Data Connect syncs to your warehouse.

Toggle syncing of Contentsquare event tables or custom events that you created individually or in bulk.

Data Connect always syncs the following pre-built tables: users, sessions, and page_views.

When you toggle off an event table that's in sync, the existing table remains in your warehouse but will no longer receive updates.

Tables that are not synced appear greyed out.

Monitor the status of your Data Connect syncs through:

• The Data Connect dashboard in the Contentsquare UI
• Email notifications for failed syncs (if configured)
• Warehouse query logs showing Data Connect activity

When a Data Connect sync fails, Contentsquare automatically retries the sync several times. If multiple retries fail, you'll receive a notification.

Work through these steps to identify the cause:

1. Check for error messages in the Data Connect dashboard
2. Verify warehouse permissions and quotas
3. Look for schema conflicts
4. Review recent changes to your Contentsquare implementation
5. Contact Contentsquare support with specific error messages and timestamps

Once resolved, syncs will resume from where they left off.

As your Contentsquare implementation evolves (adding new events or properties), Data Connect handles schema evolution automatically:

Data Connect syncs all data available based on your account's retention settings. If data is deleted in Contentsquare, it is not automatically removed from your warehouse — you need to delete it manually.

To ensure smooth operation of your Data Connect pipeline:

1. Monitor sync status regularly: Check that syncs are completing successfully
2. Plan for schema changes: Consider potential impacts when adding new properties
3. Test new warehouse queries: Verify queries after schema changes
4. Manage warehouse resources: Schedule heavy queries outside of the daily sync window
5. Document custom tables: Maintain documentation for any views or derived tables you create
6. Set up alerting: Configure monitoring for sync failures
7. Manage data volume: Archive or partition historical data as needed