Advanced Crawling Criteria

If no content is being crawled from your website, try adjusting these settings:

• Wait Time – Adds a delay before crawling to ensure the page fully loads.

• Wait for CSS – Delays crawling until a specific CSS selector is loaded.

• Execute JavaScript – Prevents pop-ups from being crawled instead of actual content.

These settings help improve crawling accuracy and ensure the right content is captured.