Key Takeaways

  • Screaming Frog SEO Spider 23.0 (“Rush Hour”) introduces major improvements including updated Lighthouse/PageSpeed Insights audits, new Ahrefs API v3 integration, crawl retention settings, semantic embedding rules, and better link visualizations.
  • The update modernizes data integrations, enhances automation, and simplifies large-scale site management.
  • This guide explains how to use each new feature, why it matters, and best practices for teams to maximize the new release.

1. Definition: What This Update Is About

Screaming Frog SEO Spider is a leading desktop tool for technical SEO auditing and website crawling. Version 23.0 is a major modernization release. It focuses on improved integrations with Lighthouse, PageSpeed Insights, and Ahrefs, while also adding storage management, semantic content mapping, and advanced visualization tools.

The result: faster, smarter, and more insightful technical audits.

2. Context: Why This Matters

2.1. Changing API and Audit Standards

Google’s Lighthouse and PageSpeed Insights audits have evolved, introducing new metrics and retiring older ones. This update ensures your site performance audits stay aligned with Google’s latest standards.

Ahrefs’ transition from API v2 to v3 also required a new connection method. The new integration keeps backlink data consistent and accessible directly within Screaming Frog.

2.2. Smarter Data Management

Large crawls can fill up storage quickly. The new crawl retention setting lets you automatically delete old crawls while locking important ones for long-term analysis.

2.3. SEO is Becoming Semantic

SEO isn’t just about links and titles anymore — it’s about content relationships. The new “Embeddings Rules” feature helps identify semantic similarities across pages, improving content strategy and internal linking.

3. Process: How to Use the New Features

3.1. Upgrading to Version 23.0

  1. Download and install the latest version from Screaming Frog’s website.
  2. Backup your crawl database if you use scheduled crawls.
  3. After installation, check for changes in your PageSpeed and API configurations.

3.2. New Lighthouse & PageSpeed “Insight Audits”

  • Navigate to the PageSpeed Insights configuration.
  • You’ll find new audits such as “Document Request Latency,” “Improve Image Delivery,” “Largest Contentful Paint Request Discovery,” and more.
  • Some older audits were removed or merged.
  • Update any custom exports or scheduled tasks that depend on old audit names.

Pro tip: Run a before-and-after comparison to understand how your scores differ with the new audit structure.

3.3. Ahrefs API v3 Integration

  • Open Configuration > API Access > Ahrefs.
  • Authenticate using the new OAuth flow.
  • Review updated metric options — some old metrics have been renamed or deprecated.
  • Use the integration to cross-analyze backlink data with crawl depth or internal link count.

3.4. Crawl Retention and Auto-Deletion

  • Go to File > Settings > Retention.
  • Choose how long to keep crawl data before automatic deletion.
  • Lock important crawls (right-click > Lock) to prevent them from being deleted.

Best use case: Set retention for short-term audits (30–90 days) while locking key audits such as pre-migration benchmarks.

3.5. Semantic Similarity Embedding Rules

  • Navigate to Configuration > Content > Embeddings.
  • Add URL patterns or rules for content comparison.
  • Run a semantic similarity report to detect overlapping or duplicate topics.

Use this feature for identifying redundant content, improving topic clusters, or planning redirects between similar pages.

3.6. Enhanced Visualizations

  • Open Visualisations > Site Visualisation or Content Cluster Diagram.
  • Right-click a node to show inlinks or outlinks.
  • You can now display all links in context, making it easier to spot linking gaps and hub pages.

Use it to plan internal link optimization, cluster validation, and content architecture presentations.

3.7. Additional Improvements

  • Limit crawl totals per subdomain for more controlled site audits.
  • Enhanced H1/H2 counts to show all heading occurrences.
  • Reorder Custom Search and JavaScript rules with simple drag controls.
  • Configurable URL percent-encoding.
  • New bulk exports like “All Error Inlinks” and “Redirects to Error” reports.

4. Benefits of Version 23.0

  • Updated audit alignment: Ensures compatibility with Google’s latest metrics.
  • Better data control: Manage crawl retention and storage effectively.
  • Improved link visualization: Understand internal link flows at a glance.
  • Semantic intelligence: Identify and consolidate similar content efficiently.
  • Stronger integrations: Keep your Ahrefs and Lighthouse data synchronized.
  • Optimized reporting: Use new exports for faster post-launch QA and debugging.

5. Best Practices

  1. Update team documentation with the new audit names.
  2. Re-baseline your key sites after the upgrade.
  3. Audit scheduled crawls for deprecated PSI fields.
  4. Lock key crawls to preserve your audit history.
  5. Use visualizations in client reports for clarity.
  6. Run “Redirects to Error” after migrations.
  7. Monitor API usage if connecting to Ahrefs or PSI frequently.
  8. Use embeddings to find semantic duplicates.
  9. Clean up old crawl files regularly.
  10. Align new audit data with your existing dashboards.

Did You Know?

Each Screaming Frog release gets a codename — version 23.0 is called “Rush Hour,” reflecting the update’s focus on managing high-traffic sites more efficiently.

Conclusion

Version 23.0 of Screaming Frog SEO Spider isn’t just an upgrade — it’s a modernization of how you audit websites. The improved integrations, retention management, semantic tools, and visualizations make it more powerful and user-friendly than ever.

If you’re serious about technical SEO, this update helps you work smarter, not harder — keeping your audits aligned, your storage clean, and your data richer.

FAQs

What platforms does version 23.0 support?

Version 23.0 is available for Windows, macOS (Intel and Apple Silicon), and Linux.

Will older PageSpeed reports break?

Some audit names have changed. If you rely on scheduled PSI exports, you’ll need to update templates or adjust mapped columns.

Is Ahrefs v3 integration required?

Yes. Ahrefs API v2 has been deprecated, so you must reconnect using the new v3 authentication method.

Can I recover deleted crawl data?

Once a crawl is deleted via the retention setting, it cannot be restored unless backed up manually.

Is the semantic similarity feature resource-intensive?

It depends on your site size. For small sites, performance impact is minimal; for large crawls, it may increase processing time slightly.

References