Screaming Frog SEO Spider Update – Version 24.0 - Screaming Frog

Screaming Frog SEO Spider Update – Version 24.0

We’re delighted to announce Screaming Frog SEO Spider version 24.0, codenamed internally as ‘bolus’.
In this release we have introduced a number of much demanded new features, as well as smaller quality of life improvements and updates.
Let’s take a look at what’s new.

1) Screaming Frog SEO Spider MCP

You can now run crawls, analyse, export and manipulate data using the SEO Spider and node.js within Claude, LM Studio and other AI chat assistants.

The MCP allows you to connect to the SEO Spider and automate countless crawl related activities using natural language. From the basic, such as running and summarising a crawl.

To more fun, and advanced tasks.

As well as performing the leg work for exporting, combining data, running the analysis pipeline and installing relevant scripts for interactive visuals for greater insight.

To set this up, please refer to our documentation on the MCP Server.

2) Auto Compare Crawls

You’re now able to select ‘Auto Compare Crawls’ for scheduled (and CLI) crawls.

This can be set up in the scheduled crawl task, which will then auto compare the last two crawls performed in a project.

This allows you to quickly view a crawl comparison of the last two crawls, including all changes that have occurred since the last crawl.

Read more in our crawl comparison tutorial.

3) View Crawl Changes in Email Notifications

If you have enabled crawl completion notifications, you will already be familiar with the email summary upon crawl completion.

This has been improved to include all issues found within the crawl in a table below the summary.

If you have selected to ‘Auto Compare Crawls’, you will see a crawl comparison of issues in the email – highlighting any changes in issues found between the last two crawls.

This is a great way to monitor and be alerted to website changes and SEO issues – such as pages being added or removed, a sudden spike in non-indexable pages, or page titles disappearing.

4) Send Crawl Export Attachments by Email

You’re now also able to send emails with export attachments automatically upon crawl completion.

For example, if you need to send a weekly broken links export to a developer, or a crawl issues summary to a stakeholder, you can input their email and the report of choice and it will appear in their inbox.

5) Find Uncrawlable Links

You can now find common uncrawlable link types in the SEO Spider.
These can be viewed in the Links tab and the new ‘Pages With Uncrawlable Internal Outlinks‘ filter.

There is a new ‘Link Crawlability’ column in the lower Outlinks tab, which helps to identify any of these link types as ‘Uncrawlable’.

To find uncrawlable link types, please ensure ‘Store’ is enabled in the ‘Config > Spider > Crawl’ menu.

Please note – Our research indicates Google will crawl anything in the HTML that looks like a link. They will attempt to parse it and use for discovery.

6) Usage Stats

If you’ve ever wondered how much time you spend crawling, then you can now see your own usage data via ‘Help > Usage Stats’.

This has limited practical use (!), but is fun and might give you a better idea of how much value you’re getting from the tool each year. This data is stored locally on your machine.

Other Updates

Version 24.0 also includes a number of smaller updates and bug fixes.

New Arm64 Linux Versions – We now have two more packages available for Arm users for both Ubuntu and Fedora.
Improved Reporting of Syntactically Invalid Links – There’s a new ‘Crawl Invalid Links‘ option under ‘Config > Spider > Crawl’.
Skip Empty Reports – You can now skip exporting empty reports in ‘Output Settings’ of scheduled crawl tasks.
Ahrefs Country Level Metrics – There’s a new ‘Options’ tab in the Ahrefs integration with filters to adjust data for countries and volume (monthly or average).
Filterable Content Clusters – You’re now able to click on the content cluster legends in the graph to enable and disable their appearance in the diagram.
Live Model Validation & List In AI Integrations – When you connect to an AI provider, the model selected in the prompt will be checked against the live model list endpoint and alert you if they don’t exist.
System Wide Prompt for AI Integrations – In the ‘Advanced’ tab of each AI provider integration, you’re able to include a chat system prompt to establish the role, personality, constraints, and overall behaviour.
AI Provider Token Usage – Token usage is now displayed on the ‘Account Information’ dialog for each AI provider when connected.
Ollama Image Generation & Request Timeout – The Ollama integration now supports their image generation and has an adjustable ‘Request Timeout’ in the advanced tab.
Renamed Looker Studio Back to Data Studio Again – For our automated crawl reports in ~~Looker~~ Data Studio.
Java 25 – One for the purists. Version 24 has been updated to Java 25.

That’s everything for version 24.0!

Small Update – Version 24.1 Released 8th June 2026

We have just released a small update to version 24.1 of the SEO Spider. This release is mainly bug fixes and small improvements –

Added recently crawled URLs to scheduled crawl seed textbox.
Added a view to ‘Usage Stats’ graph to switch away from cumulative (a running total), to standard non-cumulative data points.
Added ‘Auto-start MCP Server on application launch’ setting for cloud.
Fixed unique crashes.

Small Update – Version 24.2 Released 22nd June 2026

We have just released a small update to version 24.2 of the SEO Spider. This release is mainly bug fixes and small improvements –

List available reports when a CLI crawl fails to start due to an invalid report name.
Remove the retired Google FAQ rich result feature.
Updated to new AXE version 4.11.4 for accessibility audits.
Fixed various unique crashes.

Small Update – Version 24.3 Released 29th June 2026

We have just released a small update to version 24.3 of the SEO Spider. This release is mainly bug fixes and small improvements –

Updated our version of jackson-databind from 2.21.2 to 2.22.0 to fix several CVEs.
Fixed various unique Java crashes on Windows, unfortunately this means reintroducing a crash when using RDP.

Dan Sharp is founder & Director of Screaming Frog. He has developed search strategies for a variety of clients from international brands to small and medium-sized businesses and designed and managed the build of the innovative SEO Spider software.