I had heard about Screaming Frog SEO Spider a few times over the last few months. It’s a tool for seo and web developers that spiders a site in a similar manner to any search engine spider. As i have been testing out some of the latest seo tools out there, I asked Dan for a copy to review and he kindly agreed.
Screaming Frog is a GREAT tool to audit your website from a technical point of view, and can quickly collect a lot of the important information you need for examination purposes.
As usual, it’s available for PC and Mac. As a Mac user, I’m often confronted with software that slows my mac down a little, so it was a pleasure to see this thing run in the background in a quiet, efficient manner.
Run the spider and it will grab information on your pages, links, page titles, redirects, 404s, header tags, status codes, attribute lengths, anchor text, alt text and numbers of internal backlinks, numbers of external links on each page – and some juicy information for seo like click depth, canonical tags or meta-refresh detection, and more. You can also output this report to CSV to have a closer look at.
As the title says – highly recommended for seo and web designers everywhere. I ran this tool on a small site and on a site with more than 10,000 urls, and it worked no problems. You can get a free version too to test it out.
Run the spider and read up the general advice laid down in our on page seo topic page on this site.
There’s no affiliate links in this post by the way.
I personally use this tool to check 301 redirects. Its especially useful when dealing with a large amount of URLs. All you have to do is simply place all your old URLs in a text file and this will spider through and provide a header response allowing you to make sure the redirects are in place.