Disclosure: Hobo Web uses generative AI when specifically writing about our own experiences, ideas, stories, concepts, tools, tool documentation or research. Our tool of choice is in this process is Google Gemini Pro 2.5 Deep Research. This assistance helps ensure our customers have clarity on everything we are involved with and what we stand for. It also ensures that when customers use Google Search to ask a question about Hobo Web software, the answer is always available to them, and it is as accurate and up-to-date as possible. All content was verified as correct. Edited and checked by Shaun Anderson, creator of the Hobo EEAT tool and founder of Hobo Web. See our AI policy.
Screaming Frog is my favourite website development tool. It’s the best website crawler available at the cost, in my opinion, at least.
That’s why I have always used it for my crawl data source.
The tricky part of working with Screaming Frog (which you will have purchased separately) is just managing the files it produces, and prioritising that list in a time-efficient manner.
Hobo SEO Dashboard (available to buy here) is a Google Sheet, Sheet Template, and App Script Library, which you can add to your Screaming Frog workflow to do this exact thing.
When set up, Hobo works in concert with a Scheduled Crawl in your previously purchased licensed version of Screaming Frog.
It will autonomously cycle through loaded clients, load Screaming Frog Files, manage your Google Drive and email you (or your clients) your reports.
During beta testing, I have not run into API costs. It works perfectly on a free Personal Gmail account (tested with about 150 clients), although it will work best with a Google Workspace account.
There is no recording of data at the developer’s end (mine).
It is the real deal: a private, simple, tidy, managing and reporting script for managing SEO Audits and website development or migration projects and using Screaming Frog to its full reporting potential.
This uses your API credentials entirely through Google Sheets authorisation.
Key benefits
- Time Saver – Identifies your website development priorities in minutes.
- Time Saver – Manages, stores and collates 20-30 critical Screaming Frog crawl files saved to your Google Drive by Screaming Frog Scheduler (in minutes).
- Time Saver – Tidies your Screaming Frog files in Google Drive, moving used files to the Google Drive trash for 30 days – so restorable, too.
- Time Saver – Loads clients to a schedule you set.
- Time Saver – Built-in checks to ensure you are sending clients the correct report.
- Organiser – One report sheet is continually updated for the client report. Track your client data on one report sheet. The system creates a new document and continually updates that document for your client so it is one document you ever share with the client that is continuously updated.
- Time Saver – Emails you and your clients (optional) an SEO Audit based on Screaming Frog crawl data of the site to a schedule you set.
- Time Saver – Performs an automatic check against Screaming Frog data (which identifies content indexing issues) and a Hobo Premium SEO Checklist (guidelines laid down in Google’s documentation).
- Secure – Automatic versioning, Backup, Sharing, Access Privileges and Security built in and handled by Google.
- Library Based (App Script).
Hobo SEO Dashboard for Google Sheets, working in concert with Screaming Frog, Google Sheets, Google Drive and Gmail, helps you set up a quick and automatic SEO audit reporting system for multiple websites to a schedule you set.
Key features:
NOTE – I used AI to generate a feature list directly from the code it analysed, which is exactly the code behind Hobo SEO Dashboard for Google Sheets, and here it is below:
Hobo SEO Dashboard (Multi-Site) for Google Sheets
Hobo SEO Dashboard for Google Sheets is a tool designed to automate SEO analysis using your Screaming Frog data. It operates directly within Google Drive and Sheets, streamlining the process of identifying SEO issues such as underperforming pages and broken links. This tool is particularly efficient, working within the runtime limits of a personal Gmail account and capable of handling unlimited client sites without incurring high API costs.
By importing a copy of your Screaming Frog crawl outputs, Hobo SEO Dashboard for Google Sheets ensures a comprehensive review of your site’s SEO performance. Its automated reporting feature schedules and sends updates directly to you and your clients, saving significant time in manual reporting and analysis.
Hobo SEO Dashboard for Google Sheets emphasises efficiency and cost-effectiveness, utilising caching to minimise operational expenses and focusing on delivering direct, actionable insights to improve your site’s SEO.
It does this by:
- Retrieving and Processing Specific Files from Google Drive – Finds and processes documents from Google Drive for SEO review.
- Identifying Underperforming Pages – Highlights pages lacking engagement to signal performance issues.
- Searching for New Missing Link Errors – Detects recent 404 errors for correction.
- Finding New Server Error Instances – Identifies fresh server (500) errors affecting page display.
- Coordinating SEO Analysis Process – Manages a thorough SEO audit for site visibility improvement.
- Checking for External Broken Links – Documents broken inbound links from other websites.
- Identifying Images Without Descriptions – Finds images missing alt text for SEO and accessibility.
- Scanning for Potential Security Risks – Looks for site vulnerabilities compromising security or trust.
- Looking for Pages Missing Main Headings – Searches for pages without H1 tags, essential for SEO.
- Detecting Errors in Page Numbering Sequence – Corrects sequencing errors in page navigation.
- Identifying Errors in Navigation Links – Fixes navigational link issues for better site usability.
- Looking for Mixed Content Security Issues – Identifies secure/insecure content mix risking security.
- Finding Pages Without Titles – Pinpoints pages missing titles, crucial for search engine listing.
- Checking for Non-Searchable Reference Links – Ensures canonical links are searchable by engines.
- Identifying URL Issues with Spaces – Detects space issues in URLS causing broken links.
- Scanning for Unsecure Forms – Highlights forms with security risks.
- Looking for Mobile Page Linking Issues – Identifies indexing issues with mobile page links.
- Identifying Internal Links with No Response – Finds dead internal links affecting navigation.
- Checking for Unsafe Links Between Websites – Ensures external links are secure.
- Detecting Soft Error Links Within the Site – Identifies internal soft 404 errors.
- Finding Pages with Access Blocked to Content – Searches for pages with blocked content access.
- Identifying Redirects Within the Site – Looks for unnecessary internal redirects.
- Searching for Redirect Chains Within the Site – Identifies and corrects redirect chains.
- Looking for Non-Secure External Links – Checks external links for security issues.
- Analysing for Nearly Identical Content – Searches for duplicate content affecting SEO.
- Identifying Pages Lacking Incoming Links – Finds pages without internal linking.
- Inspecting for Pagination Issues – Checks for pagination errors affecting indexing.
- Cleans and organises Google Drive – Manages space by moving used files to trash.
- Adheres to Personal Gmail Account Limits – Ensures operation within Google’s script runtime limits.
- No Limit on Client Sites Handling – Manages SEO tasks for any number of client sites.
- Schedules and Emails Client Reports – Automates reporting to clients on a schedule.
- Automatically loads the Next Client – Moves to the next client seamlessly after task completion.
- Runs Reports According to Schedule – Automates report generation and execution.
- Uses the Cache at All Times – Reduces API costs by utilising caching for data retrieval.
More details to follow.
The Screaming Frog Integration
This is a quick video demonstrating what the Screaming Frog integration does. If you are an experienced SEO or web developer, you know organising and managing all these files can get pretty messy over time, and managing them can be an art in itself.
Hobo SEO Dashboard for Google Sheets is designed to speed all this up and take something that can take hours and do it in minutes.
Setting Up Screaming Frog
This system is designed to work with exported documents from Screaming Frog, and it should only take you moments to set this up.
When you purchase a Screaming Frog license, you can start to schedule crawls.
When you schedule a crawl, you want to save the exported files to your Google Drive. Screaming Frog Scheduler will format the files as they need to be for use in the Hobo SEO Dashboard for Google Sheets.
(Note – You can upload Screaming Frog files to your Google Drive manually, even using the free version of Screaming Frog for export files, but make sure you save the format as a Google Sheet specifically if you are hacking it like this. A rule of thumb is that the Hobo SEO Dashboard for Google Sheets files uses GREEN files in your Drive (Google Sheets), not BLUE files, even if they are named the same, possibly with a .csv extension. Just make sure your manual files are saved in Google Sheets format. )
I am currently grabbing screenshots for all this, I will update this later.
Important files to export to Google Drive from Screaming Frog will be the Crawl Overview, Internal_html, and any other files you are interested in.
A full list will be available here shortly.
Note that once the Hobo SEO Dashboard for Google Sheets system uses a Screaming Frog export file, it moves it to the Drive trash for 30 days, so it won’t be used again in future crawls.
1. Set up a Scheduled Crawl in Screaming Frog
Name your task and Project name whatever you wish, and set up the frequency of your crawls.
1.2 Configure Start Options
Add your website URL and your config file location (you save this in Screaming Frog)
You can save a configuration file in Screaming Frog for any particular client.
1.3 Set Up API Access
Set up API access in Screaming Frog (Hobo SEO Dashboard for Google Sheets uses Google Search Console data for now, but will soon also include Analytics).
2. Choose export files
Set up export files in Screaming Frog that Hobo SEO Dashboard for Google Sheets uses.
Just copy all my settings from the images:
2.1 Choose export tabs
Set up export files in Screaming Frog that Hobo SEO Dashboard for Google Sheets uses.
At the moment, it uses two Export Tabs:
2.2 Choose Bulk Exports
Set up Bulk Export files in Screaming Frog that Hobo SEO Dashboard for Google Sheets uses.
Hobo SEO Dashboard for Google Sheets uses the following Bulk Export Reports:
and:
(I’ll clarify the above asap)
2.3 Choose Reports
Set up report files in Screaming Frog that Hobo SEO Dashboard for Google Sheets uses.
It uses the following reports:
And that’s it.
Make sure you save to Gsheet format and connect Screaming Frog to Google Drive.
Required Screaming Frog Export Filenames for Hobo SEO Dashboard Integration
1. Introduction
This report delineates the specific Screaming Frog SEO Spider (SF) export filenames essential for the automated data integration with the Hobo SEO Dashboard.
The objective is to furnish a precise reference for technical SEO professionals and analysts configuring or managing this specific integration with the Dashboard and SF..
The integration between Screaming Frog and the Hobo SEO Dashboard relies on a defined automated process.
Screaming Frog’s scheduled crawl feature executes website audits at predetermined intervals. Upon completion, specific reports are automatically exported directly to a designated Google Drive folder.
Crucially, these exports must be in the Google Sheets (gsheet) format, as this is the format the Hobo SEO Dashboard is designed to access and process.
Hobo’s scripts then retrieve data from these specific Google Sheet files to populate its dashboard, identify technical issues, and potentially offer prioritised recommendations.
The accuracy and completeness of the Hobo dashboard are therefore directly dependent on the presence and correct formatting of these exact export files.
The identification of the required filenames presented herein results from a cross-referencing process.
The requirements detailed in Section 4 (“Key Screaming Frog Reports for Hobo SEO Dashboard Integration”) of the source documentation, particularly Table 1 and associated references, were compared against a specific list of potential export filenames also provided within the source material.
Only those filenames confirmed as necessary to work with Hobo SEO Dashboard through this comparison are included in this report.
2. Required Screaming Frog Export Filenames for Hobo SEO Dashboard
The successful operation of the Hobo SEO Dashboard hinges on receiving a precise set of data inputs from Screaming Frog. The following table lists the specific export filenames identified as mandatory for this integration, based on the analysis of the source documentation.
These files must be generated by Screaming Frog’s scheduled task function, configured to export directly to Google Drive in the native Google Sheets (gsheet) format.
Manual uploads or files in other formats (e.g., .csv, .xlsx) are generally incompatible unless converted to Google Sheets within Drive.
Adherence to these specific filenames and format is critical for the automated data pipeline that fuels the Hobo dashboard.
Table 1: Required Screaming Frog Export Filenames for Hobo SEO Dashboard Integration
|
Screaming Frog Menu Location |
Corresponding Report/Data Type |
Required Filename |
|
Reports |
Crawl Overview Summary Report |
crawl_overview |
|
Tabs |
Internal HTML Pages Data |
internal_html |
|
Bulk Export > Response Codes |
Internal Links to Unresponsive URLs |
internal_no_response_inlinks |
|
Bulk Export > Response Codes |
Internal Links to Client Error (4xx) URLs |
internal_client_error_(4xx)_inlinks |
|
Bulk Export > Response Codes |
Internal Links to Server Error (5xx) URLs |
response_codes_internal_server_error_(5xx)_inlinks |
|
Bulk Export > Response Codes |
Internal Links to Redirecting (3xx) URLs |
internal_redirection_(3xx)_inlinks |
|
Bulk Export > Response Codes |
Internal Links to External Client Error URLs |
external_client_error_(4xx)_inlinks |
|
Bulk Export > Page Titles |
Pages with Missing Title Tags |
page_titles_missing |
|
Bulk Export > H1 |
Pages with Missing H1 Tags |
h1_missing |
|
Bulk Export > Images |
Source Pages with Images Missing Alt Text |
images_missing_alt_text_inlinks |
|
Bulk Export > Canonicals |
Pages Canonicalized to Non-Indexable URLs (Inlinks) |
canonicals_nonindexable_canonical_inlinks |
|
Bulk Export > Pagination |
Pagination Sequence Errors (rel=next/prev) |
pagination_sequence_error |
|
Bulk Export > Pagination |
Pagination URLs Not in Anchor Tags |
pagination_pagination_url_not_in_anchor_tag |
|
Bulk Export > AMP |
AMP Pages with Non-Indexable Canonical Links |
amp_nonindexable_canonical |
|
Bulk Export > Content |
Near Duplicate Content Report |
near_duplicates_report |
|
Bulk Export > Security |
Internal Links to HTTP URLs (on HTTPS site) |
security_http_urls_inlinks |
|
Bulk Export > Security |
Pages with Mixed Content (HTTP resources) |
security_mixed_content |
|
Bulk Export > Security |
Pages with Forms Submitting to HTTP URLs |
security_form_url_insecure |
|
Bulk Export > Security |
Pages with Unsafe Cross-Origin Links |
unsafe_crossorigin_links |
|
Bulk Export > Security |
Pages Linking Out Using Protocol-Relative URLs |
protocolrelative_outlinks |
|
Bulk Export > URL |
URLs Containing Space Characters |
url_contains_space |
3. Analysis of Required File Set
The specific collection of filenames required by the Hobo SEO Dashboard reveals characteristics about the dashboard’s analytical focus and the nature of the integration.
A notable pattern is the prevalence of error reports. Files such as internal_client_error_(4xx)_inlinks, response_codes_internal_server_error_(5xx)_inlinks, security_http_urls_inlinks, and images_missing_alt_text_inlinks do not merely list the problematic destination URLS (e.g., the 404 page, the page with missing alt text).
Instead, they identify the source pages within the website that contain the links pointing to these problematic destinations or elements.
This structure strongly suggests that the Hobo dashboard is designed to facilitate remediation directly at the point of the error – the internal link itself.
By highlighting the source page, the dashboard streamlines the process of correcting broken internal navigation, updating links pointing to server errors, fixing insecure links, or adding missing alt text to images on specific pages, thereby directly improving user pathways and internal link equity flow.
Furthermore, the selection of required files demonstrates a focus on fundamental technical SEO health and core on-page elements.
The inclusion of internal_html provides core data about HTML pages, while reports covering response code errors (internal_client_error_(4xx)_inlinks, response_codes_internal_server_error_(5xx)_inlinks, internal_no_response_inlinks) , missing essential tags (page_titles_missing, h1_missing) , critical canonicalization errors (canonicals_nonindexable_canonical_inlinks, amp_nonindexable_canonical) , pagination issues (pagination_sequence_error, pagination_pagination_url_not_in_anchor_tag) , and various security vulnerabilities (security_mixed_content, security_form_url_insecure, unsafe_crossorigin_links) represents a concentration on issues with direct and significant impacts on website crawlability, indexability, user experience, and security.
Basic URL hygiene is also addressed through url_contains_space . This curated set indicates the Hobo dashboard provides a robust overview of essential technical status, prioritizing foundational checks.
The integration’s scope extends slightly beyond purely technical metrics, incorporating indicators of content quality and accessibility.
The requirement for near_duplicates_report, which identifies pages with highly similar content potentially causing keyword cannibalisation or signalling low value , and images_missing_alt_text_inlinks , which flags source pages with images lacking descriptive alt text crucial for accessibility and image SEO , suggests a more holistic approach.
By including these checks, the Hobo dashboard is equipped to surface not only technical faults but also opportunities for content improvement and enhanced web accessibility.
How to export to .gsheet format
Finally, the strict requirement for these specific filenames, exported automatically in the .gsheet format, underscores the integration’s reliance on a precise and automated data pipeline.
The system architecture anticipates Screaming Frog’s scheduler delivering these exact reports to the correct Google Drive location. Any deviation—incorrect filenames, wrong format (e.g., .csv files not converted to Google Sheets ), incorrect export location, or potentially incorrect export settings like failing to overwrite previous files —could disrupt the data flow and prevent the Hobo dashboard from functioning as intended.
This highlights the critical importance of meticulous configuration within Screaming Frog’s scheduling and export settings, ensuring strict adherence to the requirements reflected by this list of filenames.
4. Methodology and Exclusions
The list of required filenames presented in Table 1 was determined through a direct comparison methodology.
The export types explicitly mentioned as necessary for the Hobo SEO Dashboard in Section 4 of the provided source documentation were identified.
This list of requirements was then cross-referenced against a distinct list of potential Screaming Frog export filenames also included within the source material.
Only those filenames that were present in the provided list and corresponded to a documented requirement in Section 4 were included in the final compilation (Table 1).
It is important to note that the source documentation, particularly the comprehensive table in Section 4, lists numerous other reports and bulk exports available within Screaming Frog that cover a wide range of technical SEO checks (e.g., duplicate titles, duplicate meta descriptions, multiple H1/H2 tags, detailed hreflang validation errors, various sitemap issues, structured data validation details, etc.).
However, these additional export types were excluded from the final list in Table 1 because corresponding filenames were not found in the specific list provided at the end of the source material for the cross-referencing exercise, or they were not explicitly cited as required for Hobo.
The developer of Hobo SEO Dashboard, Shaun Anderson, confirms that eventually the system will manage more files.
This focused set of required files may suggest that the Hobo SEO Dashboard is designed to monitor a specific, curated group of common and often critical technical SEO issues, rather than attempting to replicate the full breadth of analysis available directly within the Screaming Frog application.
While the integration provides significant value through automated monitoring of these core areas, users requiring exhaustive analysis covering every check Screaming Frog performs (such as detailed international SEO validation or comprehensive sitemap diagnostics beyond non-indexable URLs) might need to supplement the Hobo dashboard insights with direct examination of Screaming Frog crawls or utilize other reporting solutions.
5. Conclusion
This report identifies the specific set of Screaming Frog SEO Spider export filenames required for successful data integration with the Hobo SEO Dashboard.
The analysis, based strictly on the provided documentation and cross-referencing specific requirements with a supplied list of filenames, confirms the necessity of the files listed in Table 1.
The successful implementation of the automated workflow between Screaming Frog and the Hobo SEO Dashboard is contingent upon precise configuration.
It is essential that Screaming Frog scheduled tasks are set up to export exactly these identified files, using the mandatory Google Sheets (gsheet) format, to the designated Google Drive location.
Failure to adhere to these specifications regarding filenames, format, or export destination will impede the data transfer process and prevent the Hobo dashboard from accurately reflecting the website’s technical SEO status based on the latest crawl data. Ensuring configuration accuracy is paramount for leveraging the efficiency and proactive monitoring capabilities offered by this integration.
Note – Hobo SEO Dashboard will automatically import SF files you have in your Google Drive, periodically as per the schedule that is set. This will result in either a new client in the Client tab being generated or updating a current client report in the Clients tab. This is automatic as part of Automatic Reporting in your Hobo SEO Dashbaord menu.
Multi-Site Client Website Manager System
I’ve made it as simple and cost-effective as possible to manage an SEO audit process for multiple websites, to a schedule you have full control over.
The system is designed to manage multiple websites and multiple client website data files in your Google Drive folder.
Hobo SEO Dashboard for Google Sheets will run your SEO audits, manage your crawl files, create and develop a central client report, email clients links to the central report and load your next client to process.
You can use your own template with it (and add your own branding).
You can modify the Settings tab to hack reports and change limits on reporting and such.
Once it is set up it IS an autonomous reporter and will run to your schedule.
More Hobo SEO Dashboard for Google Sheets Videos coming shortly:
- Performing A Full Manual Audit with this system (Soon)
- How it works underneath the hood, so to speak (Soon)
- Code Validation (Third-Party code validation – Soon)
- All of Screaming Frog Reports (In the next major update)
See reactions for Hobo SEO Dashboard for Google Sheets on Linkedin and Twitter.
This is very much an indie-hacker project, and I’ve built a lot of it in public if you have been following along.
This is NOT affiliated with Screaming Frog, and you need a Screaming Frog license to use it to its full potential (although it will run Search Console Reports without it).
As an SEO manager or web development manager, it will be among the best purchases you’ve made recently.
NB: Note Current Version is V6. Any systematic abuse will not be tolerated. Your use is governed by a fair use policy and can be suspended (it is very unlikely normal use would trigger such an event, and it runs autonomously all day, every day).