📖 Quick Summary
Tool: Screaming Frog SEO Spider
Difficulty: Beginner
Reading Time: 6 minutes
Business Impact: ⭐⭐⭐⭐☆
Indexability Explained
Before a page can appear in Google Search results, it first needs to be indexable. Simply creating a page on your website does not guarantee it will appear in search engines. Various technical factors, including robots directives, canonical tags, redirects and server responses, all influence whether Google is allowed to include a page within its index.
Understanding Indexability helps identify why pages may be missing from search results, allowing Technical SEO issues to be diagnosed and corrected before they impact organic traffic, leads and revenue.
Indexable
What this means
The page is eligible to be included in Google’s search index.
Why this matters to the business
Indexable pages can appear in search results and generate organic traffic, leads and revenue.
How Growth Architect AI identifies this
By analysing robots directives, canonical tags, HTTP response codes and crawl behaviour collected through Screaming Frog SEO Spider.
What to do about it
Ensure important pages remain indexable while removing unnecessary pages from Google’s index.
Related terms
Non-Indexable • Noindex • Canonical URL
Non-Indexable
What this means
The page cannot currently be included in Google’s search index.
Why this matters to the business
If an important page is non-indexable, it cannot generate organic traffic regardless of how good the content may be.
How Growth Architect AI identifies this
By analysing all technical signals affecting indexability during the crawl.
What to do about it
Determine why the page is non-indexable and remove any unintended restrictions.
Related terms
Indexable • Noindex • Robots.txt
Noindex
What this means
A Noindex directive tells search engines not to include a page within search results.
Why this matters to the business
Noindex is useful for low-value pages, but accidentally applying it to important landing pages can remove them from Google entirely.
How Growth Architect AI identifies this
By detecting Meta Robots and HTTP header directives instructing search engines not to index a page.
What to do about it
Only apply Noindex where appropriate and remove it from pages intended to rank.
Related terms
Meta Robots • Indexable • Robots.txt
Canonicalised
What this means
The page points to another page as the preferred version using a Canonical tag.
Why this matters to the business
Canonicalisation helps prevent duplicate content issues and consolidates ranking signals onto a single preferred page.
How Growth Architect AI identifies this
By analysing Canonical tags found during the Screaming Frog crawl.
What to do about it
Confirm the Canonical tag points to the correct version of the content and has not been applied accidentally.
Related terms
Canonical URL • Duplicate Content • Indexability
Canonical URL
What this means
The Canonical URL is the preferred version of a page that search engines should index.
Why this matters to the business
When multiple pages contain similar content, Canonical URLs help consolidate ranking signals and reduce duplicate content problems.
How Growth Architect AI identifies this
By analysing the Canonical tag declared on each crawled page.
What to do about it
Ensure every Canonical points to the correct preferred page and avoid conflicting Canonical signals.
Related terms
Canonicalised • Duplicate Content
Blocked by Robots.txt
What this means
The page has been blocked from crawling by your website’s Robots.txt file.
Why this matters to the business
Search engines cannot properly crawl blocked pages, limiting their ability to understand your website.
How Growth Architect AI identifies this
By comparing Robots.txt directives against every crawled URL.
What to do about it
Only block pages that genuinely should not be crawled. Ensure important pages remain accessible.
Related terms
Robots.txt • Noindex • Crawl Status
Redirected
What this means
The requested page automatically forwards visitors and search engines to another destination.
Why this matters to the business
Redirects are useful when pages move, but excessive redirects reduce crawl efficiency and create unnecessary complexity.
How Growth Architect AI identifies this
By analysing HTTP redirect responses during the crawl.
What to do about it
Use permanent redirects where appropriate and eliminate unnecessary redirect chains.
Related terms
301 Redirect • Redirect Chain • Canonical URL
Duplicate Content
What this means
Two or more pages contain identical or substantially similar content.
Why this matters to the business
Duplicate content can confuse search engines, dilute ranking signals and reduce overall search performance.
How Growth Architect AI identifies this
By comparing page content, titles, meta descriptions and Canonical relationships across the website.
What to do about it
Consolidate duplicate pages, apply Canonical tags where appropriate or rewrite content to create unique value.
Related terms
Canonical URL • Canonicalised • Near Duplicate Content
Near Duplicate Content
What this means
Pages that are not identical but are extremely similar in wording or structure.
Why this matters to the business
Near duplicate pages may compete against one another, making it difficult for Google to determine which version should rank.
How Growth Architect AI identifies this
By analysing content similarity during the Screaming Frog crawl.
What to do about it
Consolidate overlapping pages or improve each page so it provides unique value.
Related terms
Duplicate Content • Canonical URL
Soft 404
What this means
A page appears to exist but provides little or no meaningful content, causing Google to treat it like a missing page.
Why this matters to the business
Soft 404 pages waste crawl budget and provide poor user experiences, reducing the overall quality of your website.
How Growth Architect AI identifies this
By combining crawl behaviour, server responses and page content analysis.
What to do about it
Either improve the page with meaningful content or redirect it to a relevant alternative.
Related terms
404 Not Found • Thin Content • Indexability
Why Growth Architect AI uses Indexability
Growth Architect AI doesn’t simply report whether a page is indexable—it helps explain why. By combining Screaming Frog crawl data with Google Search Console impressions and Google Analytics 4 performance data, Growth Architect AI highlights pages that should be generating organic traffic but are prevented from doing so because of technical SEO issues.