Technical SEO can be daunting to those without a background in SEO or web development.
If you’re looking to hire an SEO agency or consultant then the technical side of SEO is one area where you can start to feel a bit lost and some SEO agencies will deliberately use this against you to sell you services you don’t need.
In this post I hope to signpost some technical SEO checks you can perform on your website for free before you consider hiring an SEO expert. This way you can troubleshoot technical SEO issues that you may have on your business website, and become more familiar with the terminology and concepts involved in technical SEO.
This is a kind of mini-introduction to technical SEO and by no means an extensive checklist – there is a lot that isn’t covered. My hope is that readers will find it accessible and discover some things they didn’t know about their business website and what it looks like from a search engine’s point of view.
Why it pays to learn a bit about the technical side of SEO
Many people find technical SEO daunting and confusing. Experts often don’t help things my throwing around terminology like robots.txt and client-side rendering Javascript without explaining themselves properly.
But the technical side of SEO doesn’t have to be so scary, and it is worth gaining some understanding of the technical side of SEO if your business relies on getting customers from Google and search engines.
I believe that, because technical SEO is so important, teaching clients about how it works and involving them in solving these problems as much as possible should be part of my job as a SEO specialist.
Optimising for robots
Within search engine optimisation you can divide website users into two types: humans and robots.
Technical search engine optimisation can be seen as optimising a website for robots, whereas content marketing and keyword research side of SEO are more to do with understanding human users.
Optimising for robots means looking at a website from a search engine’s point of view. The main goal is ensuring that search engines are able to crawl, access and index a website. In other words, the website has to make sense and be understood by Google’s systems in such a way that it can be stored and retrieved later in relation to a search query.
The following checks will reveal more about whether search engines have understood your website correctly or if there are technical issues that might be undermining your SEO strategy.
5 Simple Technical SEO Checks
- Check how many pages are indexed in Google (indexing)
- Use the Search Console Coverage report (indexing)
- Consider your site’s structure (crawling and ranking)
- Check your website’s performance (crawling)
- Check your 301 redirects (crawling and indexing)
1. Check how many of your pages are indexed by Google
This is a very easy check you can perform to see how many pages of your website are currently indexed in Google. using the “site:” search function.
Why this is important: If you find that there are fewer pages than you’d expect indexed then that indicates there may be a problem with either crawling or indexing of your site. If there are far more pages than you thought, including pages you didn’t intend to appear in public search results, this may also be something that should be corrected. Alternatively there may be fewer pages than you’d expect.
Go to Google search and type “site:yoursite.com”. You will be shown search results for your domain and an indication of how many results Google is indexing from your site.
Assuming you know how many pages are on your site, you can guage whether Google is under-indexing (or over-indexing) your content.
Don’t know how many pages you should have? You can check your content management system, your XML sitemap (if you have one), or use something like Screaming Frog (free up to 500 URLs) to crawl your site.
In the example above the number of pages listed in their XML sitemaps is around 400, while the pages in Google are 8700. This may indicate extra pages being indexed from somewhere that this company don’t intend to be indexed.
Some large websites and e-commerce sites often over-index in Google due to technical problems. For example, Google will crawl and index thousands of almost identical pages because of faceted navigation on e-commerce sites.
Have a look through your site’s content that is indexed by clicking through the search results after doing a “site:” search. Are these all pages you want Google to index?
You can also make additional searches using the site: parameter using keywords or in combination with other search functions. For example, type “site:yoursite.com filetype:pdf” to see if any pdfs are indexed that were just for internal use or supposed to be part of a lead generation strategy. The latter is a problem I found with the site pictured above whereby you can bypass the need to give them your email and just find their lead generation pdfs in Google.
Caveat: Don’t rely too heavily on “site:” as a “comprehensive list” of all the pages that Google has indexed (source). For our purposes here it is still a very useful starting point and suggestive measure of your number of pages.
2. Check your Search Console technical SEO report
Update: In 2020 Google re-introduced crawl stats report to Search Console. This report shows you where Google’s crawlers are focusing on your site and where they are running into problems. View crawl stats via this page.
This is another check you can do using information directly from Google – this time from Google Search Console (AKA Webmaster Tools). Google Search Console is an incredibly useful resource for your business and I urge any do-it-yourself SEO to look through all of the reports it offers.
Search Console provides information about which pages are indexed and which are not as well as telling you why, and indicating whether this has affected your visibility.
For example, above you can see a screenshot from a (very small) site in Search Console that I have permission to share here. It shows the number of pages indexed (green bars) and the impressions (i.e. how often the site is appearing in search results – blue line). You can see additional pages seem to coincide with a growing number of impressions – the content strategy seems to be working!
Check your error pages, excluded pages and warnings. If you see a drop in impressions that concerns you and this coincides with changes in indexed pages, you may have found a technical problem.
3. Check your site’s structure
This check is to do with how Google calculates authority and scores pages on the web. This is done using their PageRank algorithm which uses links between pages to determine which page has the greatest level of authority (or trust). The principle applies to links between domains as well as internal links within your website.
Checking PageRank of your internal links
As a guiding principle your most important pages should be well linked to on your site. Furthermore your site should be structured in a logical way that allows Google to crawl all of the pages via links – especially the important pages.
A good site structure will also tends to mean that Google only has to crawl a few pages deep into your site’s structure to be able to find all of the content. Poor site structure means that pages are buried deep in your website (30 clicks away from the homepage) and in some cases not linked to at all, therefore not discoverable by Google’s crawlers.
A simple and easy way you can check your site structure is by clicking around the site and looking for your most important pages. Are all of the key pages easy to get to this way or do you have to click a lot of links to find pages?
Another resource for checking internal links is Google Search Console. Go to the links report and you’ll be shown your most well-linked pages. Ideally you want these to be your key landing pages for users coming from Google.
More advanced tools for internal links analysis:
A more comprehensive way to check internal link structure is to use crawling software such as Screaming Frog, SEMRush or Ahrefs. All have limited trial versions that can be used to check your site’s internal structure.
If you use WordPress the plugin Link Whisper is a powerful tool for anlaysing site structure and automatically creating better internal linking. It isn’t free – prices start at $77 annually – but it’s cheaper than hiring a freelancer or purchasing licenses for other SEO software.
4. Check your site speed
Site speed is important. It may not have a direct impact on your rankings (source), but it has indirect consequences when users bounce away from your slow website back to search results, thereby telling Google the site isn’t what they’re looking for.
I like to suggest people do manual checks as you may have noticed. This also applies to site speed. Open your site in an incognito window in Firefox, Chrome and Edge to see if it loads within 4 seconds. Then test it on your phone using your phone’s internet connection. If it is painfully slow to load, that might be why your users are bouncing away and your rankings have dropped in Google. Check your competitors’ website as a benchmark.
To get deeper into speed checks you can use Google’s PageSpeed Insights to get some feedback on why your site may be slow. However, I would recommend speaking to a web developer or SEO specialist about the specific reasons your site may be slow. A lot of money is wasted fixing issues raised by automated speed checks with little improvement for users.
If you have a lot of pages and use Google Analytics, you can also use this to look at site speed data from real users of your site. Go to Google Analytics > Behaviour > Site Speed > Page Timings (screenshot below) to find pages that are below your site average. Are these pages also performing badly traffic and rankings wise? Then you may have found a problem.
There are many other ways to troubleshoot and fix site-speed issues using Google Analytics and PageSpeed Insights. For example, segmenting by user, device or browser. For detailed performance checks and troubleshooting I suggest considering my SEO Strategy Starter Pack or Complete Optimisation Services.
5. Check your 301 redirects
Many technical SEO checklists will talk about HTTPS and your site’s SSL certificate. This is because there is a myth that moving to HTTPS will suddenly increase your rankings.
You should implement SSL, but I haven’t included that in this list of checks because problems associated with HTTPS are more likely to come from poor implementation of a move from HTTP to HTTPS and incorrect use of 301 redirects. Moving to HTTPs itself is not a technical SEO fix, just something that has to be done with SEO in mind.
This brings us to the wider and much more important topic of 301 redirects. Understanding and implementing redirects properly is a fundamental aspect of technical SEO.
What is a redirect?
A redirect is a HTTP response status code that tells a client (e.g. a web browser) accessing a server that a URL has changed. A 301 redirect tells your browser that the page has moved permanently to the new URL.
Without this the page will return a 404 not found response instead or you may end up with two identical versions of a page for HTTP and HTTPs.
For SEO this means that both users and search engine crawlers need to be told when pages move to new URLs otherwise they will see a 404 error page. Both the user and the search engine will be lost. For search engine this means that all the old links to that URL will also no longer pass PageRank, and the new URL will not keep the PageRank score of the old page. This is why redirection is so important to get right.
How to manually check your 301 redirects for a migration or when moving from HTTP to HTTPS
If the URL of a site is changing then it is important to redirect ALL of the URLs from the previous URL to the new one.
If your site has moved domain or switched to HTTPS it may be worth checking that you have implemented redirects in this way. To manually check redirects you can use a browser plugin such as Link Redirect Trace (Chrome and Firefox) and test them yourself using your browser.
Don’t just check the homepage!
After testing the homepage, test more pages on your old domain to see if it redirects to the new one and the plugin will provide details of the redirect. For HTTPS sites, check HTTP versions of your top pages in the browser and make sure they redirect to the correct HTTPS page.
To check redirects in bulk it is best to use a tool like Ahrefs, SEM Rush or Screaming Frog which all offer limited free trial versions.
How to check 301 redirects and 404 pages in Search Console
Again, Search Console provides reporting on redirects and 404 pages that Google’s crawlers encounter on your site. Go to the Coverage report to see details of pages with 301 redirects. You can then test the links to see where the redirection takes you in your browser.
Search Console will also list 404 responses, which means that there are links resulting in an error. These may be due to URLs being changed that have not been redirected properly.
Conclusion
Hopefully this list has given you some insight into aspects of technical SEO and enabled you to do some straightforward tech SEO checks on your own website.
This post was intended as a resource for getting to grips with some common technical SEO checks but is not intended for use as advice or as a basis for making actual changes to your site. I would recommend speaking to an SEO specialist before attempting to fix technical SEO issues based on the information provided – simply because the risks of getting it wrong can be significant if your business relies on SEO traffic.
If you want to discuss technical SEO issues on your website further, use the contact form to get a FREE consultation and make sure to provide as much detail as possible about your technical SEO problem so I can look at it more closely.
