7 Ways to Improve On-Page SEO using Screaming Frog


On-page SEO makes sure that Google knows what your page is about. This can be as simple as making sure your targeted keywords are in the right places. Most of this has to do with the HTML elements of your page and how your content is structured.

I can’t emphasize the importance of on-page optimisation – we’ve significantly improved the organic performance for some of our clients just by optimising a few key on-page elements:

  • optimising title and meta description tags
  • implementing internal site linking
  • adding keyword-targeted anchor texts
  • submitting and updating XML sitemap

For this post, we also provide step-by-step instructions to use Screaming Frog Spider Tool to crawl on-site data for your website.

On-Page SEO Checklist

☑ <title> tag contains your focus keyword, ideally at the beginning

☑ <title> tag contains less than the viewable limit of 70 characters

☑ <meta name=”description”> tag contains your focus keyword and is between 165-320 characters.

☑ <h1> tag contains your focus keyword

☑ any images in your page contains an <alt> tag that contains your focus keyword

☑ there is at least 500 words in your page

☑ the content on your page is unique and not copied from other external sites or your internal pages

☑ your focus keyword appears in the first paragraph of the copy

☑ your focus keyword and related keywords (LSI, long-tailed, medium tailed) are used in several times throughout the copy

☑ your focus keyword is in the URL

☑ your focus keyword density is between 0.5% – 2.5%

Best Tool for On-Page SEO

If you haven’t already done so, download Screaming Frog on your computer now. The free version lets you crawl up to 500 URLs and is sufficient for small websites. It’s available for Windows, Linux and Mac.

Using Screaming Frog, you can crawl every URL on your site and identify on-page SEO issues you never even knew existed!

This tool was designed for in-depth technical SEO audits. For experienced users, you can even utilise advanced features such as auditing redirect chains, canonical errors, crawl path reports and more. But we’ll get into that another time.

Depending on the size of your site, crawling can take a few minutes or more than an hour. After the crawl is is finished, simply download the crawl data in .csv and start sorting! (my favourite part, actually). Don’t you just feel like an investigator?

So let’s get started, I will show you step-by-step how to use this tool and within minutes an On-Page SEO audit for your own website.

Quick Jump Menu

1. Title tag optimisation

Use your targeted keyword in the beginning of your title tag and keep it to 70 characters. Title tags show search engines the relevance of your page’s content, and significantly increases the user’s tendency to click.

Title tag audit using Screaming Frog

In the ‘page title’ tab:

  • filter by ‘over 65 characters’
    • overly long page titles are truncated on search engines.
  • filter by ‘missing page titles’
    • create two additional columns in this .csv export file, and add ‘optimised title tag’, ‘character count’
    • in the ‘character count’ column, add the formula =len(reference cell of optimised title tag)

What is a perfect title tag?

Truth is, there is no perfect title tag. But there are bad titles and some pretty kick-ass, awesome ones that we just can’t help but click. Art or science?

Here are some proven successful trends for writing great titles:

  • optimised for user’s search intent
  • contains targeted keywords
  • list-style
  • emotional

On-Page SEO title tag optimisation example

Having a good title tag not only increases CTR, but puts your targeted keywords on prime real estate on search results. We know that dwell time and clicks are factored in search rankings, and the importance of a low bounce rate (time that users users stay on your page). So optimising your titles is definitely a critical aspect of On-Page SEO!

Useful read:  This New Data Will Make You Rethink How You Write Headlines

Headline generators:

Portent Content Idea Generator 

Sumo Headline Generator 

Hubspot Blog Ideas Generator 

2. Meta description optimisation

The meta description in a page isn’t used directly in search engine rankings, but that doesn’t mean it’s not important. If the meta description contains the keyword query, then it shows up in search result snippets.

On-Page SEO meta description seo example

As you can see from the above snippet, the keyword query is bolded in the meta description which certainly helps users understand what the content of the page is about. The purpose of the meta description is to get searchers to click and bring visibility to keyword queries. Think of the meta description snippet as ad copy – summary of your page’s content, optimises for search intent and leads users to care about your page.

Meta description audit using Screaming Frog:

In the ‘internal’ tab, filter by ‘HTML’ and export all:

  • in the ‘meta description 1’ column, format cells with less than 320, and format in red colour.
  • next, check if the short meta descriptions are perfect on its own – summary of the page and contains targeted keywords. If they are, clear the red formatting on those cells as they don’t need to be changed.
  • prioritise optimising your meta descriptions which are ‘missing’, ‘duplicate’.

In the ‘meta description’ tab, filter by ‘missing, then filter by ‘duplicate’ and export.

Recently, google announced the latest update to the length of meta description displayed on SERPs.

The 320 character limit means we can take advantage of the longer snippet to sprinkle keywords naturally and write longer action-oriented copy.

What is a good meta description?

A good meta description is able to generate click throughs. Simple as that. Although there is no direct correlation with search engine results, there is an indirect affect – Click through rate (CTR). Google uses CTR as an indicator of whether your page is a good result.

Characteristics of good meta descriptions:

  • Between 160 – 320 characters
  • Contains primary keyword, and 3-4 related keywords
  • optimises for search intent
  • uses an active voice
  • should contain a call-to-action phrases, such as “try out for free”, “discover”, “find out”
  • unique and summarises the page’s content

3. H1 tag

h1 tag seo on page optimisation

The H1 element is the main heading for your content.

Why is the H1 tag important?

Header tags have always been important in search engine optimisation. In a Moz study on search engine rankings, “Page-Level Keyword & Content-Based Metrics” are the third most important ranking factor in Google’s ranking algorithm.

seo ranking survey results

Moz Search Engine Ranking Factors 2015

Audit all your pages’ H1 with Screaming Frog

In the ‘H1’ tab:

  • filter ‘missing’, h1 tags are important on-page elements for search engines.

If you’re optimising for a specific page, simply view the source-code and find “h1”.

On windows, CTRL + U

On mac, Option + CMD + U

How to create great H1:

  • Use only one H1 tag in each page.
  • Summarise your page’s content in the h1, usually the h1 can be similar to the title of the page
  • 20-70 characters in length. It should be like a title.
  • contain a medium or long-tailed keyword in your H1.
  • bold and style your h1 font size to be 2x bigger than your paragraph font. It should improve the user experience by providing structure to the body text

4. URL Structure

duplicate content url structure

Your site’s URL structure is both important for user experience and search engine optimisation. As a core guideline, your URL structure should just be really simple and easy to understand. So making them human-readable and as short as possible.

For example, if you’re searching for information about sauna, a URL like http://en.wikipedia.org/wiki/sauna will help you decide whether to click to that link. A URL like http://www.example.com/index.php id_sezione=360&sid=3a5bnd843f41daa6f849f730f1 is much less appealing to users.

Tips for creating SEO-friendly URLs:

  • Rely on static URLs.
    • Dynamic URLs are hard for users to read and understand, and web crawlers can get confused if its implementation is incorrect.
    • if your URLs are riddled with symbols, question marks, exclamation marks and other nonsense, work towards a static solution for a more identifiable and clean URL.
  • Use keywords in URLs but don’t keyword stuff. 
  • Use hyphens (-) for word separation. 
    • Not underscores ( _ ), not the pound symbol, not anything else.
  • Have only one URL for each page
  • Avoid creating overly complex URLs containing multiple parameters.
    • This creates unnecessarily high numbers of URLs pointing to the same or similar content on your site. As a result, Googlebot might use up more bandwidth than necessary or may be unable to index all the content on your site.

Common causes of problematic URLs:

Additive filtering 

When a site has many different views of the same set of items. This happens when you add filter and combine them in an additive manner (Eg: resorts on the beach and with a restaurant), causing URLs to continue driving up the number of variables in them.


Resort properties at “promotional rates”


Resort properties at “promotional rates” on the beach:


Resort properties at “promotional rates” on the beach and with a restaurant:


Sorting parameters

In e-commerce sites especially, sorting parameters are used to sort the same products in multiple ways. This also results in a large number of URLs.

Referral parameters

Brands and many websites that use affiliate marketing track affiliates with referral parameters. If that link gets shared, you don’t want it showing up as different pieces of content.

Using Screaming to check for URL Structure:

In the ‘URI’ tab,

  • filter by ‘duplicate’ to find duplicated URLs
  • filter by ‘underscores’ to find non SEO-friendly URLs

Note:* In large websites, especially e-commerce sites with multiple product categories, variations, models, etc, you’d have to manually check the complex URLs and their implications. Make sure you’re not mistakenly de-indexing or blocking web crawlers access to key product pages.

How to improve URL structure:

  • Use a canonical tag or 301 redirect for duplicated URLs to point to the original version
  • Use a robots.txt file to block googlebot’s access to problematic URLs
  • Wherever possible, shorten URLs by trimming unnecessary parameters
  • Simplify additive filtering and sorting parameters
  • Avoid using session IDs, use cookies instead

5. Canonical URLs

Unintentional content duplication due to improper URL handling can hinder SEO efforts dramatically.

Canonicalisation tells search engines which version of the URL they encounter to index. It basically says this to web crawlers: this is the original, correct version of the URL I want you to see!

As we have analysed earlier, common causes of mistaken content duplication are due to problematic URL structures.

Creating a large number of slightly different listings of the same content for the crawler to crawl and index is redundant. To remedy these problematic URL structures that might cause content duplication, you need to define a canonical tag for such content.

For instance, you can take care of product parameters that get added to tracking by adding a link element with attribute REL equals canonical to the head section of every page.

canonical URLs on-page SEO technical

So on page https://www.example.com/product I’d add a canonical with that exact URL. It’s a self-referencing canonical and should always be done with the full URL.

This way, the existence of www.example.com/product?refer=email or any infinite variations of tracking parameters will always point to the proper canonical URL in the HTML. Thus making it clear to google that there is no duplication.

How to find SEO canonical errors using Screaming Frog

In ‘reports’, find ‘canonical errors’ report and export.

  • This shows you canonical URLs that have technical issues – blocked by robots.txt, have no response, redirected, have status code 4xx (page not found error), status code 5xx (server error). Essentially, anything but 200 status code OK.
  • In the report, look to the column ‘unlinked’ with “True”- this shows you any URLs that were discovered only via a canonical, but are not linked to internally from any page in your website.

6. Word count

There was a recent study by Ahrefs showing that longer content ranks better. By analyzing rankings across 2 million keywords, they found that content with 2,000+ words dominated search engine result pages.

Thin content audit using Screaming Frog

Identify pages with thin content, so anything less than 500 words.

  • In the ‘Internal’ tab, filter by HTML, then download.
  • In the word count column, add a new rule and format cells that are less than 500 words, in red.

How to improve thin content:

At Oliver Moose, we focus on theme/topic as a whole and how the content itself answers the target keyword. Instead of just looking at word count as just a score, think of it as a benchmark for minimum quality. If a 500 word article can answer a targeted keyword fully, then by all means – sometimes less is more.

But if you would like to boost your content quality, here are 3 simple ways to implement right away:

  • Determine if these thin content pages are low-value pages, and should be deindexed with robots.txt file.
  • Create a content plan to add more words and depth to your articles.
  • Adding images, pictures and infographics boosts the quality of content too. This can be as simple as adding a few blog post graphics that summarises a paragraph.

Resources for creating blog graphics:



Icon Finder 

Creative Market

7. Image Optimisation

Optimising your images can have a huge difference in SEO and content quality.

In terms of SEO,

  • alt text: make sure your images have keyword-ed alt text associated with them so web crawlers know what your images are about.
  • image size: large image sizes increases the load speed of your site. Keep them under 100kb with image compression solutions.
  • image file names: you want google to know what your image is about. If you image is about lip balm, then it should be properly named “lip balm.jpg” instead of 1324DS.jpg

In terms of content,

Adding images enhances the quality of your content piece, afterall, a picture speaks a thousand words. If you have a long article, having visuals would help to digest information better.

Image optimisation audit with Screaming Frog

  • In the ‘Images’ tab, filter for ‘missing alt text’ to find images without alt text associated with them.
  • In the ‘Images’ tab, filter for ‘over 100kb’ to find overly large image sizes. These can be compressed, as smaller image sizes help to improve your site speed.

How to compress image size:

  • Use WP Smush or Imagify plugin for WordPress to compress any picture you upload automatically. If you want to compress your images beforehand, use online tools like TinyPNG or Optimizilla.
  • Another tip is to use JPEG files as they are smaller but not as high quality as PNG. The best option is to use SVG files which can scale image sizes without loss of quality.

There you have it, 7 on-page techniques using Screaming Frog to improve your SEO.

Was this article helpful? Leave your comments below!