How To Check Every Page On Your Site For Google Analytics / Tag Manager Tags

Warning: this content is older than 365 days. It may be out of date and no longer relevant.

Does this sound familiar? Maybe you or someone you work with made a change to your website and suddenly, your site traffic is down considerably.

One of the most common reasons for Google Analytics to show a decline in site traffic is someone removing the tracking code. If you, like me, question whether every page on your website is tagged properly – especially when you use marketing automation software – then you’ve probably wished for a tool that checks every page for Google Analytics tags.

You, like me, have Googled for such tools and found them available, but at prices that seem a little steep, especially for a task that you shouldn’t need to do more than a few times a year at most.

So what can we do?

The good news is, we have access to open source tools which let us build these tools for free. Because they’re open source, they’re also much more flexible and adaptable. Let’s walk through the steps of setting up our Google Analytics / Google Tag Manager Power Tag Checker.


  • An operating system environment that supports Python 2.7+
  • Basic working knowledge of Python
  • A text editor
  • Optional: a cloud-based server so you can set it and forget it
  • The Scrapy Python library

Step 1: Scrapy Installation

If you don’t already have Python installed, you should install Python. Consult for specific instructions for your computer/operating system.

From the command line/terminal, type:

pip install scrapy

Allow Python’s installer, pip, to set the library up.

Step 2: Start a Scrapy project

From the command line/terminal, navigate to part of your hard drive or server where you keep documents and type:

scrapy startproject TagChecker

Your computer should say something that resembles this:

New Scrapy project ‘TagChecker’, using template directory ‘/usr/local/lib/python2.7/dist-packages/scrapy/templates/project’, created in: /home/cspenn/scrapers/TagChecker
You can start your first spider with:
cd TagChecker
scrapy genspider example

This will create a folder named tagchecker, and inside that folder will be a whole bunch of files. Don’t worry about them just yet. Follow the instructions from the startproject script to navigate down into the tagchecker folder in the command line/terminal.

Step 3: Create a Tag Spider

From the command line/terminal, type:

scrapy genspider TagSpider

For my site, I typed:

scrapy genspider TagSpider

Your computer should say something like:

Created spider ‘TagSpider’ using template ‘basic’ in module:

Step 4: Configure the Spider’s Item Collection

If you’re doing this on a server, open up your SFTP/FTP client. If you’re doing this on your desktop computer, navigate to the folder and subfolders.

Find the file.


Open it in your text editor of choice. Edit it to look like this:

(you can copy and paste this right into your file, unchanged)

This is telling the spider what items we want to collect – URLs and three kinds of tags. Note that these are entirely arbitrary; you could configure this spider to look for Marketo tags, Pardot tags, Adobe Omniture tags, etc. We’ll use Google Analytics and Tag Manager because that’s what most websites use.

Save and close the file.

Step 5: Configure the Spider’s Tag Detector

Next, find and open the file in your text editor. Edit it to look like this, but don’t obviously copy my website URL. Change it to yours!

What this script does, simply put, is crawl our entire website and check for three items – the old, outdated Google Analytics classic tracking code, the Universal Analytics tracking code, and the Google Tag Manager tracking code. If it finds any one of those three, it changes an output variable to 1; otherwise, output variables are 0.

Step 6: Run the Spider!

From the command line/terminal, run the following command:

scrapy crawl -o giveyourexcelfileaclevername.csv TagSpider

This will create a CSV file which you can open in Excel. Your command line/terminal window at this point should fill with text scrolling by at an astonishing speed as the spider does its work.

Step 7: Analyze Your Site!

Find the CSV file that the spider created in its folder.


Open it in Excel. What you’ll see is something like this:


As you can see, I use Tag Manager on my site, so the first two columns after the URL – Classic and Universal – are zeroes. Let’s apply some conditional formatting to the Tag Manager column, and suddenly everything will become clear:


In the case of my blog, I’m okay with not having the tracking code on my admin login page. However, if I saw this on any other page, I’d know I had tags missing – and what pages those tags were missing on. I could then go in and fix them.


Install these tools and use them to check your site for missing tags. As mentioned earlier, when you dig into the script, you’ll see how it detects different tags. If you’d like to track other systems like Pardot, Mautic, Marketo, etc. in addition to Google Analytics, just add the appropriate lines.

Disclaimer: The gists published in this post are released under the GNU General Public License. Absolutely no warranty or support is provided.

You might also enjoy:

Want to read more like this from Christopher Penn? Get updates here:

subscribe to my newsletter here

AI for Marketers Book
Take my Generative AI for Marketers course!

Analytics for Marketers Discussion Group
Join my Analytics for Marketers Slack Group!


2 responses to “How To Check Every Page On Your Site For Google Analytics / Tag Manager Tags”

  1. Doug Jensen Avatar
    Doug Jensen

    Christopher – If you want to automate, schedule, apply rules and manage implementation specs for site tags you can use ObservePoint. Happy to give you a tour of what we’ve built. [email protected]

  2. Andrew Li Avatar
    Andrew Li

    Hi, this is really useful, however, im stuck at configuring the items file. Step 4. Can you help?

Leave a Reply

Your email address will not be published. Required fields are marked *

Pin It on Pinterest

Share This