We launched our IntelliTraffic 'not provided' solution in April after months of development. At the moment at least, it is a fairly unique proposition in the SEO software industry in that it makes use of up to 3 data sources in order to restore keyword visit estimates in the platform. Essentially, it helps you to do the following:
However, naturally, this process can become quite complex, as there are certain factors you have to deal with when dealing with mixing these very different data sources.
The 3 data sources we use are:
- Google Analytics imports
- Keyword Ranking data, plus Search Volume and (customisable) Click Through Rates (CTRs) for different positions on Page 1 of the SERPs
- Google Webmaster Tools imports
Google Analytics imports
Google first introduced support for SSL searches in October 2011. For the first 12 months, the percentage of 'not provided' (NP) visits increased for most sites pretty steadily and was averaging around 20% after a year and we had expected, at that time, to see an average of around 40% by October 2013. However, in September 2013, Google decided to move over entirely to SSL searches. All organically referred traffic from Google would show up in your Google Analytics as 'not provided'. Prior to this move, you could have still extrapolated (from the keyword visit data you could still see in Google Analytics) up to a broad 'guesstimate' of what visits you were getting for each keyword. However, as the percentage of 'not provided' traffic continued to grow, this became an extremely futile and imprecise exercise.
So, is Google Analytics a pointless data source to use? Well, it depends. For most recent sites, it won't help . If your NP percentage is too high for the period you're examining, we won't use GA data as it would lead to misleading results.
Remember, though, we can backdate your Google Analytics imports! This means that where the 'not provided' percentages (historically) are low enough for your profile, we can still extrapolate from the specific keyword visit data we were still able to access at that time.
So, at some point in time during your campaign, your keyword visit calculations may use Google Analytics as a data source and then use CTR or GWT later on. Don't worry, though! The platform will work all this out for you!
We can take your rank for a particular keyword, together with the monthly Search Volume for that keyword and a CTR figure and estimate the number of visits you can be expected to have received for ranking in that position. The IntelliTraffic configuration screen allows you to configure your own CTR rates, should you wish to do so:
We've added this extra functionality, as SEOs will disagree about the likely actual CTRs for each ranking position, so we've allowed users to set their own rates. NB: these CTRs and all the other IntelliTraffic settings are set on a site/campaign basis, so you can vary them for different sites and different markets.
However, for us to make use of this data in order to estimate your keyword visits, the keyword would need to be ranked for the period you're analysing and we would also need a monthly Search Volume figure. If we don't have this data, it cannot estimate the visit numbers from just a CTR.
If this isn't clear, I'll illustrate it with a few examples:
Possible IntelliTraffic Scenarios
| Organic Rank
| Monthly Search Volume
| Visit Estimate
It doesn't matter that there is such a large Search Volume for this keyword, because the website is unranked (beyond 100 or the first 10 SERPs), so IntelliTraffic will assume a visit figure of 0 for that keyword
Now your website is ranked for a more specific keyword, but a rank of 42 is still unlikely to bring your site any traffic, so the IntelliTraffic keyword visit is still zero.
Colour Inkjet Printer
OK, so you're on Page 2, but this still isn't good enough to expect much in the way of organic traffic, so IntelliTraffic will still give you a visit estimate of zero.
Cheap Colour Inkjet Printer
| 6,345/month or 1,464/wk
Great! You're ranking #2 for a good key phrase and there's plenty of Search Volume, so we can work out an estimate! Whether you see a monthly estimate or a weekly estimate will depend upon the date range you have selected above the Keyword Visits table, but, in order to work out a monthly visit estimate for this keyword, we simply multiply the Monthly Search Volume by the expected CTR of 42.3% (or whatever you may have changed that CTR to for this particular site). This comes out at ~6,345 organic visits per month or 1,464 per week ((6,345 * 12)/52).
Google WebMaster Tools imports
If you turn this option on, we will import GWT keyword visit data every day and store it for you. GWT will only store the data in your account for around 90 days. The above image is an example of how the data comes out. We highly recommend you switch this option on (even though it is classified as an 'Enhanced Not Provided' solution and may cost you a few quid to upgrade), simply because it is the best data you can get direct from Google which shows you how many visits you're getting for each keyword. Remember, the CTR calculations are based on assumption of CTRs for each ranking position - essentially, educated guesswork.
However, there are a couple of limitations to the data shown in your GWT account:
- It is limited. Google say it will only show your 'top keywords'. Presumably, they mean your top trafficked organic search terms, but it's Google, so who really knows? It appears to be limited to around 2,000 keywords per account, so if you are likely to be getting traffic from many more search terms, you will need to rely on CTR calculations (or GA data, if you're looking into historical numbers)
- It will only show data for the past 90 days, so is not useful if you want to analyse a trend over, say, 6 months. However, you can do this, of course, with IntelliTraffic, as we will store your GWT data and also use other data sources to estimate keyword visit numbers over a long period.
Understanding the Data
Because this is quite a complicated thing to do and we are analysing a whole myriad of different sites with different traffic levels and ranking data, you may come across some keyword visit estimates which are not, initially at least, easy to interpret.
A) Flat columns!
This looks odd, right? Why would the visits for this keyword group be so flat for that period (18th May to 22nd May). Well, the answer is pretty simple:
- The columns might be right for that website (you might well have received 1 visit for that keyword group each day)
- We don't have ranking data for that period (you might not have daily rankings enabled) and we can't use GWT data (as there is a 5 day lag, as mentioned above), so the system has to make assumptions by using past averages to estimate your expected visit numbers for those days.
Don't worry, though! These numbers will continue to get updated as we continue to import GWT data and pull in more ranking data, so you are likely to see this pattern move from day to day (i.e. it will likely show the following day as flat columns for the period of the 19th May to 23rd May, and the next day show from the 20th May to 24th May, and so on).
B) There are lots of "N/As"!! What do they mean?
N/As simply mean that the platform doesn't have the requisite data it needs to calculate those particular numbers. Note, in the above screenshot, that the "Show Dates" option is set to "Last 3 full months". For this particular website, the platform obviously has enough source data (GWT and ranking data) in order to estimate visit numbers for the current quarter (3 months), but lacks enough data in order to calculate those estimates for the previous quarter. This might be simply that:
- GA data could not be used (as the 'not provided' percentage exceeded 50% for this profile)
- Those keywords weren't being monitored in that period, so we cannot use CTR-based calculations
- You didn't have GWT connected at that time, so we weren't importing your keyword visit data from GWT
Unfortunately, there's no way to rectify this situation and change those "N/As" into an intelligible and likely estimate. However, as you can continue to use IntelliTraffic, you're less likely to see instances of these appearing in your tables.
Your platform reports a "not provided" percentage of 0% - that can't be right!
The only typical cases we see now where a site reports a "not provided" level of 0% is where someone has setup a special filter in Google Analytics to split the "not provided" numbers into specific visit numbers for each landing page. This allows for the "not provided" numbers to be broken down into something like this in your Google Analytics account:
np - /
np - /example
np - /example/example.html
Our recommendation in these cases is that you will need a normal profile which our platform can use and a filtered profile which makes use of this advanced filter. If you have only one profile in your Google Analytics account for the site in question and have applied the above advanced filter to this already, then we would recommend that you create a normal unfiltered profile which our platform can use. Please note, however, that profiles only begin to record data in Google Analytics from the date at which they're created.