Authority Website Income
Shares

Definitive – Best Keyword Research Tool to Determine Keyword Competitiveness

My last post reviewed how effective MarketMuse was at predicting a piece of contents success in Google.

This post digs deeper into MANY keyword research tools to try and show what tools can help provide us an edge!

So… What is The Best Keyword Research Tool? What tool provides a keyword competition score that gives us the best ability to predict how well an article will do in Google?

 

Analysis Summary – What Tools Competition Score Correlated With a Higher Google Rank

The list below shows the scores correlated with a higher Google Ranking.

Comment 1 – The results surprised me with both Ahrefs and SEMRush underperforming my expectations and LongTail pro overperforming my expectations. I expected SECockpit to do terrible as the competition score it provides I never find to be a reliable analysis.

Comment 2 – The accuracy of this analysis essentially means those top 3 were tied. The margin of error would be larger than the difference between the results.

Based on this analysis I recommend using one of the top 3 tools when identifying keywords to create content for (it is now what we do at ContentRefined.com)! 

This post is going to help with exactly that, giving you the ability to choose the right tool available to give you a competitive advantage in targeting keywords that you can then turn into successfully ranking articles.

There are so many different websites and companies that have or are starting to develop keyword difficulty or SEO difficulty tools. But once you input a phrase and determine the existing competition and window of opportunity what weight should the number or percentage that they provide actually hold.

Previously in “Rank in Google with Certainty” I only analyzed Ferzy, SEMRush, and SECockpit on a very basic level. For this article I expanded that research to include ahrefs, Long Tail Pro, and cognitiveSEO while also including search volume influence (based on feedback from you in my last article).

I have also analyzed competition domain authority, trust flow, headline click-through-rate, and more but look for that to be shared in another post!

 

Now for the Fun Stuff (if you are a data geek!)

Using some articles from my various money site portfolio and all of ‘Content Refined created content, I have been able to compile enough data points to provide some statistical insight into predicting the performance of ranking an article around a certain keyword.

I have broken the 6 different tools into two tiers as the results within the tiers are statistically indifferent.

So here are the results…

Worst Performing Group (Worst Ability to Predict)

  • SECockpit

SECockpit keyword difficulty tool uses a percentage system. Scoring below 30% and as close to 10% while maintaining a reasonable search volume is a typical metric we use. However, with only a moderate correlation of 17% SECockpit has the least predictive tendency of the tested tools.

 

  • Ahrefs

Ahrefs provides an absolute number score out of 100 to determine the difficulty you would encounter to rank for a searched keyword.  While it only held a statistical correlation relevance of 23%, I do like that it provides a suggested required number of backlinks to go with the score.

NOTE – I was surprised by this result, the guys at AuthorityHacker who do great work had found AHrefs tool to be the best when it came to Keyword Difficulty but we found it to not perform very well.

 

  • SEMRush

With SEMRush you get a percentage score out of 100 with being below 70-75% as a target. Still only holding a moderate 25% correlation, SEMRush is the best of the tools found in our 2nd tier.

 

Best Performing Group (Best Ability To Predict How an Article Will Rank)

  • Ferzy

Ferzy is a newer tool on the market and will provide an absolute score out of 100 with varying levels of ratings categories. At a 35% statistical correlation, it offers to be one of the strongest predictors of keyword difficulty and ability to rank of the tools tested.

 

  • cognitiveSEO

cognitiveSEO is new to me and recently came out with a new keyword difficulty and content analyzing tool. Both seem to work great and at a 37% correlation it compares and competes well with some of the more well known tools. The content analyzing tool offers a competitor to MarketMuse at a more reasonable price, but the effectiveness difference is still to be determined…(more to come on this in the future).

 

  • Long Tail Pro

Behold the Long Tail Pro, the final tool analyzed. Coming in with a statistical correlation of just under 40%, it is the most predictive tool but not by much. I was surprised by this as LongTailPro has been around for a long time and I would have thought one of the newer tools would have come out with a better predictor.

 

Question – What if we adjust for search volume?

Monthly Searches ahrefs cognitiveSEO Ferzy Long Tail Pro SECockpit SEMRush
ALL 23.07% 37.49% 35.71% 39.37% 17.31% 24.17%
< 1,000 8.5% 27.52% 28.07% 24.30% 0.57% 19.48%
> 1,000 24.59% 30.22% 30.65% 39.28% 10.43% 19.57%
> 2,500 31.48% 31.68% 35.58% 44.29% 1.27% 29.62%

 

When the varying search volumes for a keyword were separated the results varied little. The three successful tools reviewed before held their ground in being the most successful predictors.

 

Summary

In conclusion, the results shown offer that there is a significant competitive advantage available if you are using the right tool. Ensuring that you follow a strategy of 500-1000+ monthly searches and a low recommended score from the varying scoring systems from tier 1 tools, you can start your content creation process with a stronger chance of ranking well in the end!

But remember, when looking to rank there are so many factors involved that only one step of the process can only provide so much insight and competitive advantage. 

While it is necessary to find good keywords, the rest of the process must be completed at the same level of effort and quality to maintain the opportunity to rank. So combining a keyword difficulty tool with a content analyzer tool, backlink analyzer tool, and so on, is essential to maximize opportunities.

Look forward to an upcoming post where I will wrap up this round of statistical analysis by exploring additional tools and factors that help you rank like domain authority, trust flow, and headline score. Finding out that there may be an auxiliary tool for helping rank could be a difference maker.

Hope you are enjoying these more statistically driven posts!

If you have any questions or ways we could expand on this analysis please let me know!

 

UPDATE – Based on Popular Requests Two Additional Tools Added…

Mangool’s KWFinder had not been used by me or the team until it became the most popular suggested and asked-about tool in the comments. Based on the recommendations I have gone back and completed the analysis to wrap up assessing the various keyword difficulty tools.
KWFinder’s tool assess the keyword on a scale out of 100 while providing a very user-friendly dashboard that provides additional search volume and SERP charts.

 

In my short time using KWFinder thus far I am very impressed with the value. In our statistical analysis the overall correlation was 43% with keywords that have >1000+ monthly searches having a correlation of 41%. The tool provides to have the highest correlation of any tools researched and remains true across various search volumes.
The tool instantly becomes an intriguing option for any level of operation with its variable pricing and added SERP Tracking features. More time spent using the tool will provide more insight but this tool is the absolute leader of all analyzed.

Based on recommendations here is the statistical analysis for Term Explorer keyword analyzer difficulty. Term Explorer provides a score out of 10 to its users that determine the level of competition of the keyword in terms of using the keyword to rank. From our brief analysis, it appears a score below 2.00 is a good number to reference for researching and maintaining below.

Overall the correlation was 28.16% for Term Explorer. A moderate correlation that places the tool somewhere between the 1st and 2nd tiers. It’s ranking scale makes it an attractive tool to operate and its ability to maintain projects and track given keywords regularly.

About the Author Jon Haver

I am a 33 year old husband, father of 3, engineer and a huge fan of developing systems to build useful and profitable websites. The reason I build online businesses is to provide financial independence for my family and yours AND so I can spend time outside skiing and biking with my family.
Jon Haver, Online Entrepreneur

Leave a Comment:

34 comments
Rob Stretch says August 14, 2017

Any thoughts on Mango Tools?

Reply
    Jon Haver says August 14, 2017

    We haven’t used… but will look to test it out as well and update the results! Thanks! Any other tool suggestions?

    Reply
      Jesse winters says August 14, 2017

      I would also like to know!

      Reply
      Joe says August 15, 2017

      Another vote for KWFinder from Mangools please: https://kwfinder.com/

      Seems like Ferzy is the least expensive option to get started with from your report?

      Reply
        Jon Haver says August 15, 2017

        We are working on it right now… stay tuned 🙂

        Reply
          rob says August 17, 2017

          I know you are working on it but i just wanted to say that my vote and money is also on kw finder mango. 😀

          Reply
Steve Blundon says August 14, 2017

Nice article Jon. As a long time Long Tail Pro user, I’ve seen the good, the bad, and the ugly. It’s certainly not perfect, but I agree the data analysis it performs is weighted enough in the right context to make it one of the best of the bunch.

Reply
    Jon Haver says August 14, 2017

    Hi Steve, yah this is just looking at the absolute score and nothing related to the keyword suggestion/UX… glad you found the article useful!

    Reply
john says August 14, 2017

I’m using kWfinder tool it works great for me, could you consider for your next analysis,
and also, another one is termexplorer.com but only used the free plan

Reply
    Jon Haver says August 14, 2017

    Thanks for the suggestions! Will look to update with all added suggestions.

    Reply
Res says August 14, 2017

The reason you are different than Authority Hacker is likely because you are doing a different test.

If you test only informative articles, you will get different results than if you test review articles only, and different again if you test a blend.

Reply
    Jon Haver says August 14, 2017

    Possibly, I reached out to them to understand how they had run their test so we can all hopefully learn more.

    Reply
Bobbi says August 14, 2017

Interesting! We used to use Longtail Pro b/c we had heard great things. However, we found that it glitches a LOT and that our results weren’t anything close to as on target as they used to be when we used another one or two of the sites that you mention here. lol Perhaps we’re doing something wrong? We are about to test Ferzy (since you had recommended it previously) and their support team seems awesome when we asked a few questions. Thanks for some insights and just more things to test and try. 😉

Reply
    Jon Haver says August 14, 2017

    Hi Bobbi, this was just looking at the 1 aspect of the tool and not trying to say what the BEST tool was overall. The fact is the different tools have different strengths. I find myself using SEMRush and SECockpit for different tasks then Ferzy/LongTailPro.

    Reply
Bernard says August 14, 2017

Really interesting so far. I would love to know your criteria for using Secockpit when it has the worst score.

Quote: “Based on this analysis I recommend using one of the top 3 tools when identifying keywords to create content for (it is now what we do at ContentRefined.com)! ”

So what is so different that you prefer using Secockpit?
Which of the top three tools do you use primarily, and what do you use Ferzy, Longtailpro for exactly?

Reply
    Jon Haver says August 14, 2017

    SECockpit & SEMRush I find are great at coming up with keyword ideas and that is what I use. Then I prefer Ferzy for the keyword score.

    If you wanted one tool to accomplish everything then LongTailPro would be good but I don’t find it as good as SECockpit for identifying keyword ideas.

    Reply
Matthew Allen says August 15, 2017

I do the same. I use SEMRush to find keyword ideas… they come up with WAY more than tools like Long Tail Pro or Ferzy. I previously used SECockpit for the same reason, but got tired of paying for both tools and SEMRush has way more useful functionality.

I then download my list of keywords from SEMRush and upload all of them to Long Tail Pro to get KC Scores. This strategy works quite well!

Ferzy is great for quickly checking the competition levels on one keyword at a time, and their visual layout is awesome. But they don’t have an option to upload large lists of keywords.

Reply
    Bernard says August 15, 2017

    Thanks, Jon and Matthew for the detailed answers. I use Secockpit and am shocked at its prediction percentage.

    Definitely going to try the others on some keywords I expected to rank better.

    Reply
      Jon Haver says August 26, 2017

      SE Cockpit is great for a lot of what you are after in a keyword research tool… this is just looking at one aspect of a KW Research Tools performance. But yah I was a little surprised too!

      Reply
Dean says August 15, 2017

Another vote for KWFinder, though I do use Semrush as the starting point for finding good keywords from competitors.

Having used 4 of the above including LongTailPro I find KWFinder way more accurate for competitiveness, easier to use with the colour coding system and faster too. Nice research though.

Reply
    Jon Haver says August 15, 2017

    Hi Dean, seems like a popular choice and we are working on it right now to see what the data says. Hopefully, we all learn something soon!

    Reply
Peter says August 17, 2017

Thanks for another great post Jon.

A tool that can accurately, and consistently, show you the difficulty of a keyword is the holy grail for internet marketers.

As you mention, the Google ranking algorithm is so complex that no single tool is up to the task, in my experience.

The tools you mention are a great starting point, but if you are serious about your website you must manually analyse the top 10-20 results for your most important keywords.

Reply
    Jon Haver says August 26, 2017

    Hi Peter, definitely agree that tools only provide an approximation (hence why the correlation isn’t higher) and to really understand you need to analyze the top 10 results.

    Reply
Joseph says August 21, 2017

WOW! Thanks for the article. Loved every bit of it. I have a question, Can you explain more why did you rate Ahrefs okey?

Another question, What do you think about (keywordrevealer) ?

Reply
    Jon Haver says August 26, 2017

    Ahrefs was ranked as okay simply because of where the correlation of its ability to predict how well an article ranked. Haven’t tried KeywordRevealer

    Reply
Taylor says August 23, 2017

What about keywordtool.io??

Reply
    Jon Haver says August 26, 2017

    haven’t tried it… will look to get it tested.

    Reply
Robin Wiman says August 24, 2017

Hi Jon, nice roundup!

I love to see statistical studies. Have you calculated the causality (R^2) in your data? Because correlation can be both random and very deceiving.

It would also be nice to see the data plotted on a bell curve because it looks a bit all over the place in your scatter plots.

I have no reason to believe this study is inaccurate, but as you probably get paid an affiliate commission for the tools you promote, I’m just curious if there is any bias in the study.

Reply
    Jon Haver says August 26, 2017

    Did not calcuate the coefficient of determination… agree R^2 could have been a useful test as like you said the data is pretty scattered.

    Regarding the bell curve I am not sure if that would make sense to do since so many of the data points would be sitting at #1…but maybe I am wrong on this and if you have suggestions would be happy to improve the analysis.

    Maybe that has left some error on the table which I have attempted to account for by grouping the clear difference in performance between group 1 and group 2. I am not attempting to say that a tool that scored 27% is always better then a tool that socred 25%.

    Regarding the affiliate commissions there are tools I don’t make affiliate commissions on that I have ranked higher then tools I do make a commission on… the data is the data and that is how the tools were ranked.

    Reply
Tony Hilsden says August 24, 2017

Hi Jon, This is the first I have heard of you or seen your articles, so maybe I am coming in half cocked… apologies.
We are completely revamping our old “non ranking’ website, new look, new approach, new structure and it will be live soon. I have been researching and playing with a number of the tools mentioned.. have LTP (old version) and also tested KW finder, which is great plus Keywords everywhere, so this article was very timely.
So it seems that many are using these tools to find a potential niche site that can be targeted by the use of Low competition, high search keywords and then structure the site and content accordingly.
Am I right in saying that for those of us with existing websites and products, that we are looking for L T keywords that have high searches, low competition which we can then include in existing content or write content about, in order to increase our organic ranking and capture the interest of an audience that uses search terms other than the usual or “seed” terms?

maybe I have researched this TO much, so if you could guide me a little or refer me to another of your blogs on the topic, it would be appreciated.. Great work none the less.

Reply
    Jon Haver says August 26, 2017

    Hi Tony, I would say you are definitely thinking about this correctly but also agree I think you might be over-thinking this!

    Reply
Barbara DiLucchio says August 25, 2017

Hey Jon, I really enjoyed this keyword tool correlation! This is the kind of stuff I really love it’s really interesting not to mention extremely helpful to have more confidence in the tools you are using. The results that shocked me the most were the SECockpit results-really awful and I thought that was one of the best! Well, I guess when it comes time to renew and they haven’t improved it then that will save me some money.

I was a little confused about what you were saying towards the end of the article. Were you saying that KWFinder is the best for keyword difficulty but you need the best tool for each of these categories to really give yourself the best advantage? I didn’t see all those categories mentioned when you were talking about the correlation score or is that included in it? I’m still a little confused on that one. But anyway I mainly wanted to tell you how much I enjoyed this article and thank you for including it!
Barbara

Reply
    Jon Haver says August 26, 2017

    Hi Barbara, thanks for the question and agree on the shock for SECockpit as it is often my go-to tool. What I was saying regarding the tools being the best at all categories is a keyword research tool needs to do more then just identify the competition score we want to use a tool to help identify keywords to go after (this is where SECockpit is great) we also want a tool that works fast and can analyze a competitors site fast (this is where SEMRush shines) but for a quick competition score (Ferzy can be great). So unfortunately no ONE tool is best at everything and that is what I was saying at the bottom.

    Reply
georg says August 28, 2017

Question KW finder for everyone

What Keyword SEO difficulty ranges in kw finder is low enough for you to go for?

A 0-19 (go for it)

B 19-29 (easy)

C 30- 39(still easy)

D 40+

? I would very much like to see how you guys interpreted these metrics

I personally go for A 0-19

Reply
Add Your Reply