05/09/2018, Content Management Systems
What you are about to read is an SEO study gone horribly wrong.
There are several articles out there explaining which Content Management Systems (CMS) have the features necessary to optimise your site for Google. This study was supposed to be an analysis of the platforms actually performing best on Google and gaining the most organic traffic.
It turns out, this was something of a fool’s errand (with me being the fool). There are so many factors influencing a site’s performance on Google and most have little or nothing to do with the platform on which the site was built. Realising my mistake, I consulted some of the leading SEO experts to see if there was any way to salvage my study. The consensus was that it wouldn’t be possible without collecting a huge amount of additional hard-to-obtain data. Even with that data, the final analysis would still be inconclusive.
Rather than abandon the project entirely, I decided to analyse the data in a slightly different way. I obtained a list of one million websites from BuiltWith.com from eight of the most popular CMS platforms. I then used the wonderful Screaming Frog Spider to crawl the home page of all 1 million domains and the Ahrefs’ API to analyse the organic performance and backlink data of each site.
I therefore have no hypothesis to test nor can I tell you anything about with CMS performs best on Google. Instead, this is an analysis of how well (or poorly) each platform performs, irrespective of whether this was due to the CMS itself or the type of businesses using the platform.
I also wish to make it clear that I have no affiliation with any of the CMS platforms or SEO tool companies mentioned in this study. I am a provider of SEO training only.
There seemed little point including sites in this study that make no attempt to gain traffic from Google. This analysis was therefore restricted only to sites that met all the following conditions:
In addition to the above, I excluded three outlier domains that were getting so much organic traffic they would have skewed the results. That left a remaining 40,000 websites which still provided an adequate sample size across each platform. The fact that over 80% of these sites did not meet the above criteria tells you something about the sad state of small business SEO.
Link Metrics used in this study*
Although many different factors influence a site’s ranking on Google, external links are still regarded as one of the most important. This has been confirmed by Google on several occasions.
Since Google stopped publicly revealing their own PageRank metric, most of the major SEO tool companies have attempted to emulate this data with their own link quality metrics (with varying degrees of accuracy). For this study, I used the URL Rating metric provided by Ahrefs which I found to be highly correlated with organic traffic from Google (see chart below).
I generally avoid using Domain-level metrics such as Ahrefs Domain Rating or Moz Domain Authority as these provide a much weaker correlation with a site’s organic performance (see chart below)
*Read these posts to see a more detailed explanation of how Ahrefs URL Rating and Ahrefs Domain Rating are calculated and what they are designed to represent.
Ratio of HTTP to HTTPS sites by CMS
The chart below shows the ratio of HTTP (not secure) to HTTPS (secure) websites for each CMS.
Google has confirmed that sites using HTTPS with a valid SSL certificate is one of the factors their algorithm uses to rank websites. Non-secure sites now also display a “Not Secure” message in Google’s Chrome browser.
Most CMS platforms offer users the option of migrating their site to HTTPS. Clearly, some have made more effort than others. Wix and Weebly clearly need to help more of their users migrate to HTTPS.
Over 90% of Wix and Weebly websites are not secure. Click To Tweet
Avg. Server Response Time by CMS
The chart below shows the average server response time for each CMS.
Google has confirmed that page speed is a ranking factor and impacts conversions and visitor engagement. I decided not to evaluate the page load time as this would have been skewed by site owners who failed to optimise their images correctly or install heavy plugins on their site.
The CMS platforms that performed most poorly in this test are all self-hosted. It is no surprise that hosted platforms are better able to configure their server to provide the best performance environment for their own code. Self-hosted WordPress, despite its popularity, is plagued by poorly-coded plugins that have no doubt contributed to it receiving the lowest score in this test.
The performance of WordPress sites is often hampered by poorly-coded plugins. Click To Tweet
Avg. Page Size by CMS
The chart below shows the average page size (in bytes) for the home page of each CMS.
A page containing several unoptimised images or using a large amount of code will cause that page to load more slowly, particularly on mobile devices. A slow-loading page can negatively impact Google ranking and provide a poor user experience.
Without collecting a lot more data, it would be difficult to establish whether those with larger pages are due to the content added by the site owner or a poorly-coded CMS. As Shopify and Magento are more typically used by larger ecommerce businesses it likely that the home page of these sites contains more images than platforms such as Wix and Weebly.
URL Rating/Organic Traffic Correlation by CMS
The chart below groups all the sites on each platform by their URL Rating and then calculates their correlation to the organic traffic those sites received from Google.
When I analysed the correlation between URL Rating and organic traffic across all sites in this study, we see an R2 value of 0.9517. An R2 value of 1.0 would mean that URL Rating is perfectly correlated to organic traffic. An R2 value of 0 would mean there was no correlation at all.
I must however repeat the old adage that correlation does not imply causation. A strong relationship between two metrics may however give us a clue that there could be a causal link.
As you can see, platforms such as Shopify and WordPress have a strong correlation between URL Rating and organic traffic while other such as Wix and Weebly have a lower correlation. This suggests that even where Wix and Weebly users gain more links to their site, the impact those links have on their organic traffic is weaker than for other CMS platforms. This could either be due to their users being less likely to optimise their sites or possibly an indication of a limitation within the platform itself (possibly a combination of both).
Avg. No. of Ranking Keywords by CMS
The chart below shows the average number of ranking keywords for each CMS.
It is no surprise that sites built on Shopify and Magento rank for more keywords than SquareSpace and Weebly sites. A site that has more pages indexed by Google is likely to receive more organic traffic.
That said, the fact that the average Wix website is ranking for an average of only 3.46 keywords is worrying. Remember that I excluded businesses that had not make any effort to optimise their sites and had no external links. Wix has come in for a lot of criticism from the SEO community in the past and went to great efforts to defend their platform. It seems this criticism may not have been entirely unjustified.
The average Wix website ranks for only 3 keywords while WordPress sites rank for 418 keywords. Click To Tweet
Ratio of Sites with a Meta Description by CMS
The chart below looks at the number of sites on each CMS that have included a Meta Description on the home page.
A Meta Description is information about the page but is not displayed on the page itself. Google will often use the Meta Description in the organic listings on their search results pages.
Google doesn’t use the Meta Description to determine the relevancy of a page. It can however make a significant difference to the likelihood of Google’s users clicking on your search listings and visiting your site. It is therefore good practise to include a unique and compelling Meta Description on each page.
All platforms included in this study allow their users to add a Meta Description. I suspect that Wix may be adding a default description on each page which has resulted in their sites scoring well in this test. I was surprised to see SquareSpace users performing so poorly in this respect. This could be an indication that SquareSpace should do more to educate their users on basic SEO practices.
48% of SquareSpace websites have no Meta Description on their home page. Click To Tweet
Average URL Rating by CMS
The chart below shows the average Ahrefs URL Rating for each CMS platform.
Ahrefs URL Rating is a metric indicating the quality and quantity of links pointing to a page. It correlates strongly with organic traffic from Google.
In this analysis, I wanted to see which platforms’ users have been more successful in gaining high-quality backlinks to their site.
Please remember that I excluded from this study all sites with a URL Rating below 10. When I performed this evaluation across the original list of one million sites, the average URL Rating was only 9.87. Most sites have no backlinks at all or have only gained links from low-quality directories. Again, this chart tells us nothing about the platform itself but more about the ability of their users to gain quality links to their site.
Avg. Domain Rating by CMS
The chart below shows the average Domain Rating for each CMS platform.
Ahrefs Domain Rating is a metric that shows the strength of a website’s total backlink profile in terms of its quantity and quality.
Although I found a much weaker correlation between Domain Rating and organic traffic than for URL Rating and organic traffic, it is still a metric widely used in the SEO industry to evaluate the overall authority of a site.
It is interesting to see the stark differences between this chart and the URL Rating correlation chart above. It is clear that Wix and Weebly users are struggling to gain links to any page on their site other than their home page.
As I stated in the introduction, this study may tell us more about the typical users of a CMS rather than the capabilities of the platform. That said, I found the data interesting and I hope it might open up further questions and research.
I would like to show my heartfelt appreciation to the following smart people who kindly – and correctly – informed me that my original hypothesis was probably untestable. They saved me a considerable amount of wasted time and effort, but also helped me find a way to use this data in a way that might be useful to others.
Special thanks to:
Post by Danny Richman