Can you trust Trustpilot? – 9 million reviews studied

26/06/2019, SEO

Can you trust Trustpilot? – 9 million reviews studied


I studied 9 million reviews left for 198,000 businesses on the Trustpilot UK website to reveal how companies deceive consumers by manipulating their reviews on Trustpilot.

In this article, I will explain how they do it and and reveal the worst offenders.

An increasing number of people use the web to research businesses before purchasing a product or service.

A recent study by BrightLocal showed that 86% of consumers consider online reviews when making a purchase. 68% agreed that positive reviews make it more likely they will buy from the business; while 40% say that negative reviews make it less likely.

 

 

68% of consumers agree that positive online reviews make it more likely they will buy from a business; while 40% say that negative reviews make it less likely. Click To Tweet

Although the vast majority of companies do not manipulate their online reviews, consumers should be able to rely on this information when making important financial decisions.

When companies manipulate reviews to paint a false picture about their business, it has a detrimental effect on both consumers and the honest businesses that may appear less attractive in comparison.

Although businesses owners found guilty of review fraud could face a prison term of up to 10 years or a fine, prosecutions and convictions are rare and seldom policed.

In this post, I will reveal the methods these businesses use, how you can spot the manipulation yourself, and reveal the worst offenders.

1) Review Gating

What is Review Gating?

Review Gating is the process of filtering customers before asking them to leave a review for your business. The aim is to encourage reviews from only customers you believe had a positive experience with your company and ask for private feedback (or none at all) from those with a negative experience.

Although review gating doesn’t entirely remove the possibility of negative comments, it can significantly reduce the ratio of negative to positive reviews. Over time, negative reviews are pushed deeper into the review feed, and the company’s average review rating remains positive.

Most of the major review publishing platforms, including Trustpilot and Google, warn against this illegal practice in their guidelines but appear to take little action against those companies found to be in breach.

How do businesses use Review Gating?
The internal memo shown below from Yell UK (formerly Yellow Pages) reveals how staff are incentivised to “ask every happy customer to write a review on Trustpilot for Yell” and to not ask if “there’s any risk [they] may leave a negative review”. Yell reward their staff with £10 for every 5-star review and £5 for every 4-star review posted.

Internal memo to Yell employees incentivising them to request 4&5-star reviews only

This memo (sent to me by a whistle-blower at Yell) is a classic example of review gating. Despite this memo being forwarded on to Trustpilot, none of Yell’s reviews have been removed and no action appears to have been taken against the company.

Some businesses use automated review management software which analyses past email communications with the customer, internal complaints records and the history of product returns. The software then sends a review request to the customers most likely to post a positive review for the company.

 

2) Review Flagging

What is review flagging?

Trustpilot allows businesses to flag a review for removal if it breaches any of their review guidelines. The review is then removed from the Trustpilot platform if found to be in breach. There are several grounds for which a review could be flagged.

These include (but are not limited to):
• The review contains offensive language.
• The review mentions the name of an individual.
• The review describes the functionality of the purchased product.
• The service experience has not occurred within the last 12 months.

The following examples could all potentially be in breach of Trustpilot’s guidelines:

  • “I received great service from John Smith. He installed my dishwasher quickly and efficiently with no fuss” – (Names an individual).
  • “My dishwasher broke down within two weeks of being installed and couldn’t be repaired – not happy” – (Describes the functionality of the purchased product.)
  • “The dishwasher installed by Smith Appliances cleans my dishes far better than any dishwasher I have owned – I am delighted!” – (Describes the functionality of the purchased product.)
  • “The dishwasher installed by this company has been working perfectly for more than 2 years without a glitch. – (Describes a service experience that has not occurred within the last 12 months.)

As you can see, it’s just as likely that a positive review could be flagged as a negative one. This is confirmed by analysing the reviews that Trustpilot flag for comments left about their own business.

58% of the reviews that Trustpilot have flagged about their own company are positive, 37% negative and 5% are neutral. This shows that when flagging reviews fairly, a negative review is no more likely to be in breach of Trustpilot’s guidelines than a positive one.

In a Techcrunch interview in March 2019, Trustpilot’s Founder and CEO Peter Mühlmann stated, “The vast majority of companies flag questionable reviews in exactly the right way. However, there’s a small minority that are potentially ruining it for everyone by aggressively flagging bad reviews.”

He goes on to explain how, “Trustpilot now reveals how many reviews were taken down because they represented a real breach of Trustpilot’s policies, versus reviews where the poster simply didn’t respond to questions, versus legitimate reviews that were ultimately reinstated”. Mühlmann warns, “If marketers are motivated to cheat on our platform, consumers are going to see them do it.” 

In fact, my research shows that businesses are 33 times more likely to flag a negative Trustpilot review than a positive review.

Businesses are 33 times more likely to flag a negative Trustpilot review than a positive review. Click To Tweet

Let’s look at some examples of businesses that don’t appear to be flagging reviews fairly:

Parcelforce Worldwide

77 reviews flagged | 99% negative | 0% positive

 

PayPal

75 reviews flagged | 100% negative | 0% positive

 

Wren Kitchens

134 review flagged | 100% negative | 0% positive

 

Webuyanycar

77 reviews flagged | 100% negative | 0% positive

 

Yell.com

132 reviews flagged | 100% negative | 0% positive

When a review is flagged, it is then appraised by a member of the compliance team at Trustpilot. If the review is found to be in breach of their guidelines, it is removed from the Trustpilot website. Approximately, 60% of all flagged reviews are removed from the Trustpilot platform.

Selectively flagging only negative reviews is, therefore, a highly effective method of increasing the average rating of the business.

Businesses are unfairly flagging negative reviews to manipulate their Trustpilot rating. Click To Tweet

Some businesses are so keen to flag negative reviews they make little or no effort to determine whether the review is even in breach of Trustpilot’s guidelines. They flag every negative review and leave it up to Trustpilot to determine whether the review is, in fact, in violation.

For example, 75% of the reviews flagged by Parcelforce are placed back online by Trustpilot and found not to be in breach. To the best of my knowledge, Trustpilot has taken no action against Parcelforce for this extreme manipulation of their review profile.

 

75% of reviews flagged by Parcelforce are placed back online

 

75% of all reviews flagged by Parcelforce are placed back online by Trustpilot. Click To Tweet

 

3) Fake Reviews

What are fake reviews?
Most people are now aware that companies can buy fake online reviews to enhance their online profile. There have been several recent high-profile stories in the media highlighting this issue calling on Trustpilot to take action.

A quick Google search reveals a wide selection of businesses happy to supply verified reviews on Trustpilot and Google at a low cost. Many of these companies are based in East and South-East Asia.

Most of the businesses selling reviews make little effort to cover their tracks. They often post under a name uncommon in the UK, write short reviews comprised mostly of complimentary adjectives and are likely to have published only one review under the same user account. Fake reviews also tend to be posted in batches, with a disproportionate number of 5-star reviews posted on the same day.

 

Company selling fake Trustpilot reviews


Fake negative reviews

Some businesses also buy fake negative reviews for their competitors or encourage their own staff to leave fake reviews online.

Take a look at these reviews on Google left by a sales executive working for Yell from their Belfast office.

You will see he has left a total of 45 reviews including a 5-star review for his employer.

His reviews include a tattoo parlour in Belfast, an ear wax removal company in Yorkshire, an industrial lubricant supplier in Cheshire, a chimney sweep in Lancashire, a decorator in Dundee and four different removals companies all at opposite ends of the country. An unlikely pattern of reviews for any genuine customer.

 

Location of reviews left by Yell employee based in Belfast

 

You will also notice that the typical comment posted with these ratings is, “great business” or “great business to deal with”. Comments that are sufficiently ambiguous as to disguise the true nature of the relationship. 95% of the reviewed businesses also happen to be advertisers with Yell.

It may come as no surprise to learn that Yell offer a service known as ‘Reputation Manager’ which offers to improve the online profiles of their small business clients. Yell staff are incentivised to gain positive reviews for their clients.

I have evidence of at least 15 other Yell employees engaging in similar activities. Yell cannot reasonably claim that these are just the activities of a few rogue employees. One of Yell’s employees even left a negative review for my own business, despite never having been my client or even communicating with me.

 

Conclusion

While analysing the data for this article, I discovered a few other interesting facts about online reviews more generally.

There is an assumption that consumers may be more likely to leave a negative, rather than positive, review . In fact, of the 9 million reviews posted on the Trustpilot UK website, 75% are 4 or 5-star reviews. Only 10% of reviews have a 1 or 2-star rating. Of course, it’s entirely possible that may also be indicative of excessive review manipulation.

75% of Trustpilot reviews are rated 4 or 5-stars. Only 10% of reviews have a 1 or 2-star rating. Click To Tweet

The average rating across all reviews posted on Trustpilot is 4.3, and the vast majority of businesses have fewer than five reviews in total.

 

Total number of Trustpilot reviews by rating

Winners and Losers

The business with the highest number* of genuine, 5-star only reviews is Artful Hen Parties, a hen party organiser.

 

The organisation with the highest number of genuine 1-star reviews is (amusingly) The UK Parliament

The UK Parliament has highest total number of 1-star reviews on Trustpilot Click To Tweet

 

The company with the most reviews in total on Trustpilot UK is the courier, Hermes UK with an average rating of 4 stars from 173,108 reviews. 

* Only profiles with more than 100 published reviews were analysed for this article.

We are now seeing Google updates that ensure only trustworthy information appears in their search results for important health-related and financial queries. Social Media networks are also being forced to take responsibility for the “fake news” that appears on their platforms. I feel strongly that review publishers should be held equally accountable for the information that appears on their platforms.

Trustpilot charge businesses a substantial amount of money to make full use of their platform. A significant proportion of that fee should be used to ensure that their platform is being used fairly and the reviews can be trusted.

Trustpilot must make significant improvements to their fraud detection algorithms and take strict action against repeated offenders. They should not be relying on consumers to do the job of their compliance team.

Update 28th June 2019

The day after this article was published, the following message suddenly appeared on Yell’s Trustpilot review page:

Trustpilot warns consumers that Yell’s rating can’t be trusted.

It is clear that this rushed message, littered with grammatical errors, is an attempt to limit the damage from this story spreading beyond my humble blog.

It does, however, precisely demonstrate the sort of action Trustpilot should be taking were they genuinely committed to full transparency.

When companies use the techniques described in this article; a bold, yellow message like this one should appear at the top of their Trustpilot profile to make it crystal-clear that the business’s Trustscore cannot, in fact, be trusted. It shouldn’t depend on consumers noticing a small link, buried deep on a page, and then having to further click on a “details” link to then establish whether a company has been guilty of review manipulation.

Displaying a prominent message that warns the public that review manipulation has taken place (as they have now done with Yell) is, surely, the most effective method of deterring other businesses from engaging in similar activities.

Trustpilot want me to make it clear that only a minority of businesses are guilty of review manipulation. While that may be true, it does not remove the need for this issue to be made more public.

In some ways, the fact that it is only a minority of businesses makes it harder for the public to spot. If the majority of businesses were guilty of these practices, the public would know to completely disregard this information when making financial decisions.

When large brands such as Yell, Parcelforce and Webuyanycar are allowed to get away with review manipulation, it affects millions of people who rely on Trustpilot to make sure these companies play by the rules.

I am pleased that Trustpilot has finally taken this action against Yell. I would have preferred it to have been born from a genuine desire for full transparency, rather than a knee-jerk reaction to my article. I can only hope that Trustpilot now implement a more permanent, and considered, solution to make it clear to the public when review manipulation has taken place.

Update 16th September 2019

The message displayed on Yell’s Trustpilot profile warning visitors that the company had been fraudulently manipulating their reviews was removed by Trustpilot just a few short weeks later. 

Despite the removal of this warning, Yell’s Trustscore (based on this manipulation) remains unchanged. I am reliably informed that this is standard practice for how Trustpilot manages manipulated reviews.

If Trustpilot are genuinely concerned that a company’s Trustscore should accurately reflect all of the reviews received by a company, they should either leave the warning notice permanently in place or adjust the business’ Trustscore accordingly. The fact that they don’t, surely makes Trustpilot knowingly complicit in this fraudulent deception?

 

Update 21st October 2019 

I was recently interviewed by BBC Watchdog for their episode broadcast on Thursday 17th October.

The story focuses on a Watchdog viewer who lost £20,000 after investing with a company with positive reviews on Trustpilot. It turned out, the company was on an FCA warning list and classified as unauthorised and unregulated. When the Watchdog viewer posted a negative review on the firm’s page, the company reported it to Trustpilot and his review was removed.

Watchdog’s research found many more financial companies the FCA has put on its warning list on Trustpilot – and some of these had ratings of three stars or above, meaning customers would understand that they were “average”, “great” or even “excellent”. This is despite the fact that the FCA has warned people to be wary of them.

Trustpilot provided the following (edited) responses to which I have added my own comments below:

Trustpilot: “We allow anyone to post a review as long as they comply with our guidelines. No company can edit or prohibit a review from being published.”

Danny: While it is true that a company cannot prohibit a review being published, they can request for a review to be removed. In this case, the company in question had all negative reviews flagged and removed leaving only the positive reviews remaining and their Trustpilot rating intact.

Trustpilot: “Of the reviews on [the company Watchdog’s case study invested in] currently 18 percent are 1-star negatives where consumers do exactly this – share their poor experience both with the company and other consumers.”

Danny: The 18% of 1-star reviews to which Trustpilot refers have all been flagged and removed from the company’s profile. These reviews are therefore not available to any other consumers.

Trustpilot: “Regarding the positive reviews on [this company] our initial investigation in response to outreach from Watchdog has identified suspicious patterns. We are therefore conducting a fuller investigation now.”

Danny: If Trustpilot’s fraud procedures are as effective as they claim, why does it take a Watchdog investigation to prompt an investigation? Why were these “suspicious patterns” not automatically flagged by their technology or compliance team?

Trustpilot: “Since the page went live, our technology and systems have filtered 12 reviews, recognised as being fake or spam. This amounts to 57 percent of all reviews written – both positive and negative reviews that violate our rules.”

Danny: Despite Trustpilot admitting this company’s reviews are fake, their profile page is still live (as of 21/10/19) and showing only positive reviews with a 4-star rating. 

Trustpilot: “When it comes to which companies have a profile page on Trustpilot, it is consumers who decide who they want to review. A company profile page on Trustpilot is automatically created when a reviewer posts the first review about that business. This enables consumers to share their bad experiences to warn other consumers, as well as good experiences.”

Danny: This also allows scam companies to pay for fake reviews, quickly build a positive Trustpilot rating and have those reviews published without any verification from Trustpilot. In the same way that the BBC check the validity of stories before being published on their website, the onus must surely be on Trustpilot to check the validity of content published on theirs? This is especially important when consumers are making important financial and health-related decisions on the basis of these reviews.

Trustpilot: “We are constantly looking at new preventative measures, including the use of additional penalties and actions, that will be used against anyone that tries to manipulate reviews.”

Danny: When Yell were caught manipulating reviews as a result of my article, a warning message was displayed on their Trustpilot profile for only two weeks and then quickly removed. Yell’s Trustpilot rating remains unaffected. To the best of my knowledge, Trustpilot has taken no further action against Yell or any of the other companies mentioned in this article.

Trustpilot: “The vast majority of companies use us for free and a smaller percent pay us for additional services such as automated sending of invitations and consumer insights gathering.”

Danny: In fact, Trustpilot generates an estimated £100 million per year from their paid plans and online advertising. The free Trustpilot service is little more than a marketing channel that allows Trustpilot to contact businesses to upsell them additional paid services.

Trustpilot: “There will always be a few bad actors that try to game the system. Our systems and processes for tackling this are effective and we put a public consumer warning on the profile page of companies who consistently break our rules.”

Danny: It is clear that they are far from being effective . If they were, the person highlighted in this BBC report would never have lost their life savings. What use is a warning message if that warning is removed after two weeks and the company’s rating remains unaffected?

It is clear to me that Trustpilot place significantly more value on their own short-term profits than on the well-being of consumers who rely on these reviews when making important financial decisions. Ultimately, all of this negative publicity will do far more damage to Trustpilot’s finances than any additional profit they generate by not implementing effective checks and balances. Prospective investors of Trustpilot should take note!

Update 29th October 2019 

I read an interesting article by Grace Gausden published in The Daily Mail on Thursday 24th October 2019.

Grace noticed a stark difference between the TrustPilot ratings for John Lewis and Argos which seemed contrary to the perceived view of these well-known high street brands. Why would John Lewis, a company regularly praised for its high levels of customer service have a Trustpilot rating of 1.7 while Argos has a rating of 4.5?

Grace then noticed that the Trustpilot profile for Argos reveals that this company uses Trustpilot’s paid services while John Lewis does not. She wondered whether this might explain the difference.

My own analysis of 185,000 business profiles listed on the Trustpilot UK website reveals the following:

  • The average rating for businesses using Trustpilot’s free services is 3.81
  • The average rating for businesses using Trustpilot’s paid services is 4.76
  • 11.3% of all businesses listed on Trustpilot use their paid service (costing between £165-£1,000 per month).
  • The average Trustpilot rating across all businesses listed on Trustpilot is 4.03

Grace’s theory that businesses using Trustpilot’s paid services are likely to have a higher proportion of positive reviews is correct.

Trustpilot’s spokesman explained the difference by stating, “Subscribers, such as Argos, are likely to be actively encouraging all shoppers to leave reviews – and thus bagging more scores from happy punters – whereas firms that don’t subscribe, such as John Lewis, do not push for feedback from all customers and are therefore more likely to be skewed towards those seeking to review them on Trustpilot when they have had a bad experience.”

My own research suggests that this difference is more likely due to the following factors:

1) How a Trustpilot rating is calculated
The public’s perception of a Trustpilot rating is that the number is derived by calculating the mean average of all reviews. Even the Daily Mail reporter makes this same error by describing the rating as an “average rating”.

In fact, a Trustpilot rating is heavily weighted by both the recency and quantity of reviews. Businesses using Trustpilot’s paid services to send automated review invitations are significantly more likely to have a larger quantity of recent reviews and, therefore, a more positive Trustscore than a company using their free service.

2) Review Gating
Businesses using Trustpilot’s paid services are more likely to be using review management software which filters invitations to only those customers likely to leave a positive review. Customers who have made a complaint or with a history of returning items may never be asked to leave a review.

This places Trustpilot in a very strong position to persuade businesses to use their paid services should they wish to improve their Trustpilot rating. 

Related articles:

The Daily Mail – Can I trust Trustpilot’s reviews for firms that do and don’t pay it?

BBC – Watchdog viewer loses £20,000 after relying on Trustpilot reviews.

Wired – Are Purplebricks’ glowing Trustpilot reviews too good to be true?

BBC – Trustpilot tackles business review cheats

Techcrunch – Facebook and eBay told to tackle trade in fake reviews 

The Times – Estate agents and banks ‘gaming’ feedback website Trustpilot