26/06/2019, SEO

Can you trust Trustpilot? – 9 million reviews studied

I studied nine million reviews left for 198,000 businesses on the Trustpilot UK website to reveal how some businesses are deceiving consumers.

This article explains how they do it and and reveals the worst offenders. I also explain why, I believe, Trustpilot plays a significant role in allowing this manipulation to continue.

Do Trustpilot reviews matter?

A recent study by BrightLocal showed that 86% of consumers consider online reviews when making a purchase. 68% agreed that positive reviews make it more likely they will buy from the business; while 40% say that negative reviews make it less likely.

The vast majority of companies don’t manipulate their online reviews. However, consumers should be able to rely on this information when making important purchasing decisions.

When companies manipulate reviews it has a detrimental effect on both consumers and those honest businesses that may appear less attractive in comparison.

Although businesses owners found guilty of review fraud could potentially face a prison term of up to 10 years, prosecutions and convictions are rare and seldom policed.

In this post, I reveal the methods these businesses use to help you spot this manipulation and reveal the worst offenders. I also explain how Trustpilot contributes to this manipulation to the detriment of consumers and small business owners alike.

1) Review Gating

What is Review Gating?

Review Gating is the process of filtering customers before asking them to leave a review for your business. The aim is to encourage reviews from only those customers you believe had a positive experience with your company and request private feedback (or none at all) from those who had a negative experience.

Although Review Gating doesn’t entirely remove the possibility of negative comments, it significantly reduces the ratio of negative to positive reviews. Over time, negative reviews are pushed deeper into the review feed, while the company’s Trustscore rating remains positive.

Most of the major review publishing platforms, including Trustpilot and Google, warn against this illegal practice in their guidelines but take little or no action against companies found to be in breach.

How do businesses use Review Gating?
This internal memo shown below from Yell UK (formerly Yellow Pages) reveals how staff are incentivised to “ask every happy customer to write a review on Trustpilot for Yell” and to not ask if “there’s any risk [they] may leave a negative review”. Yell rewards its staff with £10 for every 5-star review and £5 for every 4-star review posted.

Internal memo to Yell employees incentivising them to request 4&5-star reviews only

This memo (received from a Yell whistleblower) is a classic example of Review Gating. Despite this memo being forwarded on to Trustpilot, none of Yell’s reviews were removed from its platform. Yell’s TrustScore rating remains equally unaffected.

Some businesses even use automated review management software to analyse email communications with customers, internal complaints records and the history of product returns. The software then issues a review invitation to only those customers most likely to post a positive review for the company.

Should invited reviews be banned?

I performed an analysis of 9 million reviews posted on Trustpilot for businesses that invited reviews from their customers.

The average star rating for all organic (uninvited) reviews was 2.87.
The average star rating for all invited reviews was 4.57.

Trustpilot claim, “Generally we find that organic reviews tend to sway more negatively because consumers are more likely to be motivated to seek out the opportunity to share their voice if they have had a bad experience. By contrast, invited reviews, both automated and manual, tend to encourage those who have had a good experience to share their thoughts, when they may not have thought to do so without being prompted.”

It is true that organic reviews sway more negatively than invited reviews. However, this says more about the manipulation of invited reviews than it does about the nature of organic reviews.

The fact that the average invited review rating is 4.57, strongly suggests that it is invited reviews that paint a false picture of how most people feel about a company.

See the image below showing the stark difference between organic and invited reviews for Hermes UK. This difference is common for most companies inviting reviews.

Trustpilot has now banned manual review invitations from its platform aware that this process is prone to widespread manipulation. Trustpilot refuses to do the same for automatic invited reviews because its business model depends on the income this feature provides.

It is inevitable that some businesses will attempt to manipulate their Trustpilot rating. The more serious issue is the flaw in Trustpilot’s system which gives invited reviews and organic reviews equal weight.

2) Review Flagging

What is review flagging?

Trustpilot allows businesses to flag a review for removal if it breaches any of their review guidelines. The review is then removed from the Trustpilot platform if found to be in breach. There are several grounds for which a review could be flagged.

These guidelines include:
• The review contains offensive language.
• The review mentions the name of an individual.
• The review describes the functionality of the purchased product.
• The service experience has not occurred within the last 12 months.

The following (fictitious) reviews could all potentially be in breach of Trustpilot’s guidelines:

“I received great service from John Smith. He installed my dishwasher quickly and efficiently with no fuss” – (Breach: Names an individual).
“My dishwasher broke down within two weeks of being installed and couldn’t be repaired – not happy” – (Breach: Describes the functionality of the purchased product.)

“The dishwasher installed by Smith Appliances cleans my dishes better than any other dishwasher I have owned – I am delighted!” – (Breach: Describes the functionality of the purchased product.)

“The dishwasher installed by this company has been working perfectly for more than 2 years without a glitch. – (Breach: Describes a service experience that has not occurred within the last 12 months.)

As you can see, it’s just as likely that a positive review could be flagged as a negative one. You can see this in action by analysing the reviews that Trustpilot has flagged for its own business. In fact, Trustpilot flags more of its own positive reviews than negative reviews.

This shows, that when flagging reviews fairly, a negative review is no more likely to be in breach of Trustpilot’s guidelines than a positive one.

In a Techcrunch interview in March 2019, Trustpilot’s Founder and CEO Peter Mühlmann stated, “The vast majority of companies flag questionable reviews in exactly the right way. However, there’s a small minority that are potentially ruining it for everyone by aggressively flagging bad reviews.”

He goes on to explain, “Trustpilot now reveals how many reviews were taken down because they represented a real breach of Trustpilot’s policies, versus reviews where the poster simply didn’t respond to questions, versus legitimate reviews that were ultimately reinstated”. Mühlmann warns, “If marketers are motivated to cheat on our platform, consumers are going to see them do it.”

In fact, my research shows that businesses are 33 times more likely to flag a negative Trustpilot review than a positive review. Most consumers are entirely unaware how to identify which type of reviews have been flagged.

Manipulative Review Flagging examples

Let’s look at examples of businesses that may not be flagging reviews fairly:

Parcelforce Worldwide

77 reviews flagged | 99% negative | 0% positive

PayPal

75 reviews flagged | 100% negative | 0% positive

Wren Kitchens

134 review flagged | 100% negative | 0% positive

Webuyanycar

77 reviews flagged | 100% negative | 0% positive

Yell.com

132 reviews flagged | 100% negative | 0% positive

When a review is flagged by a company, the reviewer will be asked to provide proof that they have had a genuine experience with that business. A large number of people will either not still have that proof nor wish to invest the time and effort of retrieving it, simply to prove that they are a genuine customer. Approximately, 60% of all flagged reviews are removed from the Trustpilot platform.

Selectively flagging only negative reviews is, therefore, a highly effective method of increasing the TrustScore of a business.

Some businesses are so keen to flag negative reviews they make little or no effort to determine whether the review was even in breach of Trustpilot’s guidelines. They simply flag every negative review and leave it up to Trustpilot to determine whether the review was, in fact, in violation.

Compare the difference between Yell’s reviews on Trustpilot (4-star rating) where Yell has a paid account and its reviews on Reviews.io (1-star rating) where they do not. Why would Yell’s customers have such different experiences depending on the platform on which the review was posted?

Trustpilot’s Chief Trust Officer, Carolyn Jameson claims, “All flagged reviews, whether 1-star or 5-star are treated identically”.

In practise, very few 5-star reviews are ever flagged by the company being reviewed. This results in many more 1-star reviews being removed from Trustpilot’s platform than 5-star reviews.

When I asked Trustpilot to provide the exact ratio of negative flagged reviews to positive flagged reviews it declined to comment.

If Trustpilot really cared about the trustworthiness of their reviews, it would insist that businesses apply the same flagging process to all reviews left on the platform and make sure they do.

3) Fake Reviews

What are fake reviews?

Most people are aware that some companies buy fake online reviews to enhance their online profile. There have been several recent high-profile stories in the media highlighting this issue calling on Trustpilot to take action.

A quick Google search reveals a wide selection of businesses happy to supply verified reviews on Trustpilot and Google at a low cost. Many of these companies are based in East and South-East Asia.

Most of the businesses selling reviews make little effort to cover their tracks. They often post under a name that is uncommon in the UK, write short reviews comprised mostly of complimentary adjectives and are likely to have published only one review under the same user account. Fake reviews also tend to be posted in batches with a disproportionate number of 5-star reviews posted on the same day.

Company selling fake Trustpilot reviews

One of the most egregious examples of fake Trustpilot reviews are those shown on the profile of online fashion company, ChicV UK Ltd.

The company is owned by Mr Yang Xingjian and operates under several different brands including:

More than 10,000 people claim to have been scammed by this company. Customers complain that orders take several weeks to arrive, don’t match the images or descriptions shown on the website and are of substandard quality. They also claim to get no response from customer services and are unable to return goods or claim a refund.

This company was first brought to my attention by Diana Buck. Diana checked this company’s Trustpilot reviews before buying some items from their website and was reassured to see a large number of positive reviews with a four or five-star rating. It was on the basis of these reviews that she felt confident enough to order some clothing from the lilicloth.com website.

As soon as Diana realised that she was the victim of a scam, she went back to look at the company’s Trustpilot profile in a little more detail.

She soon realised that the positive reviews she had seen were obviously not genuine. Most of these reviews spoke about the company in a way that lacked any real detail or used language untypical of a genuine customer.

Most of the positive reviews had been written by people with no other reviews posted on Trustpilot. (A tell-tale sign of review manipulation.)

Several identical reviews had been posted multiple times using different names.

It it clear that rather than respond to negative reviews, this company pays people to write positive reviews contradicting the complaints being made by genuine customers.

Diana contacted Trustpilot on five separate occasions over a period of eight months but received nothing but an automated response and a vague promise that someone would look into the matter. A little online research shows that Diana was far from being the first person to complain about this company to Trustpilot.

Eight months later, Trustpilot finally took some action. A warning message currently appears on Lilicloth’s Trustpilot profile page with a statement claiming that the fake reviews have now been removed.

While it is good to see Trustpilot has made some effort to address this fraud, not all of Lilicloth’s fake reviews have been removed nor should it take eight months for Trustpilot to take action. Trustpilot has taken no action on most of the other scam websites operated by this same company.

One could argue that by allowing these scammers to manipulate their platform so easily, Trustpilot should share liability for all those who have lost money by relying on these reviews.

Trustpilot claim that 38 million fake reviews were removed from its platform in 2020. When later questioned, the company confessed that 95% of those fake reviews were posted prior to 2019.

As a Trustpilot rating is heavily skewed towards recent reviews, the removal of these fake reviews will have had little or no effect on the TrustScore displayed on these companies’ profile pages. This action does little or nothing to protect consumers.

The fact that Trustpilot had 38 million fake reviews on its platform is a clear indication that its processes are deeply flawed. Nor does the removal of these fake reviews include those flagged unfairly by Trustpilot’s customers, which remain on the platform today.

Fake negative reviews

Some businesses also buy fake negative reviews for their competitors or encourage their own staff to leave fake reviews online.

Take a look at these reviews on Google left by a sales executive working for Yell from their Belfast office.

You will see he left a total of 45 reviews including a 5-star review for his employer.

His reviews include a tattoo parlour in Belfast, an ear wax removal company in Yorkshire, an industrial lubricant supplier in Cheshire, a chimney sweep in Lancashire, a decorator in Dundee and four different removals companies all at opposite ends of the country. An unlikely pattern of reviews for any genuine customer.

Location of reviews left by Yell employee based in Belfast

You will also notice that the typical comment posted with these ratings is, “great business” or “great business to deal with”. Comments that are sufficiently ambiguous as to disguise the true nature of the relationship. 95% of the reviewed businesses also happen to be advertisers with Yell.

It may come as no surprise to learn that Yell offer a service which offers to improve the online profiles of their small business clients. Yell staff are incentivised to gain positive reviews for their clients.

I have evidence of several other Yell employees engaging in similar activities. Yell cannot reasonably claim that these are just the activities of a few rogue employees. After exposing Yell’s activities, one of its employees then left a negative review for my own business, despite never having been a client or even communicating with me.

The day after this article was first published, the following message suddenly appeared on Yell’s Trustpilot review page:

Trustpilot warns consumers that Yell’s rating can’t be trusted.

It is clear that this rushed message, littered with grammatical errors, was an attempt to limit the damage from this story spreading beyond this humble blog.

This warning message was removed from Yell’s Trustpilot profile just a few weeks after it first appeared. None of the manipulated reviews were removed from Yell’s profile and their Trustpilot rating was unaffected.

How is a Trustpilot rating calculated?

The public’s perception is that a Trustpilot rating is calculated as an average of all reviews left for a business.

In fact, a Trustpilot rating is heavily weighted by both the recency and quantity of reviews. Businesses using Trustpilot’s paid services to send automated review invitations are significantly more likely to have a larger quantity of recent reviews and, therefore, a more positive TrustScore than a company using their free service.

Is Trustpilot running a “protection racket”?

Trustpilot claims that, “Organic reviews tend to sway more negatively because consumers are more likely to be motivated to share their voice if they have had a bad experience. By contrast, invited reviews tend to encourage those who have had a good experience.”

Any business refusing to sign up to Trustpilot’s paid services (starting at £200 per month) is limited in how many customers it can invite to leave reviews on the platform.

Many businesses complain that Trustpilot is therefore running what amounts to a “protection racket” in which you either pay Trustpilot £2,400 from your hard-earned profits or become subject to organic reviews that Trustpilot admits will “sway more negatively”.

The future of online reviews

As part of the EU’s “New Deal for Consumers”, the Enforcement and Modernisation Directive (2019/2161/EU) came into force on 7 January 2020 with a deadline for implementation by 22^nd November 2022.

This directive provides additional protection to consumers around how online reviews are published, with severe penalties for those who breach the regulations. To ensure harmonisation with the EU, similar penalties and enforcement action will be introduced separately in the UK, with fining powers for breaches of consumer protection legislation capped at 10% of a firm’s worldwide turnover.

This new directive will make the following activities mandatory over and above the existing regulations:

The publisher must make it clear how reviews are being filtered.
For example, a company displaying only positive reviews on its website must make this clear to visitors.
The publisher must declare whether a review has been sponsored or in any other way influenced by a contractual relationship.
For example, a company that incentivises its customers to post a review must make this clear to visitors.
The publisher must take all reasonable steps to verify that reviews of a product have been submitted by consumers who have actually used or purchased the product.
This requirement will make it mandatory for sites such as Trustpilot to verify that all reviews posted on its platform are from genuine customers, not only those reviews flagged by the company being reviewed.
Commissioning a third party to post fake reviews will be liable to a fine of up to 10% of the company’s global turnover.

While these new regulations and fines should help focus the mind of any company that fails to comply, the real test will be the degree to which these regulations are policed. Unfortunately, my experience is that most of the regulatory bodies tasked with enforcing these laws are both under-resourced and largely inept.

Conclusion

Trustpilot charge businesses a substantial amount of money to make full use of their platform. A significant proportion of that fee should be used to ensure that its platform is being used fairly and the reviews can be trusted.

Trustpilot must make significant improvements to its fraud detection algorithms and take strict action against repeated offenders. It should not be relying on consumers to do the job of their compliance team.

While Trustpilot’s business model remains dependent on the businesses who pay to use its service, it can never be a platform that consumers can wholly trust to be impartial.

Watch this video by BBC Rip-off Britain on 19/5/2021 about Trustpilot.