Consumer Survey: How do Website Visitors feel about Pop-ups?

20/07/2016, SEO

Consumer Survey: How do Website Visitors feel about Pop-ups?


Most people are aware that intrusive pop-ups may be unpopular with online visitors. This study set out to test how pop-up forms affect the degree to which online visitors feel they can trust a business or buy its products. 


You may have noticed a growing trend in the use of intrusive pop-up forms on websites and blogs persuading you to sign up to an email newsletter.


Example of a Standard Pop-up form:

standard pop-up


Some websites are now also using “negative opt-out pop-ups” which force visitors to click on a button to either accept or reject the offer before being able to view the page content.


Example of a Negative Opt-Out Pop-up form:

neg popup


Some marketing “experts” claim that the use of negative opt-out pop-ups significantly increases their newsletter sign-up rate, while then also seeming to contradict themselves by describing such tactics as “scuzzy” on the very same website. So, who are you supposed to believe?


There is little doubt that if you make it more difficult for visitors to reject a sign-up form you will (perhaps reluctantly) get more of them to sign up.

I wanted to test what impact these pop-ups have on how visitors feel about your brand. In particular, the vast majority of visitors who do not sign up to the newsletter.


I set out to test the the following hypotheses:

1) The use of pop-ups has a negative impact on the degree to which visitors feel they can trust a business.


2)  The use of pop-ups has a negative impact on the degree to which visitors feel they would buy from a business.


Testing perceptions, not behaviour

The purpose of this study was to explore the perception of online visitors towards the business, not how they might actually behave in a purchasing decision. 

Consumers often make purchasing decisions reluctantly and, only then, until a more favourable option becomes available. A single decision to purchase gives little indication as to the longevity of the relationship or the anticipated Customer Lifetime Value. Both factors of significantly more importance to any legitimate business than whether or not an individual sale has been achieved.       



Using SurveyMonkey Audience, I enlisted a total 1,000 US-based, respondents divided into three equal groups. Each group was asked to visit a generic online T-shirt store created for the purpose of the experiment.


Generic T-shirt website shown to the respondents:



Group A

Group A (the control) were asked to look at the T-shirt page with no pop-ups but with a simple newsletter sign-up form at the bottom of the page.


Group B

Group B were shown the same page but with a standard pop-up form that could be dismissed by clicking the X at the top-right corner of the form, or by clicking anywhere outside of the form on the page.

The pop-up was displayed within two seconds of the visitor landing on the page.


Group C

Group C were shown the same site but with a negative opt-out pop-up forcing them to either supply an email address or click on a link stating, “No, I do not wish to receive discounts and promotions”. 

The pop-up was displayed within two seconds of the visitor landing on the page.


Avoiding bias

None of the respondents were told which aspect of the website was being tested and pop-ups were not mentioned until after the main survey was completed.

After visiting the site the subjects were then asked the following questions:



Q1) To what degree do you feel that you would trust this business?



Q2) To what degree do you feel that you would buy from this business?



After they had completed the survey – and on a separate page – I then asked all respondents to describe how they felt generally about pop-ups on websites.

97% of all respondents felt negatively towards pop-ups. The WordCloud below shows the most frequent comments in the largest font:


pop green word clould mid



In “Disrupted,” the enlightening memoir by tech journalist and “Silicon Valley” TV show writer Dan Lyons, he recalls his experience of working at the marketing software company HubSpot. He remembers their blogging strategy this way:

“[HubSpot’s former CMO] has one goal: to get leads. If our software analytics were to indicate that our best conversion rate comes from publishing a blog post that just says the word dogshit over and over again… then [the former CMO] would publish that post…”

A blog post that just repeats “dogshit” over and over might generate more leads — most smart people would agree that it would also be terrible for the organisation’s reputation and branding.

While using intrusive  pop-ups on your website might increase sign-ups, this study suggests that it might also significantly reduce those visitors’ desire to trust or buy your business. Both metrics that you won’t find in Google Analytics.

Intrusive pop-ups are the digital equivalent of that slimeball  you meet at parties who interrupts your conversation, thrusts his (and it’s always a him) business card  in your face and then insists on letting you in on whatever “get rich quick” scheme he happens to be flogging at the time.

Perhaps you will generate a few more sales with your intrusive pop-ups, but do you really want to be “that guy”?

Special thanks to the following smart and generous people who helped with the preparation and design of this study:


Author: Danny Richman

Danny Richman is an online marketing consultant with over 15 years’ experience providing SEO Training  and marketing support to businesses seeking to improve their digital marketing and search engine visibility. Clients include Bank of England, BBC, John Lewis and Marks & Spencer.