Consumer Survey: How do Website Visitors feel about Pop-ups?

20/07/2016, SEO

Consumer Survey: How do Website Visitors feel about Pop-ups?


Most people are aware that intrusive pop-ups can be unpopular with online visitors. This study set out to test how pop-up forms affect the degree to which online visitors feel they can trust a business or would buy its products. 


You may have noticed a growing trend in the use of intrusive pop-up forms on websites and blogs persuading you to sign up to an email newsletter.


Example of a Standard Pop-up form:

standard pop-up


Some websites now use “negative opt-out pop-ups” forcing visitors to click on a button to either accept or reject the offer before accessing the page content.


Example of a Negative Opt-Out Pop-up form:

neg popup


There is little doubt that making it more difficult for visitors to reject a sign-up form will (perhaps reluctantly) get more people to sign up.

I wanted to test the impact (if any) these pop-ups have on how visitors feel towards the business. In particular, the majority of visitors who don’t sign up to the newsletter.


I set out to test the the following hypotheses:

1) The use of pop-ups has a negative impact on the degree to which visitors feel they could trust the business.


2)  The use of pop-ups has a negative impact on the degree to which visitors feel they would buy from the business.


Testing perceptions, not behaviour

The purpose of this study was to explore the perception of online visitors towards the business, not how they might actually behave in a purchasing decision. 

Consumers often make purchasing decisions reluctantly and, only then, until a more favourable option becomes available. A single decision to purchase gives little indication as to the longevity of the relationship or the anticipated Customer Lifetime Value. Both factors likely to be of more importance to a legitimate business than whether or not an individual sale was achieved.       



Using SurveyMonkey Audience, I enlisted a total 1,000 US-based, respondents divided into three equal groups. Each group was asked to visit a generic online T-shirt store created for the purpose of the experiment.


Generic T-shirt website shown to the respondents:



Group A

Group A (the control) were asked to look at the T-shirt page with no pop-ups but with a simple newsletter sign-up form at the bottom of the page.


Group B

Group B were shown the same page but with a standard pop-up form that could be dismissed by clicking the X at the top-right corner of the form, or by clicking anywhere outside of the form on the page.

The pop-up was displayed within two seconds of the visitor landing on the page.


Group C

Group C were shown the same site but with a negative opt-out pop-up forcing them to either supply an email address or click on a link stating, “No, I do not wish to receive discounts and promotions”. 

The pop-up was displayed within two seconds of the visitor landing on the page.


Avoiding bias

None of the respondents were told which aspect of the website was being tested and pop-ups were not mentioned until after the initial survey was completed.

After visiting the site the subjects were then asked the following questions:



Q1) To what degree do you feel that you would trust this business?



Q2) To what degree do you feel that you would buy from this business?



After they had completed the survey – and on a separate page – I then asked the respondents to describe how they felt generally about pop-ups on websites.

97% of all respondents felt negatively towards pop-ups. The Wordcloud below shows the most frequent comments in the largest font:


pop green word clould mid



In “Disrupted,” the enlightening memoir by tech journalist and “Silicon Valley” TV show writer Dan Lyons, he recalls his experience of working at the marketing software company HubSpot. He remembers their blogging strategy this way:

“[HubSpot’s former CMO] has one goal: to get leads. If our software analytics were to indicate that our best conversion rate comes from publishing a blog post that just says the word dogshit over and over again… then [the former CMO] would publish that post…”

A blog post that just repeats “dogshit” over and over might generate more leads — most smart people would agree that it would also be terrible for the organisation’s reputation and branding.

While using intrusive  pop-ups on your website might increase sign-ups, this study suggests that it might also significantly reduce those visitors’ desire to trust or buy your business. Both metrics that you won’t find in Google Analytics.


Special thanks to the following smart and generous people who helped with the preparation and design of this study:


Author: Danny Richman

Danny Richman is an online marketing consultant with over 15 years’ experience providing SEO Training  and marketing support to businesses seeking to improve their digital marketing and search engine visibility. Clients include Bank of England, BBC, John Lewis and Marks & Spencer.