By Ben Hogg

Online survey best practices have changed

For the past several years, buyers of survey sample have taken a consistent approach to creating their online surveys. In general, the rule of thumb has been that a survey completion time of 20 minutes is the maximum recommended, not only to elicit well thought-out and meaningful responses, but also to ensure that surveys get completed. That methodology worked well for a while, but now we’ve entered a new phase of online sampling.

With the advent of exchanges, such as Cint, things have changed. We’ve always known that shorter surveys are better, but until now, that has been just a recommendation. Yet, recent trends and marketplace dynamics have shown us that short and sweet are not just desirable, but rather have become increasingly essential.

Marketplace dynamics and survey sampling

Think of a sample marketplace as a beauty parade or a talent show. Your survey is competing with thousands of other surveys at any given time to attract the attention of the judging panel – in Lucid’s case, this is more than 300 supply partners that are integrated into our ecosystem.

Previously, all you needed to do to catch the judges’ eye was talk about your desire for world peace, plans to open a boutique, or have a dancing, juggling dog. In “survey” terms, you simply had to ensure your study was set up correctly and could be completed in approximately 20 minutes. Alas, the characteristics of a good survey have evolved in the marketplace environment. As suppliers receive survey bids from thousands of buyers, they have become more selective about the surveys to send panellists to.

Today, suppliers have specific expectations: the single most valuable commodity is their time. When a survey is too long or difficult to complete, it wastes both the supplier and their respondents’ time – which means suppliers will be quite hesitant to send respondents to surveys they are not likely to complete. If you’ve been in the industry for a while, you’ve undoubtedly heard this before. The only difference is that now, buyers are finally beginning to feel the pinch of limited survey supply. Although there are many reasons for fluctuations in supply or demand in a marketplace, one factor we can control is the quality of surveys being fielded.

What criteria are surveys judged upon?

Remember, buyers are competing against other buyers, so the best way to create a survey that receives adequate traffic is to make it appealing to both suppliers and respondents. I’ll outline the metrics suppliers are most keen on.

Appropriate survey length

Length of interview: a classic survey metric. Despite the industry continuing to preach the gospel of low LOI, some sample buyers are hesitant to embrace the 10-minute (or dare I say 5-minute) survey. Breaking up an hour-long survey into six separate parts can be difficult, I realise. So, I’d advise you to not only take my word for it; instead, look at our Marketplace metrics (below). Suppliers are showing a notable preference for shorter surveys.

Competitive survey pricing

Survey pricing is another metric that has shown some intriguing new Marketplace trends. Average cost per interview was previously on an industry-wide decline (possibly attributed to high volumes of US election-related studies), yet ACPI in 2021 is the highest it’s been in more than a year (see below).

Survey earnings per click (EPC)

What about earnings per click (EPC)? You will find the same trends occurring there as well. In fact, EPC (see below) has been consistently climbing. This is a metric that suppliers pay attention to when they evaluate which survey opportunities to send into, as it essentially tells them is how much they can expect to earn per respondent they send in to a study.That means if you, a sample buyer, are not proactively seeking out the average Marketplace EPC, you may find that your surveys cannot compete against other buyers as effectively as you’d like.

What does the perfect survey look like?

Now that I’ve shared the effect of marketplace dynamics and buyer competition on survey traffic, you may be wondering how your survey can win the beauty contest. What does the perfect survey really look like? First, as a disclaimer, I must say there is no *perfect* survey. Marketplace dynamics will always be at play, which means we will always be adapting to meet changes in supply and demand as they occur.

That said, I can share some advice to help you create consistently excellent surveys: ideally, your survey should have a low LOI (less than 10 minutes), fair CPI and EPC, limited open-ended questions, high incidence rate, proper screening questions, and a functional survey link. If your survey will require personally identifiable information (PII), label it as such. Lastly, always be sure to test your survey before launching it. (Of course, there are more factors to consider, such as your questionnaire, survey design, translations, and dropout rate, but I’m merely listing the highlights.)

In my opinion, one of the most profound considerations when launching an online survey is to remember that you are part of a global marketplace. Think of the suppliers vetting your studies, the respondents completing them, and the broader dynamics that continually invigorate the ecosystem.

Allow me to put this in even simpler terms. We used to say, “design a survey that you’d be comfortable sending to your parents, expecting them to be able to complete it.” These days, however, we must also account for the amount of time people will spend taking a survey. As such, (especially as many of our regions are coming out of Covid lockdown restrictions) perhaps better advice would be, “design a survey that you’d be comfortable sending to your parents, expecting them to complete it, but also knowing that they now have many other things to do, like seeing friends and family, heading to a restaurant or going on holiday.”

Design a survey that is not terribly difficult to understand or complete, and you’ll see even better results in the responses you receive.