Survey sampling is a statistical process that involves selecting and surveying individuals from a particular population. The population you choose to survey could be based on a range of attributes. The target audience could be the United States general population or perhaps a more specific group, like 18-25-year-olds, voters in Ohio, or female pet owners from the Midwest. Your sample options are endless, really.
By asking survey questions and collecting data on a subset of your target population (a “sample”), you can make inferences about the whole population. Suppose 48% of those in your sample say they support a particular political candidate. In that case, you can assume that 48% of the target population feels the same way, within a certain margin of error. Survey sampling is used in statistical research, economics, marketing, clinical and academic studies, and political polling.
How does survey sampling work, exactly? At the most basic level, it involves three steps:
- Sample selection: First, you must decide who will be in your sample. Many sampling methods can help you select survey respondents who are representative of your target population. This first step is crucial because a quality sample ensures your results’ validity.
- Data collection: Next, you can use online surveys, interviews, or other methods to ask open- and closed-ended questions from your sample. Their answers give you raw data.
- Estimation: After collecting data, you can make estimates about your general population using precise statistical calculations.
Conducting sampling via online surveys can be an intricate process. You must ensure accuracy and validity at every step to get reliable results. First, it is important to select a sample that truly represents the population you are researching. Your survey must prompt users with clearly-worded, unbiased questions. Then, you must use the correct statistical equations to make sense of your data and determine your margin of error, confidence intervals, and other statistics. To wade through these complexities, let’s talk more about how survey sampling is applied to research.
Why is survey sampling important?
In an ideal research world, you would be able to gather data from everyone within a target population. However, most of us do not have the budget or resources to procure that kind of data. Perhaps if your target population is small enough — say, all the residents of a small town or all the students at a particular university — that might be possible. A survey that includes 100% of your target population is called a census. However, if your population is much larger — like all of the small business owners in the U.S. — it’s harder to survey everyone. There’s a reason the U.S. Census only takes place once a decade.
That’s why survey sampling is so important. It lets you gather data on a representative portion of your target population. The operative word here is “representative.” For a survey to accurately reflect the target population, you must carefully select your sample. It must be balanced to include the same demographics as your target population and cannot be too big or too small. A well-designed sampling method will give you reliable insights into your target market, research subjects, or political constituency.
Advantages of survey sampling
Survey sampling is often the only practical way to gather data on a large population. Thanks to its many advantages, it’s usually the preferred method, even when other data collection strategies work. Sample survey advantages include:
1. It’s relatively low cost
Where conducting a census requires immense resources, a survey is much more cost-effective. You can find survey participants easily from across the web and pay just a set amount per survey taker.
2. It’s fast and convenient
It’s much easier to survey a select sample of the population than to take a census. You can ask detailed questions and receive your data quickly. A survey is pretty quick and easy for respondents, too. When you field your survey online, it’s easy for survey-takers and for you to view and analyze your data.
3. You can conduct research on a broad scale
When your sample is accurate and representative, you can make inferences about large populations. It’s challenging to study such a vast scope with a census. When working with large populations, their compositions will change by the time you finish collecting data. Others are so large that censusing them is impossible.
4. The results are accurate
If you set the right quality checks in place for your sample, it’s possible to achieve highly accurate results. By working with fewer participants, you can take more care analyzing the data to reach reliable conclusions.
5. You can gather intensive and exhaustive data
By surveying a small group, you can ask your participants more questions. If asking open-ended questions, you must work with a sample to make sense of the responses people give. You can also ask a larger quantity and variety of questions, including multiple-choice and numerical scale ratings. It’s only possible to gather and analyze such exhaustive data when working with a sample.
Disadvantages of survey sampling
While survey sampling offers many benefits, it has a few downsides as well. Those disadvantages include:
1. There’s potential for errors
Collecting high-quality sample results requires thoughtful survey design and setting the right qualifying criteria for respondents. While population subgroups can provide responses with incredible accuracy, some sampling methods create bias and errors. A sampling error occurs when the mean values in the sample are different from the mean values in the entire population. It’s challenging to calculate your sampling error because you won’t have access to the whole population’s mean values. However, when you use randomization during sample selection, you can estimate sampling error more accurately.
Other errors can affect data quality, too. If the researchers don’t understand their target population, they may survey the wrong people. If they select a sample from the wrong population subset, the results won’t be accurate. If specific individuals are more likely to respond to the survey than others, this will also introduce inaccuracies.
2. Developing a representative sample takes careful consideration
When you’re studying complex phenomena and types of people, it’s hard to develop a sample that truly represents the larger population. A thoughtful sample balance is crucial.
3. It requires thorough statistical knowledge
The researchers conducting the survey need to understand sampling well to cultivate quality results. The person doing statistical calculations based on your data needs to understand the mathematics thoroughly. Without this specialized knowledge, your survey research and conclusions can be misleading.
4. It can be more challenging for very niche or diverse populations
If you are targeting a niche population, creating a representative sample can be complicated. The same is true when there are many variations within the target population. In these cases, a census may provide more reliable results.
5. Some survey questions are less likely to yield accurate answers
Questionnaire design is also integral to the survey sampling process. For example, asking about highly controversial topics may mean participants misremember facts or let their emotions cloud their judgment. If questions are confusing, people may interpret them differently. Sometimes, the question wording may elicit specific responses, creating bias. It’s also important to be mindful of respondent experience when designing a survey. If your survey is too long or has too many open-ended questions, respondents may get fatigued and provide less thoughtful responses.
Sample selection
Sample selection is arguably the most crucial element of sampling for survey research. This step is how you decide who participates in your survey. Choosing a sample that reflects your target population is critical to getting accurate results. That’s because the statistical processes you apply later will assume that your sample is representative of that population. If it is not, you could generate erroneous data.
Many different techniques can produce a sample of your chosen population, and each one will result in a slightly different sample set. Even within each sampling method, there’s potential for variation. If you sample 500 people this week and an additional 500 people the next week, you’ll probably see slightly different results. Sample quality is integral to sound survey research on which you can draw valid conclusions.
Sampling bias
Through your sample selection, you want to eliminate as much sampling bias as possible. Sampling bias is when certain population members are systematically more likely to be selected for the sample than others. This factor limits your findings.
For example, if you’re researching consumer opinions about cable providers, you might select your sample from a list of home phone numbers. However, people who have home phone numbers are more likely to be cable subscribers. You’re less likely to get opinions from people who are less satisfied with the service.
Sampling bias is usually curbed by random sampling methods, although not eliminated. That’s because there can be bias within the sample frame. A sample frame is the entire set of units you can draw samples from. In the example above, the sample frame is the list of home phone numbers. Even if you choose the phone numbers included in the survey randomly, you’ll still have an inherent bias.
Any sampling introduces bias. If you wait outside a grocery store and ask people to fill out a survey, your sample frame will only include people who visit the grocery store. When surveying males in their early 20s, you may have an overrepresentation of a particular latent characteristic, such as media usage. In other words, the individuals in your sample use media more readily than the general population of early 20s males. If you use this sample, your results will contain biases, especially if the survey is about media.
Sample blending
At Lucid, we help researchers sample their populations accurately through sample blending. Blending lets us select from many sample providers and counteract some of the samples’ latent characteristics. We work with more than 350 sample suppliers and classify their samples on many dimensions. Then, we can blend survey participants from many providers, creating a compositionally neutral sample.
Population parameter vs. sample statistic
Population parameters and sample statistics are closely related concepts in survey sampling. Because they’re so similar, people often confuse the two. It’s crucial to understand the difference.
Population parameter
A parameter is a fixed value that describes an entire population. It can only be discovered through a census. Say you ask all 100 employees at your office what their favorite food is, and 50 people say pizza. It’s a population parameter that 50% of your office likes pizza.
Sample statistic
In most cases, it’s impossible to determine a population parameter. If you wanted to know the percentage of people living in the U.S. who have pizza as their favorite food, you couldn’t easily find a population parameter. Instead, you would use survey sampling to produce a sample statistic. It’s similar to a population parameter, except that it describes a sample. The sample statistic’s goal is to estimate the population parameter. If you survey several samples of the same population, you’ll get slightly different sample statistics each time.
Things to keep in mind when creating a sample
When sampling, you must consider many factors to ensure a valid, representative population subset. Some things to keep in mind include:
1. Diversity
A sample must be as diverse as the target population. It must represent all the variations between people naturally occurring within your population in similar proportions. If your target population is the general U.S. population, your sample must represent the many regions, income levels, ethnicities, education levels, and other demographic variations present in the U.S.
2. Transparency
As a researcher, you must know where your samples come from. Some sample providers will aggregate and white label samples from many different panels without telling the buyer where their participants come from. For example, when you buy sample from an open marketplace like Lucid, you know exactly where your data is coming from. You can improve your sample transparency, evaluate the differences between panels, and ensure more reliable conclusions.
3. Consistency
When comparing data over time, such as in a brand tracker study, inconsistencies in the sample frame are a particular concern. Survey panels change over time, meaning different people have the chance to be included in your survey. Meanwhile, each survey panel has a different composition. They come from various sources, even if they contain similar populations. Inconsistencies affect your sample’s quality significantly.
Let’s say you work with a particular survey distributor to provide sample for your study. You distribute the same survey year after year using the same distributor for five years. However, over the five years, the survey panel’s users change. Some people stop using the panel altogether, and new users have signed up. Meanwhile, the users who took the survey five years ago have now aged. Even small shifts in this sample frame could create year-over-year trends in your data that don’t reflect the real trends within your target population.
Sample blending is a great way to “future-proof” your sample. Lucid’s Quality Program data and sample blending calculator can help to establish consistency with your sample providers, even as their characteristics shift over time.
4. Sample size
One of the most important decisions related to your sampling is the sample size. It must be large enough to represent the population and all the potential variables at play. However, making the sample size too large can increase the survey’s cost and logistics. Also, a sample size that’s too large can be just as inaccurate as a sample that’s too small. If there’s any bias or flaws in the study design, a sample size that’s too big will magnify the inaccuracies. You can determine your sample size using the sample size formula.
Survey sampling methods
There are many ways to create a sample for your survey. The sampling methods divide into probability sampling, sometimes called probabilistic sampling, and nonprobability sampling, or nonprobabilistic sampling. These two sampling strategies break down even further into several sampling methods.
Probability vs. nonprobability samples
When selecting respondents for your sample, you have the choice between probability sampling and nonprobability sampling. Probability sampling is more reliable because it uses randomization to choose survey participants. The Office of Management and Budget’s Standards and Guidelines for Statistical Surveys for government agencies requires generally accepted probabilistic methods unless another method can be statistically justified.
Nonprobability sampling doesn’t use random selection. So, participants don’t have an equal chance of being included in the sample. It is often easier to undertake. Nonprobability sampling could even be the preferred method, usually in qualitative research. In some studies, researchers purposely choose certain participants because they can offer unique insights into a topic. However, for quantitative analysis, probability sampling will always be the preferred method, and statistical researchers will use nonprobability selection only for its practicality.
Probability sampling methods
In probability sampling, everyone in the sample frame has the same chance to be included in the sample. You can also calculate any member’s probability of being included in the survey. These sampling techniques are more reliable and allow researchers to make more accurate inferences about a population.
You can use several probability sampling methods, and each has distinct advantages and disadvantages. Those methods include:
1. Random sampling
Simple random sampling is the most basic form of probability sampling. It involves just one step, and each survey subject is selected independently from the other members of the population or sample frame. A standard method of random sampling is assigning every individual in the sample frame a unique number. Then, a random number generator determines who will be in the sample.
The benefit of random sampling is that each member of the population has an equal chance of being chosen for the survey. This characteristic makes a simple random sample highly representative of the target population.
However, with larger populations, it’s hard to include every individual in the random selection process. Instead, you would draw participants from a sample frame, such as a list of email addresses or phone numbers. Any sample frame will eliminate some members of the population since it’s impossible to contact everyone. It’s time-consuming to conduct random sampling with larger populations and larger sample sizes. A biased sample frame can also skew your results.
2. Systematic sampling
Systematic sampling is a little easier than random sampling and is similar in reliability. In this method, you assign everyone in the target population or sample frame with a number. Instead of using a random generator, you systematically select candidates at regular intervals. For example, you could select every fifth number or every 20th number.
A systematic sample is highly representative. However, it’s not quite as random as using a random number generator. There’s also a chance that the list’s organization could compromise randomness. When sampling systematically, it’s essential that the list doesn’t have any hidden patterns. If you’re surveying people at a company, the list could divide employees by department and sequence them by rank. It’s a good idea to shuffle a list of names in alphabetical order or otherwise organized.
3. Stratified sampling
Stratified sampling attempts to account for the demographics and traits of the larger population. It attempts to recreate the elements in the sample. For example, if you’re surveying college history majors, and you already know that 40% of history majors are female and 60% are male, you might want your sample to have the same proportions.
Before generating a sample, the researchers first decide what traits or dimensions are significant. They may want to account for gender, social class, age, religion, education level, or other characteristics. Then, they randomly sample within each chosen category. If the researchers know that 70% of their target population is not college educated, they’ll ensure 70% of their survey participants are also not college educated.
This sampling method can better replicate the demographics of the target population. It’s especially useful when one category is a small minority compared to the others. In simple random sampling, this demographic could be underrepresented or even nonexistent in the sample. Stratified sampling is complex and time consuming. It could be challenging to find participants in the target population that meet the other demographic criteria.
Nonprobability sampling methods
Nonprobability sampling methods do not use any randomization to select survey participants. Therefore, population members do not have an equal chance of being included. Some members may have no chance of being in the sample. Others may have a much higher chance, and they will have disproportionate representation in the sample. Nonprobability sampling has limited applications for quantitative researchers who need quality samples.
Some nonprobability sampling techniques include:
1. Convenience sampling
Convenience sampling includes participants based on their availability and accessibility. Essentially, it includes people who are easy to reach. If you’re an academic researcher, it’s easy to sample people from your own institution. It’s even more convenient to survey your own students or classmates.
One way to conduct convenience sampling is to wait in a crowded location and approach people to participate in a survey. When enough people agree to meet your sample size, you stop the survey.
Convenience sampling is beneficial because it lets you collect data quickly. It’s usually inexpensive to sample these participants because you can take advantage of a readily available sample. You also don’t need to follow strict rules to ensure randomization. The downside is that the survey won’t be representative of the target population.
2. Snowball sampling
Snowball sampling relies on the first survey participants to refer you to the next ones, and so on. Once you’ve found enough people to meet your required sample size, you stop the survey.
One advantage of snowball sampling is it allows surveyors to find people who are generally hard to reach. If people in the target population don’t want to be found, like those involved in illegal activity, their contact information is not readily available. The snowball method creates a kind of word-of-mouth marketing that makes it easier to find participants.
The downside is that it’s impossible to know how representative your sample is. These surveys generally create a very homogeneous sample.
3. Quota sampling
Quota sampling is similar to stratified sampling. The difference is that this method doesn’t randomly select participants. As with stratified sampling, the researchers first define categories they want to represent in their sample and choose appropriate proportions for each group. These could be equal quotas, like 100 men and 100 women, or they could seek to replicate a target population’s demographics.
Instead of randomly selected participants, the surveyors will use some form of convenience sampling. When they’ve hit the right quotas for each category, they stop the survey.
One benefit of quota sampling is that it can represent the target population more accurately than convenience or snowball sampling. This survey method can cover many different characteristics and handle a lot of complexity. While it is the most accurate of the nonprobability techniques, it’s still not as representative as probability methods.
How to determine what type of survey sampling is best for your application
Each type of survey sampling has a place in research. Nonprobability sampling, while less accurate, can be a practical way to begin the research process. Probabilistic sampling can provide more accurate results, which can be used to generalize about the target population. Here’s when to use each type of sampling:
- Random sampling: As a highly representative method, random sampling has the widest applications. It’s particularly useful for large populations. It’s not ideal for studying an uncommon characteristic in a large group, as you may not randomly select enough participants with the trait.
- Systematic sampling: If your population is relatively homogenous, you can use the faster and more convenient systematic sampling method.
- Stratified sampling: When you need to represent many individual characteristics from the target population, choose stratified sampling.
- Convenience sampling: Since convenience sampling is fast, it’s suitable for preliminary research. You can get a general idea of a more precise study’s results without the upfront time or cost of a probabilistic sample.
- Snowball sampling: When your target population is homogenous and generally hard to reach, snowball sampling can help. It’s helpful for surveying illicit drug users or those involved in illegal activity.
- Quota sampling: When a more rigid and costlier study would use stratified sampling, preliminary research can instead use quota sampling. Like stratified sampling, it’s used when the researchers can identify specific traits within the target population that they want to study. Market researchers often use this method.
Learn More About Cint
Our survey sampling software is an all-in-one tool to sample your target population from a community of more than 350 sample suppliers. With so many survey panels, each with different methods for recruiting and engaging survey-takers, your sample blend will be more diversified and more representative of your target population. You can set custom parameters based on more than 200 attributes and manage your quotas all within the marketplace tool. When you launch online surveys, subjects will see and begin answering questions immediately, so you can start collecting data right away. To learn more about our custom sampling tools, contact our team online today. We’re happy to tell you more about our solutions, demo our software, and provide you with a quote.
