Did you know survey questionnaire design can significantly impact the quality of your results? Surveys can clue you into respondents’ opinions, experiences and much more – but that can only happen if the survey questions support the goal.
Writing questions may seem like a simple task, but there are many elements to keep in mind when drafting a survey questionnaire. Of course, you’ll need to consider the wording of your questions, as it relates to the information you’re collecting. However, it’s equally as important to be mindful of the respondents who are answering your questions and how to best engage them with your survey.
Questionnaire design is something of an art form, and crafting the right survey calls for elements of psychology, technical skill, and marketing. If you want to get the highest quality results from your research, here are some best practices to follow when designing a survey.
Types of Survey Questions
Before we get too far into best practices, let’s identify the different types of questions that can be used for a survey.
Categorical Questions
These questions, also called nominal questions, can help you find percentages and basic numbers. If you’re looking to create some visuals with your data, like pie charts and graphs, be sure to include these. They allow respondents to select an answer from multiple options.
- Yes/no: These straightforward options are good for simple questions where one of these two answers is appropriate. However, always include a neutral choice for respondents who may not have a yes/no answer or feel they don’t know enough about the topic to answer the question accurately. Give respondents options such as: “yes,” “no,” and “unsure.”
- Multiple choice: If you need more options but still want users to select only one answer, multiple choice is your best bet.
- Checkboxes: Checkbox answers make sense when users may select more than one answer.
Ordinal Questions
With ordinal questions, there is a clear order to the answers. You might ask for a respondent’s age with multiple-choice options for “Under 18,” “18-24,” “24-50” and “50+.” Another common topic for this question is income. You can collect answers with multiple-choice, drop-down or ranking questions. The ranking or rating format allows users to sort responses by preference.
Interval and Ratio Questions
Interval or ratio questions allow you to collect more detailed information for calculating averages and correlations. They can provide a greater range of data for more advanced calculations and help you gauge the degree to which your respondents agree or disagree.
Interval questions involve using one of the following methods.
- Ranking scales: Align ascending or descending multiple-choice responses horizontally, allowing respondents to select the most representative option. For example, you have probably seen survey questions asking you to rank your preference for an item on a scale of one through five.
- Matrix scales: Matrix scales are similar to ranking scales, attaching the same scale to multiple items in a grid format.
Ratio questions provide a variable measurement scale to help you collect and compare quantitative data by having the respondent input a number, usually through a text box. For instance, a ratio survey question might be, “How many hours a day do you watch TV?” and the response options may be “0-1 hour,” ‘2-4 hours,” and “5 hours or more.”
Open-Ended Question
Open-ended questions allow you to get meaningful answers and more specific, qualitative feedback from respondents. If you’re looking for detailed responses to uncover insights you may not have considered, open-ended questions can be useful. However, these questions can be difficult to sort through and generally aren’t good for large datasets. Additionally, answering open-ended questions can be exhausting for respondents, so only use these questions when absolutely necessary.
How to Write Survey Questions
Survey question language is a critical aspect to convey transparency and meaning to respondents. While every survey is different, here are some factors that can help you write good questions for a survey.
- Clarity: The question should be concise and easy to understand, without characteristics like double negatives or ambiguous wording. For instance, asking, “Do you agree that current software is not unhelpful?” may mislead respondents. Language should also be appropriate for readability. We suggest writing at an average seventh-grade reading level to ensure all respondents can easily understand the question. Ambiguous, complex or advanced language and jargon can confuse or alienate your audience, leading to poor results.
- Neutrality: The way you ask a question can lead respondents toward a specific answer and influence their answers, so take care to word each question in a neutral, unbiased way. Leading questions tend to sway respondents toward giving favorable answers, making responses less reliable. Asking a question such as, “How would you rate the gorgeous picture on this TV?” is likely to result in biased answers.
- Translations: Ensure that your surveys are in a language that your audience can understand. For example, if you plan to ask non-English-speaking respondents to complete a survey that is written in English, we recommend using a professional translator to create grammatically understandable, culturally appropriate questions.
- Question type: As you begin to design questions, you may find multiple formats would fit. Try to approach your survey from your respondents’ perspective, then choose the format they would expect to see. It should also support how you plan to use the data. The question format can also influence the answers you receive, so you’ll want to ensure the questions you’re asking align with the results you want to glean.
Create Your Questionnaire With the Respondent in Mind
Respondent experience significantly impacts your survey results. When respondents can quickly complete your survey, they are likely to be more engaged, which means you’re likely to get better responses from them.
Remember, respondents are real people who are taking the time to participate in your study—without them, you couldn’t get the valuable data you need. So, the first question you should ask yourself is, “Would I want to take this survey?” If you are designing a survey you wouldn’t want to take yourself, it’s safe to assume your respondents will feel the same way.
With a poorly designed survey, you run the risk of experiencing the following.
- Lower sample quality: Faced with confusing, difficult-to-answer questions, many respondents will start speeding through the survey to get to the end. That means the results may not meet your quality standards
- Longer fielding time: Poor survey design can cause low conversion rates, increasing the time it takes to finish fielding. In addition, if your survey has a high dropout rate, suppliers are less likely to send respondents to your survey. Suppliers care about keeping their respondents interested in taking surveys, so they place a high value on preserving the respondent experience. All these factors can affect your workflow and deadlines.
- Higher sample cost: Because poorly designed surveys lead to higher dropout rates, you might have to further incentivize respondents by paying them more to complete your survey.
Survey Design Mistakes to Avoid
Many factors can contribute to people dropping out of your survey or providing unsatisfactory answers. Avoid these common mistakes when creating a new survey.
- Poor mobile optimization: Almost half the traffic in Cint tools occurs on mobile devices like smartphones and tablets, so ensuring that your survey is optimized for mobile users is essential. Try to avoid complex interfaces.
- Excessive Length Of Interview (LOI): Your respondents’ time is valuable, so be sure to keep your surveys short and sweet. Identify your goals and keep your questions focused, so you can reap the benefits of a short length of interview (LOI).
- Creating respondent fatigue: This issue often occurs on surveys with excessive open-ended questions, which call for more energy than the typical respondent wants to use. Other common culprits of respondent fatigue include repetitive questions. Grid questions, which help determine frequency and preference, also lead to fatigue if they are used unnecessarily.
- Interfering with respondents’ privacy: Personal questions about age, ethnicity, income, family life or political preferences may lead respondents to skip questions or leave the survey entirely. Ask these questions only when necessary and always provide a “prefer not to answer” option.
- Repetitive or excessive attention checks: Be thoughtful about how you conduct quality or attention checks during a survey. Respondents can become impatient when they keep encountering the same questions worded slightly differently. When researchers intentionally include repeat questions to measure the consistency of a respondent’s answers, the result is often a negative perception about the survey.
- Double-barreled questions: Including “double-barreled” questions with multiple points or ideas — such as asking about product quality and customer service in a single question — can lead to poor results. Instead, ask individual questions for each topic that you want responses for.
Remember, the goal is to make it easy for survey takers to understand and respond to your questionnaire.
How to Format Your Survey
How can you get the most survey responses? A thoughtfully developed questionnaire won’t matter if respondents can’t navigate your survey. Here are some essential steps to take when developing your survey format.
- Break up the wall of text: Readability is critical. Your survey should include engaging content and visuals and present information in a digestible, easy-to-read way. Ensure your content is scannable.
- Use open-ended questions sparingly: Since open-ended questions can be taxing for respondents, try to include fewer than five of these. Users may give up on answering questions that aren’t specific enough.
- Optimize load times: As you can imagine, long load times lead to respondent frustration and can even cause them to drop out of the survey. Load time issues usually occur when using data-heavy media like video.
- Use a rich text editor: A program with rich text editing allows you to add formatting options like font sizing, alignment, images, italicizing, superscript and more for necessary clarity on questions.
- Use branding if applicable: Add your logo and ensure your text aligns with your usual branding guidelines for a cohesive, visually interesting experience.
- Review survey efficiency: While you want to collect as much information as possible, your survey should focus on one goal to avoid confusing the respondent. Your survey should also support how you plan to use the data.
- Test your survey: Once you have completed the previous steps, be sure to test your survey — and include testing for mobile devices. This best practice will help you find any broken links or page loading issues before sending the survey to respondents. It should be the last step you take before launching your survey.
Survey Checklist
Now that you know the best practices for survey questionnaires, let’s go over a brief checklist before you go live with your survey.
- Determine the type of survey question(s) you plan to use: Think about your survey’s topic or idea and which questions would best fit your needs.
- Identify your question language: Verify that your language is well-translated and unbiased — keep it clear and concise!
- Optimize your survey format: Work out any efficiency and readability issues after you’ve written your questions and answers. Confirm the best user experience for your respondents.
- Double-check survey for errors: Ensure you have enhanced your survey to avoid frequent mistakes. You may miss something that causes a respondent to run into issues or leave your questionnaire entirely.
- Go live: Once you have established all these items, publish your survey! With Cint’s help, you can watch the responses roll in from your desired respondents.
Avoid questions that are leading or vague: Try not to introduce your own opinions by asking leading questions, such as “We are already a top-ranked company, but how can we improve further?” In addition, avoid asking yes/no screening questions, which can be vague and lead respondents to surveys they don’t qualify for.
Respect respondents’ privacy: Personal questions about age, ethnicity, income, family life or political preferences can lead respondents to skip questions or leave the survey entirely. Ask these questions only when necessary, and always provide a “prefer not to answer” option for respondents.
Opt for shorter surveys: Long surveys lead to fatigue, leading respondents to either drop out or provide less thoughtful answers. The ideal survey has a LOI (length of interview) shorter than 15 minutes – this will generate the best traffic and conversion rates. Compose your questions carefully and concisely.
Use straightforward questions: Including “double barrelled” questions with multiple points or ideas (such as asking about product quality and customer service in a single question) can lead to poor results. Instead, ask individual questions for each topic that you want responses for.
Have fair expectations of survey takers: It’s important to be thoughtful about how you conduct quality or attention checks during a survey. Respondents can become impatient when questions are asked in different ways, over and over. When researchers intentionally include repeat questions to measure the consistency of a respondent’s answers, the result is often a negative perception about the survey.
Break up the wall of text: Readability is important. Your survey should include engaging content and visuals and present information in a way that is digestible and easy to read. Avoid long, overly-detailed questions. We suggest writing questions at an average (approximately 7th grade) reading level, which will help you reach respondents who may not have a high reading comprehension.
Reach out to Learn More
We know designing a survey takes a lot of time and effort, so it’s critical to get it right the first time. If you invest time in creating a careful, thoughtful questionnaire, you can expect to have better-quality data, lower-cost studies and faster results. Contact our team if you have additional questions about improving your survey design!
