Writing a research questionnaire is no small task, especially when it comes to logic. Keeping track of question numbers, skip patterns, show/hide options, and how everything ties together in the end can be tricky. During the process you can often see the question order change, new questions and answers are added while others are being removed. At the end of it all, the survey looks nothing like it did at the beginning.
Through all these changes, logic needs to be correctly updated and checked for accuracy. But who wants to do that? To help keep things simple, it may seem easier to write logic based on question and answer text and content instead of values. That way if you write “If a ‘juggler’ or ‘magician’, show Question 15” you don’t have to change the logic if the question or answer numbers for “What is your special talent?” change. When the questionnaire is complete there is no need to double check question or answer numbers.
This seems reasonable, right? Let’s hold on a minute. Writing surveys this way feels intuitive but can create confusion when it comes time to program and test. Taking the time write the logic based on variables pays off in spades in the end, both in programming costs and testing time, not to mention overall accuracy.
When referencing something from another question, use the question and answer numbers in addition to the text descriptor. This makes it easier for programmers to know exactly what you’re referencing and what questions are part of the logic. We’re really good at programming studies, but still working on our ESP skills.
In addition to being unclear, this logic slows down the process for programmers, especially if the logic is based on questions much earlier in the study. It creates an extra step where the programmer has to go back through the study and find the question being referenced based on the question content. Even if the question number is listed, the programmer still has to go back and find the values of the answer options to complete the logic.
Here’s an example
- Unclear Logic: If respondent owns their home, do this…
- Better Logic: If S4 = 1 (respondent owns their home), do this…
If a programmer sees unclear logic like this, they have to search the survey and hope to find the correct question you intended to base the logic on. Sometimes that is easy sometimes…not so much.
Here is another example:
- Unclear Logic: If respondent likes chocolate cake, do this…
- Better Logic: If Q2=4 or (Q4=1 and Q4=2), do this…
Here the unclear example is vague, as there could be multiple questions that could set this logic. For example there may be a question that asks the respondent what their favorite cake flavor is as a single choice, then a multi-choice question after that which asks then to select other flavors they like.
So what does “like” refer to here? Does it only apply if they selected chocolate in the single choice, or could this also apply if selected in the multi-choice? That’s why logic based on values, not text, creates a much clearer picture of how the survey should work.
Hidden variables can also help with simplifying logic later in the questionnaire (but that’s a topic for another blog post).
It may make sense to use descriptors at the beginning to describe logic – that way during revisions you don’t need to update logic every time. But once the questionnaire is complete, go back and flesh out the logic by adding variables (specific questions and answer values) to the logic.
This extra work and attention to detail can result in faster programming, less back and forth, and lower costs – leaving you money in the budget for some well deserved chocolate cake!
