Virtually all researchers are aware that survey responses can be significantly impacted by the way the questions are designed. Language choice, level of detail, and the number of response alternatives can all influence the respondents’ selection. Most also understand that the ordering of response alternatives can likewise have a serious effect on survey results, and look to utilize a system of rotation for the order in which response alternatives appear within a question. There is, however, more than one way to implement such a rotation, and all ways are not created equal.

For example, let’s assume that you are designing a survey in which you want respondents to select from a list of five brands that they use frequently, but you desire to ask follow-up questions for only two of those brands. You could elect to implement randomization logic for the response alternatives, but a random assignment cannot guarantee an equal distribution across all of the alternatives. Because the assignment is truly random, it may end up using one ordering scheme more or less as compared to the others. This variable can impact the respondents’ selection across the sample, and risks introducing significant bias to the response data.

An alternative to randomization in this scenario is use of the “least-filled quota.” It utilizes computational logic within the survey to assign respondents to quota groups by always selecting the ordering scheme with the lowest current quota count. Least-filled quotas ensure that all of the response alternatives get displayed in each ordering position an equal number of times, and ultimately allow for the most diversity in the data set. View a least-filled quota demo.

Kinesis Survey™ offers standard least-filled quota logic as summarized above, as well as priority least-filled quota logic for instances where you desire to have one of the response alternatives take priority over the others, and nested group least-filled quota logic for instances where it is necessary to display response alternatives depending on the respondent (usually based on demographic assignments). The Kinesis Survey™ interface includes the necessary tools to program these types of setups with ease, thereby helping to ensure the highest quality of your research data.