Everyone who tells you to prepare your surveys for mobile administration says:

  • Make the questionnaire shorter
  • Don’t use complex grids
  • Don’t use long scales
  • Make the questions simpler and shorter

But they don’t tell you how. I will.

I started writing short, clear questionnaires without grids and long scales about twenty years ago because I had clients that wanted to save money and get better data for better decisions. We developed five and ten minute surveys with scientifically validated questions that were simple and made sense to respondents. We cut the cost of the $50,000 study in half or by more. Everything went faster, too. These questionnaires were ready for the smartphone before smartphones were cool!

There is no magic pill or app for this. The truth is that to write questionnaires for mobile administration, we have to become better consultants and better at the profession of research planning and questionnaire design.

Here are the critical steps to designing a questionnaire for smartphones:

1. Write down the well-specified decision(s) the research will support. Fill in the blank, “We will decide _______________.” Write down the criteria by which the decision will be made, and maybe even the threshold upon which the decision will be made. It takes about three rounds of back and forth with clients, discussing the written document, to nail down well-specified decisions. This is the most important step to writing short questionnaires.

2. Write down the top 5-10 information needs that will be used to make the decision(s). Few decisions are made based on more than 5-10 pieces of information. Once you isolate the prioritized key information, you are in good shape. This is the second most important step to writing short questionnaires.

3. Do qualitative research before quantitative research. First of all, you don’t know what to ask or how to ask it in a survey unless you do qualitative research first. Secondly, asking the survey to get information that is qualitative is an enormous mistake. Sometimes just a few qualitative interviews may be enough.

4. Outline the information needed before writing the questions. Please get out of the pattern of having your clients writing survey questions! They aren’t trained to do it. If they can do this as well as you, then, “Houston, we have a problem.” When you talk to clients about the information they need and show them excellent questionnaire design skills, they will never want to try to write the questions again. And they won’t ever want to replace you with a DIY survey tool.

5. Write the questionnaire using well-honed questionnaire design skills. You have to have really good questionnaire-writing skills. Training is available. It is fun and professionally rewarding.

6. Conduct real pre-testing, often called cognitive interviewing, with respondents. Real pretesting involves interviewing respondents to see how they interpreted each question, how they came up with their answers, and how they interpret each concept in the questionnaire (e.g., “What does the term, valued, mean to you?”). Every time I do this I find critical mistakes that need to be fixed. Even just two or three cognitive interviews can make a huge difference.

For anyone who says they don’t have time to follow these steps, this is faster than the traditional method of questionnaire design which involves emailing the questionnaire around for everyone to edit.

There are also technical issues for smartphone administration, such as not using a progress bar, using data pickers, not using images that require too much time to download, and pretesting the questionnaire on the devices to make sure everything is working. There are also a few issues associated with scale and buttons. But these technical issues are relatively easy, and any reputable agency that programs and hosts the survey will help out with these issues.

The discussion in the industry about mobile administration is focused on the technical issues. Yet three-quarters of getting this right is not a technical, but rather, about our professional skills. Getting this right requires us to be superb consultants on planning research to support decision-making, determining the difference between qualitative and quantitative information needs, and becoming true experts on the science and art of questionnaire design.

Published 
www.davidfharris.com