How not to ask questions
Sounds simple, but one of the hardest lessons a user researcher learns is how to ask questions to get meaningful and useful answers. A good way of learning the basics of this is looking at some of the most common mistakes people make when designing questions for a research session:
Bad question: “This website is designed to be easy to use – do you think it is?’”
Problem: If your question includes the answer you’re looking for, that’s what you’ll get.
Solution: Avoid putting any value judgement in your questions, so participants are more likely to be candid in their responses.
Better question:“What is your first impression of this website?”
Bad question: “Would you use this prototype more than the existing website?”
Problem: If your questions ask for a simple yes/no answer, then usually that’s what you’ll get. No detail and no explanation means it can be tough to get to the bottom of the issue.
Solution: Make sure questions encourage participants to explain their answers. It’s also a good idea to think about prompts you could use to encourage participants to expand on their answers.
Better question: “Compared to the existing website, what about this prototype would help you to use it?”
Bad question: “What don’t you like about this site?”
Problem: If your question is too open, you risk getting answers that are entirely off topic and don’t help you answer your research questions.
Solution: Make sure all your questions are focused on things that you want to know about and can act upon depending on the findings.
Better question: “Is there anything about this site that you think would stop you being able to use it?”
Focusing on attitudes instead of behaviour
Bad question: Do you think climate change is an important issue?
Problem: In many situations, there is a big difference between how to think you should behave and how they act. Generally, user research should be focused on how people will use services, not their attitude towards it.
Solution: Ideally you should observe behaviour first hand, but when this isn’t possible understanding past behaviour is the strongest indicator future behaviours.
Better question: “In the last year what have you changed in your
Worth saying, these points are relevant for open research sessions such as interviews or focus groups. When designing surveys or other large scale quantitative feedback research, a very different approach is required. But that’s a topic for another day.