How much user research is enough?
Fowlam’s CRO, Michael Brown, has a PhD in user centred design and has worked in the field of user research for over ten years. Here he discusses how much user testing to include on a project.
One of the most common questions asked of a user researcher is, how much research is enough? This question can be phrased is many ways:
-How many participants do we need to ‘fully’ test a product?
-How many test and redesign iterations should we run before release?
-How much time, money or resources should we plan to use on user testing?
Just like the choice of research methods and study design, there is no single correct answer for all projects. A simple website with a few pages of information could be great with just some light-touch techniques, whereas a complex safety critical system could call for a large investment in user research to ensure all elements are optimised and thoroughly tested. However, over the years I’ve come across some rules of thumb that are a solid starting point.
A good starting point that I’ve used during early project planning is the 2 to 1 rule. Basically, for every 2 hours of effort that go into developing a system you should allow 1 hour for user research and user centred design activities. That hour could be spent planning and running usability test, building and communicating personas, or investigating user needs before development begins. For example, if I am expecting a new website to take about 100 hours of development effort, I should plan to spend 50 hours split between discovery work, user centred design activities and iterative testing. This may seem like a lot, especially for team that currently engages in little or no user testing, but I’ve always found that heavily investing in user research from the start of a project ultimately leads to less overall effort going into the project and a better outcome. Teams that have a serious commitment to research-driven development save so much time by focusing on fulfilling user needs instead of developing feature that will be scrapped before release or simply never engaged with by the user.
5 participants is enough for user testing
In general, testing with five participants will give you enough data to make solid, reliable design decisions. However, it is always important to run multiple rounds of testing with different groups of users. You should also have more participants if there are divergent groups of user for your technology.
See the Nielsen-Norman website for a more in depth exploration of this rule of thumb.
Minimum 30 survey responses
Paper-based and web surveys are another useful tool that people often struggle to scale correctly. The low cost of collecting more responses via surveys (compared to say user testing or interviews) is balanced by less reliable and often less detailed responses. A guideline I’ve come to use over the years is that 30 is the bare minimum number of responses a survey should aim for. Larger samples are necessary if you are exploring divergent user groups or if high value decisions are going to be made on the basis of responses.
Of course, every user research study is different and requires careful consideration of a wide range of factors before decisions are made on the right amount and type of testing to carry out. If you would like the support to carry out effective and efficient user research that gives you actionable insights, contact our experienced team.