For many design teams, customer support feedback, server logs, online surveys and possibly A/B testing are the only sources of user based data they may collect to guide their design efforts. It is great if a team is also conducting usability tests to improve their designs. Even if it is just before launch or to benchmark designs after launch, usability tests can be helpful to uncover problems with a design. Of course a more effective time to usability test is early in the design phase when the design can still be more easily changed. Despite a growing awareness and usage of usability testing, your team may still be in the dark about essential needs of the people for whom they are designing products or services. For those who want to have more success, there are several other research methods that can be conducted to help an organization learn what people need beyond what you already provide.
Usability Tests are Great
Usability tests are a great way to learn about how people perceive and use an interface design. In-person usability tests provide not only direct observation of a person trying to complete a specific task, they also afford the opportunity to ask follow-up questions based on their behavior. Usability tests also allow you to probe and/or to expand into other issues that may affect someone’s ability to complete a task which may not have been in your list of original scenarios being tested. This can help improve your team’s understanding of the people in the specific context of the UX being tested. However usability tests still have limitations that can prevent your team from discovering other needs people may have for which you could better serve.
Limitations of Usability Tests
First a usability test typically involves having people come to either a lab facility or a conference room which is removed from their normal environment. The system you have set-up likely does not have all the applications or even the bookmarks and other digital and analog resources that a participant may normally rely on to do their daily tasks. Sometimes these additional resources make a critical difference in the performance of those tasks. To be clear, usability testing is still great for uncovering problems with the UI you are designing even with these limitations. To really discover new features or services you may not have thought of yet, you need to get out of the lab or office.
Remote Usability Not the Answer
Remote unmoderated usability testing is presented as a way to have people stay in their own environment, which may seem like it would address this gap. Unfortunately remote unmoderated usability tests have their own limitations. One of the major problems is not being able to ask follow-up questions based on their behavior. Remote sessions are also more limited in the length and number of tasks that you can expect a participant to complete.
You can conduct moderated remote usability tests, where you utilize a screen sharing tool to run the sessions just as you might in a lab. This does allow you to ask relevant follow up questions and explore issues outside of your predetermined scenarios. However you still do not get full visibility to the environment in which a person is working. You will also have to deal with issues like garbled audio and failing network connections which can disrupt the session. The main issue, as with in-person sessions, is that you are still focused on the scenarios you have predetermined to follow and not necessarily seeing other opportunities.
Alternative Research Methods
So here are some alternative research methods that can provide you the ability to gain a deeper understanding of people’s needs:
- Ethnographic Field Study: This is a method where you normally would travel to a participant’s work or home and simply observe their activities. Somewhat like Jane Goodall and her studies of gorillas, you generally would not speak to the person until after they have completed their activity. Afterwords you may ask them follow-up questions based on what you observed.
- Contextual Inquiry: This is another method where you go to the participant’s work or home location to observe how they perform some activity. Similar in some ways to a usability test, as you would ask them to show you how they perform a specific activity. This has a bit more structure compared to an ethnographic field study in that you would ask them to explain what they are doing as if you are the apprentice and they are the master of what they are doing.
- Diary Study: This is kind of like a remote version of an ethnographic study. In this method you ask a number of participants to keep a written diary as they are doing their work over a longer period of time. This might also involve a short interview afterwards which can be conducted in-person or by phone. This type of study is good for activities that may take longer than a day to complete.
The value in these types of research methods is that you see the larger context of use that people deal with beyond your existing product or service. This knowledge can help you not only understand why your product may need certain features or content and how to improve them, it may also show you entirely new features and content which could be developed to improve the experience.
Resources
The following are some of the resources I have used to learn more about these and other user research methods:
- When to Use Which User-Experience Research Methods, by Christian Rohrer. — This is a great article from the Nielsen Norman Group that maps many of the most common research methods and what research questions they are best at helping to answer
- Contextual Interviews, from the extensive Usability.gov website.
- Contextual Design, by Karen Holtzblatt and Hugh Beyer. Their book is the seminal resource to learn how to do contextual inquiries as part of a user-centered design process. There is a great video where Karen explains the Contextual Design process.