I was lucky enough to attend a free virtual event on called How to perfect remote user testing, hosted by Torchbox. The hosts Olly Willans and Luiza Frederico ran a Q&A style event. Below is a video and some of the highlights, insights and resources from this event.
The event covered:
- How user research fits into the design process
- How to carry out key activities remotely: user interviews, user testing, user co-creation
- How to manage and communicate insights effectively
- Product management: how to make the right decisions
Why do user research?
Olly started highlighting the importance of user research. He stated ‘User testing is the bedrock of user-centred design. It shouldn’t fall by the wayside because we are working remotely.’
To de-risk the process
User research is often done at the start but it can also be done throughout the double diamond. Sometimes you have some insight but don’t know how to build it effectively.
Sometimes you have some knowledge or insights, but don’t know how to build it effectively. User research helps to ensure we are building the right product or sometimes you find out that your idea is a bad idea.
Building without user research is a form of gambling. It might be okay but is it worth the risk?
What steps do you need to take to do user research?
Research plan: Have clear objectives and communicate these within the team. Make sure you know what the main questions you are trying to answer. Who from? How will you reach them? How long will the sessions last?
Recruitment: Make sure you are recruiting users that meet your target audience. Existing users might be an easy route for speaking to engaged individuals, but it is often useful to test with those who aren’t existing users. Make sure you are clear on who you want to engage, how you are going to reach these individuals and how you are going to run the sessions.
Tips for recruiting user testers include:
- Incentives often help but people are often willing to help.
- Advertise through your existing channels. e.g. Twitter, newsletter
- Make sure you screen people and get consent for data collection and use
- Automatic scheduling speeds up the process. Tools for doing this include: youcanbook.me / calendar.ly
- If recruiting user testing through an agency, you will need a 2 week lead time, approximately £90 per person for a 30 minute test via a recruiter (incentives average £1 per minute).
- Think about testing accessibility needs. Specialist user testing services include the Accessibility collective.
- Do not underestimate how long preparation for user testing sessions will take.
How much research do you need to do and how do you know if you have enough?
Do lots of little things and often to back up and de-risk the process. You should only do user testing if it is going to move forward. The team recommended reading ‘Just enough research’ by Erika Hole
User testing methods
- In person interviews allow you to really dig into things in detail and see more about an individuals body language.
- Analytics allows you to see what users are doing, but not why they are doing it and what they are thinking.
- Remote user testing allows you to conduct user research with participants in their natural environment. There are two types of remote user testing – moderated and unmoderated. Moderated involves running interviews with users using screen sharing and video software, whereas unmoderated user testing involves quick and easy tasks that users can complete without you needing to be like, much like more traditions evaluation methods like an online survey.
The practicalities of moderated remote user testing:
- The norm is 45 – 60 minutes with 30 minutes afterwards to analyse the notes. What did I learn?
- Using two staff individuals is preferable. 1 to lead on the questions and the other to take notes, but can also participate where needed (if the leads internet drops out for example)
The pros of remote user testing
- Cost-effective: This can be of lower expense because you don’t need to travel or hire rooms etc, but it still takes time and money to prepare and run the session.
- Increased accessibility: for those who have access to the internet, but it is not accessible to all.
- Increased flexibility for users: It is easy to schedule into their lives. Early in the morning or late in the evening
- Recordings: Recording and transcription can be automatic, depending on the software used. Dovetail software allows you to do analysis on the software in a dashboard. auto transcribe recordings and highlight keywords when mentioned within the recording.
The cons of remote user testing
- Lacking facial expressions and body language: Phone and video user testing lacks as much facial expressions and body language
- Data privacy: There are increased data protection considerations. You need to be really transparent about data and privacy.
- Accessibility: It is not accessible to all. Some will be unable to engage and often individuals may need more support to engage with the platform etc.
How to ask the right questions?
- Ask open questions, not closed questions. Jobs to be done methodology can be a helpful framework. The framework focuses on past experiences because past experiences is the biggest predictor of future behaviour.
- You want to know the timeline of their experience – you want to know why and where it fits in their journey? Why does it matter to them? What is it that creates the change?
Unmoderated user testing sessions
Unmoderated tests is what it sounds like. It’s not monitored or guided by an individual, so there’s no one else present during the session except the participant. The test is live and can be accessed by the participant without your team needing to be there. Torchbox suggested having a minimum of 40 people per test.
Services such as Usability hub can offer this service. The website has over 170k users to potentially engage.
- Remote tools can allow for more shared conversation. You can get everyone to contribute, rather than having domination voices in the room. Software includes Miro, Mural and others.
- Pre-workshop sessions to prep them on the software is helpful. For some it isn’t going to work
- It is on a continuum from prototypes and miro – to just talking about or jotting down ideas. E.g. a letter from the future. Note down what your product would look like in a few years time.
How to make the most of your data?
- Clear strategy: Have a clear product vision and strategy.
- Prioritise features: You won’t be able to do everything, so prioritise and create an opportunity backlog. Will it generate meaningful engagement with users? Is it feasible? Make sure you say no to things. Just because it was mention recently or is mentioned frequently, doesn’t mean that it is the right feature. Make sure you dig into the why behind user features.
- Mixed methods: Combine your user insights with mixed methods. Most insights are not that useful in isolation, combine analytics, user interviews and other methods.
- Outputs and communicating insights: Think about the outputs and why you are sharing them with the team. Software to support the communication of the findings: Dovetail, Optimal Workshop, Notion and Airtable.
You can watch the recording here: https://youtu.be/fJeyYtzYYLQ
How have you been using remote user testing? What tips and tricks do you have for others to learning from?
To finish, I will share a quick joke from Olly, “How many UX designers does it take it screw in a lightbulb? First we need to be sure that a lightbulb is the correct solution, so I suggest a research project to understand the role of light in society.”