#4 Usability Testing Setups
Once again, I’ll open with mentioning that this is part of a series of articles looking back over the UX Design Institute Professional Diploma in UX Design course. For this article I will be going through my setups for the online and in-person Competitor Usability Tests.
This was the first usability test for Project 4. One was required, but I did two for the experience online and in-person. Covid19 here in Korea was controlled enough at that time that allowed for in-person visits.
Online Setup
Tools: Let’s View (letsview.com), Zoom (zoom.us), Final Cut X (not required)
The Competitor Apps to be tested: Bangkok Airways, Cathay Pacific
There are so many different methods and software to get this task done. This is just one of them. This is what worked for me to achieve a video like that above.
I had previously been using Let’s View for mirroring my phone to the desktop. The quality was decent, and never a glitch. Easy to setup, and the free version is plenty powerful, with good resolutions and connectivity.
1. INSTALL APPS
Have the user prepare a cellphone; installed with the apps for testing — In my case, I wanted to test the splash screen and app permissions, so I reminded the user not to open the apps after installation.
2. SCREEN MIRRORING
Instruct the user to install Let’s View on both their computer and cellphone.
Open Let’s View on computer and phone. Make sure both of them are connected to the same network. My PC was connected via LAN and phone WIFI (through the same router).
Scan the barcode or enter the pin as above. And that’s it! There are some tweaks you can make for quality and bandwidth in the settings page. But it should be good to go out of the box.
And here you are with a decent projection of the phone screen on the desktop.
3. ZOOM CALL (free version)
Instruct the user to install Zoom from zoom.us. They don’t actually need to install the software as it can be done through the browser. However, for stability the app is advisable. Note that the user does not need to sign-up.
You the tester can create the zoom call. Sign up for a Zoom account if you don’t have on already. Then sign in.
You can directly start a meeting, or alternatively you can schedule the meeting or even a series of meetings. You can assign a unique PMI (meeting ID) and password for each meeting scheduled, and enable a waiting room so your user can wait for you to sign in to that meeting.
Allow your participant to Share Screen under the Security Tab.
Instruct your user to Share Screen. Then you should be viewing their desktop with the phone screen mirrored on it. If the user’s webcam isn’t already turned on, then instruct them to do so.
Make sure you can see and hear your user, see their phone screen mirror, and that they can hear you. Check the quality of the mirror as they navigate the phone. If all is good you are set to record.
*For recording the call there are some settings available for recording separate files and so on. I won’t go into it here as the default option does the job you’ll need.
As you are doing a 1 to 1 call you will have unlimited time for the call available.
CUT OUT LET’S VIEW
You could cut out Let’s View completely and have the user join the call with their computer and phone.
However, a free Zoom call lasts 40 minutes. They can go over sometimes, but I wouldn’t rely on that. If you opt for this option (it does have much easier setup). My advice is to get the Depth Interview completed, along with Task 1 (Airline App 1), end the call, and restart the call to do Task 2 (Airline App 2).
If you have the pro version of Zoom then this is obviously the best way to run the usability test without the 40 minute limit.
4. Editing
The screen shot of the usability test at the top of this article isn’t the actual output video from Zoom. The output file from zoom would be perfectly okay, but I wanted to create a template for it, to use with future Usability Tests.
I did this in Final Cut Pro, my go-to video editing software. Bottom left has date, tester, company, and user detail. I will include a timecode in the bottom right in future versions. Basically the videos are cropped and a border added.
The test ran smoothly, without any technical hiccups.
In Person Setup
Tools: Laptop, Phone, Zoom (zoom.us), O̶p̶e̶n̶ ̶B̶r̶o̶a̶d̶c̶a̶s̶t̶ ̶s̶o̶f̶t̶w̶a̶r̶e̶, Final Cut X (not required), Camcorder (not required, but wait for it..)
The Competitor Apps to be tested: Bangkok Airways, Cathay Pacific
For this usability test I had planned to use Open Broadcast software. A nice feature filled open source software, though not without usability issues.
After testing successfully at home before going to meet the user at their office, my setup failed on the day! But not to worry, I had a backup plan. Always have one at hand!
That backup plan was as I mentioned above in the ‘CUT OUT LET’S VIEW’ part. I did the free version Zoom only method. Stopping after Task 1 to call again to do Task 2. This was very easily done as it was an in-person test using my own cellphone — prepared with the required apps, and my own laptop also (supposedly) ready.
However, again something was not right when I arrived home to check out the usability test. There was no audio on the recorded zoom call. I had a laptop microphone issue that was also related to the previous Open Broadcasst Software problem. Once again though, I saved myself with the camcorder I had set up on a tripod as backup. I was able to use that audio synced to the video from the zoom call.
The Tests
The actual interviews and usability tests went really well. Considering it was the first attempt at something like this. I don’t mind the technical side of things so much as there is always a way. The difficult part was actually holding a smooth as possible conversation with the user .
The tests ran in the order of:
- Explanation to the user that we are conducting customer research and the reason ~ to identify common issues in apps and pinpoint areas for improvement.
- Tell the user about the room setup. What is being recorded, what is being used later, and how it’s going to be used.
- Depth Interview — find out the user’s background and experience in travel booking
- Task 1 — Explain the task. Have the user write down the details if required
- Task 2 — as above
The depth interview is probably the area that I need to work on more and more. Getting that important opening to the usability testing as smooth and natural as possible is really beneficial to the user testing the app. They’ll be able to relax and make a booking as they normally would, and hopefully feel confident enough to make decisions and give feedback on those decisions.
The tasks also need some good facilitation. You have to time any questions so as not to interrupt the user’s flow through the app. You need to watch carefully the users expressions. Something like timing tasks or actions can be helpful in that pinpointing areas for improvement. If the user looks like they are having some trouble, you could step in and ask what is the problem, but it’s more effective to wait and give some time for them to get through it by themselves, and ask later if desired. Like mentioned in ‘The Mom Test — “Don’t zoom in on a problem too quickly”. That book is a good read in preparation for interviewing. It helped a lot when preparing for these tests.
Keep a feedback section at the end of each task for letting the user recollect their thoughts on what was positive and negative about the whole experience.
You can learn much more by watching someone doing a task, than by actually doing the task yourself. You can get an unbiased view of the task in progress and how the user is delighted by the task or just plain pissed off!
Up next, is the beginning of the Analysis Phase where I try to make sense of the research carried out so far.
