#2 Further Research — Surveys
This is the second article in a continuing series, of a look back through my progress throughout the UX Design Institute’s Professional Diploma in UX Design.
In the previous article I presented the first project, the Competitive Benchmark project. As I mentioned, this was a really good exercise and start project as it really helped me familiarize myself with some flight apps. I could spot some regular patterns along the journey I made through the apps. Common faults stood out; ones I can further research soon. Each app was successful in a number of ways, too, with some sharing these successful features. It was an initial first-person look into understanding the user and the competitors. It’s not enough to rely on my own experience with the apps, though. It has been effective in highlighting the positive and negatives and will help solidifying my goals and prioritizations later on.
In my own mind from benchmarking, I have spotted some features of each app that seem to work and which I consider pain points. By using some open ended questioning I can find out if users are having a similar experience as I did, backing up my benchmark data. These specific points that match up, i found a little later, will be great use for following projects.
The Survey
The survey contained a mix of structured and unstructured questions. From this survey I wanted to find out more about the overall goals of people that use airline or travel apps, the things that help them along their goals, and also the hurdles they come across on that journey. I used survey monkey for this as it does a nice job with its presentation, both for the survey and the outputted data afterwards. I compiled this later for the affinity diagram note taking (some of which is presented below).
The Benchmark project was largely qualitative research, comparing my user experience regarding the techniques, processes, interactions, journeys, and more, throughout their apps. This survey would pull in a mix of both qualitative and quantitive data through multiple choice questioning and open ended questioning. The survey would also include the ‘Three Golden Questions‘ as taught in the course.
- Why did you visit… ?
- Were you able to complete your task? Why not?
- What would you change…? What improvements would you make… ?
To further and build on that qualitative based benchmarking, surveys or interviews are needed to learn how people are interacting with and experiencing the apps. Question one above, should help in learning about thew users’ goals. The second should guide me on how well those goals have been achieved, and which, if any, hurdles were encountered along the way. The third question above, at first seems ineffective as it’s likely going to generate a lot of users’ opinions, product feedback, and design feedback. Some of that is true and not useful data, however this question generated the most feedback. It gives the respondent control, feeling free to make suggestions and comments, not just on the improvements or changes, but also on problems they encountered. Those problems are the valuable data here. Respondents could have mentioned them in question 2 above, though with more control given with the completely open ended question users evidently were free to write more about their problems encountered.
The Questions
#1
This introduction question was included in the example provided by the course. I wondered about the effectiveness or use of this question. In one way, it can be used as a type of screener question. We can learn how fresh an experience is in the user’s mind and how frequently they use travel booking apps in general. This question may have been alternatively worded as in: How often have you used a flight booking app, website or aggregator in the past year? Though, in the case of my survey I wanted to find out about user feedback and the most effective feedback would have been the freshest in the users’ minds.
Results below show my data is a bit outdated and not so fresh. This was largely due to the pandemic being in full swing at the time (April 2020). I expected the answers to this question would be a little different from this time last year, as proved when checking some of the UX Design Institute Sample Research Data.
#2
From the competitive benchmark, I found that most airlines brought the user directly to the Home Screen. The option to log in, or sign up, was not so clearly laid out. Korean air and Aer Lingus both had multiple instances of their login and sign up links on their Home Screens, though worded multiple ways. It was not so easily accessible. I wanted to check how people use the type of sites in question. The purpose, shown later, being providing an upfront, easy way for users to choose their route, while prioritizing one way (guest) over the other (log-in).
There was no notable difference in how respondents used the sites.
#3
The next question, one of the most important for sure is related directly to finding the users‘ goals. For this project it generated expected results:
‘Book a flight’ being the obvious primary use-case. Though, this question did also result in some good indications of secondary and tertiary use-cases. With a larger number of respondents I would be able to have solid secondary and tertiary use cases; to find what else I should add to a quick menu or static menu on the Home Page.
#4
This question, being a yes or no question yielded unexpected results. 93% of respondents said they had no problems while using the site. Though, many of the ‘Yes’ respondents seemingly contradicted themselves in their Q.10 answer as many of them showed they had problems or failed in the their use of the site. Or, the site failed in its provided service, I should say.
I find it hard to believe that 93% of users completed their task after my own experience using their apps and sites. I‘m not really making an assumption, though, as my last question points out a lot of problems users experience.
#5
This question is about importance when booking. I was thinking about Flight Search Results mostly though. Different airlines display results in so many varied ways. Prioritizing search result data is important in the flow through a flight booking. Also, ease of sorting that result data differently is. I’ll come to this a bit later.
Price proved to be the most important to users when booking a flight. Followed by flight time and flight duration.
Airline mileage wouldn’t usually be related to flight search information, but I wanted to include something related to mileage in the survey. After all, this question is primarily about what’s important when booking.
#6
From the benchmarked apps and generally my experience with travel apps, i found it difficult to share the information i wanted, and when I wanted. I could hare flight data with Aer Lingus, for example, but only after selecting the flight. And when I shared from that app it shared a long link message with minimal information except the link, which linked not to the app, but to the web version. Pretty much useless UX since I am sending it through a mobile messaging service (Kakao Talk). Further, it brought me out of the Aer Lingus app, never to retutn automatically. Again, total failure.
So this app along with the others pushed for this specific question: regarding sharing flight details. The results show that most users share (or want to share) flight details before booking. Though many users resort to a manual screenshot or text message, rather than the app’s own share function.
Skyscanner did allow flight information sharing which linked to the app, though it did not contain any information within the message about the actual flight. In this case the person I am sharing to must either have, or install the app to see the flight information.
#7
The benchmarked apps showed huge variations in how extras were handled, and where exactly they were placed on the user journey. I specifically wanted to find out if users cared enough for booking seats and baggage as part of the booking.
Some airlines, have seat booking as part of their journey on their website and mobile website, but emitted the possibility to do this on their app.
KLM in particular emitted the possibility to book seats on their app at the time of benchmarking. Though, if I booked through Skyscanner (which directs me through the KLM mobile app site to finalize booking) it allows me to book seats before checkout.
Results are divided on this one. A slight majority feel that they can do this after checkout. This would lead me to including the possibility to book seats and baggage, but also giving the user an easy choice in continuing without it.
Q8
I simply wanted to find out which payment methods are common, and which payment methods people want. The results included mainly credit card, but also bank transfer and debit card. Many sites do not include bank transfer ability or even debit card.
Here in Korea, for online shopping and travel booking bank transfer is a common service to offer as it’s easily done within an app.
#9
I realized soon after that this question is basically made redundant by preceding questions, and the final question. The results: 50:50 — neither enjoyable or unenjoyable.
#10
This final question proved the juiciest, as I mentioned at the beginning. The results of this question gave me a lot of unstructured data, which was great to align with my own thoughts.
Still the research phase of the course. It has all been interesting to see the connections developing between both research projects so far, with more to come in the following projects.
The next projects on the course were the note-taking and usability tests. I will combine the next article to include both as the usability project obviously had note-taking involved anyway. On to the next….
