In the previous post I described the test methods of user experience test. In this part I share the test results and my recommendations based on them.
One of the main goals of the UX test is to capture how people engage with GNOME when they interact with it for the first time. After conducting the test with 7 participants I noticed several patterns of users behavior that I would like to mention in the beginning.
During the warm-up period the volunteers had some time to explore the system. There were a few most popular things people did after the computer was launched:
- Launch a browser.
- Change settings to personalize the computer.
- Look through pre-installed applications.
- Launch “Software” to check out available applications and search to download browsers users normally use.
After this, participants started doing the scenario tasks. Each participant completed all three tasks: Managing files, Using a browser, and Checking email.
What was hard for users
I consider welcome setup to be the stage of user onboarding — that’s the first thing people see when a new computer starts. After each tester logged in to a fresh test account, they had to follow a number of steps — select language, keyboard layout, privacy settings, and connect online accounts. For most users it was difficult to make decisions in each step, they either didn’t understand clearly where it would lead, or they didn’t know the difference between the available options.
So during the initial setup users skipped all the steps that could be skipped.
Using email client
The biggest difficulties users had with the Evolution, the email client. Firstly, most users had issues with the mail authentication. After submitting the user name and password, they didn’t move forward, instead the same mail authentication request window appeared again.
No notifications of the origin of the problem or recommendations occurred on the screen, and users supposed the problem was with the password they entered, so they tried different passwords. After several attempts to log in they became annoyed because were unsure if the application worked at all.
Besides, wrong password wasn’t always a reason of the issue, for a couple of testers who had Gmail account Gmail didn’t allow Evolution to access their accounts. Testers got to know it from notification that came to their email.
When finally logged in, the testers claimed that the interface of Evolution was quite outdated. After checking email and exploring Evolution most testers found it uncomfortable, some looked through settings and said it was “too much”. One of the testers created a note, but after saving it he couldn’t find it.
In general, the testers didn’t express desire to use Evolution. It is worth noting that all the participants normally use web email client and none of them ever had to use any desktop client (but a few tried and didn’t like it).
What was easy for users
The participants had no problems with managing files, as the interface is intuitive and similar to the systems that they usually use. They could easily navigate through the file system, copy files from the USB drive to the computer, create and delete folders.
Using a browser
All participants considered using GNOME Web very simple. The testers managed to open a few websites that they normally visit without any difficulties. However, some users commented that the browser lacks some liveliness and a colorful theme.
Directly after each UX test, I asked the participants the following questions. Not every participant had a response to every questions, and some overlapped, but here are the general responses:
Who can you imagine using GNOME?
IT specialists that use Unix systems.
Young people, as they tend to learn new things faster.
Anyone, as it can fit the needs of people of various occupations, age and interests.
If you had the choice of choosing this system over your current one, which would you choose? Why?
I am happy with my current system, and now I don’t have a reason to switch to this one. It doesn’t have any obvious advantages for me.
As a person who doesn’t like adopting and getting used to new systems, I wouldn’t like to use GNOME.
If I had to switch to this system (for example at work), I wouldn’t be upset.
Do you think it’s attractive?
The system is attractive enough, but it’s too monochrome and lacks colors.
I would like the design style to be more modern. It feels out of date and clumsy for me.
I use Unix, so Fedora with GNOME solves the tasks I normally do. The interface design is not very different, and the differences are not critical for me.
The design is understandable and looks good enough.
There are a few recommendations for GNOME based on what I observed in the test:
The most often testers commented on the design style. As Windows and Mac users, they are used to more colorful and comfortable user interface, so the UI of GNOME seemed dull and outdated to them. The users claimed the main reason why they wouldn’t choose GNOME over their current system was the design style, so I would recommend to modernize it. Differentiation in design can give GNOME more competitive advantages over other systems and help acquire users for which design is one of the top priorities.
The first launch of a computer
I recommend to reduce number of steps in welcome wizard. Language settings are must have, but privacy settings and connecting online account makes people hesitate and don’t give much value on this stage of engagement. Users do want to personalize the computer, but they are happier when they are not forced to do it.
There are two scenarios connected with passwords that make users confused:
- When setting a new password, the system doesn’t accept “low-security” passwords, and only replies with “Sorry, that didn’t work. Please try again”.
- When a user enters wrong password, a form reloads without any notifications and disclaimers.
In both situations it is hard to understand what’s the problem and how to fix it. My recommendation is to clarify requirements for a password.
Keyboard input language
For people who communicate in several languages it is important to see the selected input language and to switch it easily. Windows and Mac users are used to seeing the language indicator in the taskbar, and they find it complicated to follow a number of steps in settings to do such a simple thing.
Place the input language settings where people would normally look for it.