This week has been great, I’ve finally completed my 7 persons user experience test, hurray! In this post I’d like to share my first impressions and briefly describe what went well during the test and some difficulties I had to face.
Test had a total of 7 participants, which is not many, but enough to get information on how people perceive and relate to the product and uncover important usability problems.
There is an article by Nielsen that explains how many test users we need in a usability study, it provides a mathematical formula and a chart:
Nielsen assumes that after the fifth user, you are wasting your time by observing the same findings repeatedly but not learning much new.
Frankly, I would be happy to test more users, as the process is really exciting and fun, and the testing results are surprising, but last month I moved to a new city and even appointing seven tests was a little challenging. It also took a little bit longer than I expected (people are busy in general and it’s not easy to find them available).
I had the chance to welcome volunteers from different fields, with different computer expertise and expectations. I learned that person’s background matters, and it was very interesting to observe how behavior of a thirteen-year-old differed from behavior of adults, and how a designer’s perception differed from programmer’s and manager’s.
It appeared that conducting UX tests is not as easy as it might seem, we must be really well prepared. And it’s essential to do a dry run of the text by yourself — in my case I had to do it twice before I made sure everything would work properly during the test session.
The participants didn’t hesitate to speak their mind when they were confused or something bothered them during the test, which made me happy, because they said really useful and sometimes unexpected for me things.
Unlike the usability test that Renata and Ciarrai performed, the UX test didn’t require participants to do “get down to work” kind of tasks. Rather, I allowed each participant to explore the system, and pick some tasks, based on what they tend to use their computer for.
After the testers interacted with GNOME and its applications, I asked them some questions about their experience, which you can find in my previous post. I also added more questions aimed at understanding and meaning, suggested by Alan:
Who can you imagine using this [GNOME]?
Do you imagine men or women prefer it?
Old or young?
What kind of jobs do you think they might have?
If you had the choice of choosing this system over your current one, which would you choose? Why?
Do you think it’s attractive?
In general the tests ran smoothly, and lasted for about 35 minutes each. I did audio recording during most of the test sessions.
I am pleased with how the tests went, and the volunteers seemed to be pleased and enthusiastic too! Most of them found that they liked GNOME and indicated interest in using it in the future.
In the next post I will write the analysis and get into details of the tests. I’ll also present an analysis of the tester’s engagement, using the emoji-with-counts method.