For the last few weeks I’ve been preparing for the user experience testing. In this post I share the final version of my usability test: the opening script, scenario tasks, pre-test and wrap-up interview.
I plan to start each session with a brief welcoming part that explains to participant how the test is going to work. Below is the current version of the script that I’m going to read during the introductory part of the test.
You have been asked to participate in a “first experience” test for the GNOME desktop. Just like Windows is a desktop and MacOS is a desktop, GNOME is a free software desktop. If you have any further questions about GNOME I’d be happy to try and answer them.
For this test, we are interested in what people think of their first experience with GNOME. We are looking for people who haven’t used GNOME before, so we don’t expect that have used GNOME before today. We want to see what you think of GNOME when you use it for the first time.
This is entirely a test of GNOME. We are not testing you and there is no wrong answer, so please do not feel pressured by time or anything else. All we’re looking for is what you think about your first experience with GNOME.
For this first experience, I’ll ask you to login using a test account. I’ll give you some time to experiment with GNOME. Use it like you would use a new computer for the first time. To help guide you, I’ll ask you to do a few sample tasks that mimic how most people would probably use a new computer.
But before we begin, I’d like to learn a few things about how you would use a new computer:
Let’s say at work or at home you have a new computer with pre-installed operating system which is new for you.
You have booted this computer for the first time. How would you use it at first?
Scenario tasks are the main part of the test. For each task I’ll give a participant a copy of the scenario and read it aloud. In some tasks I’ll give additional information, like login credentials, or a USB drive with sample folders and files.
First I’ll start with a short intro:
Okay, let’s have you use the computer now. Here’s a username and a password you can use to login. No one else has used this account before, and no one else will use it when you’re done here today. (I’ll delete any data you leave behind when we’re done.) To help you explore the system, here are a few tasks that we think most people would do with a new computer.
Then I’ll give a participant some context of each task:
Task 1: Managing files
You’ve booted a new computer for the first time. Let’s say this USB fob drive has files from your old computer. Please copy the files to the new computer. Put them wherever makes sense to you.
Task 2: Using a browser
After you used your new computer for a while, you want to browse the internet and open some of the sites you visit more frequently. Please open a few websites that you would normally visit, like Google or Facebook.
Task 3: Checking email
After you start up your new computer, you want to check your email. Go ahead and check your email. I’ll delete everything after we’re done today, and you are the only person who will use this account, so please access your email however you normally do it at home.
I have some questions prepared for the wrapping up part of the test to help me get the participants’ general impressions of the experience:
What things were really easy to figure out?
What things were harder to figure out? Why?
Can you summarize your first experience today in a single word, like an adjective? What one word describes the test today?
Jim suggested an interesting method — asking testers to summarize different parts of their experience using an emoji. It will help testers express their own feelings, but because we’ll provide a set of emojis (testers won’t make their own) it will be easier to collate results.
Think back to the start of the test today. If you had to pick one emoticon (from this list) to describe the first time you used GNOME today, what emoticon would that be? What emotion does that represent to you?
After you got settled into GNOME, and had played around with it for a while, what emoticon would you use to describe that part of the experience? What emotion does that represent to you?
I’m excited to perform this UX test very soon, and I’m definitely open to your thoughts and feedback on any part of this test preparation!