Usability testing is not a science, here how to do it yourself


Benefits of Usability testing:

  • Helps design teams make faster and informed decisions.
  • Gets the right people involved in the design process
  • Makes the design better for the user and for the business goals

Steps to start usability testing

1. Usability test planning

Get together with the people involved in the project and collect the requirements and determine the focus for the project.

Getting everybody involved at the beginning of the process eliminates miss communication issues

a) Defining the requirements (scope):

  • What are you testing?
  • What do you want to learn? (Define the issues if any)
  • What stage are you in the process?
  • Identify user goals and tasks

b) Testing early in the project:

  • Identify users’ goals and tasks
  • Focus on competitors designs
  • Low fidelity mock-ups

c) Testing later in the project:

  • Focus on the details
  • Examine specific design elements
  • High-fidelity prototypes

d) How many users to test?

  • Simple applications: 4 to 5 users to test, small rule: 80% of usability problems can be found with four or five users [Robert Virzi, 1992]
  • Complex applications: The best way is to incorporate the testing in ongoing bases and not in specific stages of the project. Break down each section/functionality and test.

e) User recruitment

  • Identify your users

Who are your primary users?

Even tho we wan to cater to all the users we need to identify our two primary users.

With your team, identify who are your two main targets that you want to satisfy,  figure out what their primary goals are and mission to come to your site, and what you need to provide them to accomplish those goals.

  • Develop a user profile

To develop a user profile we can focus on 3 factors:

  1. Tool experience: familiarity with how to use the website or application. What type of experiences on the web? purchasing? reading blogs?
  2. Domain Experience: We want to bring people that has knowledge about the content provided by the site. We want people that will enjoy/appreciate the content.
  3. Interpersonal skills: We want to work with people who share their thoughts easily in an articulated way. Cooperative and talkative.
  • Find and screen users

Make the process of recruiting an ongoing process, overtime you meet somebody that will match your target, ask them if they could come in for a test.

Look for places online and offline where people who fit your target share information and/or get together.

Research about user recruitment firms if you have problems finding users for the testing.

  • Where to start?

Family, friends, co-workers.

Usability Testing :)

Usability Testing 🙂

2. Task Design

It is important we come out with the right tasks for the test to ensure we get the users doing what they actually come to do on the site.

How do we ensure this when there are  many things to do on the site?

Bringing the right users with the right tasks will provide us great information to improve our site or application.

Good tasks create realistic context to see the problems actual users encounter.

Users show different behaviors depending on the amount of interest on the content, the ones invested in the content:

  • Search more vigorously through out the content
  • They are more likely to read details
  • They are more likely to wait for downloads

There are two types of tasks designs:

Verb-based tasks

Verb-based tasks are created to:

  • Asking users to do something specific
  • Yield information about functionality
  • Traditional method to test software
  • Most applicable to web applications and least applicable to information-rich sites

How to design the verb-based tasks?

  • Make a list of all the things the design allows the user to do.
  • List all the action items that the user can complete in the application
  • Identify the most critical action items
  • Create tasks around them and provide some context to make it easier for the user to realize the tasks

Advantages of doing verb-based tasks:

  • Work very well to test functionalities
  • Ask large numbers of users to do the same tasks and see the differences that come across with different users.

Dis-advantage of doing verb-based tasks:

  • We are asking the user to do tasks to test functionalities but we are not sure if these tasks are something a user would really do.

Scavenger-hunt tasks

These tasks are designed to:

  • Challenge users to find specific information in our site
  • Test users’ understanding of information
  • Suggested to test rich content sites. Not suggested to test software or web apps
  • Scavenger-hunt tasks are not designed to test functionality.

How to design scavenger-hunt tasks:

  • Navigate your website and look for areas that you think your users will be most interested in.
  • Define the content on your site that you want people to find
  • Create a tasks to see if people can find that important info.
  • The tasks usually start with the word: Find, we want the users to find a particular piece of content. Example for a travel site: You and your family are planning a trip to New York,  find the cheapest flight to New York from your city (It is important to create the scenario to provide a reason to the user why to accomplish the tasks)
  • Don’t use ‘Give away’ words when wording the tasks to the user (Specific names of pages, categories or options available)

Advantages of using scavenger-hunt tasks

  • Allows us to see how well the users can find and comprehend the information
  • Allows us to measure how easy or difficult it is for different users to find the same information

Dis-advantages of using scavenger-hunt tasks

  • We are assuming that the users will perform the tasks when coming to the site, same as verb-based tasks

Interview-based tasks

You give the users the opportunity to accomplish tasks that they would like to get done instead of doing tasks designed for us.

How to create the interview-based tasks

  • Interview the users before they come to the test center to ensure that they are interested in the content.
  • During the testing help the user create their own tasks by talking and interviewing them on the fly. (Ask what they would do in the site, what are the areas look interesting for them, what do they want to learn)

Advantages of interview-based tasks

  • They provide a realistic assessment on how the users are accomplishing their own goals.
  • The interview-based tasks provide better results to test the experience provided by our site or application

3. How to conduct the usability test

a) Pre-test briefing with the users and the observers. Here we make sure the users understand we are testing the user interface and not them. We almost make sure our team understand the goals of the tasks on the test.

b) Introduce users and observers, users know they are been watched and it makes it easier for them to meet the people who will be observing.

c) Explain testing protocols to users (Example: time for the test)

d) Conduct the test

e) Mini Q&A session to the user after each task

f) General Q&A session to the user at the end

g) Debrief observers, talk about things learned and how to improve the design.

4. Setting up a lab for usability testing

Usability professionals usually use a one room or two room configuration to perform the tests.

Two room set up:

  • One way mirrors
  • Using inexpensive digital cameras and video monitors to project to observers in a separate room

One room set up: (Suggested set up)

  • In a conference room with design team members present
  • Testing is for observers
  • Dedicated space

What to record during the test?

  • Record all the actions happening on the screen
  • Audio of the users and observers.
  • Showing recording to stakeholders are not as valuable as having the stakeholders at the moment of the test is the best. Encourage watching the tests

Recording tools:

  • TechSmith’s Morae
  • Camtasia (Suggested)

Remote usability testing:

  • Great alternative if you can’t bring in the users for the test
  • Remote testing can be done using online tools like: WebEx, GoToMeeting,  Adobe Connect Meeting
  • Remote usability tools: Ethnio, UserVue
  • There are two types of remote usability tests: Automated or Moderated (suggested)

Advantages of remote usability testing:

  • Easier o reach users in many different geographic locations
  • Easier to test more users
  • Reduce travel costs for development teams
  • More natural environment for users since most of them participate from their own home or office.

Dis-advantages of remote usability testing:

  • The design and development teams are not physically observing the user behavior.
  • Additional costs for online meeting services

5. Facilitating a test

The facilitator needs to take on 3 different roles during the test:

a) Flight attendant role:

Make sure the user feels comfort while taking the test by briefing the user

  • Make sure the user knows we are testing the site and not them.
  • Make sure the user know can stop or take a break at any time
  • It’s fine if they don’t complete a tasks
  • Any problems found are not their fault
  • Make sure the user knows what they are doing is valuable for the team
  • Make sure they know they are allowed to give up on any task requested

b) Sportcaster

This role is to make sure we maximize the information gotten during the test

  • Verbally reinforce the actions taking by the user, observers might not be able to see everything
  • Respond to non-specific vocalizations, look for situations where the user fells stuck and ask what is stopping them.
  • Take advantage of time when the user is waiting for a page load or when they are not doing anything to summarize the user’s recent actions and to ask them what they will do next
  • Suggested questions:

Is that what you expected?

What are you thinking right now?

What are you trying to do?

What does ___ mean to you?

Why are you frowning?

Can you say more about that?

c) Scientist

Analyze and disseminate the results of the test, getting the teams onboard

  • Focus on the users’ actions and not opinions on the design
  • Disseminate the results quickly by:

I. Debrief often

II. Send out interesting findings to development team throughout the testing process, this ensures the team will note any findings.

III. Host an email discussion list for findings

IV. Avoid writing detailed reports (No one read them)

6. Showing the results

  • Avoid writing detailed reports
  • Use presentations to show results and walk through the findings.
  • Workshops where you can relive the tests
  • Newsletters lists

Important tips:

Have different types of tasks for testing your website or application depending on the stages the project is on. (Napkin sketches, wireframes, prototypes, full developments)

Make sure all tasks requested can be accomplished, we don’t want to set the user for failure when doing the test

Rehearse the tasks and rehearse the testing process

Users will always give you their first impressions on the site, but those impressions are usually not the right ones to take in consideration to make modifications

I am originally from Venezuela but I have been living in Austin, Texas for the last 8 years. Web development has been part of my life for more than a decade; It started in the middle of my College years and by the time I got my bachelors degree in Electronics Engineering I was already known for delivering complex projects while delivering a good user experience.

Be first to comment