Testing: Test & Summary

100 points

Overview

The purpose of this assignment is to give you experience in recruiting the participants, conducting the test, summarizing the results.

The most effective way of understanding what works and what doesn’t in an interface is to watch people use it and is the essence of usability testing. When the right participants attempt realistic activities, you gain qualitative insights into what is causing users to have trouble. These insights help you determine how to improve the design. Also, you can measure the percentage of tasks that users complete correctly as a way to communicate overall usability.

You will conduct a usability observation using one of the inquiry methods we have read about and discussed in class. The materials to conduct the test will be provided for you.

At this point, we aren’t trying to replicate a specific user populations or personas. You should select least three participants who can be family members, friends, neighbors, or online acquaintances to participate in this inquiry.

This is a simple test with the NCSU internship website. The purpose here is not to arrive at some startling new discovery about usability, but rather for you to learn the basics:

  • recruiting
  • administering
  • reporting

A list of questions or methods you will use prior to the test will be provided to you. The goals and the questions/methods should be included in your final report. Prepare the report following Barnum (pp. 277 ­- 320) and our discussions in class. Your audience for the report is the director of internships as well as chair of the English program.

Getting Started

First, familiarize yourself with screen casting technologies. As a university student you have access to Zoom, which is a great way to record usability sessions.

Second, familiarize yourself with the project and the project materials you will be using during this test.

Third, think about recruiting folks as participants.

What Users Need To Be Able To Do

In order to observe participants you need to give them something to do. These assignments are frequently referred to as tasks. (During testing call them “activities” to avoid making the participants feel like they’re being tested). Rather than simply ordering participants to “do X” with no explanation, it’s better to situate the request within a short scenario that sets the stage for the action and provides a bit of explanation and context for why the user is “doing X.” Before you can write the task scenarios used in testing, you have to come up with a list of general user goals that your site, application, or document may have. Ask yourself: What are the most important things that every user must be able to accomplish or figure out with this product or content? For websites, these tasks are frequently things like on boarding or purchasing for products like coffee makers, it can be tasks like cleaning or making a single cup of coffee.

Making the Task Realistic

  • User goal: Browse product offerings and purchase an item.
  • Poor task: Purchase a pair of orange Nike running shoes. (Too specific, doesn’t let user think)
  • Better task: Buy a pair of shoes for less than $40.

Making the Task Actionable

  • User goal: Find movie and show times.
  • Poor task: You want to see a movie Sunday afternoon. Go to www.fandango.com and tell me where you’d click next.
  • Better task: Use www.fandago.com to find a movie you’d be interested in seeing on Sunday afternoon.

Avoid Giving Clues and Describing the Steps (i.e. do not give procedural instructions)

  • User goal: Look up grades.
  • Poor task: You want to see the results of your midterm exams. Go to the website, sign in, and tell me where you would click to get your transcript.
  • Better task: Look up the results of your midterm exams.

Methodology

Using the Usability Test Plan, recruit and execute a usability test for two to three participants. Invite four. Someone always can’t make it at the last minute.

Write up your in a report should consist of the following

  • Time on task: How long did the participant take to accomplish their goal
  • Qualitative evaluations
    • Description of completion conditions
    • Descriptions of non-critical errors
    • Description of critical errors

Deliverables

  • A report summarizing testing process and results

Length

  • 7 – 10 pages not including appendices, please note you should integrate materials from the Usability Test Plan into your report with appropriate attribution.

Submission

  • all drafts and work should be submitted as shared G Suite links through through Moodle
  • Peer Feedback Evaluation Form

Rubric

  • Rubric (Prepare the report following Barnum (pp. 277 ­- 320) and our discussions in class.)