User Testing

Failing to plan is planning to fail. There's a common misconception that running into the closest coffee shop will count as user testing and validate your idea.

Sure, any time spent with users can prove invaluable, and an average person may be able to raise simple usability issues, but will they understand the terminology?

Has a fashion designer on her coffee break ever drawn down her pension on a tablet before? More than likely no.

A User testing plan is invaluable and probably the only way you will get budget approval for long-term testing and platform optimisation.

The most common user testing plan contains what you are going to do, how you are going to conduct the test, what metrics you are going to capture, number of participants you are going to test, and what scenarios you will use.

Once everyone has commented and the final plan is agreed upon, you can get going and create a report to reflect the final outcomes of each test.

Elements of a Test Plan

• Scope: Indicate what you are testing e.g the navigation; a particular flow, or a section of the site to assess conversion

• Purpose: Identify the concerns, questions, and goals for this test. e.g Can users navigate to product pages from the check-out basket?"

• Schedule and Location: Indicate when and where you will do the test. Before each sprint? Remote testing or Lab testing.

• Session description: Will they be assisted? Will the session be timed? If so how long? Equipment: Indicate the type of equipment you will be using in the test; desktop, laptop, mobile/Smartphone, monitor size and resolution, operating system, browser etc. Indicate any accessibility tools required also such as screen readers and audio descriptors.

• Participants: Indicate the number and demographic of participants to be tested.

• Scenarios: Indicate the number and types of tasks included in testing.

• Metrics:

  • Subjective metrics: Include the questions you are going to ask the participants prior to the sessions

  • Quantitative metrics: Indicate the quantitative data you will be measuring in your test (e.g., successful completion rates, error rates, time on task).

• Roles: Include a list of the staff who will participate in the usability testing and what role each will play.