Usability Engineering Basics - Part 3 (First)

In this time, I thought writing some words about usability engineering as I promise you some months back. If we need to highlight real usability issues which customers have, we need to conduct a practical usability test with appropriate infrastructure and tools. Then we can collect data to be analyzed and get the correct actions upon solving any usability issues. So we called it as Moderated Usability Test and let’s discuss how we can plan and do it.

1. Planning a Moderated Usability Test

Planning a test should be the first part and it is very vital when we conducting this kind of complex test. And the testing scope will get differ upon the usability requirements.

What you should test?

First, as usability engineer we need to decide what the functionalities that we are going to take are. In this case, we can collect valuable information from BA, design and development teams and select features that are new, often used, error prone and specially critical. After that we should prioritize them and write scenarios which represents typical user work flow. Scenarios should be:

• Small. Time is costly during usability testing.
• To the point. The meaning of the scenario should have a specific DOD.
• Realistic. The scenario should be a normal activity that the average user do.
• Scenario should be on user’s context. Selected scenarios should be related to the user’s context and we need to understand the participant’s connection with the system.

Following is an example for a scenario:

You need to add a contact with first name, last name and contact image in your company’s CRM web application. Add those information and click on “Save” button. Make sure to let me know when you are done.

Who is going to evaluate the Application?

Who you choose to evaluate the application will have a massive effect on the outcome of the test. Imagine that you’re creating an application that reconcile accounts. Your customers are people who deal with account reconciliation. That is a huge group of people. Narrow your focus to a particular user profile.

While you’re creating the user profiles, you may realize that you have two or more equally important subgroups like people who control the system (Business Admin) and people who use the system (Normail User).

Test with proper participants – Rule of thumb is testing no more than 5 users and running small tests as you can afford.

According to the industry standards, the common curve for usability test is as follows:

If we try to read the curve, as soon as you collect data from a first 3 users, you have already captured almost 50% of the usability issues. After you have usability insights from 5^th user almost all of the usability issues were captured and no need to spend more time and money. In other words, as you add more and more users, you learn less and less because you will keep seeing the same things again and again.

Who is going to Observe and how?

When talk about the observation, we have 2 main observation types. First one is ‘Obtrusive observation’ method and the other one is ‘Unobtrusive observation’ method which concentrate on observing what the test user does and refrain as much as possible from influencing her/him by explaining the design or asking questions. In ‘Unobtrusive observation’ method, we can have following combination as observers:

•         At least 2 developers.
•         At least 1 tester
•        And a BA

Where Are You Going to Test?

Now we know, what we are planning to test and who is going to evaluate the system. Now we have another question about the place which we going to conduct the test. The location of the test can be as simple as a meeting room or as complex as a purpose-built facility. If we need to conduct an advanced usability test we need video or audio recording equipment for analysis. Conduct formal tests in an environment that simulates normal use as much as possible. Usability tool like Morae is a good example for a usability tool which we can use to do such tests with advanced data analysis and presentation of results.

In coming articles, I’ll continue this and hope to address topics such as “How we going to create a script”, “Run the tests” and “How we gonna analyze results and prepare the report”.