In this time, I thought writing some words about
usability engineering as I promise you some months back. If we need to
highlight real usability issues which customers have, we need to conduct a
practical usability test with appropriate infrastructure and tools. Then we can
collect data to be analyzed and get the correct actions upon solving any usability
issues. So we called it as Moderated Usability Test and let’s discuss how we
can plan and do it.
1. Planning a Moderated Usability Test
Planning a test should be the first part and it is very
vital when we conducting this kind of complex test. And the testing scope will
get differ upon the usability requirements.
What you should test?
First, as usability engineer we need to decide what the
functionalities that we are going to take are. In this case, we can collect
valuable information from BA, design and development teams and select features
that are new, often used, error prone and specially critical. After that we
should prioritize them and write scenarios which represents typical user work
flow. Scenarios should be:
- Small.
Time is costly during usability testing.
- To
the point. The meaning of the scenario should have
a specific DOD.
- Realistic.
The scenario should be a normal activity that the average user do.
- Scenario should be on user’s context. Selected scenarios should be related to the user’s context and we need to understand the participant’s connection with the system.
Following is an example for a scenario:
You need to add a contact with first name, last name and
contact image in your company’s CRM web application. Add those information and
click on “Save” button. Make sure to let me know when you are done.
Who is going to evaluate the Application?
Who you choose to evaluate the application will have a
massive effect on the outcome of the test. Imagine that you’re creating an application
that reconcile accounts. Your customers are people who deal with account
reconciliation. That is a huge group of people. Narrow your focus to a particular
user profile.
While you’re creating the user profiles, you may realize
that you have two or more equally important subgroups like people who control
the system (Business Admin) and people who use the system (Normail User).
Test with proper participants
– Rule of thumb is testing no more than 5 users and running small tests as
you can afford.
According to the industry standards, the
common curve for usability test is as follows:
If we try to read the curve, as soon as you collect data from a first 3 users, you have already captured almost 50% of the usability issues. After you have usability insights from 5th user almost all of the usability issues were captured and no need to spend more time and money. In other words, as you add more and more users, you learn less and less because you will keep seeing the same things again and again.
Who is going to Observe and how?
When talk about the observation, we have 2
main observation types. First one is ‘Obtrusive observation’ method and the
other one is ‘Unobtrusive observation’ method which concentrate on
observing what the test user does and refrain as much as possible from
influencing her/him by explaining the design or asking questions. In ‘Unobtrusive
observation’ method, we can have following combination as observers:
- At least 2 developers.
- At least 1 tester
- And a BA
Where Are You Going to Test?
Now we know, what we
are planning to test and who is going to evaluate the system. Now we have
another question about the place which we going to conduct the test. The
location of the test can be as simple as a meeting room or as complex as a
purpose-built facility. If we need to conduct an advanced usability test we
need video or audio recording equipment for analysis. Conduct formal tests in
an environment that simulates normal use as much as possible. Usability tool
like Morae is a good example for
a usability tool which we can use to do such tests with advanced data analysis and
presentation of results.
In coming articles, I’ll
continue this and hope to address topics such as “How we going to create a
script”, “Run the tests” and “How we gonna analyze results and prepare the
report”.
