|
The type of evaluation activity used depends on the stage of the design and/or development of the actual system.
Evaluations of:
- a design concept via a paper or electronic prototype are used to refine the design
- the completed and or deployed system can benchmark its performance and inform redesign.
Earlier stage evaluations are low cost, relatively quick and the results feed directly back into the next design iteration. The costs of later stage evaluations depend on the scope of the testing activity and number of participants.
Early stage evaluation techniques are:
Later stage evaluations are:
Evaluations after system deployment:
»Top of page
Usability walkthrough
A Usability Walkthrough involves going step by step through a system design and getting reactions from relevant staff - typically target users, business sponsors and technical representatives.
More detailed and structured walkthroughs, often involving a specified stakeholder group (eg architects, developers or users) can also be done and involves stepping through a data populated task scenario, discussing and evaluating each element of the interaction.
»Top of page
Expert usability review
An expert usability review is done by a usability consultant putting themselves in the role of a typical user. Usability issues are identified based on usability design principles, relevant standards and the reviewer’s experience. It is fast, low cost technique. It is often done in preparation for usability testing with end users, and sometimes as a standalone activity if there are time and budget constraints.
The usability issues uncovered are given a severity rating to enable developers and project team members to prioritise which issues are to be fixed first.
»Top of page
Standards compliance check
Standards compliance checks ensure compliance with industry and/or relevant corporate 'lool and feel' standards. They are generally done as part of an expert usability review using a standards compliance checklist.
Ensuring that controls such as buttons and drop down lists behave consistently and in line with user expectations minimises learning time and cognitive load.
»Top of page
Accessibility Review
As part of a usability review or standards compliance check, an interface is often judged for it’s ability to support users with sight, hearing, physical or cognitive disabilities, and to prevent occupation health and safety (OH&S) issues such as repetitive strain injury.
There are a number of automated tools for the web that evaluate a sites accessibility and provide compliance reports.
The World Wide Web consortium (W3C) provides guidelines on how to support people who need to use assistive technologies
The degree of compliance targeted depends on the user audience. As a general rule, applications that are:
- used by the public need to pay closer attention to disability compliance
- designed for internal corporate users need to pay more attention to OH&S issues.
»Top of page
Usability testing
Usability testing is a technique for ensuring that the intended users of a system can carry out the intended tasks efficiently, effectively and satisfactorily. It involves directly or remotely observing users performing typical tasks.
For a graphical user interface (GUI) or web site, users are often encouraged to explain their decisions by thinking out loud.
Usability testing can be conducted on:
- paper prototypes, which can be hand drawn or mocked up in PowerPoint to look more realistic,
- electronic prototypes, provided by the development team
- the actual system.
GUI testing can be done:
- in any suitable room with a minimum of equipment
- using PC based usability screen capture software and small videocams
- in a specialist usability laboratory with one way mirrors, numerous video cameras and optionally eye tracking software can be used.
For a speech system or voice user interface (VUI), the user can be tested in a lab and observations of the interaction between the user and the speech system can be recorded. Alternatively users can be tested remotely over the telephone
VUI testing can be done:
- as a Wizard of Oz test where a person or wizard simulates the speech recognition system
- on a prototype of the developed speech system.
The number of test participants can vary depending on budget and test objectives. Any testing is better than none with 4 to 8 participants being common for GUIs.
Usability Review Post Deployment
In a review of a deployed system, users are observed completing normal daily tasks. They are typically asked to complete questionnaires to gauge perceptions of the application’s performance and suitability.
If the review is done in the first few weeks of deployment, information on learnability and areas that confuse users can be gathered. If done several months after launch, the review can identify areas that cause users on going concern, or perhaps missing functionality.
Gathering metrics on the times for users to complete tasks can be used for benchmarking against other systems.
»Top of page
> |