Glossary
Automatic data collection equipment
hardware and/or software for automatic collection of usability test data. Examples of such data collection include screen capture and video, audio, and keystroke recording, often with time stamping.
Configuration
the specific collection of hardware and software, including model and version numbers, that constitutes the computing environment on which the product was tested. The configuration includes the platform.
the organization considering the purchase of the product.
Debriefing
a posttest interview with a participant.
Experimental design
a plan for assigning experimental conditions to participants and the statistical analysis associated with the plan. Typically, this includes a specification of the independent variables, dependent variables, number of participants and sampling strategy, procedure for assigning participants to experimental conditions, and order in which test tasks are given.
Face validity
the apparent validity of an instrument, as determined by an observation-based review.
a type of usability test whose goal is learning about the design to improve its next iteration. (Contrast with summative test)
Independent variable
a variable manipulated by the tester to set up the test conditions. (Contrast with the dependent variable, which is a variable that is being measured by the study.)
Likert scale
a rating scale designed to measure user attitudes or reactions by quantifying subjective information. It typically contains a set of favorable and unfavorable statements about the product, to which a respondent indicates his or her level of agreement or disagreement. A Likert questionnaire may have an odd or even number of points, is often numerically numbered, and typically has at least two anchors describing the extremes of the scale dimension. Example:
This product was easy to use.
|
Strongly |
Somewhat |
Slightly |
Neutral |
Slightly |
Somewhat |
Strongly |
I felt frustrated while using this product..
|
Strongly |
Somewhat |
Slightly |
Neutral |
Slightly |
Somewhat |
Strongly |
the arithmetic ‘average’ of a set of quantities. Specifically, the sum of the quantities divided by the number of quantities in the set.
Nondisclosure agreement
an agreement between the supplier and the consumer, in which they recognize that their cooperation requires them to divulge some of their proprietary information to each other, and in which each agrees not to disclose any of the other’s proprietary information to anyone else.
One-way mirror
a piece of glass that appears as a mirror from one side and a window from the other, depending on the lighting conditions. This device allows testers to observe a usability test without themselves being observed by the participants.
Participant
a person who participates in a usability test as a representative of the product s target user population. A participant performs the test tasks and follows other instructions given by the tester.
Pilot study
the initial test study period, lasting approximately 2.5 years and run by the NIST Usability Workshop. The Pilot Study consists of pairs of consumer and supplier companies who will use the report templates to test a previously agreed-on product. The test will operate under the guidelines described in the NIST Usability Workshop s ‘White Paper’.
the kind(s) of computer (operating system and hardware) used for the testing. Examples of common platforms include Windows on COMPAQ, MacOS on Power Macintosh, UNIX on Sparc or HP/UX, Linux on Dell, and OS/2 on IBM.
Procurement decisionmaker
a person responsible for making decisions about which products to acquire for the consumer organization.
Replication
the collection of two observations under identical test conditions. In this document, it is used to mean that the consumer organization should be able to repeat the test described in the report with its own sample of participants.
Semantic differential
a rating scale designed to measure user attitudes or reactions by quantifying subjective information. It typically contains a set of bipolar adjective pairs (e.g., good/bad, easy/hard) that describe some characteristic of the product. A semantic differential questionnaire typically has a seven- to ten-point scale. Example:
This product was
|
easy to use |
1 |
2 |
3 |
4 |
5 |
6 |
7 |
hard to use |
|
frustrating |
1 |
2 |
3 |
4 |
5 |
6 |
7 |
satisfying |
Special needs
user needs for special hardware, software, or design features to accommodate disabilities in vision, hearing, speech, or mobility.
Standard deviation
the square root of the variance. Provides a measure of the variability of the distribution of measurements.
Standard error of the mean
square root of (variance divided by sample size). Estimates the likely closeness of the sample mean to the population mean.
Statistically significant
unlikely to have occurred by chance (i.e., likely to reflect real differences).
a usability test of a completed product to determine how well it meets its usability objectives. (Contrast with formative test)
the producer of a product being considered for purchase.
the presentation and description of a task as part of a realistic, very short story.
Taxonomy
the systematic classification of the objects in a field of study.
Test protocol
a list of the steps to be followed in the test.
a person who designs and/or conducts a usability test. (The tester is not the person who interacts with the product to exercise task scenarios; that person is a test participant.)
Validation
a determination of the validity of a measure such as a questionnaire or a usability test.
the sum of squared differences of a set of measurements from their mean, divided by the sample size.
Usability Professional/Specialist/Engineer
person who engages in the practice of usability engineering/testing. Such practitioners know how to design and execute summative performance tests with human subjects and how to analyze the resulting data. Their training is usually an advanced degree in human factors, behavioral science, industrial engineering, human-computer interaction, industrial design, computer science, or a related field.