|
|
|
Mary Theofanos |
|
Sharon Laskowski |
|
|
|
|
|
|
What is Usability? |
|
What is usability testing? |
|
Why does usability matter? |
|
What is the CIF? |
|
How do we incorporate usability and the CIF in
the procurement process? |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Effectiveness -- a measure of user productivity,
how well a user can perform his job accurately and completely. |
|
Efficiency -- a measure of how quickly a user
can perform work, the resources expended to accomplish the task. |
|
Satisfaction--The degree to which users like the
product – a subjective response in terms of ease of use, frustration, and
usefulness. |
|
|
|
|
|
|
|
|
|
Development costs can be reduced |
|
User-centered design reduces re-work because
interaction design is done prior to coding, focusing functionality &
reducing “churning” |
|
Deployment costs can be reduced |
|
Less training and support required |
|
Vendors products are evaluated prior to
selection based on user goals and tasks |
|
Cost of doing business can be reduced |
|
Improved users’ performance across the following
dimensions: |
|
Ease of learning |
|
Efficiency |
|
Memorability |
|
Accuracy |
|
Satisfaction |
|
|
|
|
|
|
|
|
|
Example of an Operational Definition for
Usability for a |
|
travel system: |
|
|
|
On their first try, within 12 minutes, 75% of
Government travelers shall be able to correctly- |
|
Create a travel request form |
|
Select one departure flight and one return
flight |
|
Designate one hotel |
|
Reserve one rental car |
|
Forward the travel request form for approval . . |
|
|
|
By their second try, within 20 minutes, 90%
shall be able to complete the tasks correctly |
|
|
|
|
|
|
|
|
|
|
|
|
Objective measures of effectiveness and
efficiency, and recommends including subjective satisfaction data |
|
Measures for effectiveness may include: |
|
Completion Rate |
|
Number of Errors |
|
Number of
Assists |
|
Measures for efficiency may include: |
|
Task time |
|
Completion rate efficiency |
|
Number of references to the manual |
|
Enough description of the test to allow
replication |
|
Portion of the product tested |
|
Test participants sampled |
|
Design of the test |
|
Procedures, materials, instructions, equipment,
etc. |
|
Data scoring procedures |
|
Statistical analysis |
|
|
|
|
|
|
|
Two target audiences: |
|
usability professionals |
|
stakeholders in an organization |
|
Usability professionals develop a CIF |
|
Stakeholders use a CIF to: |
|
Make informed decisions concerning the release
of software products |
|
Compare usability test results among products
for the procurement of products. |
|
Many organizations have incorporated the CIF
into their standard Computing Architecture Standard and have created
company-wide procedures and processes |
|
|
|
|
|
|
|
|
|
|
|
|
Federal agencies have virtually no visibility of
software product usability before we make procurement decisions |
|
We do not know how to compare products or to
plan for or measure usability costs |
|
As dependence on COTS and the web increases, so
will the impact of usability problems and costs |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
For Purchased Software: |
|
Require that suppliers provide usability test
reports in CIF format |
|
Analyze for reliability & applicability |
|
Replicate within agency if required |
|
Use data to select products |
|
For Developed Software: (In-house or
subcontract) |
|
Define measurable usability goals |
|
Conduct formative usability testing as part of
user interface design activities |
|
Conduct summative usability test using CIF to
ensure goals have been met |
|
|
|
|
|
|
|
|
Usability testing for the CIF found: |
|
|
|
Usability bug = time saving of 2 hrs, 12 minutes
per custom upgrade |
|
|
|
Bug is now fixed in all subsequent versions |
|
|
|
|
|
|
|
|
|
|
|
|
Join IUSR at: |
|
http://www.nist.gov/isur |
|
|
|
http://www.techstreet.com/cgi-bin/detail?product_id=918375 |
|
|
|
|
Mary Theofanos |
|
|
|
Sharon Laskowski |
|
|
|
Email: iusr@nist.gov |
|
|
|
|
|
|
|
|
|
Notes