NIST Web Metrics Logo

Technical Overview

NIST Logo

Good usability is critical to the success of a website. Usability evaluation has traditionally been a slow, labor-intensive process which makes it difficult to apply to websites. The dynamic nature of the Web poses problems for usability evaluation. Development times are rapid and changes to websites occur frequently, often without a chance to re-evaluate the entire site. Advances in web-based user interfaces change user expectations. Finally, the potential audience for a website may be geographically dispersed and encompass a wide range of demographic groups.

The challenge then is to determine how best to provide automated support to the usability engineer. Automated techniques cannot entirely supplant manual testing; the intuition of a good usability engineer is still vital. However, automated techniques can enhance traditional approaches and provide additional information to the developer as well.

The objective of the NIST Web Metrics Testbed is to explore the feasibility of a range of tools and techniques that support rapid, remote, and automated testing and evaluation of website usability. The prototypes are used to support the usability engineering research of the Visualization and Usability Group (VUG). As part of the Information Access Division of the Information Technology Laboratory at the National Institute of Standards and Technology, VUG encourages industry to use and/or commercialize its ideas.

The NIST Web Metrics testbed was undertaken in 1997 to explore solutions to the problems described above. We released version 1 (consisting of simpler versions of WebSAT, WebCAT, and WebVIP) in June of 1998. Since that time, we have enhanced the original software and added FLUD and VisVIP.

We are now (October 2002) releasing a new version of Web Metrics with even more improvements. Please take a look at our What's New page for the details.

Prototype Tools from the Web Metrics Testbed-- Download
Static Analyzer Tool The Web Static Analyzer Tool (WebSAT) checks the HTML of web pages against usability guidelines, either its own, or a set of IEEE Std 2001-1999 guidelines. It can check individual pages or an entire website.
Category Analysis Tool
The Web Category Analysis Tool (WebCAT) lets the usability engineer quickly construct and conduct a simple category analysis across the web. It is a variation upon traditional card sorting techniques. The usability engineer establishes a set of categories and a number of items which are to be assigned by test subjects to those categories. The engineer can then compare the actual assignments with the intended usage to make sure that the categories match users' intuitions.
Variable Instrumenter Tool The Web Variable Instrumenter Program (WebVIP) lets the usability engineer rapidly instrument a website so as to capture a log of user behavior on the site. Once the site is instrumented, test subjects are asked to complete tasks typical of those for which the site was designed. Their interactions (such as navigating among pages and manipulating buttons and checkboxes) are captured in a log file for later analysis. Note that even remote users can be tested.
Framework for Logging Usability Data The Framework for Logging Usability Data (FLUD) is a file format and an associated parser for representation of the behavior of website users. The captured log data can be valuable for analyzing and improving the usability of web-based applications. This log data is quite complex and therefore a common file format is needed to allow various software components (such as recorders, parsers, analyzers, and visualizers) to exchange information.
FLUD file Visualizer The FLUDViz tool lets the usability engineer (UE) visualize and analyze a single usability session. The x-axis is used for time, and the y-axis for the types of behavior. The UE can zoom and pan in the time dimension by manipulating a time slider at the bottom of the display. The UE can also filter activity type in the Y dimension through the control menu.
Visual Instrumenter Tool The VisVIP tool lets the usability engineer visualize (in 3D graphics) and analyze the navigational paths of website users as captured in a FLUD file. VisVIP automatically lays out a 2D graph of the website, and then overlays the paths of selected subjects to show which pages were visited. A vertical bar indicates how much time users spent at the various pages.
Tree Decorator The TreeDec tool supports the representation of a website as a single logical tree. It automatically adds links (such as so-called "breadcrumbs") to each page of the website to allow easy navigation to nearby nodes of the tree.

Generic Topics

Resources

Project Team

  • Sharon Laskowski, Project Manager
  • John Cugini
  • Paul Hsiao
  • Joe Konczal
  • Emile Morse
  • Jean Scholtz
  • Charles Sheppard
  • Former team members: Laura Downey, Jason Lunn


Version 3.0
Page last modified: 11 May 2005
Comments and questions to webmetrics@nist.gov