Looking at testing
the design and operational characteristics of
products without
formal standards or standardized criteria
Hewlett-Packard Accessible Solutions (hpas)
Abstract:
Standards have not been developed by Industry or the Government to reflect quantitative parameters, conformance testing or certification for specific rules associated with Section 508. This paper address different ways, through functional tests, demonstrations or inspections, to evaluate a product and to provide information on what is and what is not accessible in relationship to a technical standard i.e., 1194.23a and a functional performance standard i.e., 1194.31a associated with Section 508. The findings from these types of tests can be used to incorporate new criteria into future designs and operational parameters of products. Most of the comments are derived from previous testing accomplished on printers and their associated host computer, but can apply, in most cases, to any type of product. As the reader progresses through the concepts of testing for accessibility it becomes apparent that individuals who have disabilities should accomplish the testing or at least be very involved in the development and execution of the tests and assistive technology devices be integrated into the equipment configurations.
Background:
Moving a mouse to select different printing options associated with a printer driver is considered as a normal, routine operation for a person without a disability; however, those selections can become very time consuming and major tasks when the end user has a mobility or dexterity disability that does not allow for the use of a mouse or other pointing devices. In some situations people without disabilities, especially users of laptops, complain that the movement of the mouse is difficult and “power user” have a great dislike for mice.
Looking for information streamed from a printer to a host Personal Computer (PC), is a routine function for an end user with no vision disabilities, but can become very confusing and intimidating to a user who is blind or who has limited vision.
Knowing what state the printer is in, e.g., power on, paper status, etc. all constitute basic information that must be provided to the end users, regardless of their capabilities, to ensure products can be produced and distributed.
Many users who have no disabilities struggle with some of the information flow provided by products needed to maintain a normal operating environment, so it is imperative to also provide all of the information to the user with a disability. Don’t provide just a few of the items a designer may feel are necessary.
This paper is an attempt to provide an insight into how products can be tested for accessibility, when formal industry standards and criteria are not available, which in turn provides a more usable product for the whole community. Sometimes, due to the lack of a standard the results become subjective, but the results do provide an insight as to what is accessible and what is not.
The following are situations to consider when making decisions about the testing of certain products.
Objective:
Determine the product’s accessible characteristics through rigorous testing and evaluations. Provide changes and solutions as to what can be done to make it more accessible. Evaluate its compatibility with assistive technology devices. This should not be limited to one type of assistive device, but should provide information on several types of devices for vision, hearing and mobility disabilities. If possible tests should also include devices cognitive, learning, speech impairments and seizures (this is especially important for screen flickers). There are several avenues that must be explored and developed to ensure the objective is met. They are:
5. Share all results with individuals responsible for making design decisions. Inform them of any changes, large or small, that could benefit the product from an accessibility and usability standpoint. What may be a minor problem to them may be an enormous hurdle for an individual with a special disability.
Approach:
These tests were conducted in 2000 while performing accessibility and usability tests sequences for studies associated with the Customer Solutions Test Group. The main emphasis of the test was to determine the compatibility of the PC with the printer using only the PC keyboard as the input interface to the printer and a screen reader as the assistive device. All printer drivers and printer functions were controlled by use of the mouse during the first phase. All PC operating systems and applications were controlled using a mouse. As the tests progressed the configuration changed by deleting the mouse and adding assistive devices to the PC.
Test 1. Expected Results:
All functions of the test would perform normally since navigation tests had been performed previously on a like product.
The person conducting the test had no disabilities. There were only 5 discrepancies noted during the test. Three of the discrepancies were compatibility issues with a newer version of a printer driver, and the other two were associated with selections required during the installation of a driver. The newness of the test and the unfamiliarity of the product may have contributed to several of the discrepancies. Movement of the mouse to those areas requiring action by the technician was not encumbered in any way and all functions could be selected. The test time for this particular portion took approximately 4 hours to complete, which included re-running several of the steps to gain the desired results.
Test 2: Expected Results:
All functions of the test would perform the same as the first test resulting in the same amount of defects. This thought process was predicated upon the fact that the only difference between the two configurations was that the mouse was disconnected and would not be a factor for input to the printer.
The test was conducted by an individual with an upper body mobility disability that did not allow for use of the mouse or other pointing devices. Surprisingly the test was completed in about 3.5 hours because the technician was a “Power User” and very familiar with the necessary keystrokes required for movement to the desired locations on the screen. The same 5 minor defects noted during the first test were still present, but there were also 7 more defects noted. Five of the new defects were associated with keyboard strokes by not allowing the user to access the correct portion of the driver interface and the two others were associated with the movement in the “Start” menu of the operating system. These discrepancies were not noted during the first test and indicated a flaw in some of the design criteria and in some of the software logic used with a specific driver. A minor modification was made to the printer’s driver software and the applicable portion of the test was re-accomplished. The keystrokes discrepancies associated with the first run of this test were corrected; however, the two associated with the ‘Start’ menu resides within the operating system were still present. Documentation was provided to the manufacture of the operating system and they have been corrected. The ability of the tester to use the keystrokes instead of the mouse aided greatly in discovering the discrepancies resulting in changes that will enhance the accessibility of the product.
Test 3: Expected Results:
Since all of the discrepancies had been noted or fixed all functions of the test would perform the same as the second minus those fixed defects. This particular product had never been evaluated using a screen reader so the outcome was not know, even though the software had been tested against tools on the WEB to determine its compatibility.
A person who has a minor mobility disability, but prefers to use keystrokes instead of a mouse, and is also blind conducted the third test. Screen reader software, familiar to the tester, was loaded onto the host PC and the mouse was left disconnected. The two defects noted during the second test with the ‘Start” menu were still present as well as 6 new defects were noted with the use of the screen reader. Four of these new defects were associated with a software product residing within the printer that provides printer status back to the host PC and the remaining two were associated with the capability of the driver providing focus for the screen reader during a driver installation. This test was also conducted in about 3.5 hours.
The test was conducted again, but this time the configuration was with screen reader software unfamiliar to the tester. The same defects from the other tests were still present, but the 6 defects associated with the screen reader had increased to 9. It was determined later that the three extra defects were the result of using unfamiliar software. The time for this test was about 4 hours.
Discussion:
1. Mobility/Dexterity:
Using accessibility attributes of operating systems can provide individuals with the capability to operate products in a normal day-to-day operation, but designers need to re-think their position as to relying on the operating system. Software systems developed for printers and like products should keep accessibility in the forefront of their development efforts. Results during the first test revealed no discrepancies for conducting day-to-day operations or for several unique requirements when using a mouse as the input interface to the printer software. As discovered during the second test, moving around in the PC/printer environment was successful for the day-to-day operations, but lacked the necessary interfaces and compatibility for those items required on a non-routine basis.
2. Vision:
With the third test another layer of interaction/compatibility was introduced with the incorporation of the screen reader. Once again the user was able to navigate through the operating system and the applications used for the test, but only on a normal day-to-day basis. This test scenario was unique because it encompassed testing for two disabilities that may be uncommon in most situations.
3. No tests were conducted for hearing disabilities or other disabilities.
Conclusion:
1. Developers of software and applications need to be aware that the disabled user is not just interested in doing the normal day-to-day routine tasks associated with their equipment. They want to be self-sufficient and produce their required output without the aid of others.
2. Individuals building specific tests for accessibility need to realize that testing products without using the benefit of a person with a disability will cause the product to fail in many areas once the product enters the market. Technicians/testers with disabilities should be used whenever possible. Testing with a non-disabled person gives a perspective or benchmark of the product but testing with the correct community provides the necessary information ensuring good design practices for future products.
3. Computers, the Internet and business Intranets have increased an individual’s capability to communicate throughout the world. They have improved upon the quality of life and aided in providing a more informed world community. Testers need to be aware of all the interfaces and capabilities available to persons with disabilities. Merely designing a product to produce or print documents using a word processing program is not designing for accessibility. They need to expand their scope and include accessibility for all aspects and functions of any given product.
4. Information
Technology Companies, Designers, developers and testers need to form alliances
with the assistive technology industry.
Without the use of an assistive devices in the tests many discrepancies would have gone unnoticed
until a user in the field discovered them.
These devices make it possible for individuals with disabilities to
function in all aspects of our society and become more productive citizens. If manufactures of Information Technology products
don’t become familiar with assistive devices and educate their work force on
how to use these products and how to test with them they will lose to those
companies that have planned ahead.
Appendix 1
Sample Test Cases
Installing a driver:
Pre-conditions:
1. Clean the system to ensure no print
drivers installed
2.
PC Configured per Tier One items 12,19,37,38,39,50, and Tier Two item 10
Applications (Tiers located in Customer Solutions Test, Test Description
Specification Appendix "A")
3. Install CD
4. Disconnect the mouse
5 Ensure Printer connected to applicable
PC
Steps:
1. From the START menu select setting
2. Select printer and add printers
3. From the add printer wizard menu select
my computer, next, LPT1, and next
4. From the add printer wizard menu select
from the manufactures list and select "have disk"
5. Install from the CD-ROM (All drivers,
i.e., PS, PCL)
6. Type in the name of the driver in the
menu window
7. Select printer as the default
8. Select not shared
9. Print a test
page and verify print
Pre--conditions:
1.
Ensure printer driver installed
2.
Disconnect the Mouse
3.Ensure
PC Configured per Tier One items 12,19,37,38,39,50, and Tier Two item 10
4.
Ensure no network configurations are in place
5.
Ensure Printer connected to applicable PC
6.
Delete all previously selected selections for Accessibility Options
Steps:
1.
Select Start, Settings, Control Panels, Accessibility Options
2.
Select Keyboard Tab; under sticky keys options check "Press modifier key
twice to lock"
3. Under Notification select 'Make Sound When
Modifier Key is Pressed, then select Okay
4. Select Sound Tab, under SoundSentry select
"use SoundSentry and under setting pull down "Flash Active Window,
then select Okay
5. Under ShowSounds select "Use
ShowSounds", Select Okay
6. Select Mouse Tab, Select 'Use Mouse Keys;
Under Settings select a comfortable level, Select "Use, MouseKeys when NumLock is "ON",
then
select Okay.,
7. Select Display Tab: Select Use short Cut,
select use high contrast, select white on black.
8. Select General Tab: Under automatic reset
select "turn off accessibility feature after idle for 30 minutes
9.
Under notification select "make sound when turning a feature on or off,
select apply and Okay.
10. Repeat TC DA # 07 and #029. (These are
pre-canned test developed for certain applications)
11. Verify all controls were accessible by the
tab key and shift tab, that the tab order was down and right, all hot keys
worked correctly, All visual and audio functions for the "Accessibility
Options" functioned correctly.
12. Re-configure the PC to ensure no Accessibility Functions are still selected