Workshop on Usability and the Web
November 4-5, 2002
By Sharon Laskowski
Goal of the workshop:
To develop activities and plans for a proposed W3C Usability Interest Group
Monday, November 4, 2002
First we listened to our invited speakers. See the workshop Web site at http://zing.ncsl.nist.gov/uig_w3c/ for slides and position papers. Then we brainstormed about issues and topics for further discussion on Tuesday.
We need to decide if we want a working group or an interest group and we need to update the charter.
Some issues that should be considered are:
A discussion about the scope of the group:
- Usability of sites
- Usability of products
- Possible audiences
- Essential: underlying technology, the specifications need to get review because if they cause bad usability, they can't be fixed
- Make sure the technology doesn't preclude usability
- E.g., the WAI: different interest areas; Web site review, technology review, usability of product review
- Testing the usability of a specification: it can't be tested
- Work with the technology people to get usable implementations
- User scenarios are a way to get at the usability of implementations
- Usability professionals have expertise in getting people to talk about their intentions of how the technology would be used, but this sounds "soft"
- Should have 2 example implementations of a specification: but even then, something like html is a blank slate so it got confused by people that didn't read the specs
- What is the # of professional developers who go to the site?
- How do we promote this?
- Who has time?
How to fit into the W3C structure:
- What kinds of objects would we be reviewing: W3C site itself, critique of proposed technology, and documentation of the technology, Web practice itself (guidelines, tools)?
- Establishing an authority? Organizations such as UPA, ACM SIGCHI? But, these are not destination sites, they can only provide a thin layer of collaboration
- Relationship to the TAG (Technical Architecture Group)? Have references to usability, but not focused on it
- Documentation for the general public, supporting material
- Web practice itself-too large
- Specifications and recommendations; shouldn't this be woven into QA?
- Getting people better tools for making better Web sites
Subgroups for Tuesday, areas to focus on:
- How can the group play a role for process/organization change? Might usability be introduced into existing processes? Can a usability group affect this? But, this is hard to do. What techniques we could employ to effect organizational change?
- Usability is not core to W3C, no one in the TAG is a usability person
- Role for outreach to business, other organizations to educate them and create more alignment? The hope is that communication is two-way, getting the information out and also getting folks to look for the information. Source of funding?
- Issues for redesign: W3C says no broken urls, site is not in a content management system
- Are Web statistics available? Could be available; maybe too large for analysis tools
- Guidelines for whom: developers?
- Navigational structure - consistency, how?
- Focus on user and be custodian of main user activities
- Critique of proposed technology and design (#1)
- Specifications, documentation of the technology (#2)
- W3C site itself, (#3)
- Web practice itself (guidelines, tools) (#4)
- Scope: determine users
- Can we be successful? Techie organization: but WAI, and internationalization look at users; how do we integrate or do we ask the questions in the 4 areas above
- Look for low hanging fruits in each of the areas
Tuesday, November 4, 2002
Before we began our discussion we asked about time commitments for an Interest Group (IG). For IG's most meetings are by telephone and perhaps an annual face-to-face meeting could be arranged around the time of the CHI conferences perhaps. To get more participation we can send out notices to various listservs.
The group decided that they preferred not to break into subgroups for discussion. Everyone wanted to hear all opinions and it was felt that the group was small enough to be manageable given our experience on Monday.
The GOAL TODAY is to create a charter describing work we are willing and able to do.
Based on yesterday's discussions, the areas we want to focus on can be collapsed into 2 initiatives:
Audience: internal, developers
Benefits: fewer technical mistakes, better understanding of usability issues
Web site practice
Benefits: Web usability never has had an institutional backing. This will be good for the profession. It benefits the W3C in terms of encouraging funding/resources. That is, by working on public side, e.g., making documents more easily digestible, it would encourage more company membership.
What kind of expertise and level of expertise is needed?
How to objectively evaluate: What is a usability review of a technology? Is RDF usable? What is the impact on usability, e.g. frame example? Easier to evaluate from an authoring standpoint, because you can implement to test. Had there been a frame spec, would we have found the problems?
What if we have a Web site and try the new technology? Or, look at a need as a starting point, i.e., we need to do these things. What do we need to build usable, accessible interfaces? Then specifications get produced and then we see if needs are met. Start with requirements, so we are proactive.
Consider frame problems: can we look at previous efforts to show pitfalls; go back and identify issues; maybe we could engage the community. To show our value, we need to pick a specific task, fairly easy, and show what we can do. An example is XML, e.g., do a small task well. We need to look at an in-progress specification then look at a task, e.g., how to do navigation. Then we could provide guidance on how to implement using the specification.
Work is underway to redo frames: some features are good, easy to implement for authors; a classic tradeoff. Is there anything in the current technology that would preclude a usable product?
Not convinced that we are all talking about the same thing and that we have enough expertise in the room. There are different types of usability coming together here. Frames are a good example: many uses of frames were bad, introducing bookmarking problems and accessibility problems. There is the technology and the USE of the technology. It will be difficult to have the expertise, but we can work with people who do. We do have expertise to say "for a good use of frames" here are the requirements. There are people available with the mindset and technical knowledge as long as we set up the mechanisms. We could ask for specifications AND implementations. Then we can look at what issues exist, how does it satisfy usability issues?
Note about W3C: during the CR (candidate requirement) time is when implementations are built. At last call there is not an implementation. User scenarios are mandated, e.g., Xframes. Introduction should talk about usability, a specific section of a document. List the negatives, but not the requirements the document should meet. Internationalization and accessibility have to be reviewed but not always in a dedicated section.
Focusing on saying the first thing the IG wants to do is change your specifications is probably not a good idea. We need to be helpful and prove ourselves, by helping to make a specification better by explicitly taking into consideration usability.
We need to identify one low hanging fruit that would have some value.
We should look at stuff that is aimed at the user experience. Documentation, format, CSS, forms, SMIL, etc are places to start, (as opposed to SOAP and machine to machine communication.)
Anyone can start making comments on the public specifications, e.g. QA specification guidelines, (Lynne Rosenthal from QA, in attendance, says we are more than welcome to comment.) We need a group of us to make comments and submit them as a group after discussion.
We would like to get in as early as possible, when they are building the technology. The charter should call this out. Should be in on the requirements doc. But we should also pay some attention to current products.
What do other groups do? The Internationalization Activity Operates in "police" mode, but starting guidelines to get closer to the action, to the beginning, to build it into the product. Issues you struggle with as you comment should be noted and that starts the guidelines.
What are the procedures that all reviewers should follow?
Summary of technology discussion:
Resources to write better specs at the W3C:
- Be involved early with requirements documents and technology design
- Review last call documents for the technology for the usability of the specification
- Review at CR (candidate requirement)
- Feedback experience into second editions
- Document experience, techniques into a default
- Need to be part of the working groups
- Need Tools for different set of users
- As a group we add value to the end product
- Is spec clear, is technology usable, and is the end product usable...?
- First stage: requirements review. Document best practice and new process
- Value: is end product usable? This is where we should focus: better specs, more usable technology, more usable end product
- Identify things that would PRECLUDE a usable product.
QA framework specification guidelines
W3C Publication rules
Manual of Style
Best practice, guidelines discussion
What are we going to put into the charter? We should have a real deliverable. We could pick 1 or 2 products to show value, that is, here are the usability issues, and here are our suggestions. We can even start before the charter.
Counterpoint: There is a concern: guidelines are a huge area. What we can/should do is find all the guidelines and pull them out. There are several types of guidelines:
We could do a series of brief guidelines: for online documents, for W3C, and a larger audience.
- NCI-style, very broad, very deep
- WAI-style, content guidelines at the W3C, how to implement so as not to get into the way of accessibility, e.g., how to better implement CSS to be better with respect to usability
There is a line between design and development; we are more on development side here. On the development side, there are some things that can happen that affect usability, but this is not a direct relationship. The next level is if you use this technology, this is what will happen. The next level is Web site development.
Also, how could we make the guidelines more usable, how to test them for usability?
With the WAI guidelines, there are some differences between what they are trying to accomplish and us; some are being created to satisfy government requirements. For usability maybe a different approach, perhaps more geared towards scenarios would be feasible. There is a concern. Usability professionals find it difficult to endorse guidelines, because "it depends". On the other hand, there is a huge hunger for this info.
Who are we writing the guidelines for? Who are the target users? Specification writers? The world at large? Who is our user base? Will designers accept guidelines? With bookmarks can almost make it a standard, but some guidelines are hard to agree on. Guidelines are an evaluation method (for technology and Web sites). You could imagine guidelines for all the web pages in the world. You could imagine a guideline on how to assess usability. There probably is a small core set of guidelines for web pages. For areas where there is dispute, we could at least call out areas rather than specify a guideline.
Types of guidelines:
A collection of lessons learned could raise the visibility of usability to institutionalize it. "Lessons learned" are more compelling than guidelines.
- General for people developing specifications,
- Very specific to a technology (html, Xframes),
- Back button problem: Xframes not the place to start, but could produce a short usability guide for back buttons
- Document to accompany Xframes document,
- General area of usability: for content authors
Guidelines are more successful, if there are tools to support it.
Providing of templates can also help: WAI tries to do it, but someone always says, "That's a usability issue."
Important to have a real example of problems/benefits, then contribute to lessons learned,
Especially given limited resources. This could serve as a set of guidelines.
Who are the guidelines for? If they are part of the public face of W3C, this is more palatable to people from industry.
We will need a decision-making process for the group, to avoid commercial groups using this as a lever. We shouldn't let our fear of doing a wrong thing prevent us from doing right thing.
We could promulgate existing guidelines and leave it open-ended that there might be value in developing a W3C set of guidelines (and maybe this is a WG). The WAI started with a stake in the ground and a resource portal. Start with a central repository. This will establish legitimacy. Monitor public and W3C reaction. This sends a message that this is a real entity.
We could look at the usability of the W3C Web site: could derive lessons learned.
We could put together a repository of tools to help people improve usability.
From a commercial viewpoint a W3C document on this would go a long way in selling usability to a company.
Summary of guidelines discussion
Public guidelines are important: how do we put it in the charter?
An IG starts a presence and a repository. We could generate a top ten list by going over existing documents and pulling this out. BUT, need to figure out a way to ADD value. See STC Usability SIG site for an example: http://www.stcsig.org/usability/index.html
STC adds value through the Usability Resources pages on its site.
We could have a users voting approach and go through W3C review/endorsement.
For the charter, we think that we want to produce guidelines for the public, using the WAI approach. IG will build towards achieving by creating a repository of existing knowledge, a top 10 list (maybe) and building toward a WG.
We also want to invite others to participate, not just from the usability community.
Usability of the W3C Web site discussion
It is difficult to manage such a large site. There is no CMS (Content Management System).
Better search functionality would make a monumental difference in improving the site. Looking at the information architecture and web design would go a long way.
Fidelity Labs could do a usability test of the site, with help from this group.
Who owns the site? The Communications team. But there are 70 webmasters, all individuals. There are no rules except valid html, accessibility, and internationalization. It's nice that W3C people all do their own thing. A corporate look would stifle the creativity. Maybe a few "guidelines" would be appropriate.
Caution: this is a very large site. Even a 10% revision is major. A lot of value would come from some summaries. STC volunteers? This is a valuable short-term project.
It would be good to get feedback from users and designers. This is a mind shift from what do we want to do to what do the users need, not easy but we should do it.
Maybe we build our own IG presence that is very usable and start from that base.
Are there any guidelines at the W3C? Template for working groups.
Show by comparison some improvements.
Why would someone volunteer to help?
We need a model for how to allow for growth and still keep it usable.
Think about branding usability: out of the IG. BUT, it is hard to brand because you can't insure it. What are the incentives? Criteria need to have good rationales in order to be branded.
There are 230,000 html web pages on the W3C site. We could get validation for the guidelines by seeing how they work on the site.
What is the process for applying branding, templating, etc?
We could say: here's the top 10. Each owner says whether they conform or not. Then improved sites will pull the others along.
W3C site is much like an intranet, but exposed to the outside world. Might be interesting to look at the evolution intranets go through.
So, what's wrong with the W3C site, what are the specific problems? Give us some tasks.
The Top Ten sites are: homepage, validator, htmlwg, xmgl,css wg, xml, trpage, trs, wai, htmlwgpage
W3C site has good google karma because so many links to hit.
What do we want to say for the charter vis a vis the Web site? IG itself will maintain a Web site that will include activities, and repository. We will act to advise on request and participate in any redesign activities and further evolution of the W3C Web site. Goal: if someone does a google search on web usability, the page is at the top.
We would like to get feedback from users. Structured usability questionnaire?
We should note what we did, the process, to make it a usable site, including user testing.
This could lead to branding. Branding could go to the report on usability testing for the site.
Summary of W3C Web site usability discussion
Specifications usability discussion
- Why? Eating our own dog food
- Investigate the possibility of branding
- Low hanging fruit: home page, templates/advice, resources
We want to help W3C with things they want to publish by reviewing specification formats for usability. We would need templates for consistency and a review process. Reminder point: who is the audience?
Is STC involvement possible? This is a technical communication exercise.
A possible start could be the QA specifications for writing specifications. Are they usable and are the specs produced usable? This is low hanging fruit and high impact because everyone had to read the specs. It is difficult to understand the specs and high-level intros needed.
We could expand to include documentation, but define the range of documentation. A primer needs to be usable where as technical spec need not be as usable. But, we don't want voluminous documentation-minimalism is important. We can borrow from some other models of success for documentation. At least make authors think about usability by requiring a usability section.
W3C doesn't have technical editors (there are 2 that do at best, a review). In QA they have a volunteer that helps them. A technical editor can help with terminology and duplicate terms. W3C does have a glossary: the manual of style has a glossary. QA is working on a glossary: to have consistency among TRs and for translations. Susan Lesh reads all specs from a technical writer's point of view. The glossary is at http://www.w3.org/QA/2002/01/Glossary-req
For the Charter: Pinpoint it as low hanging fruit. Or do we partner with QA? Note that
Web site usability recommendations are quite different from recommendations for technical documents. We should stay away from documents written by and for highly technical people, whereas WAI-type guidelines need to be written for a very broad audience. QA framework is a really good start. Let's review this. If we can help at all, this will help to lead to some really good specs. But, user scenarios should be priority one.
There are clear dependencies: QA and accessibility. But don't want to get the scopes confused. Groups can handoff usability stuff or we need coordination groups
Summary of specifications usability discussion
Editing the Charter
- Low hanging fruit
- High impact
- There may be some additional documentation that is needed
- Identification of missing support material
Name is Usability Interest Group.
We will form the IG now, but we are building towards creating a WG.
We should add a background section on why we did it this way.
We want to move away from opinions, to best practice, "research based", and community consensus.
We need to comply with the QA operational guidelines; put this under dependencies.
Process: the Advisory Committee size is 500 people. They get the charter and the briefing package saying why usability is important and why this work is important.
The mission of the Usability Interest Group (UIG) is to work within the context of the W3C to improve the Web through better usability.
The UIG will support the Web community in making the Web more usable. It will:
Background: define "usability"
- Identify and encourage adoption of usability best practices
- Provide a resource for usability information
- Support the W3C in integrating usability into its processes and products
- Be an advocate for users of the Web (end users, developers, designers)
- Take a leadership role in recommending improvements to Web technology to enhance usability
- First version of the resource
- Review of applicable specifications (technology)
- Issues: see current version
- Analysis of selected W3C products (usability of documentation, Web site)
- Usability WG
- Duration 24 months
Long term -internationalization if we collaborate on guidelines, documents
Implicitly coordination with other groups in the W3C
Contact with External groups ACM SIGCHI, UPA, STC, HFES
Criteria for success:
- Exchange of information
- Founding of a WG
- Improved usability of W3C products
- Within the group: monthly telecon, maillist, homepage, IRC channel, Face to face: WWW or CHI conference once per year
- With W3C : as in draft
- Public: homepage, and
- UIG may also participate in external mailing lists, conferences and other activities to advocate for web usability.
How to handle reviews of documents that are not public; an IG can only discuss public documents?
Voting: draft not completely true.
Issues will normally be decided by consensus. When consensus cannot be reached, the UIG chair will make the decision.
One change: UIG members are strongly encouraged to take advantage of frequent opportunities to participate in the ongoing activities of the Interest Group.
Scrap mention of "invited experts"
Under IPR, scrap "invited experts" and add "organizations and people with IPR."
ACTION ITEMS and FUTURE WORK
- Create a mailing list for creating the charter, 1 week
Steven will email the instructions to Sharon
Sharon will email contacts from workshop
- Create the charter, end of Dec., QA will review the charter
- Push the charter through the W3C process, end of March
- Consider Chi meetings: Florida in April 2003, Vienna in April 2004; UPA
meetings: Arizona in June 2003, Minnesota in June 2004.
Workshop Home Page
Last updated: 12/06/2002, W3C Communications Team <email@example.com>, and Paul Hsiao, NIST.