Abowd, Coutaz, and Nigay "Structuring
the space of interactive system properties." In Engineering for
Human-Computer Interaction, edited by Larson and Unger,
pp. 113-129, 1992. Andriessen, J. H. E. The Why, How and What to
Evaluate of Interaction Technology: A Review and Proposed
Integration." In CSCW Requirements and Evaluation,
P. Thomas editor, chapter 8, pp. 107 - 124, 1996. Bass, L. "Mini-workshop: scenarios for CSCW
systems." In Engineering for Human-Computer Interaction: proceedings
of the IFIP TC2/WG2.7 working conference on engineering for human-computer
interaction, Yellowstone Park, USA, August 1995, editors Bass,
L. J. and Unger, C. Champman & Hall, 1996, pp. 333-338.
Key Features
EWG/ICV Home
Teleconferencing, groupware, electronic meeting systems - these
terms indicate technologies that claim to be of great importance
for the support of information exchange and cooperation in
organizations. Particularly, the development of high-capacity
infrastructures, often subsumed under the buzzword "electronic
highway", is hailed as the major driving force for the evolution
of these systems. However, their design and evaluation is often
technology-driven. The development and introduction of
multimedia groupware would progress more adequately if they
would be evaluated systematically in the light of real tasks and
contexts of cooperating people. Roughly speaking, one encounters
four contexts of social interaction and three types of
interaction-supporting technology.
Campbell, R. L. "Will the real scenario
please stand up?" in SIGCHI Bulletin, vol. 24, no. 2,
pp. 6-8, April, 1992.
Carroll, J. M. (ed.) Scenario-Based
Design: Envisioning Work and Technology in System
Development. John Wiley and Sons, 1995.
Carroll, J. M. Book preview, "The scenario
perspective on system development," in interactions, vol. 2,
no. 2, pp. 79-83, 1995.
Carroll, J. M. and Rosson, M. B. "Getting
around the task-artifact cycle: how to make claims and design by
scenario," in ACM Transactions on Information Systems,
vol. 10, no. 2, April 1992, pp. 181-212.
We are developing an "action science" approach to human-computer
interaction (HCI), seeking to better integrate activities directed at
understanding with those directed at design. The approach leverages
development practices of current HCI with methods and concepts to
support a shift toward using broad and explicit design rationale to
reify where we are in a design process, why we are there, and to guide
reasoning about where we might go from there. We represent a designed
artifact as the set of user scenarios supported by that artifact and
more finely by causal schemas detailing the underlying psychological
rationale. These schemas, called claims, unpack wherefores and whys of
the scenarios. In this paper, we stand back from several empirical
projects to clarify our commitments and practices.
Clarke, L. "The use of scenarios by user interface
designers," in Proceedings of the HCI'91 Conference on People and
Computers VI, Scenarios and Rationales in Design, pp. 103-115,
1991.
There are many factors which drive software design. This paper
focuses on the use of scenarios by designers and examines the effect
this has on the design process. Scenarios are examples of user
interactions with systems which are used to structure and communicate
information about how a design might be used in the real world. The
use of scenarios was examined in a study of designers in a commercial
setting engaged in the design of an air traffic control system. The
findings are used to make recommendations about how design should be
done and also about the tools required to support design.
Clark, H. H. and Brennan S. E. "Grounding in
Communication." Perspectives on Socially Shared Cognition.",
(pp. 127-149), 1991.
Constantine, L. L. "Essential modeling:
use cases for user interfaces." interactions, vol. 2, no. 2,
pp. 34-46, 1995.
Essential modeling is a systematic process for designing user
interfaces that more simply and fully support what users are trying to
accomplish. Essential use cases are abstract, generalized scenarios
representing the essential patterns of interaction between users and a
system. Use context models are abstract designs representing
collections of capabilities and resources that a user interface must
present to users in support of particular use cases. By keeping the
focus on intent and purpose, essential models lead to user interfaces
with improved usability built in from the start, thereby reducing the
demands on usability testing, inspections, and iterative refinement.
Dewan, P. "Multiuser architectures." In
Proc. of IFIP WG2.7 Working Conference on Engineering for
Human-Computer Communication, Chapman & Hall, London, pages
247-270, 1996.
We have developed a framework for characterizing the design space of
architectures of multiuser applications. It includes a generic
multiuser architecture that captures properties common to the points
in this design space, a set of issues that arise in the design of an
instance of this architecture, a discussion of alternate approaches to
resolving these issues, and several criteria for comparing the
approaches. The generic architecture divides the (programmer-defined
and system-provided) modules of an application into multiple layers,
some of which interact with a single user while others are shared
among all users. The design issues include the nature of these layers,
replication, concurrency, and distribution of the layers, the nature
of collaboration events exchanged among the layers, and how
collaboration modules are added to existing single-user layers. The
approaches make different tradeoffs in providing a balance between
flexibility, automation, code reuse, and performance. In this paper,
we discuss the evaluation criteria, generic architecture, design
issues, approaches, and tradeoffs.
Dewan, P., Choudhary, R., and Shen, H. "An
editing-based characterization of the design space of collaborative
applications." Journal of Organizational Computing, vol. 4,
no. 3, pp. 219-240, 1994.
The design space of collaborative applications is characterized using
the notion of generalized multiuser editing. Generalized multiuser
editing allows users to view interactive applications as editors of
data structures. The model includes several collaboration functions,
which allow users to collaboratively edit application data
structures. These functions include coupling, concurrency control,
access control, and multiuser undo/redo. Coupling allows the users to
share editing changes, access control and concurrency control prevent
them from making unauthorized and inconsistent changes, respectively,
and multiuser undo/redo allows them to collaboratively undo/redo
changes. These functions must be performed flexibly to accommodate
different applications, users, phases of collaboration, and bandwidths
of the communication links. In this paper, we characterize the design
space of each of these functions, describing some of the principles,
issues, alternate approaches, and tradeoffs related to the design of
these functions.
Dewan, P. "Principles of Designing Multi-User
User Interface Development Environments." In Proceedings of the
IFIP TC2/WG 2.7 Working Conference on Engineering for Human-Computer
Interaction, Ellivuori, Finland, pp. 35-50, 1992.
Domain-independent, high-level, flexible, and efficient user interface
development environments (UIDEs) are required for easing the task of
developing multi-user interfaces. This paper describes several
principles of designing such environments, gives motivation for
supporting them, and identifies existing systems that follow
them. Some of these principles are also well-accepted principles of
designing single-user UIDEs-multi-user interaction provides new
reasons for accepting them. Others are single-user principles that
have been adapted for the multi-user case, while still others are new
principles that have no counterparts in the single-user case. These
principles address the functionality and architecture of a multi-user
UIDE and the programs supported by it.
Dix, A. "Que sera sera - the problem of the
future perfect in open and cooperative systems." In Proceedings of
the HCI'94 Conference on People and Computers IX, edited by
Cockton, Draper, and Weir, 1994, pp. 387-408.
Dix, A. "Beyond the interface." In
Engineering for Human-Computer Interaction, edited by Larson
and Unger, pp. 171-190, 1992.
Finn, K., Sellen, A., and Wilbur, S.
"Video-Mediated Communication: Testing, Evaluation, and Design
Implications." In Proceedings of ACM CSCW'94 Conference on
Computer-Supported Cooperative Work, p. 2, 1994.
Video-mediated communication (VMC) has been touted as an
invaluable tool for such applications as distance learning,
collaboration, and communication. In trying to compare,
evaluate, or improve upon these systems, various studies have
found widely conflicting conclusions, marked by the absence of
a common language or set of metrics. There were two main goals
of this workshop: (1) to resolve discrepancies in research
findings by comparing methods, metrics, and interpretations of
results; and (2) to draft a set of guidelines for designers of
VMC systems based on the results of our analysis of the
research. The main topics addressed were: VMC evaluation
methods, metrics, and terminology; interpretation of research
results and assessment of impact of VMC on conversation and
collaboration; and implications for design.
Grudin, J. "Why CSCW Applications Fail:
Problems in the Design and Evaluation of Organizational Interfaces, in
Proceedings of ACM CSCW'88 Conference on Computer-Supported
Cooperative Work, Perspectives on Evaluation, pp. 85-93, 1988.
Many systems, applications, and features that support cooperative work
share two characteristics: A significant investment has been made in
their development, and their successes have consistently fallen far
short of expectations. Examination of several application areas
reveals a common dynamic: 1) A factor contributing to the
application's failure is the disparity between those who will benefit
from an application and those who must do additional work to support
it. 2) A factor contributing to the decision-making failure that leads
to ill-fated development efforts is the unique lack of management
intuition for CSCW applications. 3) A factor contributing to the
failure to learn from experience is the extreme difficulty of
evaluating these applications. These three problem areas escape
adequate notice due to two natural but ultimately misleading
analogies: the analogy between multi-user application programs and
multi-user computer systems, and the analogy between multi-user
applications and single-user applications. These analogies influence
the way we think about cooperative work applications and designers and
decision-makers fail to recognize their limits. Several CSCW
application areas are examined in some detail.
Houwing, E. M., Wiethoff, M., and Arnold,
A. G. "Usability Evaluation from Users' Point of View: Three
Complementary Measures." In Proceedings of the Fifth International
Conference on Human-Computer Interaction, IV. User Issues,
vol. 1, pp. 475-480, 1993.
In the European ESPRIT project 'MUSiC', metrics, methods and standards
are developed for industrial use. In this paper a validation study is
reported in which metrics of cognitive workload, performance and
subjective usability are tested. Subjects were studied working with a
menu oriented and with a graphical object-oriented wordprocessor. The
main hypothesis of the study is that the packages induce a different
level of user efficiency. All measures added unique information to the
usability evaluation, and the combination of the domains of
measurement is most informative. Contrary to the expectations, it
could not be concluded unambiguously, that user efficiency is better
when individuals use a package with a graphical object-oriented
interface.
Hutchins, E. Cognition in the wild. MIT
Press, Cambridge, Mass, 1995.
SUBJECT :
Cognition -- Social aspects -- Case studies
Cognition and culture -- Case studies
Navigation -- Psychological aspects
Karat, C., Campbell, R., and Fiegel, T.
"Comparison of empirical testing and walkthrough methods in user
interface evaluation," in Proceedings of ACM CHI'92 Conference on
Human Factors in Computing Systems, Usability Walkthroughs,
pp. 397-404, 1992.
We investigated the relative effectiveness of empirical usability
testing and individual and team walkthrough methods in identifying
usability problems in two graphical user interface office systems. The
findings were replicated across the two systems and show that the
empirical testing condition identified the largest number of problems,
and identified a significant number of relatively severe problems that
were missed by the walkthrough conditions. Team walkthroughs achieved
better results than individual walkthroughs in some areas. About a
third of the significant usability problems identified were common
across all methods. Cost-effectiveness data show that empirical
testing required the same or less time to identify each problem when
compared to walkthroughs.
Karat, C. and Karat, J. (editors) "Some dialogue on scenarios,"
in SIGCHI Bulletin, vol. 24, no. 2, pp. 7 - 17,
Oct. 1992. Contributing authors include Kyng, M.; Young, R. M. and
Barnard, P. J.; Wright, P.; Nardi, B. A.; Reisner, D.; and Campbell, R. L.
Kuutti, K. "Activity theory and its applications to
information systems research and development." In Proceedings of
the IFIP TC8/WG 8.2 Working Conference on the Information Systems
Research Arena of the 90's Challenges, Perceptions, and Alternative
Approaches. Copenhagen, Denmark, 1991.
Lewis, R. L. "Psychometric evaluation of an
after-scenario questionnaire for computer usability studies: The
ASQ." SIGCHI Bulletin, vol. 23, no. 1, pp. 78-81, 1991.
A three-item after-scenario questionnaire was used in three related
usability tests in different areas of the United States. The studies
had eight scenarios in common. After participants finished a scenario,
they completed the After-Scenario Questionnaire (the ASQ). A factor
analysis of the responses to the ASQ items revealed that an
eight-factor solution explained 94 percent of the variability of the
24 (eight scenarios by three items per scenario) items. The
varimax-rotated factor pattern showed that these eight were clearly
associated with the eight scenarios. The benefit of this research to
system designers is that this three-item questionnaire has acceptable
psychometric properties of reliability, sensitivity, and concurrent
validity, and may be used with confidence in other, similar usability
studies.
Malone, T. W. "Designing organizational
interfaces." In Proceedings of ACM CHI'85 Conference on Human
Factors in Computing Systems, Panel, pp. 66-71, 1985.
This paper argues that it will become increasingly important to extend
our concept of user interfaces for individual users of computers to
include organizational interfaces for groups of users. A number of
suggestions are given for how to develop a theoretical base for
designing such interfaces. For instance, examples are used to
illustrate how traditional cognitive points of view can be extended to
include information processing by multiple agents in
organizations. Examples of design implications from other perspectives
such as motivational, economic, and political are also included.
Markus, M. L. and Connolly, T. "Why CSCW
applications fail: problems in the adoption of interdependent work
tools." In Proceedings of ACM CSCW'90 Conference on
Computer-Supported Cooperative Work, pp. 371-380, 1990.
McCarthy, J. C., Miles, V. C. and Monk,
A. F. "An experimental study of common ground in text-based
communication." In Proceedings of ACM CHI'91 Conference on Human
Factors in Computing Systems, The Use of Video in Remote Group
Work, pp. 209-215, 1991.
An experiment was performed to examine predictions from
Clark's contribution theory of discourse. Pairs were asked to
use a text-based synchronous messaging system to solve a
problem involving the layout of a bank. Contribution theory
suggests that in such text-only communication common ground
will be difficult to achieve. This was shown to be the case. A
parallel system, where participants could use a common report
space in addition to the messaging space, significantly
reduced these problems. The implications for design are
discussed in terms of providing additional channels for
communicating the results of discussion separate from the
conversation itself.
McCarthy, Miles, Monk, Harrison, Dix and
Wright. "Four generic communication tasks which must be supported
in electronic conferencing." ACM SIGCHI Bulletin, vol. 23,
no. 1, pp. 41-43, 1991.
McCarthy, J. C., Miles, V. C., Monk, A. F.,
Harrison, M. D., Dix, A. J. and Wright, P. C. "Text-based on-line
conferencing: a conceptual and empirical analysis using a minimal
prototype." Human-Computer Interaction, vol. 8, no. 2,
pp. 147-183, 1993.
This article is concerned with an analysis of the requirements
for text-based on-line conferencing. From a system
perspective, text-based on-line conferencing can be viewed as
either message passing or data sharing. These complementary
views give rise to different design dimensions. For example,
the message-passing view is concerned with granularity,
channels, message labels, and so on. The data-sharing view is
concerned with the access different individuals have to the
text: read only, appending, editing, pointing, and so on. A
deliberately sparse prototype was built and placed in this
design space. This minimal prototype has limited functionality
so that the real problems experienced by users can show
through. Relevant literature from disciplines such as social
psychology, conversational analysis, and linguistics is
briefly reviewed in terms of three generic communication
tasks: synchronizing communication, maintaining structural
coherence, and maintaining referents. An empirical analysis of
subjects' use of the sparse prototype was analyzed to
establish the relevance of the generic communication tasks to
text-based on-line conferencing. Possible forms that support
for these tasks might take are discussed.
1. Prototyping with a Minimal System
2. Text-Based Communication
3. The Prototype and Design Space
3.1. The Prototype
3.2. The System as a Message Passer
3.3. The System as a Shared Data Structure
4. Methodology
5. Generic Communication Tasks
5.1. Synchronizing Communication: Knowing What Others Are
Doing
Illustrations From the Data
Design Issues
5.2. Maintaining Structural Coherence: The Relationships
Between Messages
Illustrations From the Data
Design Issues
5.3. Maintaining Referents: Establishing a Shared Focus
Illustrations From the Data
Design Issues
5.4. Summary of the Task Space
6. Conclusions: Design Space Revisited
McCarthy, J. C. and Monk, A. F. "Measuring the
Quality of Computer-Mediated Communication." Behavior and
Information Technology, vol. 13, no. 5, pp. 311-319, 1994.
There is a growing literature of experiments whose purpose is
to compare different configurations for computer-mediated
communication. If the results of these experiments are to be
useful they must: (i) use the right experimental tasks; and
(ii) measure the right dependent variables. This paper is
concerned with the latter problem which is illustrated using
data collected in experimental comparisons of three
configurations of a text-based conferencing system. No
significant differences were found using a measure of task
outcome. This accords with numerous previous
findings. However, a number of process-related dependent
variables were devised that did show significant
effects. These included common ground, as measured by shared
recall, and references to the topic of one message in the next
available turn. Another, the use of first and second person
pronouns in conversation approached significance. Finally, an
approach to the selection of measures for use in studies of
computer-mediated communication is commended.
McGrath, Joseph E. Groups: Interaction
and Performance Pentice-Hall Inc. New Jersey, 1984.
Monk, A. "How and when to collect behavioural
data." Empirical Evaluation of User Interfaces, pp. 138-143, 1984.
Monk, A. "Statistical evaluation of
behavioural data." Empirical Evaluation of User Interfaces,
pp. 143-146, 1984.
Monk, A., McCarthy, J., Watts, L., and Daly-Jones, O.
"Measures of Process." In CSCW Requirements and Evaluation,
chapter 9, 1996.
Monk, A., Nardi, B., Gilbert, N., Mantei, M. and
McCarthy, J. "Mixing oil and water? Ethnography versus experimental
psychology in the study of computer-mediated communication." In
Proceedings of ACM INTERCHI'93 Conference on Human Factors in
Computing Systems, Panel, pp. 3-6, 1993.
Monk, A. Perspectives on HCI. Computers and
People Series, A. Monk and B. Gaines, editors, 1995.
The field of human-computer interaction (HCI) has attracted
researchers with interests as diverse as formal mathematics,
ethnography, cognitive psychology, and conversation analysis. Each
approach has different assumptions and perspectives on the practical
problem of designing more effective systems. Perspectives on HCI:
Diverse Approaches contains 10 tutorial chapters, each written to
illustrate a particular approach to HCI. The authors are all
international authorities in their particular fields. The book will be
of interest to researchers and students in computer science,
management science, psychology, human factors, ergonomics, and
sociology who want to know how other disciplines approach the
practical problem of making computers into more effective tools for
people to use.
Nardi, B. A. "Some reflections on scenarios."
In Scenario Based Design: Envisioning Work
and Technology in System Development, edited by Carroll,
J. M. pp. 397-399, 1995.
Nielsen, J. and Mack, R. L. Usability
Inspection Methods Wiley, New York, 1994.
Nielsen, J. "Usability engineering at a
discount." In Designing and Using Human-computer Interfaces and
Knowledge Based Systems, edited by G. Salvendy and M. J. Smith,
1989, pp. 394-401.
Nielsen, J. Usability
Engineering. Academic Press, Inc. 1993.
Potts, C. "Using schematic scenarios to
understand user needs," in DIS95, Ann Arbor, MI, pp. 247-256,
1995.
Scenarios are narrative descriptions of interactions between users and
proposed systems. The concreteness of scenarios helps users and
designers develop a shared understanding of the proposed system's
functionality; but concreteness leads to a potentially unbounded
number of scenarios for a system. To help designers develop a limited
set of salient scenarios, we propose a schema similar to story
schemata. Like stories, scenarios have protagonists with goals, they
start with background information already in place, and they have a
point that makes them interesting or tests the reader's
understanding. The scenario schema provides a structural framework for
deriving scenarios with slots for such teleological
information. Scenarios are derived from a description of the system's
and the user's goals, and the potential obstacles that block those
goals. In this paper, we describe the scenario schema and a method for
deriving a set of salient scenarios. We illustrate how these scenarios
can be used in the analysis of user needs for a multi-user office
application.
Ramage, M. "CSCW evaluation in five types."
Unpublished, 1996. [HTML]
One of the potentially confusing aspects of evaluation within
computer-supported cooperative work (CSCW) is that there are many
activities one might wish to carry out at different times that
evaluate socio-technical systems. I identify five ideal types: the
effects of a new computer system in an organization; the formative
development of a piece of software; the evaluation of conceptual
developments; the evaluation of a cooperative system where factors
other than the computers are more interesting; and the determination
of which piece of software to buy.
Rauterberg, M. "AMME: An `Automatic Mental
Model Evaluator' to measure complexity of user behaviour recorded on
logfiles." in Proceedings of the Fifth International Conference on
Human-Computer Interaction, Poster Sessions: Abridged
Proceedings, Cognitive Aspects of HCI, vol. 3, p. 191, 1993.
Rauterberg, M. "Quantitative Measures for
Evaluating Human-Computer Interfaces," in Proceedings of the Fifth
International Conference on Human-Computer Interaction,
V. Methodologies, vol. 1, pp. 612-617, 1993.
Rauterberg, M. "Four different measures
to quantify three usability attributes: 'feedback', 'interactive
directness' and 'flexibility'." In P. Palanque & R. Bastide (Eds.), Design
Specification and Verification of Interactive Systems (DSV-IS'95),
pp. 209-223. Wien New York: Springer, 1995.
There currently are 4 different views on human computer interaction in
measuring interactive qualities: (1) the interaction-oriented view,
(2) the user-oriented view, (3) the product-oriented view and (4) the
formal view. Two different possibilities of measurement within the
product-oriented view are introduced in this paper. Different types of
user interfaces can be described and differentiated by the concept of
"interaction points". Regarding to the interactive semantic of
"functional interaction points" (FIPs), 4 different types of FIPs must
be discriminated. Based on the concept of FIPs, the dimensions
"[visual] feedback" and "interactive directness" can be quantified.
Rauterberg, M. "Usability evaluation:
an empirical validation of different measures to quantify interface
attributes." In T. Sheridan (Ed.), Analysis, Design and Evaluation of
Man-Machine Systems, Vol. 2, pp. 467-472. Oxford: Pergamon, 1995.
Rengger, R. "Indicators of Usability Based
on Performance," in Proceedings of the Fourth International
Conference on Human-Computer Interaction, Congress II: Design and
Implementation of Interactive Systems: USABILITY EVALUATION;
Development of Usability Metrics, vol. 1, pp. 656-660, 1991.
As part of the ESPRIT MUSiC Project, literature recently published on
measuring the performance of Human Computer Interaction (HCI) was I
surveyed. 'Indicators of usability based on performance' were
identified and assigned to generic groups. Using the experience of NPL
and the HUSAT Research Institute in Loughborough, coupled with
evidence from the literature survey, a set of indicators of usability
based on performance were proposed for validation. The NPL work was
jointly supported by the CEC and by the UK Department of Trade and
Industry.
Ross, S., Ramage, M. and Rogers, Y. "PETRA:
participatory evaluation through redesign and analysis."
Interacting With Computers, vol. 7, no. 4, pp. 335-360,
1995.
Compared with single user-computer interactions, evaluating CSCW is
difficult. We argue for multiplicity - of theory, method and
perspective - in CSCW evaluation. This allows us to address both
theoretical concerns and practical design issues, and to incorporate
the expertise and experiences of both evaluators and participants. We
propose the PETRA framework, incorporating a theoretically-driven
"evaluators' perspective" (ETA) to investigate the collaborative
activity, and a practical, user-focused "participants' perspective"
(ETR) to evaluate the interface of the supporting tool. Our particular
instantiation of PETRA focused on collaborative writing, both in a
face-to-face context, and supported by a computer-based group editor,
ShrEdit. We investigated the development of shared understanding in
the two different mediated settings; and used a PD-inspired rapid
prototyping session to elicit participant reactions to and redesigns
of the tool interface. Our ETA findings show that computer-supported
shared understanding develops technologically, using social
coordination as a repair mechanism; the ETR findings show that the
collaborative tool must be particularly sensitive to issues of
awareness, communication, focus and ownership.
Salvador, T., Scholtz, J. and Larson, J.
"The Denver model for groupware design." SIGCHI Bulletin,
vol. 28, no. 1, pp. 52-58, January, 1996.
Sellen, A. J. "Speech patterns in
video-mediated conversations." In Proceedings of ACM CHI'92
Conference on Human Factors in Computing, Studies of Media
Supported Collaboration, pp. 49-59. Monterey, CA: ACM Press, 1992.
This paper reports on the first of a series of analyses aimed at
comparing same room and video-mediated conversations for
multiparty meetings. This study compared patterns of spontaneous
speech for same room versus two video-mediated
conversations. One video system used a single camera, monitor
and speaker, and a picture-in-a-picture device to display
multiple people on one screen. The other system used multiple
cameras, monitors, and speakers in order to support directional
gaze cues and selective listening. Differences were found
between same room and video-mediated conversations in terms of
floor control and amount of simultaneous speech. While no
differences were found between the video systems in terms of
objective speech measures, other important differences are
suggested and discussed.
Sellen, A. J. "Remote conversations: the
effects of mediating talk with technology." Human-Computer
Interaction, vol. 10, no. 4, pp. 401-444, 1994.
Three different videoconferencing systems for supporting
multiparty, remote conversations are described and evaluated
experimentally. The three systems differed by how many
participants were visible at once, their spatial arrangement, and
control over who was seen. Conversations using these systems were
compared to same-room (Experiment 1) and audio-only (Experiment 2)
conversations. Specialized speech-tracking equipment recorded the
on-off patterns of speech that allowed objective measurement of
structural aspects of the conversations, such as turn length,
pauses, and interruptions. Questionnaires and interviews also
documented participants' opinions and perceptions in the various
settings. Contrary to expectation, systems in which visual cues
such as selective gaze were absent produced no differences in
turn-taking or in any other aspect of the structure of
conversation. In fact, turn-taking was unaffected even when visual
information was completely absent. Overall, only the same-room
condition showed any significant differences from any other
condition; people in the same room produced more interruptions and
fewer formal handovers of the floor than in any of the
technology-mediated conditions. In this respect, the audio-only
and video systems examined in these studies were
equivalent. However, analyses of participants' perceptions showed
that participants felt that visual access in mediated
conversations was both important and beneficial in
conversation. Further, there were indications that the particular
design of the different video systems did affect some aspects of
conversational behavior, such as the ability to hold side and
parallel conversations.
Sharples, M. "A Study of Breakdowns and
Repairs in a Computer-Mediated Communication System." Interacting
with Computers, vol. 5, no. 1, pp. 61-77, 1993.
There have been few naturalistic studies of synchronous
computer-mediated communication. For two years a telewriting
system was used for tutoring students at the UK Open
University, connecting up to ten study centres and adding
graphic communication to voice-only teleconferencing. This
paper presents the findings of a study of the operation of the
system, concentrating on breakdowns in communication and the
technical and organizational changes needed to overcome
them. The paper concludes with recommendations for the design
of future telewriting systems and guidelines for the
deployment of computer-mediated communication systems as part
of working life.
Smith, M. J. and Salvendy, G. Proceedings of the
Fifth International Conference on Human-Computer Interaction
jointly with the Ninth Symposium on Human Interface (Japan),
pp. 1042+1125 Elsevier, Orlando, Florida, August 8-13, 1993.
Volume 1: Applications and Case Studies
I. Manufacturing
II. Special Applications
III. Case Studies
IV. User Issues
V. Methodologies
VI. Longitudinal Studies
VII. Health Issues
VIII. Psychosocial and Stress Issues
Volume 2: Software and Hardware Interfaces
Plenary Paper
I. Software Knowledge
II. Software Tools
III. Media
IV. Help and Learning
V. Hardware Interfaces
van Harmelen, M. "Exploratory user
interface design using scenarios and prototypes." In Proceedings
of the HCI'89 Conference on People and Computers V. pp. 191-201,
1989.
Exploratory user interface design consists of the experimental
construction of, or depiction of, the use of a user interface to an
interactive system, and the subsequent evaluation and incremental
improvement of that interface in an iterative fashion, in a way that
is similar to the incremental change that occurs during exploratory
programming. Two uses for exploratory user interface design are to
investigate ideas of the scope and function of a system being
designed, and to try out alternative user interface
designs. Exploratory user interface design is best performed using
tools to construct scenarios of interactive system use, or to rapidly
construct prototypes of interactive systems. Observations are made
about this technique and the use of scenarios and prototypes in two
projects; one being the development of a large Integrated Project
Support Environment, and the other being a small editor for a MIDI
sound source. In these projects the construction of scenarios and
prototypes formed a means of exploratory user interface design that,
it is postulated, forms a valuable part of user interface designers'
design techniques. Finally some existing exploratory user interface
design tools are examined in the light of a discussion of desirable
features for these tools.
Watts, L., Monk, A., and Daly-Jones,
O. "Inter-Personal Awareness and Synchronization: Assessing the Value
of Communication Technologies." International Journal of
Human-Computer Studies, vol. 44, no. 6, pp. 849-873, 1996.
How may we discriminate between the multitude of
point-to-point communication facilities currently available?
To take just one aspect of communication, how can we assess
the fluency of coordination that results from using some
communication technology? This paper describes two groups of
measures with this general purpose. The measures described
have been devised to be used in a particular approach to
evaluation for the design of communication systems that
borrows from experimental and ethnographic methods. This
approach is promoted as a practical and rigorous way of
assessing design alternatives. The first group of measures are
subjective ratings that assess someone's awareness of the
attentional status of their conversational partner, such
awareness is necessary for the successful coordination of
conversation. The rating scales are shown to be sensitive in
that they distinguish between video and audio mediated
conversation in a short experiment. The second group are
measures derived from video records of communicative behavior
using "activity set" analysis. This can be used to assess
coordination in communication directly. An activity set is a
mutually exclusive and exhaustive set of behavioral states. A
publicly available tool, Action Recorder, makes it possible to
score the tapes in near real time. "Simple statistics" are
extracted from a single activity set, examples are: the
proportion of time spent looking towards the video monitor and
the average duration of these glances. "Contingent statistics"
are extracted from two or more activity sets, for example, the
proportion of time both members of a pair are looking towards
their video monitors. A way of assessing the synchronization
evident in two people's behavior is presented that makes use
of these contingent statistics. Inter-observer reliabilities
are given for all the measures generated.
Wright, P. and Monk, A. F. "Evaluation for
Design." In Proceedings of the HCI'89 Conference on People and
Computers V, pp. 345-358, 1989.
In an iterative design methodology prototypes or mock-ups are
built and evaluated by having typical users work through
realistic tasks. Data are elicited from users in order to
refine the prototype. This paper is concerned with the types
of data which may be obtained and how they may be used. Two
commonly used forms of data are compared, a behavioral record
of system use, in the form of a time stamped system log, and
verbal protocols. In each case the objective is to illustrate
practical methods of data collection and the kind of usability
problems likely to be revealed. It is argued that an account
of system usage alone provides insufficient data for the
identification of many important usability problems.
Wright, P. C. and Monk, A. F. "A Cost-Effective
Evaluation Method for Use by Designers." International Journal of
Man-Machine Studies, vol. 35, no. 6, pp. 891-912, 1991.
A strong case has been made for iterative design, that is,
progressing through several versions of a user interface
design using feedback from users to improve each
prototype. One obstacle to wider adoption of this approach is
the perceived difficulty of obtaining useful data from
users. This paper argues that quantitative experiment methods
may not be practical at early stages of design, but a
behavioural record used in conjunction with think-aloud
protocols can provide a designer with the information needed
to evaluate an early prototype in a cost-effective
manner. Further, it is proposed that a method for obtaining
this data can be specified which is straightforward enough to
be used by people with little or no training in human
factors. Two studies are reported in which trainee designers
evaluated a user interface by observing a user working through
some set tasks. These users were instructed to think aloud as
they worked in a procedure described as "cooperative
evaluation". The instruction received by the designers took
the form of a brief how-to-do-it manual. Study 1 examines the
effectiveness of the trainee designers as evaluators of an
existing bibliographic database. The problems detected by each
team were compared with the complete set of problems detected
by all the teams and the problems detected by the authors in a
previous and more extensive evaluation. Study 2 examined the
question of whether being the designer of a system makes one
better or worse at evaluating it and whether designers can
predict the problems users will experience in advance of user
testing.
Last modified on Fri May 2 15:02:17 1997 by
Emile Morse