Matrix for Evaluation of Ubiquitous Computing Workshop

 

 

System Level/ Experimental Component

Workload

System Under Test

Data Collection/ 
Experimental Methods

User

Makela

Dey (Intel)

Ganoe (VATECH)

Sasojevic (HP)

Quintana (UMICH)

Richter (GATECH)

Task

Funk

 

 

 

Service

Basu (BU)

Castro (UCLA)

Burnett (DSTO)

Arnstein (U. Washington)

OS/Network

 

 

 

Castro (UCLA)

Burnett (DSTO)

 

Physical Device

 

 

 

 

Matrix eplanation –

Workload:  for evaluation to occur we need to define a workload (this could be a scneario for the user to carryout or a set of applications that we are going to evaluate the workload from the system perspective)   So in this case, we could consider having different systems and seeing how parameters in the system could be changed to give better performance – while holding the workload constant. 

 

System under test:  This would actually be testing the system – either from the user perspective to determine performance, usefulness, usability.  We might use different workloads to vary this.

 

Data collection/ methodologies – at each level we need to define what we’re going to collect and how we’re going to collect it.   

 

The one thing that isn’t addressed in the matrix is the dimension of time:  what can we do early in the design of the system and how, and what has to be done later and how?