a comparison study: sketch-based interface versus wimp interfaces in three-dimensional modeling...

8/9/2019 A comparison study: sketch-based interface versus wimp interfaces in three-dimensional modeling tasks

1/8


2/8

Science. The improvement of the sketch-based interface is

considered one of the most exciting and challenging

areas.

In the work presented by Seok-Hyung Bae and

colleagues in [25], an interface metaphor of pencil and

paper created for professional designers, called

ILoveSketch. The users can draw curves freely and

directly on the screen, and connect them through camera

rotations. Although the results are 3D models, all objects

are built through the users strokes, with no system

interpretation. The researchers tested the prototype with

the collaboration of a specialist with 12 years experience

in design in the automobile industry and with toys and

movies. The choice was justified by the decision to

project the system for professional users with a high level

of experience. The user carried out an intense evaluation

of the system after a one-hour of training. The main

conclusion was that the user was satisfied with the great

numbers of features of the system.The software Fibermesh [3] is an evolution of

Teddy. It brings even more power to the user in the task

of creation of 3D models. The original curve stroke lies in

the model. It makes possible to manipulate the object in a

practical way with operations directly applied in the curve

by user actions. The researchers presented also a non-

formal evaluation of their system with novice users and

artists. They concluded that Fibermesh is an easy to use

tool, which permits evolution in the creativity tasks

executed by the users.

In the SESAME project [12], James Lin and

colleagues studied ways to provide support to the work ofdesigners in the initial stages of the designing process.

SESAME was created to explore different visions to solve

conceptual design problems in three dimensions. The

work was presented in two phases: the first presented a set

of guidelines to create collaborative systems for

conceptual designs, and the second compared SESAME

against 3DS max. The main goal of the evaluation was to

analyze how designers could make a creative complex

design in the least amount of time, and what sets of

operations they need to execute during the task.

With the GODZILLA [26] system, S. Tano and

colleagues presented experimental systems where theusers can make 2D drawings, which are recognized and

exhibited as 3D sketches in a display (stereo vision TV).

The user can later modify the drawings, as viewed from

many view points (2D or 3D). In order to evaluate the

system, they compared it against pencil and paper, and a

3D CAD (Computer Aided Design) tool. The results

revealed that the ideas in terms of numbers of sketched

are much closer in this system to the numbers of sketched

in traditional pencil and paper combination.

Marcus Wacker and colleagues developed The Virtual

Dressmaker [18], a Virtual Reality application to design

clothes. The system supports advanced interactive

techniques with six degrees of freedom. The researchers

argue that sketch-based techniques are more natural than

the traditional desktop techniques. They started with a

pilot test, where the user needed to position clothes on an

avatar in three different systems: the Virtual Dressmaker,

Maya and the CosmoWorlds. They evaluated variables

such as the time for task completion and precision

achieved by the users. The results showed that the users

performance was better with VirtualDressmaker. They

also presented points to improve the users tasks in future

versions of the software.

In this work [16] Kamran Sedig and colleagues

presents a methodology to evaluate the impact of using

geometry learning software in the learning geometry

process, for children at a basic educational level. The

research compared three versions of the same softwarebuilt to teach geometry transformations. The goal was to

find ways to design effective tools to ease the knowledge-

building process in learning. The research revealed that

the present interface style brings implications in the

education by the way users interact with the tools.

Another conclusion was that the HCI elements could

improve the cognitive capabilities of users who use the

software, although it can also affect the same capabilities.

This work served also as an inspiration to us, since it

advances the idea that sketch-based interfaces have to be

more investigated in order to identify gains and eventual

losses that the technique can offer in different contexts.

Takeo Igarahashi and colleagues introduced the

software Teddy [10], a gesture based system where the

users draw on a white screen with strokes in 2D (input

data), and the result of this interaction is a 3D model

(output data). Basically, all operations are a result of a set

of actions (gestures) like: creation, paint, extrusion, cut,

smooth, bend, etc.

3. Methodology

We used a qualitative and quantitative methodology

[4] to conduct our comparative study, described below.

3.1. Pilot Tests

We executed a great number of pilot tests [18] in order

to decide the tools to be used, to evaluate and improve our

methodology. After the pilot sessions, the collected data

was analyzed and the methodology was adjusted when

necessary. We repeated this procedure until we decided


3/8

that the methodology was ready to be executed in a real

context.

In our pilot tests we had the collaboration of a

computer science student familiar with traditional

software for 3D modeling (Maya in this case). One

identified necessity was the reduction of the experiments

length, to avoid the user becoming tired. Although there is

no limit on the time to execute the tasks, we planned it to

consume a minimum amount of the users time, without

jeopardizing the goals of our study.

An important decision taken during the pilot tests was

the definition of the Teddy system as the selected tool to

evaluate the sketch-based modeling technique. Our other

choice, the software Fibermesh (which has more

interaction possibilities than Teddy), was rejected due to

great instability in the prototype version available. The

details of the methodology are showed below.

3.2. Hypotheses

To evaluate our study, we considered the following

hypothesis:

H1: modeling with the use of sketch (as presented in

Teddy), demands less effort from the user than modeling

with the use of WIMP-like interfaces (as presented in

Maya or 3DS Max).

H2: the sketch-based modeling approach (as presented

in Teddy) reduces the users time to complete tasks.

H3: the sketch-based modeling technique (as presented

in Teddy) produces satisfactory results.

The effort (H1) and the satisfactory users results (H3)

in this study were evaluated through the users answers

collected in a survey related to the two techniquespresented. The time (H2) was verified through the video

register of the users activities.

3.2.1. Dependent and Independent Variables. The

independent variables involved in this study were:

the technique utilized (Sketch or WIMP); the target object to be modeled (a bear); the executed task.

The dependent variables were the following:

users effort; number of tries to realize the task; time to execute the tasks; users satisfaction; Hierarchical tasks models (HTA).

The user effort and satisfaction with the results were

collected through the same survey. All the other variables

have their results computed after the analysis of the users

activities.

3.2.2. Subjects. The profile defined users who are

studying or working in the Design, Art or Technology

fields with experience in Computer Graphics products

such as Maya or 3DSmax. They should also have basic

understanding of the modeling process of these tools. All

the users were recruited as volunteers in academic or

technical schools, or in design, games, and technology

companies.

3.2.3. Collected Data. To collect the data, we defined

25 users. In the study here presented we used only 6 users

to show qualitative data, our quantitative analyze

continues and will be shown in a future work.

3.2.4. Users Mental Model. We used the user speech

to analyze, in an hierarchical form, their activities in

Teddy and Maya or 3DS Max. The main objective was to

build two trees of analyses, one for each approach (sketch

and WIMP). By analyzing each tree, we can extract

details of the users modeling activities, such as usability

gains, needs and requirements for designing a system

based on our findings.

3.2.5. Survey (personal data). A simple survey wasused to collect information about the users, such as

occupation and familiarity with the WIMP tools defined

for the study.

3.2.6. Questionnaire (System Usability Scale). The

use of a questionnaire was necessary to collect, through

the users replies, the measures for the three subjective

hypotheses (H1, H2, H3), which are related to the

easiness of use and the satisfaction with the observed

results. The questionnaire was adapted from the available

model developed by SUS System Usability Scale [5]

and applied to both techniques for comparison effect (the

adaptation is available at

www.cin.ufpe.br/~tlam/sus_adaptation ).3.2.7. Users Comments. Through the users answers

we collected qualitative data related to their opinions

about satisfaction with the created 3D models, the use of

creativity in the tools, and ease of use with the software.

3.3. Procedure

The test sessions were composed of two phases: one

dedicated to introduce Teddy, and another oriented to

tasks execution and answering the questionnaire. In the

first phase, the goal was to make Teddy more familiar to

the users. They filled a simple questionnaire about their

experience with the tools and their occupations.The users had time to test the system functions with a

tutorial help available in: http://wwwui.is.s.utokyo.ac.

jp/~takeo/teddy/teddy/tutorial.html.

In the second phase, the users had to execute three

tasks, one only for modeling and the other two dedicated

to editing the model previously created. Each task was

performed with both Teddy and Maya (or 3DS Max).


4/8

There was not a time limit to conclude the tasks, and

the only rule was that the user needed to execute each task

in each tool. The tasks were as follows:

Creation - The user had to reproduce a teddy bear

(Figure 1) model presented in a reference picture. This

reference was used only to show a direction of how the

users test could start and not to follow the reference

exactly as seen in the picture.

Figure 1. The reference picture of a teddy bear.

Editing - Using the bear model created in the previous

task (did not need to be complete), the user was asked to

make a drawing of a four-point star at any point on the

bears surface. Following, they should erase this star and

draw a five-point star instead. After that, the user needed

to cut one of the bears ears, and create a little cavity in

the bears body.

Pointed ear - The user was asked to deform the bears

ears to make it look like a cats ear. To do this task, the

previously created ears must be used.

After the conclusion of the third task, the users filled in

a questionnaire about the tests and talked about their

experience when performing the tasks.

The tasks were defined in this way to cover a set of

basic operations presented in the 3D modeling systems

evaluated.

4. Results

For our qualitative study, we used the Hierarchical

Task Analyses (HTA) technique. Based on the full video

and audio recorded during the sessions, we developed a

HTA related to each task executed by the users. The

HTAs were generated with the trial version of the

Software Task Architect [27]. We adopted the number of

units generated in the tasks to define the usability gains of

the techniques. We will use the definition NS to denote the

number of operations in a sketch-based interface for

modeling (Teddy in this case) and the definition NW to

denote the number of operations in a traditional desktop

interface system (Maya or 3DSMax).

We collected data from six subjects, all of whom were

volunteers for the study. They were recruited in graduate

courses (Computer Science, Design and Arts) and in

game companies. All of them had little knowledge about

sketch-based systems, but some experience with Maya or

3DSMax in different levels, varying from beginner to

professional, according to their use of these tools in their

leisure time or in their professional lives.

Unfortunatelly we didnt reach a large number of

users (we defined 25) to generate enough data.

We have a consulting with a specialist in statistics and

the recommendation was following this study until we get

the specified amount of users data.

According to the specialist, an analyze with only 6

users is insufficient to give effective results.

By these reasons this paper section analyze our

qualitative data exclusively.

We continue to work with the quantitative part of the

study and soon as possible we will reveal all the achieved

results and hypotheses comments.

4.1 Task results

4.1.1. Creation. NS < NW

In task one - the creation of a 3D bear model - all the

users used less operations in the sketch-based system than

in the wimp based one(Figure 2 and Figure 3). The

creation of the model in Teddy was straightforward. The

users started this task in one of two ways: drawing

directly on the screen with the mouse, or using the

example sphere, which starts the software. All the modelelements such as arms and legs, were generated with the

extrusion set of gestures available in Teddy.

In the WIMP systems, the users used geometric

references to construct the model. The manipulation of

these references, in order to build the model, forces the

users to think in some pre-defined ways. The users have

to adapt their ideas to the object seen currently on the

screen, thus resulting in additional tasks.

Figure 2. A pseudocode example of the creationtask in a sketch interface (Teddy) done by one of our

test users.


5/8

Figure 3. A pseudocode example of the creationtask in a WIMP interface (Maya) done by one of our

users test.

4.1.2. Editing. NS ~= NW

In this task, the number of operations was close in both

approaches. We will present the result of this task

according to their subtasks: cut operation, drawing (and

erase) a star, and creating a cavity.

Editing: cut operation. NS ~= NW

In this subtask, with only a single stroke in Teddy, the

users can cut the ear off the bear. In the WIMP-based

system, two techniques were used: some preferred to use

the delete face function, whereas others preferred the

Boolean operation to cut the ear.

Editing: drawing (and erase) a star. NS~= NW

Most of the results in this subtask showed that the

number of operations is closer in both systems. In fact, all

the users used the same approach in the sketch-basedsystem, i.e., they made strokes directly on the models

surface to create the stars and then scribbled to erase it. In

the WIMP-based systems, the users defined a few points

to connect edges in the surface model. The stars were

built in this way. To erase it, they used the delete

command. Another user preferred to use the function

Staravailable in the system. But this function added more

tasks related to typing the number of vertices to form the

star which increases the effort (NS < NW) in an WIMP

interface.

Editing: creating a cavity. NS ~= NW

To create a cavity in the sketch-based system the users

only needed to apply the extrusion function. In theWIMP-based systems the users executed the task by

selecting some vertices and pushing one to define the

cavity.

4.1.3. Pointed ear. NS > NW

In this task, the number of operations was greater in

the sketch-based system (Figure 4 & and Figure 5). In

the sketch approach almost all users used cut and extrude

operations to give the ears of the bear a pointed view

look. One user tried the bend function, the most common

alternative to this task, but he did not finish his action

because a system fail locked the system. Another usertried the bend function with success.

In the WIMP approach, all users selected a set of

vertices or faces defining the ear, and pulled these to give

the bears ear with a pointed view look.

Figure 4. A example of the first and second try inthe Pointed ear task in a sketch interface (Teddy).

The user used the bend function without success, so

he made the ear pointed with an extrusion function.


6/8

Figure 5. The pointed task in the WIMPapproach.

5. Conclusions

In this section we present our main conclusions,

grouped by our impressions regarding: the reducedinstruction set gestures presented by Teddy in our test

sessions, the creation and the editing phase of the

procedure, and their implications in the user activities.

5.1 Reduced Instruction Set Gestures

The sketch-based system Teddy has a reduced

instruction set of gestures. This characteristic implies that

the user can generate 3D models in a faster and easier

way. They can interact with the computer environment

without loss of time and concentration in long searches

over interface menus.

This reduced set of instructions can be repetitive andtedious in great projects or in the users time task along a

computer, but it could not be observed in the tasks

analyzed.In our sessions, we could observe that the users

created the model basically with a simple extrude

function.

In the case of WIMP-based systems, the use of several

menus and dialog boxes cause some mistakes and

difficulties to get a solution for a specified task. The users

have to acquire a great knowledge of the options

presented in the system interface even to create simple

objects.

.

5.2 Creation

We could observe that the best usability gain that the

sketch-based interface technique brings to the 3D

modeling task, is in the creation phase. On this task, the

number of user operations was consistently less than

when realizing the same tasks in a WIMP-based

interface. Creating a model in this way was very practical

to our users, since the sketch technique is a

straightforward way for users to express their ideas,without having to make technical decisions about the

object to be generated.

In the WIMP-based systems, the users need to plan

ahead, or to have a well-defined idea to minimize theirtime and effort in the selection sets to construct their

models. The technical background required to manipulate

these interfaces is also higher, if the user wants to make

full use or their creativity throughout the project. A high

level experience in the use of these interfaces is needed to

reach the same level of interaction as in the sketch-based

systems.

5.3 Editing

The editing operations revealed more usability gains in

the WIMP-based systems. In task 2 (Editing) of our

experiment, we could observe that the number of

operations was very close, but in the task 3 (Bend) we

observed that number of operations was greater in thesketch-based system. The possibility of direct

manipulation of vertices and faces, and the use of

keyboard commands to copy and paste, showed that these

familiar computational and geometry instructions are

more remarkable than some new gesture-based

instructions.

6. Requirements and Future Work

We are developing a 3D geometric modeling prototype

using sketch-based ideas for the interface, and we plan to

use the gathered results from this study as requirements

for this prototype. The evaluation of the HTAs and the

user comments, showed us that a system combining the

two approaches (sketch and WIMP) could be built with

the following characteristics:

Manipulationofvertices and faces - We observed that

the manipulation of vertices and faces that define the

models, is an operation for editing objects very popular

with by the users. Editing a model at the level of a vertex

or a face was very useful and simple in our tests sessions

in the WIMP approach. In the sketch approach, although

the interaction can be very simple too, the users have to

make some effort to find the right stroke which will

generate the desired modification.

Copy and paste functions - Another functionality which

reduces the users' efforts is the copy and paste functions.

All users use it to avoid modeling similar objects from

scratch several times. In a WIMP context, this use is very

simple with selected options through the graphical

interface or keyboard commands. But in the sketch

software, this function was not available and the users had

to develop similar objects from the scratch every time

they needed to use them.

This study did not yet reach a representative number

of users to run the quantitative analysis as presented in the

Methodology session. We continue to collect data from

new users tests and we hope to show the results in a

future as a natural evolution of this research.

7. Acknowledgements

We would like to thanks CNPQ for finatial support, the

members of our research groups Cincias Cognitivas e

Tecnologias Educacionais and Mdia & Interao,


7/8

Playlore Gameworks for the users, the teacher Richard

Lane and Cultura Inglesa for the technical review of this

paper, the teacher Renata Souza for consulting and all the

users who participated in a volunteer way of this study.

8. References

[1]3DS MAX, 2008. Autodesk,

http://www.autodesk.com/3dsmax.

[2] Alice, Carnegie Melon University, http://alice.org/

[3] Andrew Nealen , Takeo Igarashi , Olga Sorkine , Marc

Alexa, FiberMesh: designing freeform surfaces with 3D

curves, ACM SIGGRAPH 2007 papers, August 05- 09,

2007, San Diego, California

[4] Batista, Makilim Nunes., Corra de Campos, Dinael.

Metodologias de Pesquisa em Cincias: Anlises

Quanitativa e Qualitativa. Rio de Janeiro: LTC, 2007

[5] Brooke, J. (1996) SUS: a "quick and dirty" usability

scale. In P. W. Jordan, B. Thomas, B. A. Weerdmeester &

A. L. McClelland (eds.) Usability Evaluation in Industry.

London: Taylor and Francis

[6] C. Alvarado and R. Davis. Resolving ambiguities to

create a natural computer-based sketching environment.

In Proceedings of the Seventeenth International Joint

Conference on Artificial Intelligence, pages 13651374,

2001.

[7] DIETRICH, Carlos A.; NEDEL, Luciana P.;

COMBA, Joo L.D. A Sketch-based Interface to Real-Time Strategy Games based on a Cellular Automaton.

Games Programming Gems 7. : Charles River Media,

2008, p. 59-67

[8]Dunham G., Forbus K., and Usher J. nuWar: A

Prototype Sketch-based Strategy Game. Northwestern

University, IL. USA

[9]Google SketchUP, http://sketchup.google.com/, 2009

[10]Igarashi, T., Matsuoka, S., and Tanaka, H. 1999.

Teddy: A sketching interface for 3D freeform design. In

ACM SIGGRAPH, 409416

[11]Ivan E. Sutherland, Sketch pad a man-machine

graphical communication system, Technical Report,

University of Cambridge, September 2003

[12]James Lin , Mark W. Newman , Jason I. Hong, James

A. Landay, DENIM: an informal tool for early stage web

site design, CHI '01 extended abstracts on Human factors

in computing systems, March 31-April 05, 2001, Seattle,

Washington

[13]Ji-Young Oh, Wolfgang Stuerzlinger, John Danahy,

SESAME: Towards Better 3D Conceptual Design

Systems, Proceedings of the 6th conference on Designing

Interactive systems, 2006, University Park, PA, USA

[14]Joseph J. LaViola, Jr. , Robert C. Zeleznik,

MathPad2: a system for the creation and exploration of

mathematical sketches, ACM Transactions on Graphics

(TOG), v.23 n.3, August 2004

[15]Joseph Jacob Cherlin , Faramarz Samavati , Mario

Costa Sousa , Joaquim A. Jorge, Sketch-based modeling

with few strokes, Proceedings of the 21st spring

conference on Computer graphics, May 12-14, 2005,

Budmerice, Slovakia

[16]Kamran Sedig , Maria Klawe , Marvin Westrom,

Role of interface manipulation style and scaffolding oncognition and concept learning in learnware, ACM

Transactions on Computer-Human Interaction (TOCHI),

v.8 n.1, p.34-59, March 2001

[17]Luke Olsen and Mrio Costa Sousa and Faramarz

Samavati and Joaquim Armando Pires Jorge, A

Taxonomy of Modeling Techniques using Sketch-based

Interfaces,Eurographics, Apr. 2008 , Eurographics

Association

[18]Markus Wacker , Stanislav L. Stoev , Michael

Keckeisen, Wolfgang Straer, A comparative study on

user performance in the Virtual Dressmaker application,Proceedings of the ACM symposium on Virtual reality

software and technology, October 01-03, 2003, Osaka,

Japan

[19]Maya, 2008. Autodesk,

http://www.autodesk.com/maya

[20]Matthew Thorne , David Burke , Michiel van de

Panne, Motion doodles: an interface for sketching

character motion, ACM SIGGRAPH 2004 Papers, August

08-12, 2004, Los Angeles, Califrnia

[21]Microsoft Office PowerPoint,office.microsoft.com/PowerPoint, 2009

[22]Phun, 2009, http://www.phunland.com/wiki/Home

[23]Robert C. Zeleznik , Kenneth P. Herndon , John F.

Hughes, SKETCH: an interface for sketching 3D scenes,

Proceedings of the 23rd annual conference on Computer

graphics and interactive techniques, p.163-170, August

1996


8/8

[24]Robert Zeleznik , Timothy Miller , Chuanjun Li,

Designing UI techniques for handwritten mathematics,

Proceedings of the 4th Eurographics workshop on Sketch-

based interfaces and modeling, August 02-03, 2007,

Riverside, California

[25]Seok-Hyung Bae, Ravin Balakrishnan, and Karan

Singh, ILoveSketch: As-natural-as- possible sketching

system for creating 3D curve models,. (To appear) ACM

Symposium on User Interface Software and Technology

2008 (Monterey, CA, USA, October 19-22, 2008)

[26]Tano, S. et al. Godzilla: Seamless 2D and 3D sketch

environment for reflective and creative design work,

INTERACT03 (2003) 311- 318.

[27]Task Architect, 2009,

a comparison study: sketch-based interface versus wimp interfaces in three-dimensional modeling...

Documents