a comparison study: sketch-based interface versus wimp interfaces in three-dimensional modeling...
TRANSCRIPT
-
8/9/2019 A comparison study: sketch-based interface versus wimp interfaces in three-dimensional modeling tasks
1/8
-
8/9/2019 A comparison study: sketch-based interface versus wimp interfaces in three-dimensional modeling tasks
2/8
Science. The improvement of the sketch-based interface is
considered one of the most exciting and challenging
areas.
In the work presented by Seok-Hyung Bae and
colleagues in [25], an interface metaphor of pencil and
paper created for professional designers, called
ILoveSketch. The users can draw curves freely and
directly on the screen, and connect them through camera
rotations. Although the results are 3D models, all objects
are built through the users strokes, with no system
interpretation. The researchers tested the prototype with
the collaboration of a specialist with 12 years experience
in design in the automobile industry and with toys and
movies. The choice was justified by the decision to
project the system for professional users with a high level
of experience. The user carried out an intense evaluation
of the system after a one-hour of training. The main
conclusion was that the user was satisfied with the great
numbers of features of the system.The software Fibermesh [3] is an evolution of
Teddy. It brings even more power to the user in the task
of creation of 3D models. The original curve stroke lies in
the model. It makes possible to manipulate the object in a
practical way with operations directly applied in the curve
by user actions. The researchers presented also a non-
formal evaluation of their system with novice users and
artists. They concluded that Fibermesh is an easy to use
tool, which permits evolution in the creativity tasks
executed by the users.
In the SESAME project [12], James Lin and
colleagues studied ways to provide support to the work ofdesigners in the initial stages of the designing process.
SESAME was created to explore different visions to solve
conceptual design problems in three dimensions. The
work was presented in two phases: the first presented a set
of guidelines to create collaborative systems for
conceptual designs, and the second compared SESAME
against 3DS max. The main goal of the evaluation was to
analyze how designers could make a creative complex
design in the least amount of time, and what sets of
operations they need to execute during the task.
With the GODZILLA [26] system, S. Tano and
colleagues presented experimental systems where theusers can make 2D drawings, which are recognized and
exhibited as 3D sketches in a display (stereo vision TV).
The user can later modify the drawings, as viewed from
many view points (2D or 3D). In order to evaluate the
system, they compared it against pencil and paper, and a
3D CAD (Computer Aided Design) tool. The results
revealed that the ideas in terms of numbers of sketched
are much closer in this system to the numbers of sketched
in traditional pencil and paper combination.
Marcus Wacker and colleagues developed The Virtual
Dressmaker [18], a Virtual Reality application to design
clothes. The system supports advanced interactive
techniques with six degrees of freedom. The researchers
argue that sketch-based techniques are more natural than
the traditional desktop techniques. They started with a
pilot test, where the user needed to position clothes on an
avatar in three different systems: the Virtual Dressmaker,
Maya and the CosmoWorlds. They evaluated variables
such as the time for task completion and precision
achieved by the users. The results showed that the users
performance was better with VirtualDressmaker. They
also presented points to improve the users tasks in future
versions of the software.
In this work [16] Kamran Sedig and colleagues
presents a methodology to evaluate the impact of using
geometry learning software in the learning geometry
process, for children at a basic educational level. The
research compared three versions of the same softwarebuilt to teach geometry transformations. The goal was to
find ways to design effective tools to ease the knowledge-
building process in learning. The research revealed that
the present interface style brings implications in the
education by the way users interact with the tools.
Another conclusion was that the HCI elements could
improve the cognitive capabilities of users who use the
software, although it can also affect the same capabilities.
This work served also as an inspiration to us, since it
advances the idea that sketch-based interfaces have to be
more investigated in order to identify gains and eventual
losses that the technique can offer in different contexts.
Takeo Igarahashi and colleagues introduced the
software Teddy [10], a gesture based system where the
users draw on a white screen with strokes in 2D (input
data), and the result of this interaction is a 3D model
(output data). Basically, all operations are a result of a set
of actions (gestures) like: creation, paint, extrusion, cut,
smooth, bend, etc.
3. Methodology
We used a qualitative and quantitative methodology
[4] to conduct our comparative study, described below.
3.1. Pilot Tests
We executed a great number of pilot tests [18] in order
to decide the tools to be used, to evaluate and improve our
methodology. After the pilot sessions, the collected data
was analyzed and the methodology was adjusted when
necessary. We repeated this procedure until we decided
-
8/9/2019 A comparison study: sketch-based interface versus wimp interfaces in three-dimensional modeling tasks
3/8
that the methodology was ready to be executed in a real
context.
In our pilot tests we had the collaboration of a
computer science student familiar with traditional
software for 3D modeling (Maya in this case). One
identified necessity was the reduction of the experiments
length, to avoid the user becoming tired. Although there is
no limit on the time to execute the tasks, we planned it to
consume a minimum amount of the users time, without
jeopardizing the goals of our study.
An important decision taken during the pilot tests was
the definition of the Teddy system as the selected tool to
evaluate the sketch-based modeling technique. Our other
choice, the software Fibermesh (which has more
interaction possibilities than Teddy), was rejected due to
great instability in the prototype version available. The
details of the methodology are showed below.
3.2. Hypotheses
To evaluate our study, we considered the following
hypothesis:
H1: modeling with the use of sketch (as presented in
Teddy), demands less effort from the user than modeling
with the use of WIMP-like interfaces (as presented in
Maya or 3DS Max).
H2: the sketch-based modeling approach (as presented
in Teddy) reduces the users time to complete tasks.
H3: the sketch-based modeling technique (as presented
in Teddy) produces satisfactory results.
The effort (H1) and the satisfactory users results (H3)
in this study were evaluated through the users answers
collected in a survey related to the two techniquespresented. The time (H2) was verified through the video
register of the users activities.
3.2.1. Dependent and Independent Variables. The
independent variables involved in this study were:
the technique utilized (Sketch or WIMP); the target object to be modeled (a bear); the executed task.
The dependent variables were the following:
users effort; number of tries to realize the task; time to execute the tasks; users satisfaction; Hierarchical tasks models (HTA).
The user effort and satisfaction with the results were
collected through the same survey. All the other variables
have their results computed after the analysis of the users
activities.
3.2.2. Subjects. The profile defined users who are
studying or working in the Design, Art or Technology
fields with experience in Computer Graphics products
such as Maya or 3DSmax. They should also have basic
understanding of the modeling process of these tools. All
the users were recruited as volunteers in academic or
technical schools, or in design, games, and technology
companies.
3.2.3. Collected Data. To collect the data, we defined
25 users. In the study here presented we used only 6 users
to show qualitative data, our quantitative analyze
continues and will be shown in a future work.
3.2.4. Users Mental Model. We used the user speech
to analyze, in an hierarchical form, their activities in
Teddy and Maya or 3DS Max. The main objective was to
build two trees of analyses, one for each approach (sketch
and WIMP). By analyzing each tree, we can extract
details of the users modeling activities, such as usability
gains, needs and requirements for designing a system
based on our findings.
3.2.5. Survey (personal data). A simple survey wasused to collect information about the users, such as
occupation and familiarity with the WIMP tools defined
for the study.
3.2.6. Questionnaire (System Usability Scale). The
use of a questionnaire was necessary to collect, through
the users replies, the measures for the three subjective
hypotheses (H1, H2, H3), which are related to the
easiness of use and the satisfaction with the observed
results. The questionnaire was adapted from the available
model developed by SUS System Usability Scale [5]
and applied to both techniques for comparison effect (the
adaptation is available at
www.cin.ufpe.br/~tlam/sus_adaptation ).3.2.7. Users Comments. Through the users answers
we collected qualitative data related to their opinions
about satisfaction with the created 3D models, the use of
creativity in the tools, and ease of use with the software.
3.3. Procedure
The test sessions were composed of two phases: one
dedicated to introduce Teddy, and another oriented to
tasks execution and answering the questionnaire. In the
first phase, the goal was to make Teddy more familiar to
the users. They filled a simple questionnaire about their
experience with the tools and their occupations.The users had time to test the system functions with a
tutorial help available in: http://wwwui.is.s.utokyo.ac.
jp/~takeo/teddy/teddy/tutorial.html.
In the second phase, the users had to execute three
tasks, one only for modeling and the other two dedicated
to editing the model previously created. Each task was
performed with both Teddy and Maya (or 3DS Max).
-
8/9/2019 A comparison study: sketch-based interface versus wimp interfaces in three-dimensional modeling tasks
4/8
There was not a time limit to conclude the tasks, and
the only rule was that the user needed to execute each task
in each tool. The tasks were as follows:
Creation - The user had to reproduce a teddy bear
(Figure 1) model presented in a reference picture. This
reference was used only to show a direction of how the
users test could start and not to follow the reference
exactly as seen in the picture.
Figure 1. The reference picture of a teddy bear.
Editing - Using the bear model created in the previous
task (did not need to be complete), the user was asked to
make a drawing of a four-point star at any point on the
bears surface. Following, they should erase this star and
draw a five-point star instead. After that, the user needed
to cut one of the bears ears, and create a little cavity in
the bears body.
Pointed ear - The user was asked to deform the bears
ears to make it look like a cats ear. To do this task, the
previously created ears must be used.
After the conclusion of the third task, the users filled in
a questionnaire about the tests and talked about their
experience when performing the tasks.
The tasks were defined in this way to cover a set of
basic operations presented in the 3D modeling systems
evaluated.
4. Results
For our qualitative study, we used the Hierarchical
Task Analyses (HTA) technique. Based on the full video
and audio recorded during the sessions, we developed a
HTA related to each task executed by the users. The
HTAs were generated with the trial version of the
Software Task Architect [27]. We adopted the number of
units generated in the tasks to define the usability gains of
the techniques. We will use the definition NS to denote the
number of operations in a sketch-based interface for
modeling (Teddy in this case) and the definition NW to
denote the number of operations in a traditional desktop
interface system (Maya or 3DSMax).
We collected data from six subjects, all of whom were
volunteers for the study. They were recruited in graduate
courses (Computer Science, Design and Arts) and in
game companies. All of them had little knowledge about
sketch-based systems, but some experience with Maya or
3DSMax in different levels, varying from beginner to
professional, according to their use of these tools in their
leisure time or in their professional lives.
Unfortunatelly we didnt reach a large number of
users (we defined 25) to generate enough data.
We have a consulting with a specialist in statistics and
the recommendation was following this study until we get
the specified amount of users data.
According to the specialist, an analyze with only 6
users is insufficient to give effective results.
By these reasons this paper section analyze our
qualitative data exclusively.
We continue to work with the quantitative part of the
study and soon as possible we will reveal all the achieved
results and hypotheses comments.
4.1 Task results
4.1.1. Creation. NS < NW
In task one - the creation of a 3D bear model - all the
users used less operations in the sketch-based system than
in the wimp based one(Figure 2 and Figure 3). The
creation of the model in Teddy was straightforward. The
users started this task in one of two ways: drawing
directly on the screen with the mouse, or using the
example sphere, which starts the software. All the modelelements such as arms and legs, were generated with the
extrusion set of gestures available in Teddy.
In the WIMP systems, the users used geometric
references to construct the model. The manipulation of
these references, in order to build the model, forces the
users to think in some pre-defined ways. The users have
to adapt their ideas to the object seen currently on the
screen, thus resulting in additional tasks.
Figure 2. A pseudocode example of the creationtask in a sketch interface (Teddy) done by one of our
test users.
-
8/9/2019 A comparison study: sketch-based interface versus wimp interfaces in three-dimensional modeling tasks
5/8
Figure 3. A pseudocode example of the creationtask in a WIMP interface (Maya) done by one of our
users test.
4.1.2. Editing. NS ~= NW
In this task, the number of operations was close in both
approaches. We will present the result of this task
according to their subtasks: cut operation, drawing (and
erase) a star, and creating a cavity.
Editing: cut operation. NS ~= NW
In this subtask, with only a single stroke in Teddy, the
users can cut the ear off the bear. In the WIMP-based
system, two techniques were used: some preferred to use
the delete face function, whereas others preferred the
Boolean operation to cut the ear.
Editing: drawing (and erase) a star. NS~= NW
Most of the results in this subtask showed that the
number of operations is closer in both systems. In fact, all
the users used the same approach in the sketch-basedsystem, i.e., they made strokes directly on the models
surface to create the stars and then scribbled to erase it. In
the WIMP-based systems, the users defined a few points
to connect edges in the surface model. The stars were
built in this way. To erase it, they used the delete
command. Another user preferred to use the function
Staravailable in the system. But this function added more
tasks related to typing the number of vertices to form the
star which increases the effort (NS < NW) in an WIMP
interface.
Editing: creating a cavity. NS ~= NW
To create a cavity in the sketch-based system the users
only needed to apply the extrusion function. In theWIMP-based systems the users executed the task by
selecting some vertices and pushing one to define the
cavity.
4.1.3. Pointed ear. NS > NW
In this task, the number of operations was greater in
the sketch-based system (Figure 4 & and Figure 5). In
the sketch approach almost all users used cut and extrude
operations to give the ears of the bear a pointed view
look. One user tried the bend function, the most common
alternative to this task, but he did not finish his action
because a system fail locked the system. Another usertried the bend function with success.
In the WIMP approach, all users selected a set of
vertices or faces defining the ear, and pulled these to give
the bears ear with a pointed view look.
Figure 4. A example of the first and second try inthe Pointed ear task in a sketch interface (Teddy).
The user used the bend function without success, so
he made the ear pointed with an extrusion function.
-
8/9/2019 A comparison study: sketch-based interface versus wimp interfaces in three-dimensional modeling tasks
6/8
Figure 5. The pointed task in the WIMPapproach.
5. Conclusions
In this section we present our main conclusions,
grouped by our impressions regarding: the reducedinstruction set gestures presented by Teddy in our test
sessions, the creation and the editing phase of the
procedure, and their implications in the user activities.
5.1 Reduced Instruction Set Gestures
The sketch-based system Teddy has a reduced
instruction set of gestures. This characteristic implies that
the user can generate 3D models in a faster and easier
way. They can interact with the computer environment
without loss of time and concentration in long searches
over interface menus.
This reduced set of instructions can be repetitive andtedious in great projects or in the users time task along a
computer, but it could not be observed in the tasks
analyzed.In our sessions, we could observe that the users
created the model basically with a simple extrude
function.
In the case of WIMP-based systems, the use of several
menus and dialog boxes cause some mistakes and
difficulties to get a solution for a specified task. The users
have to acquire a great knowledge of the options
presented in the system interface even to create simple
objects.
.
5.2 Creation
We could observe that the best usability gain that the
sketch-based interface technique brings to the 3D
modeling task, is in the creation phase. On this task, the
number of user operations was consistently less than
when realizing the same tasks in a WIMP-based
interface. Creating a model in this way was very practical
to our users, since the sketch technique is a
straightforward way for users to express their ideas,without having to make technical decisions about the
object to be generated.
In the WIMP-based systems, the users need to plan
ahead, or to have a well-defined idea to minimize theirtime and effort in the selection sets to construct their
models. The technical background required to manipulate
these interfaces is also higher, if the user wants to make
full use or their creativity throughout the project. A high
level experience in the use of these interfaces is needed to
reach the same level of interaction as in the sketch-based
systems.
5.3 Editing
The editing operations revealed more usability gains in
the WIMP-based systems. In task 2 (Editing) of our
experiment, we could observe that the number of
operations was very close, but in the task 3 (Bend) we
observed that number of operations was greater in thesketch-based system. The possibility of direct
manipulation of vertices and faces, and the use of
keyboard commands to copy and paste, showed that these
familiar computational and geometry instructions are
more remarkable than some new gesture-based
instructions.
6. Requirements and Future Work
We are developing a 3D geometric modeling prototype
using sketch-based ideas for the interface, and we plan to
use the gathered results from this study as requirements
for this prototype. The evaluation of the HTAs and the
user comments, showed us that a system combining the
two approaches (sketch and WIMP) could be built with
the following characteristics:
Manipulationofvertices and faces - We observed that
the manipulation of vertices and faces that define the
models, is an operation for editing objects very popular
with by the users. Editing a model at the level of a vertex
or a face was very useful and simple in our tests sessions
in the WIMP approach. In the sketch approach, although
the interaction can be very simple too, the users have to
make some effort to find the right stroke which will
generate the desired modification.
Copy and paste functions - Another functionality which
reduces the users' efforts is the copy and paste functions.
All users use it to avoid modeling similar objects from
scratch several times. In a WIMP context, this use is very
simple with selected options through the graphical
interface or keyboard commands. But in the sketch
software, this function was not available and the users had
to develop similar objects from the scratch every time
they needed to use them.
This study did not yet reach a representative number
of users to run the quantitative analysis as presented in the
Methodology session. We continue to collect data from
new users tests and we hope to show the results in a
future as a natural evolution of this research.
7. Acknowledgements
We would like to thanks CNPQ for finatial support, the
members of our research groups Cincias Cognitivas e
Tecnologias Educacionais and Mdia & Interao,
-
8/9/2019 A comparison study: sketch-based interface versus wimp interfaces in three-dimensional modeling tasks
7/8
Playlore Gameworks for the users, the teacher Richard
Lane and Cultura Inglesa for the technical review of this
paper, the teacher Renata Souza for consulting and all the
users who participated in a volunteer way of this study.
8. References
[1]3DS MAX, 2008. Autodesk,
http://www.autodesk.com/3dsmax.
[2] Alice, Carnegie Melon University, http://alice.org/
[3] Andrew Nealen , Takeo Igarashi , Olga Sorkine , Marc
Alexa, FiberMesh: designing freeform surfaces with 3D
curves, ACM SIGGRAPH 2007 papers, August 05- 09,
2007, San Diego, California
[4] Batista, Makilim Nunes., Corra de Campos, Dinael.
Metodologias de Pesquisa em Cincias: Anlises
Quanitativa e Qualitativa. Rio de Janeiro: LTC, 2007
[5] Brooke, J. (1996) SUS: a "quick and dirty" usability
scale. In P. W. Jordan, B. Thomas, B. A. Weerdmeester &
A. L. McClelland (eds.) Usability Evaluation in Industry.
London: Taylor and Francis
[6] C. Alvarado and R. Davis. Resolving ambiguities to
create a natural computer-based sketching environment.
In Proceedings of the Seventeenth International Joint
Conference on Artificial Intelligence, pages 13651374,
2001.
[7] DIETRICH, Carlos A.; NEDEL, Luciana P.;
COMBA, Joo L.D. A Sketch-based Interface to Real-Time Strategy Games based on a Cellular Automaton.
Games Programming Gems 7. : Charles River Media,
2008, p. 59-67
[8]Dunham G., Forbus K., and Usher J. nuWar: A
Prototype Sketch-based Strategy Game. Northwestern
University, IL. USA
[9]Google SketchUP, http://sketchup.google.com/, 2009
[10]Igarashi, T., Matsuoka, S., and Tanaka, H. 1999.
Teddy: A sketching interface for 3D freeform design. In
ACM SIGGRAPH, 409416
[11]Ivan E. Sutherland, Sketch pad a man-machine
graphical communication system, Technical Report,
University of Cambridge, September 2003
[12]James Lin , Mark W. Newman , Jason I. Hong, James
A. Landay, DENIM: an informal tool for early stage web
site design, CHI '01 extended abstracts on Human factors
in computing systems, March 31-April 05, 2001, Seattle,
Washington
[13]Ji-Young Oh, Wolfgang Stuerzlinger, John Danahy,
SESAME: Towards Better 3D Conceptual Design
Systems, Proceedings of the 6th conference on Designing
Interactive systems, 2006, University Park, PA, USA
[14]Joseph J. LaViola, Jr. , Robert C. Zeleznik,
MathPad2: a system for the creation and exploration of
mathematical sketches, ACM Transactions on Graphics
(TOG), v.23 n.3, August 2004
[15]Joseph Jacob Cherlin , Faramarz Samavati , Mario
Costa Sousa , Joaquim A. Jorge, Sketch-based modeling
with few strokes, Proceedings of the 21st spring
conference on Computer graphics, May 12-14, 2005,
Budmerice, Slovakia
[16]Kamran Sedig , Maria Klawe , Marvin Westrom,
Role of interface manipulation style and scaffolding oncognition and concept learning in learnware, ACM
Transactions on Computer-Human Interaction (TOCHI),
v.8 n.1, p.34-59, March 2001
[17]Luke Olsen and Mrio Costa Sousa and Faramarz
Samavati and Joaquim Armando Pires Jorge, A
Taxonomy of Modeling Techniques using Sketch-based
Interfaces,Eurographics, Apr. 2008 , Eurographics
Association
[18]Markus Wacker , Stanislav L. Stoev , Michael
Keckeisen, Wolfgang Straer, A comparative study on
user performance in the Virtual Dressmaker application,Proceedings of the ACM symposium on Virtual reality
software and technology, October 01-03, 2003, Osaka,
Japan
[19]Maya, 2008. Autodesk,
http://www.autodesk.com/maya
[20]Matthew Thorne , David Burke , Michiel van de
Panne, Motion doodles: an interface for sketching
character motion, ACM SIGGRAPH 2004 Papers, August
08-12, 2004, Los Angeles, Califrnia
[21]Microsoft Office PowerPoint,office.microsoft.com/PowerPoint, 2009
[22]Phun, 2009, http://www.phunland.com/wiki/Home
[23]Robert C. Zeleznik , Kenneth P. Herndon , John F.
Hughes, SKETCH: an interface for sketching 3D scenes,
Proceedings of the 23rd annual conference on Computer
graphics and interactive techniques, p.163-170, August
1996
-
8/9/2019 A comparison study: sketch-based interface versus wimp interfaces in three-dimensional modeling tasks
8/8
[24]Robert Zeleznik , Timothy Miller , Chuanjun Li,
Designing UI techniques for handwritten mathematics,
Proceedings of the 4th Eurographics workshop on Sketch-
based interfaces and modeling, August 02-03, 2007,
Riverside, California
[25]Seok-Hyung Bae, Ravin Balakrishnan, and Karan
Singh, ILoveSketch: As-natural-as- possible sketching
system for creating 3D curve models,. (To appear) ACM
Symposium on User Interface Software and Technology
2008 (Monterey, CA, USA, October 19-22, 2008)
[26]Tano, S. et al. Godzilla: Seamless 2D and 3D sketch
environment for reflective and creative design work,
INTERACT03 (2003) 311- 318.
[27]Task Architect, 2009,