web standards and (the future of) user interface design
DESCRIPTION
Seminario Martedì 22 aprile 2008, h. 15:00 Aula Seminari, seminterrato Dipartimento di Scienze dell'Informazione Bologna. Web Standards and (the future of) User Interface Design. St éphane Sire ([email protected]) École Polytechnique Fédérale de Lausanne CGC/MEDIA. Plan. - PowerPoint PPT PresentationTRANSCRIPT
Web Standards and (the future of)
User Interface Design
Stéphane Sire
École Polytechnique Fédérale de Lausanne
CGC/MEDIA
Seminario
Martedì 22 aprile 2008, h. 15:00
Aula Seminari, seminterrato
Dipartimento di Scienze dell'Informazione
Bologna
Plan
Web Standards and (the future of) User Interface Design
Part 1
Part 2
Part 3
Rich User Interaction
SVG and alike
Iterative Design and supportive technologies
Part 1
(the future) of User Interfaces
Rich User Interaction
4
Evolution of User Graphical Interaction Styles
Desktop Web Mobile
Command Line
Direct (1983)Manipulation
Point & Click(hypertext)
HTML Forms
WAP
Rich User Interaction
WIMP
Rich Desktop Application (RDA)
Rich Internet Application (RIA)
iPhone and alike (touch based)
5
Characteristics of Rich User Interaction
• Better Graphics• Extensive use of Animations • Creative layout models• Borrowings from other interaction styles• Better feedback times (AJAX)
A new vocabulary for design
6
Better Graphics (1)
• Layers– Painter's model of rendering– Compositing operators
• (Porter & Duff)
– Clipping and masking
• Applications– Layout based on layers (see later)– Non rectangular windows
• Objects with holes
– Layer's effects Source: Digistrip (CENA)
7
Better Graphics (2)
• Color models– Alpha channel (RGBA)– Gradient colors– Texture filling
• Applications– Light and shadows effects– Glossy effect– Reflection (cow.neondragon.net/stuff/reflection/)
– etc
8
• Shapes & lines– Stroke styles
• Joints• extremities
– Bézier paths• Very efficient algorithm to draw it (De Casteljeau
algorithm)
• Applications– Rounded shapes
Better graphics (3)
9
Better Graphics (4)
• Pixel Effects– Anti-aliasing
– Filters (blur, bump mapping, etc.)
Source www.corzo.com
Source www.treebuilder.de
10
Extensive use of animations
User's guidance
System state change
Visual state transition
Source www.joehewitt.com/iui/samples/music.html Source www.lemonde.fr(choose Journal Electronique)
Source www.panic.com/goods/
11
Animation Tricks
• "Exageration"
• Pace• Application: Kinetic Scrolling (video on youTube)
QuickTime™ and aMPEG-4 Video decompressor
are needed to see this picture.
QuickTime™ and aMPEG-4 Video decompressor
are needed to see this picture.
Author's made movie from www.laredoute.fr Author's made movie from www.youarethemnodel.com
12
New Layout Models (1)
• Layered information
Source www.laredoute.fr
Source www.natoora.com
Source maps.google.com
13
New Layout Models (2)
• Contraction/dilatation– Accordion
• Continuous surfaces– Linear (ribbon) : www.gucci.com
– Two dimensions : www.goruneasy.com
– Unconventional : www.etsy.com (choose Time Machine)
Source www.goruneasy.com Source www.yomiuri.co.jp
14
New Layout Models (3)
• Fixed layout + Animation– Scrolling Ribbon
• more information with constant space within the page
• Layers + Animation– Drawer windows– Docks
Source www.tf1.fr
15
Borrowings from other styles
• Direct manipulation– Drag & drop but still marginal
• Post-WIMP techniques– Bifocal menus
• Video-games– Rotative menus
Source www.amazon.com
Source www.wat.tv
16
Better Feedback Times with AJAX
• No more page reload and full page refresh– Increase reactivity– Requires special tricks to get user's attention
• Applications– Auto-suggestion input field (see Google Autosuggest)
– Dynamical queries• allows some forms without no "Submit" button
• or allows to pre-filter results before submitting
Sourcewww.trivop.com
Sourcewww.darty.com/nav/achat/telephonie/telephonie_mobile/telephone_portable/guide.html
17
Rich User Interaction (summary)
• Graphical design– To increase intuitivity– To create emotions– Emotions increase perceived usability
• See Don Norman last book "Emotional Design"
• Requires multi-disciplinary teams – User experience (interaction) designer– Visual designer, Motion designer– Developers– etc
(video on youTube)
Part 2
User Interface Design
Iterative Design and supportive technologies
19
Product Development Process
Reprinted from "Sketching User Experiences" (Buxton, 2007)
20
Iterative Design
21
Close to Agile Software Development
• Agile Manigesto (source agilemanifesto.org)
– Individuals and interactions over processes and tools
– Working software over comprehensive documentation
– Customer collaboration over contract negotiation – Responding to change over following a plan
• See also Extreme Programming• Web version
– the permanent Beta
22
Examples
Source: Nielsen
Source: Hiser
Source: Alex Poole
24
Classical development cycle
Users
SketchesStoryboardWireframesIA diagrams
Pro
gram
me
rs
Interaction,Visual,Motion,…Designers
ExecutableCode
BrowserPluginRuntime Env.
Feedback loop
How to increase the number of iterations ?
25
Model Driven Development Cycle
Users
SketchesStoryboardWireframesIA diagrams Programmers
Interaction,Visual,Motion,…Designers
Graphical modelsAnimation modelsOther models…
Browser, PluginRuntime Env.
Executable code
27
How to represent a model ?
• eXtensible Markup Language (XML)
<g id="Root"><g id="Back">
<path id="Background"><g id="Zones"><g id="CatchZones">
<text id="Callsign"><text id="RWY"><text id="Gate"><path id="Catch">
<g id="DMAN"><text id="STA">
28
How to Edit a Model ?
• Specialized editors – Example for graphics: Adobe illustrator
SVG File
29
A short Example
Interaction Designer
Visual Designer
Programmer:LET's DO IT !
Graphical ModelFile(s)
Example from Adobe dev web site
30
Adding More Models….
Interaction Designer
Visual Designer
Graphical Model
Behavior Modelling with State Machines
Finite State Machine Models
31
Example from
Visual Design
Paper Mockup
Prototyping
Integration
Airnet Project
32
Model Driven Summary
Models
Models(XML)
CodeCode
• Models– Specialized editors– Easy to move from
one platform to another
• Code– Programming skills– Hard to maintain– Less portable
Part 3
Web Standards
for User Interface Design
SVG and alike
34
W3C standards (1)
• Mainly "Models" – XML based languages– From "Draft" to "Recommendation" status– Models both for output and input modalities– HTML, XHTML, XForms, SMIL, MathML, VoiceXML,
SCXML, InkML, EMMA, …, and SVG
• Some "Code" too– Application Programming Interfaces (APIs)– DOM (3 levels)
• API for manipulating models loaded into memory as trees• Many more APIs (events, load and save, progression, etc.)
– XMLHttpRequest• Ajax programming
35
Web Standards and MDA
Markup Only
Code Behind
Markup Behind Code Only
XML
XML
ImperativeProg. Lang.
Imperative Prog. Lang.
Definition of Interactive ComponentsInstantiation
of Intera
ctive C
omp
one
nts
Today Web applications
Future of applications
Today Desktop applications
36
W3C and Rich User Interaction
(*) simplified with specialized toolkits (scriptaculous, jQuery, etc)
Characteristics Standard solution
Better graphics SVG or CSS3 (with HTML)
Animation SMIL/SVG animation module
DOM Manipulation with Javascript(*)
Creative layout SVG or CSS (with HTML)
DOM Manipulation with Javascript(*)
Better feedback time XMLHttpRequest
Cross domain XHR
Borrowing from other styles All of the above
37
SVG History
• Scalable Vector Graphic• Started in 1998• 1st draft: February 1999• SVG 1.1 recommendation 2003
– http://www.w3.org/TR/SVG11/
• SVG 1.2 Tiny candidate recommendation 2006– http://www.w3.org/TR/SVGMobile12/– SVG 1.2 Full to come later
• Working Group with main software editors– Sun, Adobe, Apple, Microsoft, Corel, Ilog, HP, Canon,
Autodesk, etc.
38
SVG vs. Flash
Property SWF SVG
XML - Std W3C No Yes
Metadata Yes, propietary Yes, RDF compatible
Format Binary Text
Size < SVG > SWF : but can be compressed with gzip
by about 80% (*)
Runtime Plugin (200K) Native (Opera, Safari, Firefox, …)
Source No Yes
Audio/Video Yes Yes (SVG 1.2 Tiny)
(*) see http://www.w3.org/TR/SVG/minimize.html
39
Authoring Tools
• Graphical Editors– Sodipodi, Illustrator, Corel Draw, OmniGraffle Pro, …
• Specialized Editors– Ikivo Animator : animations
• Automatic generation – Client-side Javascript generation
• Example from http://www.destatis.de/
– Client or server size XSLT generation• demo from www.treebuilder.de/default.asp?file=250484.xml
• Automatic conversion from/to other formats– From MathML, AutoCAD, Visio, etc.– To PDF, PNG, etc.
• Manual authoring :)
40
Players
• Adobe to stop its plugin support in 2009…– Adobe to switch to Flash/AIR ?
• … but becoming native in Web browsers– Opera, Safari, Firefox, …– open source rendering libraries (Cairo)
• Not directly available in IE…– Microsoft to switch to XAML/WVG with silverlight plugin
• Standalone players– Batik (Java) - contains a useful utility to generate SVG fonts
• SVG Tiny 1.2 on mobile phones– www.bitflash.com/mobile_primer.html– www.ikivo.com/02player_svg.html
– And more…• See list of implementations www.svgi.org/
41
Drawing Basic Shapes
• Shapes– Rectangles <rect> – Cercles <circle>– Ellipses <ellipse> – Lines <line>– Polylines <polyline>– Polygons <polygon> – Text <text>
• Attributes– x, y, width, height, rx, ry– fill, stroke, stroke-
linecap, stroke-linejoin, stroke-miterlimit, stroke-width, …
– opacity, stroke-opacity, fill-opacity
– ...
<rect x="200" y="300" width="90" height="40"
stroke-linejoin="bevel" stroke-width="3"/>
fill="lime" stroke="blue"
42
Drawing Arbitrary Shapes
• Path <path d="path data" >– outline of a shape which can be filled, stroked,
used as a clipping path – defined by connected lignes, arcs and curves.
• Base attribute– "path data" : complex vocabulary – "," and EOL are allowed in "path data", spaces are
optional between a command and a number (to compress data)
<path d="M 100 100 L 300 100 L 200 300 z"
fill="red" stroke="blue" stroke-width="1" />
43
Path data vocabulary
• Each command is named by a letter (M: moveto L: lineto, etc.) followed by a coordinate, either
– Absolute if capital (e.g. "M")– Relative to the previous point if lowercase (e.g. "m")
• M and m:– Move the point to the given coordinate– Create a new sub-path starting at the origin– Syntax: M|m (x y) (x y)*– The second group (x y)* is interpreted as lineto commands– Ex: M100 100 200 200
• L and l:– Draw a line from the current point to the given coordinate and draw lines
between subsequent given points– Syntax: L|l (x y) (x y)*
• H and h (V and v):– Draw a horizontal line (vertical) from the current point to the current point
incremented with the given number– Syntax: H|h (x) et V|v (y)
• Z and z:– Close current subpath with the last point defined with M|m
M 100,100
L 200, 200
x
Cur point
44
More path data vocabulary
Command Args (can be repeated)+ Description
A, a rx ry x-axis-rotation direction sweep x y
Elliptical arc
C, c x1 y1 x2 y2 x y Cubic Bézier curve from current point to x y with control points x1 y2 and x2 y2
S, s x2 y2 x y Cubic Bézier curve with control point as the reflection of the reflection of second control point of a C, c command
Q, q x1 y1 x y Quadratic Bézier curve from current point to x y with control point x1 y1
T, t x y Quadratic Bézier curve with control point as the reflection of previous control point of a Q, q command
See details on http://www.w3.org/TR/SVG/paths.html
45
Defining reusable graphical components
• Definition of a component– Group element: <g id="name">– Container element for grouping graphics elements– Can be nested– Its attributes are inherited by its descendants
• Instantiation of a component– <use xlink:href="#name"/>
<g id="shape" stroke="blue">
stroke-width="0.1cm" fill="none"/>
stroke-width="0.1cm"/>
<use xlink:href="#shape" x="70" y="60"/><use xlink:href="#shape" x="140" y="20"/></g> <rect x="10" y="50" width="50" height="1cm" <path width="1cm" d="M 0 50 35 25 70 50 Z"
46
Declaring graphical components without drawing them
• Everything that appears in a <defs> section will not be drawn– It will be available to be reused in other parts– This is a way to group resources (at the beginning
of the file)
<defs> <g id="shape" stroke="blue">
stroke-width="0.1cm" fill="none"/>
stroke-width="0.1cm"/>
</defs>
<use xlink:href="#shape" x="70" y="60"/><use xlink:href="#shape" x="140" y="20"/> </g> <rect x="10" y="50" width="50" height="1cm" <path width="1cm" d="M 0 50 35 25 70 50 Z"
47
Positioning graphics
• Most elements can be positioned with x and y within their parent container
• Groups <g> are also used to apply transformations– <g transform="translate(x,y)">– Other transformations include rotate, skewX|Y, scale and matrix– This is equivalent to associating a transformation matrix with each
group <g> (and to redefine a new coordinate system for the descendants)
• Transformation are cumulative in nested groups– Current transformation matrix at each <g>– Obtained as the product of all transformation matrices up to the
parent <g>
<g transform="rotate(20, 200, 300)">
fill="lime" stroke="blue" stroke-width="3"/>
</g><rect x="200" y="300" width="90" height="40"
48
Painting and Coloring
• Colors applies with different attributes to– Strokes (attribute "stroke")
– Shapes interior (attribute "fill")
• Colors are rendered by a paint server– The attribute URI reference selects a paint server
– Solid colors (e.g. "orange" or "#50A619")
– Gradients or Patterns (e.g. "url(#myGradient)")
<defs>
<rect x="200" y="300" width="90" height="40"
fill="url(#Gradient01)" stroke="blue" stroke-width="3"/> </defs> </linearGradient> <stop offset="90%" stop-color="#F3F" /> <stop offset="20%" stop-color="#39F" /> <linearGradient id="Gradient01">
49
Masking and Clipping
• Masking– Attribute
• mask="url(#mask)"
– Draw #mask in an offscreen buffer
– Convert it to gray level image
– Composite with the masked layer
Layer1
Layer2Masked by Layer3
Result
Layer3MaskNot visible
<g mask="url(#Mask)"> <use xlink:href="#logo" transform="scale(1 -1"/></g>
50
Animating Graphics (2)
• Each graphical attribute can be animated– animate element– Animation description based on SMIL2
(Synchronized Multimedia Integration Language level 2), W3C: http://www.w3.org/TR/smil20/
• Animation describing– Attribute value variation– Time variation
• Including time effect (slow-in, slow-out, etc.)
<rect x="10" y="10" width="200" height="20"> <animate attributeName="width" attributeType="XML" from="200" to="20" begin="0s" dur="5s" fill="freeze" /> </rect>
51
Animating Graphics (2)
• Other types of animation– Animation along a path (motion tweening)
• animateMotion
– Special instruction for color animation• animateColor
• More advanced animation can be built with more programming (e.g. Morphing)– Exemple: Dragicevic, Artistic Resizing (UIST, 2005)
Click image to see movie from Web (if still online…)
52
Metadata
• Images can be describe with – <desc>
• Can contain any RDF data
– <title>• Standard element to give a title
• This can be useful for – Accessibility (screen readers, …)– Indexing image file (do not forget it's text files)
<g> <title> Company sales by region </title> <desc> This is a bar chart which shows company sales by region. </desc> <!-- Bar chart defined as vector data --> </g>
53
Extra features
• Pixel filers • Markers• Glyphs• Text along a path• Adding Interactivity with Javascript Programming
– SVG APIs • SVG DOM API: for scripted interactivity• Network communication• …
• Much more … explore by yourself – Mozilla SVG home: http://developer.mozilla.org/en/docs/SVG– Lot of fascinating examples: http://www.treebuilder.de/– SVG-Wiki: http://wiki.svg.org/– W3C SVG Home: http://www.w3.org/Graphics/SVG/– User Interfaces examples developped by IntuiLab with its own toolkit: http://www.intuilab.com/gallery/
54
Conclusion: RIA Standards and Technologies
X11
1987 1992 1994 2002 2004
ShockwaveHTML HTML 4.0
Flash/SWF
X-HTMLSVG
Javascript DOM
XMLHttpRequest Ajax
CSS
IFrame
1990 1996 1998 2000 2006
OpenGL
Direct 3D OpenGL2.0
Mosaic NetscapeIE
FirefoxKonfabulator AIR
Silverlight
XAMLFlex/MXMLXUL
WVG
Credits
Web sources indicated directly on the slides, otherwise (many thanks to them):
Illustrations
p7 CENA Digistrip www.tls.cena.fr/divisions/PII/toccata/composants/digistrips.html
p24 IntuiLab DMAN strips (courtesy of Céline Schlienger) – www.intuilab.com
p32 IntuiLab Airnet (courtesy of Stéphane Vales) – www.intuilab.com
Demos
p10 SVG Slot Machine – www.treebuilder.de
p30 Adobe Airbus A321 Panel demo – www.adobe.com/svg/demos/main.html
Book
Sketching User Experience, Bill Buxton, Morgan Kaufman by Elsevier inc. 2007 – www.billbuxton.com
Article
Combining SVG and models of interaction to build highly interactive user interfaces, S.
Chatty, A. Lemort, S. Sire, J-L. Vinot, SVGOpen 2005 – www.svgopen.org/2005/papers/CombiningSVGModelsBuildInteractiveUserInterfaces/
56
Extra Slides
57
Complete SVG File Example
<?xml version="1.0" encoding="utf-8"?><!DOCTYPE svg [ <!-- entities etc. here --> ]><svg version="1.1" baseProfile="full" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" width="8cm" height="6cm"> <defs> <!-- resources to be reused --> <linearGradient id="Gradient01"> <stop offset="20%" stop-color="#39F" /> <stop offset="90%" stop-color="#F3F" /> </linearGradient> <rect id="shape" width="1cm" height="1cm" stroke="blue" stroke-width="0.1cm"/> </defs> <!-- content --> <rect x=".1cm" y=".1cm" width="7.9cm" height="5.9cm" fill="none" stroke="black" stroke-width="1px" /> <use x="1cm" y="1cm" xlink:href="#shape" fill="#BBB"/> <use x="4cm" y="1cm" xlink:href="#shape" fill="url(#Gradient01)"/> <use x="1cm" y="4cm" xlink:href="#shape" fill="url(#Gradient01)"/> <use x="4cm" y="4cm" xlink:href="#shape" fill="blue"/> </svg>
58
Web Standardization
• W3C Agenda for Rich User Interaction– Markup languages
• HTML 5 ? XHTML 2 ?• SVG Tiny … SVG 1.2
– Audio and Video
– Styling with CSS3– Application Programming Interfaces (APIs)
• XMLHttpRequest (AJAX)• Cross-domain XHR
– No integrated programming environment !
• W3C is not alone…– Microsoft: XAML/WVG
• Execute in browser plugin (Silverlight)
– Adobe: FLEX/SWF• Execute on desktop runtime environement (Apollo) and browser
plugin (Flash)
59
Client Side Javascript Toolkits
• Fix "incompatibilities" between browsers– Adaptation layer
• More efficient DOM manipulation• Animation support
– Prototype + scriptaculous– Mochikit– Jquery– Yahoo UI– Dojo Toolkit– etc.
• Major sites starting to adopt them– E.g. Prototype + Scriptaculous on Apple