![Page 1: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/1.jpg)
ANVIL: A Multi-Level Video Coding and Analysis Tool
Michael Kipp DFKI, Germany
http://www.anvil-software.de
![Page 2: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/2.jpg)
Overview
Introduction to ANVIL – who knows it? – who has used it?
Interesting (advanced) features, including – Analysis possibilities – Working with Motion Capture
Today: – Lots of demos – Please ask questions any time...
![Page 3: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/3.jpg)
Introduction to ANVIL
![Page 4: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/4.jpg)
What is ANVIL? Manual annotation of digital videos
– „Notepad“ for events in time – See temporal relationships between events
Examples – gesture and speech – human-computer interaction – parent and child interaction – dance analysis – movie analysis – sign language analysis – oceanography – …
Java, version 5 beta 16 Originally: Gesture research (incl. avatars) > 2000 research licenses in 25+ countries. Standard tool in
European projects (e.g. HUMAINE), corpus collection initiatives (e.g. LDC, GEMEP, MUMIN, OTIM), research labs (MIT, LIMSI, NII…), individuals
![Page 5: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/5.jpg)
Annotation Elements
Welcome to Anvil words
head movements
gestures
Elements within one track cannot overlap!
Tracks Track Elements
![Page 6: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/6.jpg)
words
head movements
gestures
Tracks Track Elements
token = „Welcome“
gesture class = deictic handedness = LH repetition = false certainty = 4 token: String
gesture class: ValueSet {deictic, symbolic, iconic} handedness: ValueSet {RH, LH, 2H} repetition: Boolean certainty: Number(1,4)
Elements are Complex
![Page 7: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/7.jpg)
Why multiple, typed attributes?
Multiple attributes – encode many aspects of one „event“
alternative: use more tracks Types
– efficient input: type-specific interfaces • number => number slider • set of tokens => menu • yes/no => check box • alphanumeric string => text field
– fewer errors • input restricted (options, check box etc.)
![Page 8: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/8.jpg)
<attribute name=„type“> <value-el>preparation</value-el> <value-el>stroke</value-el> <value-el>hold</value-el> <value-el>retraction</value-el> </attribute>
Type-specific Interface
![Page 9: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/9.jpg)
Type-specific Interface
String Boolean
Number
Link
![Page 10: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/10.jpg)
Welcome to Anvil words
p n adv
Track Types
gesture sequence
gesture phases
sentences
gestures
part-of-speech
primary track singleton track span track
secondary tracks
![Page 11: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/11.jpg)
deictic beat conduit gesture
NEW: primarypoint primary track singleton track span track primarypoint track
apex high low medium
![Page 12: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/12.jpg)
NEW: subdivision
gesture phases
gestures
gestures
gesture phases
Bottom-Up Coding with „span“ track
Top-Down Coding with „subdivision“ track
![Page 13: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/13.jpg)
Summary: Track Types
Base types: – primary – primarypoint
Secondary types: – singleton – span – subdivision
DEMO (JL, tracks)
![Page 14: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/14.jpg)
Welcome to Anvil words
p n adv
Groups
gesture sequence
gesture phases
sentences
gestures
part-of-speech
![Page 15: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/15.jpg)
Welcome to Anvil words
p n adv
seq
phases
sentences
phrases
part-of-speech
gesture
Groups
![Page 16: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/16.jpg)
Welcome to Anvil words
p n adv
Groups
sentences
part-of-speech
gesture
![Page 17: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/17.jpg)
main window
![Page 18: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/18.jpg)
video window main window
video controls
![Page 19: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/19.jpg)
video window
annotation board
tracks track elements
playback line
active track
video controls
record line
main window element viewer
![Page 20: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/20.jpg)
New Annotation Element
value sets
free-form comments
boolean type
link type
string type
access to coding manual
DEMO (prof)
![Page 21: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/21.jpg)
Coding Scheme = Specification
Specify – tracks name + type + ref
– groups name + contained tracks/groups
– each track: attributes name + type + values
Bad news: XML file! Good news: Graphical editor in Anvil 5
![Page 22: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/22.jpg)
Specification Editor
ANVIL 5
![Page 23: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/23.jpg)
Annotation Data vs. Coding Scheme
Store in one file – pro: all data in one place – pro: consistency between scheme and data
guaranteed – con: modify scheme modify all files!
Store in separate files – pro: modify scheme modify 1 file – pro: trivial to check whether N files have equal
scheme
![Page 24: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/24.jpg)
data 1
video 1
spec 1
data 2
video 2
spec 1
data 3
video 3
spec 1
data 4
video 4
spec 2
data 5
video 5
spec 2
data 6
video 6
spec2
data 7
video 7
spec 3
![Page 25: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/25.jpg)
data 1
video 1
data 2
video 2
data 3
video 3
data 4
video 4
data 5
video 5
data 6
video 6
data 7
video 7
spec 3
spec 1 spec 1 spec 1 spec 2 spec 2 spec2 spec 3
data 1
video 1
spec 1
data 2
video 2
spec 1
data 3
video 3
spec 1
data 4
video 4
spec 2
data 5
video 5
spec 2
data 6
video 6
spec2
data 7
video 7
spec 3
![Page 26: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/26.jpg)
Interesting Features
![Page 27: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/27.jpg)
Cross-level Links
problem code relations between elements in one or across
tracks examples
gesture and word(s) have same meaning two gestures refer to same object/person/location
solution allow putting logical links into an attribute
![Page 28: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/28.jpg)
Links
Welcome to Anvil words
head movements
gestures
category = „deictic“ lexical affiliate = { }
token = „Anvil“
lexical affiliate: MultiLink
![Page 29: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/29.jpg)
Reciprocal Links
Welcome to Anvil words
head movements
gestures
category = „deictic“ lexical affiliate = { }
token = „Anvil“ gesture = { }
lexical affiliate: ReciprocalLink(gesture)
![Page 30: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/30.jpg)
Non-Temporal Elements
Some things do not change for whole video – identity of person – writing on blackboard – object on table …
Refer to these items with link – special container for „non-temporal elements“ – called a SET
![Page 31: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/31.jpg)
a „Set“ containing non-temporal elements
![Page 32: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/32.jpg)
Multiple Videos
Multiple videos displayed in synchrony – multiple camera set-up
(e.g. one for screen / face / overall ..) Video sync (offset) possible
– no graphical interface Video that is first opened is „master“
– Spatial annotation only on master
ANVIL 5
![Page 33: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/33.jpg)
Spatio-Temporal Annotation
What does an encoded element refer to… spatially? whole frame(s)
Restrict to screen region/point: – mark up face region – mark up a distance (e.g. hand to hand) – mark up movement path (gesture)
Options – store single location (constant time) – store moving location (timestamps) – region may change size (morphing)
Where store it? – object-centric – interval-centric
![Page 34: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/34.jpg)
Current State
What – single points
Where – attribute in element (= interval)
How – draw directly on video frame
What you get – attribute myPath = < (t1, x1, y1) … (tn, xn, yn) >
Feature „under construction“
![Page 35: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/35.jpg)
Example: Hand-to-hand Distance
hand-to-hand distance = d
measure shoulder width = s to normalize value
normalized: d* = d / s = 0.4
DEMO, MR
![Page 36: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/36.jpg)
Projects
So far: annotations for single video/session Usually: corpus = many videos+annotations Definition:
– project = annotation files with same specification Project Tool
– overview: metadata, progress – search – analysis over corpus – export for external analysis
![Page 37: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/37.jpg)
contained annotation files
File info: • coder name • content description • last modification time • num. of elements for each track (coding progress !)
Project Tool search engine
![Page 38: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/38.jpg)
Search
include exclude & view options
![Page 39: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/39.jpg)
hit list
click to jump/load
![Page 40: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/40.jpg)
data 1
video 1
data 2
video 2
data 3
video 3
data 4
video 4
data 5
video 5
data 6
video 6
data 7
video 7
spec 3
![Page 41: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/41.jpg)
Import from PRAAT
Import PRAAT interval data – words, syllables, phonemes…
Import pitch + intensity – data files are linked up
![Page 42: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/42.jpg)
Analysis
![Page 43: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/43.jpg)
Analysis
Work on single annotations and projects Features
– Histograms – Agreement – Transition Diagrams – Association – Export
![Page 44: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/44.jpg)
Analysis in context menu
Analysis in main menu
![Page 45: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/45.jpg)
Track Analysis
descriptive stats – num. of elements – coverage (time) – elements / sec – elements / min
„activity“ – gesture rate – smile rate ...
![Page 46: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/46.jpg)
Histogram
Distribution of labels in a single attribute („controlled vocab“) ANVIL 5
![Page 47: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/47.jpg)
Histogram
![Page 48: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/48.jpg)
Agreement
Measure agreement between coders Needs confusion matrix Which elements to compare? Solution: slice + compare
A B A
B B A
B A
YES YES NO ANVIL 5
![Page 49: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/49.jpg)
Agreement Results
ANVIL 5
![Page 50: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/50.jpg)
Results
Cohen‘s kappa[Cohen 1960]
Corrected kappa [Brennan, Prediger 1981]
Segmentation only: treats all values as 1
Also: compare whole projects
![Page 51: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/51.jpg)
Beta feature: View multiple tracks
Right-click here: import track...
Annotations by two different coders
![Page 52: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/52.jpg)
Transition Diagrams
Models transitions between states – e.g., changing hands
(LH, RH, 2H) Compute bigram
(Markov) model from data
Transition probablity from state A to B given as percentage on edge
ANVIL 5 DEMO, JL
![Page 53: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/53.jpg)
Association
Compare association between events on two tracks
Example: gesture + emotion
Step 1: decide which entities to compare SQL query
Step 2: build contingency table
chi square analysis compare tables for
precise interpretation
LH RH 2H
Happy 12 4 1
Angry 5 2 20
LH RH 2H
Happy 6.6 2.3 8.1
Angry 10.4 3.7 12.9
ANVIL 5
![Page 54: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/54.jpg)
Export / Analysis
Single track text table Many tracks text table Import to Excel, Statistica, SPSS Export formats:
– element-by-attribute matrix – framenumber-by-attribute matrix – WEKA (machine learning software)
Alternatives: – use XML stylesheets – Anvil plug-ins (Java)
![Page 55: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/55.jpg)
![Page 56: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/56.jpg)
![Page 57: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/57.jpg)
ID start end lexeme handedness path f0e0 0.76 2.32 Cup LH straight f0e1 5.76 7.52 Frame 2H straight f0e2 13.08 15.36 Wipe LH straight f0e3 15.36 16.04 Beat LH straight f0e4 16.04 17.16 PointingHere LH straight f0e5 17.36 18.72 PointingHere LH straight f0e6 18.72 19.24 Erruptive 2H curved f0e7 19.24 20.32 ProgressiveSym 2H curved ... ... ...
Element-by-attribute matrix
![Page 58: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/58.jpg)
WEKA Export
WEKA – Decision Trees – SVM Classifiers – Rule Learning – …
Format: ARFF
@relation gesture.phrase
@attribute handedness { none, LH, RH, 2H } @attribute path { none, straight, curved } @attribute handshape { none, open-flat-relax, open-flat-relax-thumb, open-flat-tense, ... }
@data 2H, straight, open-flat-relax LH, straight, open-flat-relax LH, curved, open-flat-relax LH, straight, open-flat-tense …
ANVIL 5
![Page 59: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/59.jpg)
More ANVIL 5 Features
![Page 60: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/60.jpg)
Online Manual Inside
ANVIL 5
![Page 61: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/61.jpg)
Keyboard Shortcuts
Drastic speed-up in coding! Shortcuts for placing start / end
(F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories
– track-specific! – e.g. p=preparation, s=stroke, r=retraction
ANVIL 5
DEMO, JL
![Page 62: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/62.jpg)
Database Integration
Annotation mapped to temporary SQL DB – Track = table – Annotation element = row – Attribute = column
SQL queries for track elements (by attribute) – [ mytrack , ( att1 = 2H OR att1 = LH ) AND anotherAtt <> null ]
SQL query for temporal relation: – R[overlaps, .8] [ firstTrack, hand = 2H ] [ otherTrack, hand <> null ]
ANVIL 5
restricts track A restricts track B temporal relation
![Page 63: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/63.jpg)
Mocap Viewer
Accepts data in BVH and ASF/AMC – ASF: skeleton morphology – AMC: motion data
Java3D Synchronization with video possible Allows full 3D view on person 3D traces (curves) of joints
– position (x / y / z) – velocity (x / y / z / total) – acceleration (x / y / z / total)
ANVIL 5
![Page 64: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/64.jpg)
Mocap Viewer
ANVIL 5
DEMO
![Page 65: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/65.jpg)
Motion capture
![Page 66: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/66.jpg)
Skeleton-based annotation
Idea – add 3D key poses – Interpolate in-betweens
Potentials – animation – motion templates – motion analysis
ANVIL 5+ ?
![Page 67: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/67.jpg)
Gantool
![Page 68: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/68.jpg)
User participation
![Page 69: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/69.jpg)
ANVIL and ELAN
ANVIL: user interface + many features ELAN: stable, many videos formats supported Use best of both:
– import ELAN files to ANVIL Why?
– kappa statistics on multiple files – search over multiple files – export for WEKA – add 2D annotations – mocap annotation
Future: ELAN exporter
![Page 70: Introduction to ANVIL - MoDyCocorpus.modyco.fr/tutoriels/kipp-Anvil.pdf · Introduction to ANVIL . ... (F1 + F2) (on a Mac: 1 + 2) Shortcuts for categories ... (Exmaralda-like) –](https://reader031.vdocuments.us/reader031/viewer/2022022614/5b9ff67909d3f2385c8c9eb4/html5/thumbnails/70.jpg)
Future Work
Semi-automatic coding: exploit motion capture – determine handedness automatically
(simple rule, 83% correct, 269 items) – detect movement phases (preparation, stroke...)
Sign language transcription – sentence-oriented transcription (Exmaralda-like) – connection to animation
Interoperability – lessons from import/export – towards a unified European framework
(ELAN + Exmaralda + ANVIL + ... )