multi-touch navigation engine
DESCRIPTION
Multi-Touch Navigation Engine. Detailed Design Review. Presented by Team Extra Touch: Chris Jones Shuopeng Yuan Nathan Wiedeback. 1. The Plan. Make it interesting…. Challenge. Solution. Compare to previous frame Low resolution (240 x 180) for proof-of-concept - PowerPoint PPT PresentationTRANSCRIPT
Multi-Touch Navigation Engine
Presented by Team Extra Touch:Chris Jones
Shuopeng YuanNathan Wiedeback
Detailed Design Review
1
THE PLAN
2
Make it interesting…
Challenge• Phantom touches• “[40K] ought to be enough
for anybody!”• Need “tap points” for
intermediate data• We’re not really
programmers…
Solution• Compare to previous frame• Low resolution (240 x 180)
for proof-of-concept• Make each stage’s data
available at the output• Get help!
3
Serial Input
4
Serial Stream Parser
•74 byte “packets” from the screen
•Interrupt-driven I/O on ARM -- until an entire packet received•Then process it to get 32 sensor values, average, etc.
•Normalize by subtracting the lowest value from each •Variations of ~100-200
5
(Little-endian)Average: 0x488B = 18,571First sensor: 0x4886 = 18,566
Interpolation
6
Interpolator
•Input: 32 x 2-byte sensor levels•Output: “Image” frame -- n x m x 1 bit pixel array
•Proof-of-concept: 240 x 180 (memory constraints)
•Find intersection points of sensors (16 x 16)•Interpolate linearly based on distance – first in X, then in Y (bilinear)•Set a threshold and discretize at the end
•Each pixel ends up as a 1 or a 0
7
Image source: Wikipedia
Phantom Filtering
8
Phantom Filter
•Input: Frame•Output: Filtered frame (240 x 180 x 1 bit)
•Intersections method -> phantom touches •Remember what was touched first – i.e., save and compare to the last frame•Proof of concept -- only the simple case covered
9
Pattern Recognition
10
Pattern Recognizer: Flowchart
11
Input: Whole frame (60*20)
12
First step: Blob classification
13
(Visio files provided separately)
14
Output: the outline of each single blob
15
The X-Y Coordinates in separate arrays x y
7 312 36 414 45 515 56 615 66 715 76 814 86 914 96 1014 106 1114 117 1213 127 1314 137 1413 147 1513 157 1613 168 1713 179 1812 18
x y
32 334 331 435 431 534 5
x y
39 1140 1138 1241 1239 1341 13
16
Voronoi Skeleton Algorithm 1
17
The block diagram
18
Voronoi Skeleton Algorithm 2 (The ideal case)
19
Voronoi Skeleton Algorithm(The actual result)
6 5 4.5 6.5 2.5 7.5 -0.5 8.5 -4.5 9.5 -9.5 10.5 8 5 9.5 5.5 10 6.5 10.1053 6.65789 10 7.5 9.5 9.5 7.5 10.5 5.5 12.5 3.5 13.5 0.5 14.5 -3.5 15.5 10 8.5 10 12.5 10.0833 13 10 14.5 8.5 15.5 10 13.5 10 9.33333 10.3 15.5 10 14.5 10 14.75 10.3333 15.6667 10.5 16 11.5 16.5 11.75 6 10 8.5 10 9.33333 10.5 9.5 12.5 10.5 14 12 13 13 14.5 14.5 16.5 15.5 19.5 16.5 13.5 5.5 12.5 6 13.5 6.5 15.5 8.5 17.5 9.5 20.5 10.5 26.5 12
20
Shape recognition
21
Slope algorithm
If a skeleton is linear, according to geometry, angle a1 = a2 = a3 = a4 Therefore, only linear shaped skeleton can pass this filter. To the touch panel application, it means only the side of a palm instead of a palm shape can pass.
22
Channel Assignment
23
Channel AssignmentData In: Center coordinates (X,Y)
Processing:Data gets transformed into a 2d array, and the compared against each channel frame to determine its channel. Channel frames are defined before compile.
Data Out:(X,Y) coordinates with channel data attached.
Details:Aiming for 2 channels initiallyShould be simple to add channels in the final iteration.
24
Channel Assignment – Flow Chart
25
Channel Assignment - Visual Representation
0 0 0 0 0
0 0 1 1 0
1 1 0 1 0
1 1 0 0 0
1 1 0 0 0
Channel 0 Frame0 0 1 1 1
0 0 0 0 1
0 0 0 0 1
0 0 1 1 1
0 0 1 1 1
Channel 1 Frame
Channel 2 Frame1 1 0 0 0
1 1 0 0 0
0 0 1 0 0
0 0 0 0 0
0 0 0 0 0
Output to PC: Coordinates 3,3 Channel 2
Input: Coordinate Data (3,3) touched
We have a match at Channel 2!
26
Questions?
27