taras chaykivskyy - computer vision in front-end
TRANSCRIPT
eleks.com
Computer VisionIn Front-End (only)by Taras Chaykivskyy
Q&ABut js is slow … ?
Why ?Cross-platformNo installation
No huge video trafficWorks offline (cached)
JavaScript CV instrumentsARToolkit.jsARUco.jsTracking.jsJSFeat
Results :● QR marker (3 / 30 FPS)● Color QR markers (1 / 5 FPS)● Image matching (0 / 1 FPS)
“Web-Based Augmented Reality:To adopt or not to adopt ?”
* (LG Optimus L5 / Nexus 5 )
Results :● Laptop/Desktop (15-30 FPS)● Smartphone* (5 FPS)
Virtual-Makeup
* Nexus 5
How to boost CV on Web Application ?
*dlib’s implementation**with -O3 optimization flag
SURF* on C++
height 128 256 512
C++** (ms) 0.77 6.2 29.67
asm.jsIs low-level subset of JavaScript.
Emscripten (source-to-source compiler)
clang main.cpp -o mainem++ main.cpp -o mainModule.js
asm.js
C codesize_t strlen(char *ptr) { char *curr = ptr; while (*curr != 0) { curr++; } return (curr − ptr);}
asm.js codefunction strlen(ptr) { ptr = ptr|0; var curr = 0; curr = ptr; while (MEM8[curr]|0 != 0) { curr = (curr + 1)|0; } return (curr − ptr)|0;}
*dlib’s implementation**with -O3 optimization flag
SURF on asm.js
height 128 256 512
C++** (ms) 0.77 6.2 29.67
Asm.js** (ms) 3.5 21.5 104
Call js from C++ code
Run js code from c++#include <emscripten.h>
int main() { EM_ASM( alert('hello!'); throw 'all done'; ); return 0;}
Arguments…EM_ASM_({Module.print('I received:'+$0);}, 100);…
WebGLIs a JavaScript API for rendering interactive 3D computer graphics
WebGL
http://webglstats.com/
WebGL
js codevar gl = canvas.getContext("webgl");...var shaderProgram = gl.createProgram();gl.attachShader(shaderProgram, vertexShader);gl.attachShader(shaderProgram, fragmentShader);gl.linkProgram(shaderProgram);gl.useProgram(shaderProgram);...
*dlib’s implementation**with -O3 optimization flag***upper bound
height 128 256 512
C++** (ms) 0.77 6.2 29.67
Asm.js** (ms) 3.5 21.5 104
+WebGL*** (ms)
~15 ~25 ~75
SURF on asm.js + WebGL
What’s next ?
