computer vision - local-features-tutorial.github.io · differentiable computer vision an...
TRANSCRIPT
![Page 1: computer vision - local-features-tutorial.github.io · differentiable computer vision an introduction to kornia Edgar Riba Open Source Vision Foundation - OpenCV.org Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fcb0b764f5e1555df3e8d59/html5/thumbnails/1.jpg)
differentiable computer visionan introduction to kornia
Edgar RibaOpen Source Vision Foundation - OpenCV.orgComputer Vision Center (CVC-UAB) - Institut de Robotica Industrial (CSIC-UPC)
![Page 2: computer vision - local-features-tutorial.github.io · differentiable computer vision an introduction to kornia Edgar Riba Open Source Vision Foundation - OpenCV.org Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fcb0b764f5e1555df3e8d59/html5/thumbnails/2.jpg)
Q: > Where is Classical Computer Vision in Deep Learning ? A: > ...
![Page 3: computer vision - local-features-tutorial.github.io · differentiable computer vision an introduction to kornia Edgar Riba Open Source Vision Foundation - OpenCV.org Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fcb0b764f5e1555df3e8d59/html5/thumbnails/3.jpg)
Q: > Where is Classical Computer Vision in Deep Learning ? A: > Black hole area
![Page 5: computer vision - local-features-tutorial.github.io · differentiable computer vision an introduction to kornia Edgar Riba Open Source Vision Foundation - OpenCV.org Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fcb0b764f5e1555df3e8d59/html5/thumbnails/5.jpg)
+2400 stars
+260 forks
+50 contributors
Apache 2 Licence
wiki
![Page 6: computer vision - local-features-tutorial.github.io · differentiable computer vision an introduction to kornia Edgar Riba Open Source Vision Foundation - OpenCV.org Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fcb0b764f5e1555df3e8d59/html5/thumbnails/6.jpg)
1. Differentiable
2. Transparent API
3. Parallel Programming
4. Distributed
5. Production ➝ JIT
Core features
![Page 7: computer vision - local-features-tutorial.github.io · differentiable computer vision an introduction to kornia Edgar Riba Open Source Vision Foundation - OpenCV.org Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fcb0b764f5e1555df3e8d59/html5/thumbnails/7.jpg)
Core features
![Page 8: computer vision - local-features-tutorial.github.io · differentiable computer vision an introduction to kornia Edgar Riba Open Source Vision Foundation - OpenCV.org Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fcb0b764f5e1555df3e8d59/html5/thumbnails/8.jpg)
Basic functionality
Data augmentation
Image enhancement
Color space conversions
2D feature detection
Image filtering
Edge detection
Geometric transformations
3D geometry
Vision loss functions
![Page 9: computer vision - local-features-tutorial.github.io · differentiable computer vision an introduction to kornia Edgar Riba Open Source Vision Foundation - OpenCV.org Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fcb0b764f5e1555df3e8d59/html5/thumbnails/9.jpg)
Data augmentation
- Random sampling using torch.distributions
- Compatible with torchvision
- Batched, GPU
- Return and chain spatial transforms
![Page 10: computer vision - local-features-tutorial.github.io · differentiable computer vision an introduction to kornia Edgar Riba Open Source Vision Foundation - OpenCV.org Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fcb0b764f5e1555df3e8d59/html5/thumbnails/10.jpg)
Image Enhancement
Gamma
- Image tensors normalization
- ZCA mean/whiten
- Image Histogram 1d/2d
- contrast, brightness, gamma, hue, saturation
![Page 11: computer vision - local-features-tutorial.github.io · differentiable computer vision an introduction to kornia Edgar Riba Open Source Vision Foundation - OpenCV.org Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fcb0b764f5e1555df3e8d59/html5/thumbnails/11.jpg)
RGB to Grayscale
Color space conversions
- RGB, RGBA, Grayscale
- HSV, HLS
- Luv, Lab
- XYZ, YCbCr, Yuv
![Page 12: computer vision - local-features-tutorial.github.io · differentiable computer vision an introduction to kornia Edgar Riba Open Source Vision Foundation - OpenCV.org Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fcb0b764f5e1555df3e8d59/html5/thumbnails/12.jpg)
2D feature detection- Harris, Hessian, DoG
- Scale Space framework
- NMS , ConvSoftMax2d/3d
- Local Affine Frames (LAF)
- Differentiable SIFT, Deep descriptors
![Page 13: computer vision - local-features-tutorial.github.io · differentiable computer vision an introduction to kornia Edgar Riba Open Source Vision Foundation - OpenCV.org Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fcb0b764f5e1555df3e8d59/html5/thumbnails/13.jpg)
Image filtering
Gaussian Blur
- Filter 2D / 3D
- Kernels API: gaussian, laplacian, sobel
- Blurring: median, box, gaussian, motion
![Page 15: computer vision - local-features-tutorial.github.io · differentiable computer vision an introduction to kornia Edgar Riba Open Source Vision Foundation - OpenCV.org Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fcb0b764f5e1555df3e8d59/html5/thumbnails/15.jpg)
Geometric transformations
- Rotate, translate, scale, shear, resize
- Gaussian pyramid, PyUp, PyrDown
- Crop: center crop, crop and resize
- Flip: horizontal/vertical, rot180
![Page 16: computer vision - local-features-tutorial.github.io · differentiable computer vision an introduction to kornia Edgar Riba Open Source Vision Foundation - OpenCV.org Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fcb0b764f5e1555df3e8d59/html5/thumbnails/16.jpg)
Geometric transformations
- Rotate, translate, scale, shear, resize
- Gaussian pyramid, PyUp, PyrDown
- Crop: center crop, crop and resize
- Flip: horizontal/vertical, rot180
![Page 17: computer vision - local-features-tutorial.github.io · differentiable computer vision an introduction to kornia Edgar Riba Open Source Vision Foundation - OpenCV.org Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fcb0b764f5e1555df3e8d59/html5/thumbnails/17.jpg)
Geometric transformations
- Rotate, translate, scale, shear, resize
- Gaussian pyramid, PyUp, PyrDown
- Crop: center crop, crop and resize
- Flip: horizontal/vertical, rot180
![Page 18: computer vision - local-features-tutorial.github.io · differentiable computer vision an introduction to kornia Edgar Riba Open Source Vision Foundation - OpenCV.org Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fcb0b764f5e1555df3e8d59/html5/thumbnails/18.jpg)
Geometric transformations
- Rotate, translate, scale, shear, resize
- Gaussian pyramid, PyUp, PyrDown
- Crop: center crop, crop and resize
- Flip: horizontal/vertical, rot180
![Page 19: computer vision - local-features-tutorial.github.io · differentiable computer vision an introduction to kornia Edgar Riba Open Source Vision Foundation - OpenCV.org Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fcb0b764f5e1555df3e8d59/html5/thumbnails/19.jpg)
Geometric transformations
Warp Affine
Warp Perspective
- warp_affine
- warp_perspective
- get_perspective_transform
- get_rotation_matrix2d
![Page 20: computer vision - local-features-tutorial.github.io · differentiable computer vision an introduction to kornia Edgar Riba Open Source Vision Foundation - OpenCV.org Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fcb0b764f5e1555df3e8d59/html5/thumbnails/20.jpg)
3D Geometry- Pinhole and perspective camera API
- Conversions: homogeneous, euclidean, rotation matrix, quaternion, axis-angle, normalize_coordinates
- Subpixel: conv/soft_softargmax2d/3d, conv_quad_interp3d
- Epipolar, Lie algebra and SfM utilities
![Page 21: computer vision - local-features-tutorial.github.io · differentiable computer vision an introduction to kornia Edgar Riba Open Source Vision Foundation - OpenCV.org Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fcb0b764f5e1555df3e8d59/html5/thumbnails/21.jpg)
Vision loss functionsTotal Variation
SSIM, PSNR
Focal Loss
…
Specific Loss functions
- Image reconstruction- Semantic segmentation- Heatmaps
![Page 22: computer vision - local-features-tutorial.github.io · differentiable computer vision an introduction to kornia Edgar Riba Open Source Vision Foundation - OpenCV.org Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fcb0b764f5e1555df3e8d59/html5/thumbnails/22.jpg)
1. Easy to install
2. Easy to use
3. OpenCV syntax
4. Ecosystem Integrationdevelopment
![Page 23: computer vision - local-features-tutorial.github.io · differentiable computer vision an introduction to kornia Edgar Riba Open Source Vision Foundation - OpenCV.org Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fcb0b764f5e1555df3e8d59/html5/thumbnails/23.jpg)
1. Easy to install- Install from pip or source
kornia is dependency-free - ONLY PyTorch
From pip:
pip install kornia
From source:
python setup.py install
![Page 24: computer vision - local-features-tutorial.github.io · differentiable computer vision an introduction to kornia Edgar Riba Open Source Vision Foundation - OpenCV.org Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fcb0b764f5e1555df3e8d59/html5/thumbnails/24.jpg)
1. Easy to install
2. Easy to use
3. OpenCV syntax
4. Ecosystem Integrationdevelopment
![Page 25: computer vision - local-features-tutorial.github.io · differentiable computer vision an introduction to kornia Edgar Riba Open Source Vision Foundation - OpenCV.org Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fcb0b764f5e1555df3e8d59/html5/thumbnails/25.jpg)
2. Easy to import and use- Import from any Python >= 3.6 script- Compatible with any torch.Tensor operator
![Page 26: computer vision - local-features-tutorial.github.io · differentiable computer vision an introduction to kornia Edgar Riba Open Source Vision Foundation - OpenCV.org Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fcb0b764f5e1555df3e8d59/html5/thumbnails/26.jpg)
1. Easy to install
2. Easy to use
3. OpenCV syntax
4. Ecosystem Integrationdevelopment
![Page 29: computer vision - local-features-tutorial.github.io · differentiable computer vision an introduction to kornia Edgar Riba Open Source Vision Foundation - OpenCV.org Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fcb0b764f5e1555df3e8d59/html5/thumbnails/29.jpg)
1. Easy to install
2. Easy to use
3. OpenCV syntax
4. Ecosystem Integrationdevelopment
![Page 31: computer vision - local-features-tutorial.github.io · differentiable computer vision an introduction to kornia Edgar Riba Open Source Vision Foundation - OpenCV.org Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fcb0b764f5e1555df3e8d59/html5/thumbnails/31.jpg)
3. Compatibility
![Page 32: computer vision - local-features-tutorial.github.io · differentiable computer vision an introduction to kornia Edgar Riba Open Source Vision Foundation - OpenCV.org Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fcb0b764f5e1555df3e8d59/html5/thumbnails/32.jpg)
3. Compatibility
![Page 33: computer vision - local-features-tutorial.github.io · differentiable computer vision an introduction to kornia Edgar Riba Open Source Vision Foundation - OpenCV.org Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fcb0b764f5e1555df3e8d59/html5/thumbnails/33.jpg)
Future plans
![Page 34: computer vision - local-features-tutorial.github.io · differentiable computer vision an introduction to kornia Edgar Riba Open Source Vision Foundation - OpenCV.org Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fcb0b764f5e1555df3e8d59/html5/thumbnails/34.jpg)
We need your help and feedback !
3D transformations
TorchScriptable
Structure From Motion
![Page 35: computer vision - local-features-tutorial.github.io · differentiable computer vision an introduction to kornia Edgar Riba Open Source Vision Foundation - OpenCV.org Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fcb0b764f5e1555df3e8d59/html5/thumbnails/35.jpg)
User chat for Q&A:
https://discuss.pytorch.org/c/vision/kornia
Official documents:
https://kornia.readthedocs.io/en/latest
For users
![Page 36: computer vision - local-features-tutorial.github.io · differentiable computer vision an introduction to kornia Edgar Riba Open Source Vision Foundation - OpenCV.org Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fcb0b764f5e1555df3e8d59/html5/thumbnails/36.jpg)
Check out our contributions call
https://github.com/kornia/kornia/issues/53
or
Check out docs, and our issues marked as “contributions welcome”:
https://github.com/kornia/kornia/blob/master/CONTRIBUTING.rst
For developers
![Page 37: computer vision - local-features-tutorial.github.io · differentiable computer vision an introduction to kornia Edgar Riba Open Source Vision Foundation - OpenCV.org Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fcb0b764f5e1555df3e8d59/html5/thumbnails/37.jpg)
Examples:
https://github.com/kornia/kornia-examples
Tutorials:
https://kornia.readthedocs.io/en/latest/tutorials/index.html
Extra material
![Page 38: computer vision - local-features-tutorial.github.io · differentiable computer vision an introduction to kornia Edgar Riba Open Source Vision Foundation - OpenCV.org Computer Vision](https://reader036.vdocuments.us/reader036/viewer/2022081523/5fcb0b764f5e1555df3e8d59/html5/thumbnails/38.jpg)
code: https://github.com/kornia/kornia
twitter: @kornia_foss
www.kornia.org