"leveraging computer vision and machine learning to power the visual commerce revolution,"...
TRANSCRIPT
![Page 1: "Leveraging Computer Vision and Machine Learning to Power the Visual Commerce Revolution," a Presentation from Sight Commerce](https://reader034.vdocuments.us/reader034/viewer/2022052312/55d03605bb61ebc0768b468d/html5/thumbnails/1.jpg)
Transforming
Beauty & Fashionusing Computer Vision & Machine Learning
Satya Mallick, Ph.D.Co-Founder, Sight Commerce
![Page 2: "Leveraging Computer Vision and Machine Learning to Power the Visual Commerce Revolution," a Presentation from Sight Commerce](https://reader034.vdocuments.us/reader034/viewer/2022052312/55d03605bb61ebc0768b468d/html5/thumbnails/2.jpg)
Nothing is as powerful as an idea whose time has come
![Page 3: "Leveraging Computer Vision and Machine Learning to Power the Visual Commerce Revolution," a Presentation from Sight Commerce](https://reader034.vdocuments.us/reader034/viewer/2022052312/55d03605bb61ebc0768b468d/html5/thumbnails/3.jpg)
Vision Graphics Learning
Beauty & Fashion
DISRUPT
![Page 4: "Leveraging Computer Vision and Machine Learning to Power the Visual Commerce Revolution," a Presentation from Sight Commerce](https://reader034.vdocuments.us/reader034/viewer/2022052312/55d03605bb61ebc0768b468d/html5/thumbnails/4.jpg)
Why?
Pour quoi?
Por quê?
Zergatik?
Per ché?
Warum?
क्यों ?
Почему?
![Page 5: "Leveraging Computer Vision and Machine Learning to Power the Visual Commerce Revolution," a Presentation from Sight Commerce](https://reader034.vdocuments.us/reader034/viewer/2022052312/55d03605bb61ebc0768b468d/html5/thumbnails/5.jpg)
Beauty & Fashion Shopping is Visually Inspired
![Page 6: "Leveraging Computer Vision and Machine Learning to Power the Visual Commerce Revolution," a Presentation from Sight Commerce](https://reader034.vdocuments.us/reader034/viewer/2022052312/55d03605bb61ebc0768b468d/html5/thumbnails/6.jpg)
Intel
• Speed : 3.50 GHz
• Cache : 20MB
• Number of Cores : 4
• Instruction Set : 64-bit
• Power : 140 W
![Page 7: "Leveraging Computer Vision and Machine Learning to Power the Visual Commerce Revolution," a Presentation from Sight Commerce](https://reader034.vdocuments.us/reader034/viewer/2022052312/55d03605bb61ebc0768b468d/html5/thumbnails/7.jpg)
Intel
• Speed : 3.50 GHz
• Cache : 20MB
• Number of Cores : 4
• Instruction Set : 64-bit
• Power : 140 W
Victoria’s Secret
![Page 8: "Leveraging Computer Vision and Machine Learning to Power the Visual Commerce Revolution," a Presentation from Sight Commerce](https://reader034.vdocuments.us/reader034/viewer/2022052312/55d03605bb61ebc0768b468d/html5/thumbnails/8.jpg)
Intel
• Speed : 3.50 GHz
• Cache : 20MB
• Number of Cores : 4
• Instruction Set : 64-bit
• Power : 140 W
Victoria’s Secret
![Page 9: "Leveraging Computer Vision and Machine Learning to Power the Visual Commerce Revolution," a Presentation from Sight Commerce](https://reader034.vdocuments.us/reader034/viewer/2022052312/55d03605bb61ebc0768b468d/html5/thumbnails/9.jpg)
Help create Beautiful Imagery
![Page 10: "Leveraging Computer Vision and Machine Learning to Power the Visual Commerce Revolution," a Presentation from Sight Commerce](https://reader034.vdocuments.us/reader034/viewer/2022052312/55d03605bb61ebc0768b468d/html5/thumbnails/10.jpg)
Communication is Visual
1.8B Photos / Day
![Page 11: "Leveraging Computer Vision and Machine Learning to Power the Visual Commerce Revolution," a Presentation from Sight Commerce](https://reader034.vdocuments.us/reader034/viewer/2022052312/55d03605bb61ebc0768b468d/html5/thumbnails/11.jpg)
Communication is Visual
![Page 12: "Leveraging Computer Vision and Machine Learning to Power the Visual Commerce Revolution," a Presentation from Sight Commerce](https://reader034.vdocuments.us/reader034/viewer/2022052312/55d03605bb61ebc0768b468d/html5/thumbnails/12.jpg)
Communication is Visual
![Page 13: "Leveraging Computer Vision and Machine Learning to Power the Visual Commerce Revolution," a Presentation from Sight Commerce](https://reader034.vdocuments.us/reader034/viewer/2022052312/55d03605bb61ebc0768b468d/html5/thumbnails/13.jpg)
Image data is abundant!
![Page 14: "Leveraging Computer Vision and Machine Learning to Power the Visual Commerce Revolution," a Presentation from Sight Commerce](https://reader034.vdocuments.us/reader034/viewer/2022052312/55d03605bb61ebc0768b468d/html5/thumbnails/14.jpg)
Extreme Personalization
of the shopping experience is Inevitable
![Page 15: "Leveraging Computer Vision and Machine Learning to Power the Visual Commerce Revolution," a Presentation from Sight Commerce](https://reader034.vdocuments.us/reader034/viewer/2022052312/55d03605bb61ebc0768b468d/html5/thumbnails/15.jpg)
Extreme Personalization
of the shopping experience is Inevitable
Image based Recommendations
![Page 16: "Leveraging Computer Vision and Machine Learning to Power the Visual Commerce Revolution," a Presentation from Sight Commerce](https://reader034.vdocuments.us/reader034/viewer/2022052312/55d03605bb61ebc0768b468d/html5/thumbnails/16.jpg)
CVML @Sight Commerce • Virtual Makeover
• Jewelry try-on
• Virtual Nails
• Virtual Clothing for ecommerce
• Product recommendation based on facial analysis
• Apparel recommendation based on photo
![Page 17: "Leveraging Computer Vision and Machine Learning to Power the Visual Commerce Revolution," a Presentation from Sight Commerce](https://reader034.vdocuments.us/reader034/viewer/2022052312/55d03605bb61ebc0768b468d/html5/thumbnails/17.jpg)
Photo Enhancement
![Page 18: "Leveraging Computer Vision and Machine Learning to Power the Visual Commerce Revolution," a Presentation from Sight Commerce](https://reader034.vdocuments.us/reader034/viewer/2022052312/55d03605bb61ebc0768b468d/html5/thumbnails/18.jpg)
• Lipstick • Lipgloss • Lipliner • Blush • Bronzer • Foundation • Concealer • Eyeliner • Macara • Eyeshadow • Contacts
Virtual Makeover
![Page 19: "Leveraging Computer Vision and Machine Learning to Power the Visual Commerce Revolution," a Presentation from Sight Commerce](https://reader034.vdocuments.us/reader034/viewer/2022052312/55d03605bb61ebc0768b468d/html5/thumbnails/19.jpg)
• Lipstick • Lipgloss • Lipliner • Blush • Bronzer • Foundation • Concealer • Eyeliner • Macara • Eyeshadow • Contacts
Virtual Makeover
![Page 20: "Leveraging Computer Vision and Machine Learning to Power the Visual Commerce Revolution," a Presentation from Sight Commerce](https://reader034.vdocuments.us/reader034/viewer/2022052312/55d03605bb61ebc0768b468d/html5/thumbnails/20.jpg)
![Page 21: "Leveraging Computer Vision and Machine Learning to Power the Visual Commerce Revolution," a Presentation from Sight Commerce](https://reader034.vdocuments.us/reader034/viewer/2022052312/55d03605bb61ebc0768b468d/html5/thumbnails/21.jpg)
User Uploads a Photo
![Page 22: "Leveraging Computer Vision and Machine Learning to Power the Visual Commerce Revolution," a Presentation from Sight Commerce](https://reader034.vdocuments.us/reader034/viewer/2022052312/55d03605bb61ebc0768b468d/html5/thumbnails/22.jpg)
User Uploads a Photo
Facial Feature Detection1. Active Appearance Model2. Supervised Descent Method3. Deep Learning based
![Page 23: "Leveraging Computer Vision and Machine Learning to Power the Visual Commerce Revolution," a Presentation from Sight Commerce](https://reader034.vdocuments.us/reader034/viewer/2022052312/55d03605bb61ebc0768b468d/html5/thumbnails/23.jpg)
User Uploads a Photo
Facial Feature Detection1. Active Appearance Model2. Supervised Descent Method3. Deep Learning based
Head Pose Estimation Facial features used to estimate pose relative to a canonical 3D shape
![Page 24: "Leveraging Computer Vision and Machine Learning to Power the Visual Commerce Revolution," a Presentation from Sight Commerce](https://reader034.vdocuments.us/reader034/viewer/2022052312/55d03605bb61ebc0768b468d/html5/thumbnails/24.jpg)
User Uploads a Photo
Facial Feature Detection1. Active Appearance Model2. Supervised Descent Method3. Deep Learning based
Head Pose Estimation Facial features used to estimate pose relative to a canonical 3D shape
Facial Contour Estimation Optimization problem based on detected landmarks and image gradient
![Page 25: "Leveraging Computer Vision and Machine Learning to Power the Visual Commerce Revolution," a Presentation from Sight Commerce](https://reader034.vdocuments.us/reader034/viewer/2022052312/55d03605bb61ebc0768b468d/html5/thumbnails/25.jpg)
User Uploads a Photo
Facial Feature Detection1. Active Appearance Model2. Supervised Descent Method3. Deep Learning based
Head Pose Estimation Facial features used to estimate pose relative to a canonical 3D shape
Facial Contour Estimation Optimization problem based on detected landmarks and image gradient
Photo Quality Estimation 1. Noise 2. Illumination
![Page 26: "Leveraging Computer Vision and Machine Learning to Power the Visual Commerce Revolution," a Presentation from Sight Commerce](https://reader034.vdocuments.us/reader034/viewer/2022052312/55d03605bb61ebc0768b468d/html5/thumbnails/26.jpg)
User Uploads a Photo
Facial Feature Detection1. Active Appearance Model2. Supervised Descent Method3. Deep Learning based
Head Pose Estimation Facial features used to estimate pose relative to a canonical 3D shape
Facial Contour Estimation Optimization problem based on detected landmarks and image gradient
Photo Quality Estimation 1. Noise 2. Illumination
Skin Detection Samples of skin taken from detected face region and a matting problem is set up.
![Page 27: "Leveraging Computer Vision and Machine Learning to Power the Visual Commerce Revolution," a Presentation from Sight Commerce](https://reader034.vdocuments.us/reader034/viewer/2022052312/55d03605bb61ebc0768b468d/html5/thumbnails/27.jpg)
User Uploads a Photo
Facial Feature Detection1. Active Appearance Model2. Supervised Descent Method3. Deep Learning based
Head Pose Estimation Facial features used to estimate pose relative to a canonical 3D shape
Facial Contour Estimation Optimization problem based on detected landmarks and image gradient
Photo Quality Estimation 1. Noise 2. Illumination
Skin Detection Samples of skin taken from detected face region and a matting problem is set up.
Render Makeup1.Specular diffuse separation2.Illumination preserving rendering that
approximates BRDF of makeup / skin.
![Page 28: "Leveraging Computer Vision and Machine Learning to Power the Visual Commerce Revolution," a Presentation from Sight Commerce](https://reader034.vdocuments.us/reader034/viewer/2022052312/55d03605bb61ebc0768b468d/html5/thumbnails/28.jpg)
User Uploads a Photo
Facial Feature Detection1. Active Appearance Model2. Supervised Descent Method3. Deep Learning based
Head Pose Estimation Facial features used to estimate pose relative to a canonical 3D shape
Facial Contour Estimation Optimization problem based on detected landmarks and image gradient
Photo Quality Estimation 1. Noise 2. Illumination
Skin Detection Samples of skin taken from detected face region and a matting problem is set up.
Render Makeup1.Specular diffuse separation2.Illumination preserving rendering that
approximates BRDF of makeup / skin.
Hair Segmentation1. Bayesian Matting2. Poisson Matting3. Spectral Matting.
![Page 29: "Leveraging Computer Vision and Machine Learning to Power the Visual Commerce Revolution," a Presentation from Sight Commerce](https://reader034.vdocuments.us/reader034/viewer/2022052312/55d03605bb61ebc0768b468d/html5/thumbnails/29.jpg)
User Uploads a Photo
Facial Feature Detection1. Active Appearance Model2. Supervised Descent Method3. Deep Learning based
Head Pose Estimation Facial features used to estimate pose relative to a canonical 3D shape
Facial Contour Estimation Optimization problem based on detected landmarks and image gradient
Photo Quality Estimation 1. Noise 2. Illumination
Skin Detection Samples of skin taken from detected face region and a matting problem is set up.
Render Makeup1.Specular diffuse separation2.Illumination preserving rendering that
approximates BRDF of makeup / skin.
Hair Segmentation1. Bayesian Matting2. Poisson Matting3. Spectral Matting.
Hair coloring Matching color histograms in the PCA space
![Page 30: "Leveraging Computer Vision and Machine Learning to Power the Visual Commerce Revolution," a Presentation from Sight Commerce](https://reader034.vdocuments.us/reader034/viewer/2022052312/55d03605bb61ebc0768b468d/html5/thumbnails/30.jpg)
Automatic Facial Feature Detection
![Page 31: "Leveraging Computer Vision and Machine Learning to Power the Visual Commerce Revolution," a Presentation from Sight Commerce](https://reader034.vdocuments.us/reader034/viewer/2022052312/55d03605bb61ebc0768b468d/html5/thumbnails/31.jpg)
Interactive Alpha Matting
![Page 32: "Leveraging Computer Vision and Machine Learning to Power the Visual Commerce Revolution," a Presentation from Sight Commerce](https://reader034.vdocuments.us/reader034/viewer/2022052312/55d03605bb61ebc0768b468d/html5/thumbnails/32.jpg)
Interactive Alpha Matting
![Page 33: "Leveraging Computer Vision and Machine Learning to Power the Visual Commerce Revolution," a Presentation from Sight Commerce](https://reader034.vdocuments.us/reader034/viewer/2022052312/55d03605bb61ebc0768b468d/html5/thumbnails/33.jpg)
Interactive Alpha Matting
![Page 34: "Leveraging Computer Vision and Machine Learning to Power the Visual Commerce Revolution," a Presentation from Sight Commerce](https://reader034.vdocuments.us/reader034/viewer/2022052312/55d03605bb61ebc0768b468d/html5/thumbnails/34.jpg)
Virtual Hairstyle
![Page 35: "Leveraging Computer Vision and Machine Learning to Power the Visual Commerce Revolution," a Presentation from Sight Commerce](https://reader034.vdocuments.us/reader034/viewer/2022052312/55d03605bb61ebc0768b468d/html5/thumbnails/35.jpg)
Virtual Makeover for E-Commerce
![Page 36: "Leveraging Computer Vision and Machine Learning to Power the Visual Commerce Revolution," a Presentation from Sight Commerce](https://reader034.vdocuments.us/reader034/viewer/2022052312/55d03605bb61ebc0768b468d/html5/thumbnails/36.jpg)
Product images are not enough!
![Page 37: "Leveraging Computer Vision and Machine Learning to Power the Visual Commerce Revolution," a Presentation from Sight Commerce](https://reader034.vdocuments.us/reader034/viewer/2022052312/55d03605bb61ebc0768b468d/html5/thumbnails/37.jpg)
Compelling Imagery Sells Products
![Page 38: "Leveraging Computer Vision and Machine Learning to Power the Visual Commerce Revolution," a Presentation from Sight Commerce](https://reader034.vdocuments.us/reader034/viewer/2022052312/55d03605bb61ebc0768b468d/html5/thumbnails/38.jpg)
Create a Look
![Page 39: "Leveraging Computer Vision and Machine Learning to Power the Visual Commerce Revolution," a Presentation from Sight Commerce](https://reader034.vdocuments.us/reader034/viewer/2022052312/55d03605bb61ebc0768b468d/html5/thumbnails/39.jpg)
Create a look
![Page 40: "Leveraging Computer Vision and Machine Learning to Power the Visual Commerce Revolution," a Presentation from Sight Commerce](https://reader034.vdocuments.us/reader034/viewer/2022052312/55d03605bb61ebc0768b468d/html5/thumbnails/40.jpg)
10 models x 100 lip products x 100 eye products x 10 blush
![Page 41: "Leveraging Computer Vision and Machine Learning to Power the Visual Commerce Revolution," a Presentation from Sight Commerce](https://reader034.vdocuments.us/reader034/viewer/2022052312/55d03605bb61ebc0768b468d/html5/thumbnails/41.jpg)
1,000,00010 models x 100 lip products x 100 eye products x 10 blush
![Page 42: "Leveraging Computer Vision and Machine Learning to Power the Visual Commerce Revolution," a Presentation from Sight Commerce](https://reader034.vdocuments.us/reader034/viewer/2022052312/55d03605bb61ebc0768b468d/html5/thumbnails/42.jpg)
1,000,00010 models x 100 lip products x 100 eye products x 10 blush
![Page 43: "Leveraging Computer Vision and Machine Learning to Power the Visual Commerce Revolution," a Presentation from Sight Commerce](https://reader034.vdocuments.us/reader034/viewer/2022052312/55d03605bb61ebc0768b468d/html5/thumbnails/43.jpg)
![Page 44: "Leveraging Computer Vision and Machine Learning to Power the Visual Commerce Revolution," a Presentation from Sight Commerce](https://reader034.vdocuments.us/reader034/viewer/2022052312/55d03605bb61ebc0768b468d/html5/thumbnails/44.jpg)
Sight Commerce Works
![Page 45: "Leveraging Computer Vision and Machine Learning to Power the Visual Commerce Revolution," a Presentation from Sight Commerce](https://reader034.vdocuments.us/reader034/viewer/2022052312/55d03605bb61ebc0768b468d/html5/thumbnails/45.jpg)
Beyond Makeup
![Page 46: "Leveraging Computer Vision and Machine Learning to Power the Visual Commerce Revolution," a Presentation from Sight Commerce](https://reader034.vdocuments.us/reader034/viewer/2022052312/55d03605bb61ebc0768b468d/html5/thumbnails/46.jpg)
Virtual Nails
![Page 47: "Leveraging Computer Vision and Machine Learning to Power the Visual Commerce Revolution," a Presentation from Sight Commerce](https://reader034.vdocuments.us/reader034/viewer/2022052312/55d03605bb61ebc0768b468d/html5/thumbnails/47.jpg)
Virtual Sunglasses
![Page 48: "Leveraging Computer Vision and Machine Learning to Power the Visual Commerce Revolution," a Presentation from Sight Commerce](https://reader034.vdocuments.us/reader034/viewer/2022052312/55d03605bb61ebc0768b468d/html5/thumbnails/48.jpg)
Virtual Jewelry
![Page 49: "Leveraging Computer Vision and Machine Learning to Power the Visual Commerce Revolution," a Presentation from Sight Commerce](https://reader034.vdocuments.us/reader034/viewer/2022052312/55d03605bb61ebc0768b468d/html5/thumbnails/49.jpg)
Virtual Clothing
![Page 50: "Leveraging Computer Vision and Machine Learning to Power the Visual Commerce Revolution," a Presentation from Sight Commerce](https://reader034.vdocuments.us/reader034/viewer/2022052312/55d03605bb61ebc0768b468d/html5/thumbnails/50.jpg)
Virtual Accessories
![Page 51: "Leveraging Computer Vision and Machine Learning to Power the Visual Commerce Revolution," a Presentation from Sight Commerce](https://reader034.vdocuments.us/reader034/viewer/2022052312/55d03605bb61ebc0768b468d/html5/thumbnails/51.jpg)
Image Based Recommendation
“35 percent of product sales result from recommendations” — Amazon
![Page 52: "Leveraging Computer Vision and Machine Learning to Power the Visual Commerce Revolution," a Presentation from Sight Commerce](https://reader034.vdocuments.us/reader034/viewer/2022052312/55d03605bb61ebc0768b468d/html5/thumbnails/52.jpg)
Photo Analysis + Shopping Data =
Extremely Personalized Recommendation
![Page 53: "Leveraging Computer Vision and Machine Learning to Power the Visual Commerce Revolution," a Presentation from Sight Commerce](https://reader034.vdocuments.us/reader034/viewer/2022052312/55d03605bb61ebc0768b468d/html5/thumbnails/53.jpg)
User Photo Analysis• Skin Tone • Eye Color • Lip Color • Hair Color • Ethnicity • Face Shape • Wearing Glasses ?
![Page 54: "Leveraging Computer Vision and Machine Learning to Power the Visual Commerce Revolution," a Presentation from Sight Commerce](https://reader034.vdocuments.us/reader034/viewer/2022052312/55d03605bb61ebc0768b468d/html5/thumbnails/54.jpg)
Color & Pattern based Apparel Recommendation
![Page 55: "Leveraging Computer Vision and Machine Learning to Power the Visual Commerce Revolution," a Presentation from Sight Commerce](https://reader034.vdocuments.us/reader034/viewer/2022052312/55d03605bb61ebc0768b468d/html5/thumbnails/55.jpg)
Wish List Face Processor
Landmark detector Skin detection
Color MeasurementInexpensive
Pattern Classification
![Page 56: "Leveraging Computer Vision and Machine Learning to Power the Visual Commerce Revolution," a Presentation from Sight Commerce](https://reader034.vdocuments.us/reader034/viewer/2022052312/55d03605bb61ebc0768b468d/html5/thumbnails/56.jpg)
Sight Commerce
![Page 57: "Leveraging Computer Vision and Machine Learning to Power the Visual Commerce Revolution," a Presentation from Sight Commerce](https://reader034.vdocuments.us/reader034/viewer/2022052312/55d03605bb61ebc0768b468d/html5/thumbnails/57.jpg)
Color & Pattern based Apparel Recommendation