integrating small data, synthetic data in ai strategy for ... · © 2019 else corp s.r.l. !3...
Post on 22-May-2020
4 Views
Preview:
TRANSCRIPT
© 2019 ELSE Corp S.r.l.
Integrating Small Data, Synthetic Data in AI Strategy for Fashion Retail
a.golub@else-corp.it
Co-Founder & CEO ELSE Corp, a Virtual Retail
company
Andrey Golub
© 2019 ELSE Corp S.r.l. !2
BIG DATA X
FASHION
Today’s Technology Is Shaping Retail’s Future: Predicting fashion trends using machine learning
© 2019 ELSE Corp S.r.l. !3
Marketers Need Artificial Intelligence to Reach the Segment of One
SMALL DATA X
FASHION
Big Data Economy & Challenges
© 2019 ELSE Corp S.r.l. !5
(Big) Data Annotation Economy
The global market size is expected to reach US$ 1.6 billion by 2025.
Grand View Research
As companies are increasingly embracing artificial intelligence, a growing industry is emerging in which workers are employed to 'train' algorithms to recognize various types of data.
© 2019 ELSE Corp S.r.l. !6
(Big) Data Annotation Economy
“The largest technology companies don’t want to be in the business of training data, they want to own customer relationships [and] are using partners and procurement wisely,” Le i la Janah , founder and CEO o f Samasource
© 2019 ELSE Corp S.r.l. !7
(Big) Data Annotation Economy
Big Data challenge: Garbage In, Garbage Out
- Data labelers are sometimes referred to as “AI’s workforce” or the “Invisible workers of the AI era”
- A McKinsey report from 2018 listed data labelling as the biggest obstacle to AI adoption in the industry
- The data labelling industry spreads globally, specially in lower-income countries
- Teams are capable to annotate 10,000 images in eight days with 95% of accuracy (for a single task)
- An hour of video takes 8 hours to annotate
•Pixel - Wise Semantic Annotation•3D Semantic Annotation•Pixel - Wise Object Instance Annotation•Fine - Grained Road Segmentation•Moving Object Trajectory•High - Precision GPS / IMO Information
Managed Data Labelling Services
© 2019 ELSE Corp S.r.l. !8
Why Small Data is Important for Advancing AI
Small data refers to data that humans can observe and process personally.
One individual’s CV is small data. A database with millions of CVs is big data.
© 2019 ELSE Corp S.r.l. !9
Unprecedented Personalisation with Small Data AI
1
Product fit is typically elaborated through simulations. Individualised simulation cannot be achieved through big data, the simulation model needs to incorporate customer feedback on an individual basis. This is a small data job.
Big Data 2 Algorithm 3 Algorithm
6 Small Data 5 Big Data 4 Big Data
TightGood
© 2019 ELSE Corp S.r.l. !10
Unprecedented Personalisation with Small Data AI
1 Big Data
© 2019 ELSE Corp S.r.l. !11
Unprecedented Personalisation with Small Data AI
2 Algorithm
© 2019 ELSE Corp S.r.l. !12
Unprecedented Personalisation with Small Data AI
3 Algorithm
© 2019 ELSE Corp S.r.l. !13
Unprecedented Personalisation with Small Data AI
4 Big Data
© 2019 ELSE Corp S.r.l. !14
Unprecedented Personalisation with Small Data AI
5 Big Data
© 2019 ELSE Corp S.r.l.
TightGood
!15
Unprecedented Personalisation with Small Data AI
6 Small Data
© 2019 ELSE Corp S.r.l. !17
ELSE Corp, a VIRTUAL RETAIL Company, since 2014
12 patents filled 5 already granted 16 trademarks registered
© 2019 ELSE Corp S.r.l. !18
UVP: End-To-End Digital Ecosystem for Mass Customisation
3D CAD to Cloud PDM
3D Scanning
Virtual Retail Corner
else.shoes™, an end to end industry solution for the footwear sector, enabling Mass Customization, Industrial Made to Measure and On Demand Production.
Cloud & Hybrid ManufacturingVirtual Fitting
3D Photo Realism
© 2019 ELSE Corp S.r.l. !19
CAD-to-Cloud Integration, Automated Cloud Rendering
Cloud Rendering: A tool to bring CAD 3D geometry in Cloud, via CloudPDM and Cloud Materials Library. Full control of scheduled rendering tasks
© 2019 ELSE Corp S.r.l. !20
3D Consumer Engagement
https://vimeo.com/318865684
© 2019 ELSE Corp S.r.l. !21
New PHYGITAL Customer Experience: DIGITAL CONSUMER experience
ELSE Technology Hub
© 2019 ELSE Corp S.r.l. !22
Materials & Colors Personalisation in Action
ELSE
© 2019 ELSE Corp S.r.l. !24
Key Factors in Training Neural Nets
Data Size
Optimisation Algorithm
Implications of Small Data
Bad Generalisation
Data Imbalance
Difficult Optimisation
Loss Function
Parameter Initialisation
© 2019 ELSE Corp S.r.l. !25
Fashion and Data: Brand’s DNA
Do you know your brand’s DNA?In the worlds of marketing, advertising and PR, this is how campaigns are structured on a daily basis. It’s a useful tool to help any business owner to sell their services, any entertainment artist to sell their art, and even any employee to sell themselves. Once you know the difference, it's much easier to distinguish what you do from the competition.
Forbes
© 2019 ELSE Corp S.r.l. !26
Fashion and Data: Product DNA
Do you know your
product’s DNA? • Not visible material
• Non visual computations
• Designer signature
• Services
• Qualities
• Other declarative parameters by the producer
© 2019 ELSE Corp S.r.l. !27
Fashion and Data: Product Anatomy
Visual Computing & Visual Parameters
•Shape
•Components
•Color
•Material
•Brand
•Logo
© 2019 ELSE Corp S.r.l. !28
Artificial Intelligence Startups
Horizontal AI Vertical AI
These are companies that sell tools to help their customers implement AI
Companies solve a specific vertical problem using AI
© 2019 ELSE Corp S.r.l. !29
3D Automation Tools & Technology Stack
Matlab & Simulink Blender UI Blender Scripting
Python Automation
3D to Machine Learning Framework
© 2019 ELSE Corp S.r.l. !31
3D to ML
Bad Generalisation
Data Imbalance
Difficult Optimisation
Implications of Small Data
Data Generation
Data Augmentation
Regularisation
Ensembling
3D Synthetic Data
3D Design Level Annotation & Augmentation
Ridge - Last - Elastic Net
Bagging - Boosting
Balancing Data Set Design Level Scaling
Weight Assigning
Transfer Learning
Problem Reduction
© 2019 ELSE Corp S.r.l. !32
3D to ML - Model Training
Generative Adversarial Networks (GAN)
Web Blender Customization
Transfer Learning: Mask-RCNN Semantic Segmentation
Transfer Learning - Classification
ELSE Edge Cluster
Supported Models
•ResNet•AlexNet•SqueezNet•Inception•Unet•DeepLab
© 2019 ELSE Corp S.r.l. !33
3D to ML - Data Generation
Transfer Learning
Transfer Learning
Big Data
3D Rendered Images &
Annotations
Small Data (Target Images)
Traditional ML Transfer Learning
Isolated, single task learning: K n o w l e d g e i s n o t r e t a i n e d o r accumulated. Learning is performed w.o. considering past learned knowledge.
Learning of a new tasks relies on the previous learned tasks: Learning process can be faster, more accurate and / or need less training data.
Learning System Task 1
Dataset 1
Dataset 2
Learning System Task 1
Knowledge
Learning System Task 1
Dataset 1
Dataset 2Learning System Task 1
VS
© 2019 ELSE Corp S.r.l. !34
3D to ML - Data Generation
https://vimeo.com/289164520
Random Background
Different Annotations
Multiple Model Support
- Mask - RCNN - Semantic Segmentation - Landmark Detection - Transfer Learning Classification
Different Configurations
Different Material
Multiple View Points
© 2019 ELSE Corp S.r.l. !35
Instance Segm (Classification + Semantic Segmentation) with MRCNN
How the model sees an input.
© 2019 ELSE Corp S.r.l. !36
Layer Visualisation of Mask R CNN, TensorBoard
In the new activation layers it can be seen that the network sees the shoe almost in every layer.
© 2019 ELSE Corp S.r.l. !37
ELSE.ai Layer Functionalities
B a c k g r o u n d Segmentation2Input1 3 Recognition 4 Use Case / 3D
Visual Search
Properly Trained Model with Conditional Random Fields
Multi Layer + CRF Model
False Positive / False Negative Bad Mask Localization
Challenges
Mask R-CNN Scene Segmentation /
Background RemovalResnet Shape Classification
Semantic Segmentation Component Detection
3D Configuration MappingInput Use Case
Recommendation Results
Closest 3D Configuration
Component Array
- Laces - Toe Cap - Tongue - Outsole
3D Model Pre-Population
© 2019 ELSE Corp S.r.l. !38
Data preparation Pipeline and Training Steps
Reinforcement and Retraining
Component Level Feature Map Analysis Feature Map
Initial Future Map Analysis
Design Level Annotation + Augmentation
Component Based
Features
Code Level Augmentation +
Annotation
Initial Dataset
General AgentEvaluation Parameter Satisfied Special Agent
(Expert Reinforced)
Component + Design Level Metadata Reinforcement
Designers + Fashion Fields Experts- Customized 3D/CAD data generation
- Supervised by fashion experts and designers
- Reinforced by field knowledge
- Multi layer evaluation
- Code level and design level knowledge transfer
© 2019 ELSE Corp S.r.l. !39
Synthetic Data, Data Augmentation
Forcing AI to «see» the SHAPES, not just textures/ pixels
- Smart small data to big data transformation
- Automated augmentation and Annotation
- Annotation format adoptive (Box – Mask – Polygon , …)
- Coupled with weight and Bias analysis
- Coupled with Evolutionary Algorithms for Hyper Parameter Optimization
- Adoptive towards other parameter optimization methods
© 2019 ELSE Corp S.r.l. !40
Reinforcement by Field Experts, Humans to AI Knowledge Transfer
1.Expert defined style groups for clustering
2.Style and design based feature (component) analysis
3.Customized 3D / CAD data generation based on feature analysis
Designers + Fashion Field Experts
Component + Designer Level Metadata Reinforcement
Design Level Annotation + Augmentation
Component Based Features
Feature Map
Evaluation Parameter Satisfied
Component Level Feature Map
Analysis
Initial Feature Map Analysis
Reinforcement and Retraining
Input Image
Data
Featured Map
Data
Combined Feature Map
Input Image
Data
Featured Map Featured Map
Data
© 2019 ELSE Corp S.r.l. !41
Train on the Cloud Deploy on the Edge
• Accelerated Computing at the Edge
• Powerful Computing for Data Centres
• Democratization, from Data Centre to the Cloud
© 2019 ELSE Corp S.r.l. !42
3D to ML - Data Generation
3D rendering
Color Coded Multi - Label Annotation of Components
Landmark Annotation
3D CAD Model
© 2019 ELSE Corp S.r.l. !43
3D to ML - Edge Deployment, Re-Training & Fine Tuning
Pre-Trained Model with Synthetic Data
The fine tuning is done in-store by a sales assistant. AI knowledge is not needed in this step
In-Store Fine - TuningELSE.Ai
Real Shoe Image Gathering
© 2019 ELSE Corp S.r.l. !44
Re-Training/ Fine Tuning
Approach 1: ELSE Edge Data Collection App
Data Collection
Fine Tuning
Approach 2: ELSE Edge as a PC
Training Progress
Data Collection App UI Manual Directory Creation
© 2019 ELSE Corp S.r.l. !45
Fine Tuning Process
Before Fine Tuning During Fine Tuning After Fine Tuning
Evaluation Parameters After Fine Tuning (5 Epochs in 1 Hour)
Entering Epochs (Between 5 - 10)
Starting the Fine Tuning after
Adding Images
Some Product Images are Added (Around
30-60 each class)
Pre-Trained Model (Accuracy Increases to 72% in less than 1 Epoch)
CASE STUDY: 3D Visual Search
© 2019 ELSE Corp S.r.l. !47
Visual Style Search
• Image recognition and other A. I. technologies • It index, search, tag and find images without
using words • Gives recommendations showing visually similar
products • Generates the perfect look based on brand
style guides and available stock
• Visual search tool
• Users can use the application and upload a screenshot from Instagram, a photo taken from a magazine, or to take a picture from the app to get results of shoppable items.
• Visual recognition tool called Style Match
• It uses Google Lens to take pictures of clothing and then offers up similar shoppable items from around the Internet
http://wide-eyes.it http://www.asos.com https://www.racked.com
© 2019 ELSE Corp S.r.l. !48
Amazon in here: StyleSnap
Amazon launches AI-powered ‘Shazam for clothes’ fashion search
- Amazon unveiled a new feature in its app where customers can upload a photograph of a fashion look that they like and Amazon will find the closest match on its site.
- Asos, Wayfair and Target have similar tools in place
© 2019 ELSE Corp S.r.l. !49
3D Visual Search: Customer ScenarioC
usto
mer
Jou
rney
Gen
era
l Arc
hite
ctur
e
analysing… Detecting type…
what are we searching for today?
TYPE DETECTED
Detect ModelDetecting Model…
what are we searching for today?
CLASSIFICATION RESULT
TYPE: KED LOW
MODEL: XXX
REMOTE SEARCH
LOCAL SEARCH
CLASSIFICATION RESULT
TYPE KED
MODEL XXX
REMOTE SEARCH
LOCAL SEARCH
3D VS engine
Product gets recognized
© 2019 ELSE Corp S.r.l. !50
Overall Scenario
3D Visual Search On «Small Data»
- Synthetic 2D&3D data generation - Automated Annotation and Augmentation
pipeline - Internal definition of Style in machine language - Reinforcement by field experts - Retraining by knowledge transfer - Evaluation by cross-field fashion & AI experts
(humans & AI)
A reinforcement learning agent, builds knowledge by constantly interacting with an environment which allows it to record and replay past experiences in a more efficient way than traditional supervised models.
© 2019 ELSE Corp S.r.l. !51
Data Set: Train / Validation / Test Split x Supervised Learn
Training Set:
Synthet ic data, rendered images f rom 3D configurator coupled with auto annotation and augmentation pipeline
Validation Set:
Real product images, gathered from fashion brands & retailers
Test Set:
Real product photos, made from physical products
© 2019 ELSE Corp S.r.l. !52
Components Detection with Mask R CNN
Original Image Tested
•The model trained was able to detect the objects giving the mask and the bounding box and to categorize the objects.
•The categories are grouped into supercategories, in our case all the categories belong to the shoe supercategory.
•The Neural Network was trained with one class only (Tennis shoes). A further step consisted in the creation of a custom dataset with several classes (ked low, ked high, slip on, tennis shoe)
•The images used were the rendered shoes on the grey background.
The masks proposed by Mask R CNN for the original image
© 2019 ELSE Corp S.r.l. !53
Components Recognition and Configuration Mapping Mask R CNN
Detected & recognized components forms an array which then is mapped to a 3D configuration package.
© 2019 ELSE Corp S.r.l. !54
Configuration Detected: The Results
Component array is mapped to 3D package and results are visualized for further customization
© 2019 ELSE Corp S.r.l. !55
3D to Visual Search in the Lab
Ked-High Ked-Low Slip-On Tennis
Sneaker Classification
© 2019 ELSE Corp S.r.l. !56
Technology Stack
- Supervised Learning
- Deep Learning
- Computational Vision
- Computational Intelligence
- Data Mining & Data Analysis
- Reinforcement Learning
Initial Dataset
Code Level Augmentation + Annotation
General Agent
Initial Feature Map Analysis
Feature Map
Reinforcement and Retraining
Component Level Feature Map Analysis
Component + Design Level Metadata ReinforcementDesign Level
Annotation + Augmentation
Designers + Fashion Field Experts
Component Based
Features
Evaluation Parameter Satisfied
Special Agent (Expert
Reinforced)
© 2019 ELSE Corp S.r.l.
THANK YOU! СПАСИБО! GRAZIE ;)
a.golub@else-corp.it
Co-Founder & CEO ELSE Corp, a Virtual Retail
company
Andrey Golub
© 2019 ELSE Corp S.r.l.!58
Мастер-класс «Искусственный интеллект в индустрии моды»: РАНХиГС, 11/10/19
© 2019 Andrey Golub
VIRTUAL COUTURE FASHION «…Творческий потенциал человечества и
лучшие умы будут привлечены к машинному проектированию и автоматизации, которое станет основой для новой системы производства, --но что же в таком случае произойдет с творчеством и рукотворным мастерством?»
Ашок Сом , профессор Глобальный стратегии и со-директор бизнес-школы ESSEC-Bocconi EMiLUX Program
top related