a video analytics solution for discovering customer shopping behavior in retail … · 3 why...
TRANSCRIPT
1 © 2012 The MathWorks, Inc.
A Video Analytics Solution for
Discovering Customer Shopping Behavior
in Retail Stores
Authors: Ayushmaan Basu; Harikrishna G. N. Rai; Shreya Barsaiyan; K Sai Deepak; P. Radha Krishna; Lokendra Shastri
Speaker:
Harikrishna Rai
Research Analyst
Infosys Labs, Bangalore
2
Presentation Overview
Why analytics in retail stores?
Requirements & Assumptions
Challenges in Computer vision
Solution overview
SCID design overview
Algorithm overview
Matlab benefits & toolkits used
Solution benefits
3
Why Analytics in Retail Stores ?
Retailers need personalized and location based
services to improve their customers shopping
experience
Retailers & CPG companies need depth knowledge of
customers data – Shopper Behavior, Buying Patterns and Shelf Activity
– Enable Strategic Marketing (product placement)
Customer profiling is key to offer any such customer
engagement solutions
In-store analytics help stores to take better decisions
about how to tailor sales, coupons and other
promotional campaigns
Get insights into how consumers react to various
displays and digital merchandising scenarios
In-store analytics
Shopper path
Stop points
Floor map
analysis
Time spent on products
RFID & WSN solutions
• Expensive as separate hardware need to attached to each cart
• Maintenance is costly
• Chances of damage to these hardware devices
• Poor scalability
4
Solution Requirements & Assumptions
Assumptions
• Typical retail store setup with aisles for shopping
• Wall/rack mounted camera
• Camera fixed at height of 8-10 feet
• Multiple presence of carts
• Customer movement in moderate speed
• Reasonable lighting conditions
Requirements
• Solution to be simple to use and cost effective
• Zero maintainace
• Robust and highly accurate
• Scalable solution without change in store setup
• Provide additional value added services
• Light weight solution with minimal processor requirement
5
Challenges for computer vision approach
Low resolution images
Varied Lighting condition
Shadows
Occlusion
Orientation
Scale
Perspective projection
6
Demo 1 - - Single Cart Scenario
Demo 2 - - Multiple Cart Scenario
7
Solution Overview vid
eo
str
ea
m
field of view(FOV)
X
Y
Z
Unique Shopping Cart ID
(SCID)
camera distance
Detection of SCID
Recognition
Ca
rt ID
& L
oca
tio
n
Em
be
dd
ed
De
vic
e
Shopper Path
8
Configuration details
Camera Specifications
– a maximum of 2 megapixels resolution at 1FPS
Environment (height, lighting, distance)
– height of camera is fixed at around 8 feet
– fluorescent light
– distance of cart from camera can be between 3-20 Feet
SCID/Marker design specifications
– SCID contains a 3x3 rectangular grid of size is 6” * 6” with unique
symbols
– Black border in SCID is used for detection and it is 0.5” thick
Typical Retail Store with Shopping Carts
9
SCID Overview
ID Color Shape Orientation
9 Green Square NA
5 Blue Rectangle 180 deg
0 Red Triangle NA
- - - -
• Symbols are easily differentiable and identifiable with
combination of 3 primary colors
• Symbols have maximum separation in terms of shape such as • Length and Breadth (e.g., just a ratio may suffice)
• Color (a small number of easily distinguishable colors, e.g., red and blue)
• Orientation along major axis (e.g., horizontal and vertical)
• Redundant coding to enable detection even in cases with
occlusion
• SCID code book can support up to 1000 unique carts on a 3x3
grid
Sample SCID = 509
A B C
B C A
C A B
Code book example for decoding ID 509
10
Start
Grab I’th frame from camera
DETECTION
RECOGNITION
Store Cart ID
Entry
I = I+1
Frame I = 1
Algorithm Overview – Detection & Recognition
11
START
Read Cart ID Entry I
Entry I = 1
Entry(I) ==
END?
WAIT
Is Valid Cart?
I = I+1
PLOT PATH
yes
no
no
yes
Algorithm Overview – Tracking
12
Effect
Sample
Occlusion
(Partial row & column)
Occlusion
(Entire row or column )
Strip Removal
Occlusion scenarios
13
Solution highlights
Computer Vision based recognition system adopting a symbolic
recognition system
Simple encoding with high accuracy
Derogatory effects tackled using redundancy and image
enhancement techniques
Considerable ease of implementation with no customer interaction
Minimal hardware requirements (Porting to DSP Processor -
planned)
Low power/low resolution camera
No change in present environment setup
Completely automated
14
Why Matlab for SCID implementation?
Enabled quick prototyping
Simple to use interface
Availability of ready to use advanced image processing
library
Easy for code walkthrough and debugging
Easy to package as a stand alone solution
15
Matlab Toolbox used
Image Processing
Statistics
Compiler
Builder
16
Image Processing Techniques Used
Morphological Operations
Non Linear Filtering
Image Segmentation
Color Invariance Operations
Feature Extraction and Analysis
17
Infosys SCID - Solution Benefits
Cost reduction – Cheaper vision based software solutions as it requires
only low cost cameras
Low maintenance – Do not require frequent service for hardware devices
Scalability – Highly scalable as only camera need to be procured and
installed for increasing the coverage
Versatility – Same camera set up can be used for other purpose like
surveillance and safety etc
Rich Analytics –Analyses shopper behavior, buying patterns, store map,
most visited aisles, time spent on products etc
In-store analytics solution powered by Infosys SCID* in Matlab environment
* patent pending
18
Thanks
Q&A