commentary-based video categorization and concept discovery
DESCRIPTION
Commentary-based Video Categorization and Concept Discovery. By Janice Leung. Agenda. Introduction to Video Sharing Sites Current Problem Previous Works Commentary-based Video Clustering Conclusion Future Works. Video Sharing Sites. Allows users to upload videos Shares videos worldwide - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Commentary-based Video Categorization and Concept Discovery](https://reader036.vdocuments.us/reader036/viewer/2022081515/56815928550346895dc64fe1/html5/thumbnails/1.jpg)
Commentary-based Video Categorization and Concept Discovery
By Janice Leung
![Page 2: Commentary-based Video Categorization and Concept Discovery](https://reader036.vdocuments.us/reader036/viewer/2022081515/56815928550346895dc64fe1/html5/thumbnails/2.jpg)
Agenda
Introduction to Video Sharing Sites Current Problem Previous Works Commentary-based Video Clustering Conclusion Future Works
![Page 3: Commentary-based Video Categorization and Concept Discovery](https://reader036.vdocuments.us/reader036/viewer/2022081515/56815928550346895dc64fe1/html5/thumbnails/3.jpg)
Video Sharing Sites
Allows users to upload videos Shares videos worldwide Example:
Dailymotion YouTube MySpace
![Page 4: Commentary-based Video Categorization and Concept Discovery](https://reader036.vdocuments.us/reader036/viewer/2022081515/56815928550346895dc64fe1/html5/thumbnails/4.jpg)
De Facto
YouTube More than 65,000 new videos every day 100 million videos views daily 20 million unique visitors per month
![Page 5: Commentary-based Video Categorization and Concept Discovery](https://reader036.vdocuments.us/reader036/viewer/2022081515/56815928550346895dc64fe1/html5/thumbnails/5.jpg)
Immense amount of videos
Incredible growth of videos
How to search for desired video?
YouTube: Tags + simple Categorization
![Page 6: Commentary-based Video Categorization and Concept Discovery](https://reader036.vdocuments.us/reader036/viewer/2022081515/56815928550346895dc64fe1/html5/thumbnails/6.jpg)
YouTube
Predefined categories Videos
Title Description Tags Category Comments
Provided by the one who uploads the video
Provided by many users
![Page 7: Commentary-based Video Categorization and Concept Discovery](https://reader036.vdocuments.us/reader036/viewer/2022081515/56815928550346895dc64fe1/html5/thumbnails/7.jpg)
![Page 8: Commentary-based Video Categorization and Concept Discovery](https://reader036.vdocuments.us/reader036/viewer/2022081515/56815928550346895dc64fe1/html5/thumbnails/8.jpg)
![Page 9: Commentary-based Video Categorization and Concept Discovery](https://reader036.vdocuments.us/reader036/viewer/2022081515/56815928550346895dc64fe1/html5/thumbnails/9.jpg)
Related Works
Classify videos: Video features: color, grayscale
histogram, pixel information Keywords from description Tags
Find user interests: Object fetching information Tags
![Page 10: Commentary-based Video Categorization and Concept Discovery](https://reader036.vdocuments.us/reader036/viewer/2022081515/56815928550346895dc64fe1/html5/thumbnails/10.jpg)
Problems
Video features Cannot tell exactly what the video is
about No users interest is considered
Keywords from description Description provided by the one who
uploaded the video Not sufficient information
![Page 11: Commentary-based Video Categorization and Concept Discovery](https://reader036.vdocuments.us/reader036/viewer/2022081515/56815928550346895dc64fe1/html5/thumbnails/11.jpg)
Problems (Cont.)
Tags Not sufficient information May reflect users feelings on videos but
too brief to represent the complex idea of the videos
Object fetching information Reflects users interests but no
information about the videos at all
![Page 12: Commentary-based Video Categorization and Concept Discovery](https://reader036.vdocuments.us/reader036/viewer/2022081515/56815928550346895dc64fe1/html5/thumbnails/12.jpg)
Video Categorization and Concept Discovery
Site: YouTube Videos: involving Hong Kong singers
![Page 13: Commentary-based Video Categorization and Concept Discovery](https://reader036.vdocuments.us/reader036/viewer/2022081515/56815928550346895dc64fe1/html5/thumbnails/13.jpg)
Comment vs Tag
Comments Given by many users
Can be large amount Express users opinions Rich words describe
fine-grained level ideas
Tags Given by only one
person (the one who uploaded the video)
Few tags Describe the video in a
very brief way Singer name Song name
![Page 14: Commentary-based Video Categorization and Concept Discovery](https://reader036.vdocuments.us/reader036/viewer/2022081515/56815928550346895dc64fe1/html5/thumbnails/14.jpg)
![Page 15: Commentary-based Video Categorization and Concept Discovery](https://reader036.vdocuments.us/reader036/viewer/2022081515/56815928550346895dc64fe1/html5/thumbnails/15.jpg)
Comments
Include: Video content
Music styles Music ages
Singer description Appearance Style News etc.
![Page 16: Commentary-based Video Categorization and Concept Discovery](https://reader036.vdocuments.us/reader036/viewer/2022081515/56815928550346895dc64fe1/html5/thumbnails/16.jpg)
Commentary-based Video Categorization
Objective: Categories videos based on user interests and discover the concept of videos
Cluster videos by using comments Group videos based on user interests Find video concepts Clustering algorithm: multi-assignment
NMF
![Page 17: Commentary-based Video Categorization and Concept Discovery](https://reader036.vdocuments.us/reader036/viewer/2022081515/56815928550346895dc64fe1/html5/thumbnails/17.jpg)
Video clustering
Bi-clustering: videos and words Clusters videos and words into k
groups by matrix factorization Video-word matrix X as input
Video-word matrix X is derived by tf-idf
![Page 18: Commentary-based Video Categorization and Concept Discovery](https://reader036.vdocuments.us/reader036/viewer/2022081515/56815928550346895dc64fe1/html5/thumbnails/18.jpg)
Tf-idf
Term frequency (tf) Suppose there are t distinct terms in
document j
where fi,j is the number of occurrence of term i in document j
![Page 19: Commentary-based Video Categorization and Concept Discovery](https://reader036.vdocuments.us/reader036/viewer/2022081515/56815928550346895dc64fe1/html5/thumbnails/19.jpg)
Tf-idf (Cont.)
Inverse document frequency
where N is the total number of documents in dataset and ni is number of documents containing term i
![Page 20: Commentary-based Video Categorization and Concept Discovery](https://reader036.vdocuments.us/reader036/viewer/2022081515/56815928550346895dc64fe1/html5/thumbnails/20.jpg)
Tf-idf (Cont.)
Importance weight of term i to document j
Matrix X as input to NMF is defined as
![Page 21: Commentary-based Video Categorization and Concept Discovery](https://reader036.vdocuments.us/reader036/viewer/2022081515/56815928550346895dc64fe1/html5/thumbnails/21.jpg)
Video Clustering (Cont.)
Decompose X into non-negative matrices W and H by minimizing
where
Ref. : Document Clustering Based On Non-negative Matrix Factorization (Xu et al SIGIR’03)
0, HW
![Page 22: Commentary-based Video Categorization and Concept Discovery](https://reader036.vdocuments.us/reader036/viewer/2022081515/56815928550346895dc64fe1/html5/thumbnails/22.jpg)
Video Clustering (Cont.)
NMF decomposition for video clustering
![Page 23: Commentary-based Video Categorization and Concept Discovery](https://reader036.vdocuments.us/reader036/viewer/2022081515/56815928550346895dc64fe1/html5/thumbnails/23.jpg)
Video Clustering (Cont.)
Suppose Number of videos: N Number of distinct terms: M Threshold: β
W in size M x K wn,k: coefficient indicates how video n
belongs to cluster k
![Page 24: Commentary-based Video Categorization and Concept Discovery](https://reader036.vdocuments.us/reader036/viewer/2022081515/56815928550346895dc64fe1/html5/thumbnails/24.jpg)
Video-cluster assignment
Videos can belongs to multiple groups Multi-cluster assignment Video n belongs to cluster k if Set of clusters that video n belongs to:
where K is set if all clusters
kknw ,
![Page 25: Commentary-based Video Categorization and Concept Discovery](https://reader036.vdocuments.us/reader036/viewer/2022081515/56815928550346895dc64fe1/html5/thumbnails/25.jpg)
Video-cluster assignment (Cont.)
Threshold, β Many irrelevant videos for each cluster Coefficient distribution varies for different
clusters Coefficient distribution dependant Different for different clusters
![Page 26: Commentary-based Video Categorization and Concept Discovery](https://reader036.vdocuments.us/reader036/viewer/2022081515/56815928550346895dc64fe1/html5/thumbnails/26.jpg)
Concept Discovery
Matrix H in size of K x M hk,m: how likely term m belongs to
cluster k Term belongs to a cluster describes
the videos in that cluster Concept words of cluster k videos
Top 10 words of cluster k
![Page 27: Commentary-based Video Categorization and Concept Discovery](https://reader036.vdocuments.us/reader036/viewer/2022081515/56815928550346895dc64fe1/html5/thumbnails/27.jpg)
Experiment
19305 videos 102 Hong Kong singers 7271 users Number of cluster, k: 20
![Page 28: Commentary-based Video Categorization and Concept Discovery](https://reader036.vdocuments.us/reader036/viewer/2022081515/56815928550346895dc64fe1/html5/thumbnails/28.jpg)
Experiment (Cont.)
Threshold, β Coefficient distribution dependant Threshold for cluster i is defined as
![Page 29: Commentary-based Video Categorization and Concept Discovery](https://reader036.vdocuments.us/reader036/viewer/2022081515/56815928550346895dc64fe1/html5/thumbnails/29.jpg)
Experiment (Cont.)
Video coefficients may distribute in an extremely uneven manner
Cause poor result To compensate, threshold can be set
as
![Page 30: Commentary-based Video Categorization and Concept Discovery](https://reader036.vdocuments.us/reader036/viewer/2022081515/56815928550346895dc64fe1/html5/thumbnails/30.jpg)
Experiment (Cont.)
0.700.300.00
0.310.640.05
0.130.220.65
0.010.640.35
0.120.230.65
0.700.300.00
0.310.640.05
0.130.220.65
0.010.640.35
0.120.230.65
Mean Coef.
0.33 0.406 0.254
Mean + SD Coef.
0.631 0.622 0.526
C1 C2 C3
V1
V2
V4
V3
V5
![Page 31: Commentary-based Video Categorization and Concept Discovery](https://reader036.vdocuments.us/reader036/viewer/2022081515/56815928550346895dc64fe1/html5/thumbnails/31.jpg)
Experiment (Cont.)
![Page 32: Commentary-based Video Categorization and Concept Discovery](https://reader036.vdocuments.us/reader036/viewer/2022081515/56815928550346895dc64fe1/html5/thumbnails/32.jpg)
Experiment (Cont.)
![Page 33: Commentary-based Video Categorization and Concept Discovery](https://reader036.vdocuments.us/reader036/viewer/2022081515/56815928550346895dc64fe1/html5/thumbnails/33.jpg)
Experiment (Cont.)
![Page 34: Commentary-based Video Categorization and Concept Discovery](https://reader036.vdocuments.us/reader036/viewer/2022081515/56815928550346895dc64fe1/html5/thumbnails/34.jpg)
Concept Words vs Tags
![Page 35: Commentary-based Video Categorization and Concept Discovery](https://reader036.vdocuments.us/reader036/viewer/2022081515/56815928550346895dc64fe1/html5/thumbnails/35.jpg)
Concept Words vs Tags
Percentage of videos with tags covering concept words across groups
![Page 36: Commentary-based Video Categorization and Concept Discovery](https://reader036.vdocuments.us/reader036/viewer/2022081515/56815928550346895dc64fe1/html5/thumbnails/36.jpg)
Singer Relationship Discovery
Comments on videos may talk about singers
Singer styles, appearance, news Singer clustering using comments Reveals relationships between singers Discovers hidden phenomenon
![Page 37: Commentary-based Video Categorization and Concept Discovery](https://reader036.vdocuments.us/reader036/viewer/2022081515/56815928550346895dc64fe1/html5/thumbnails/37.jpg)
Singer Relationship Discovery (Cont.)
![Page 38: Commentary-based Video Categorization and Concept Discovery](https://reader036.vdocuments.us/reader036/viewer/2022081515/56815928550346895dc64fe1/html5/thumbnails/38.jpg)
Conclusion
Captures user interests more accurately and fairly than that of the human predefined categories
Categories can be changed dynamically, user interest changes from time to time
Obtain clusters with fine-grained level ideas Ease the task of video search by
categorizing videos and refining index
![Page 39: Commentary-based Video Categorization and Concept Discovery](https://reader036.vdocuments.us/reader036/viewer/2022081515/56815928550346895dc64fe1/html5/thumbnails/39.jpg)
Future Works
Extend to user clustering Obtain relationships videos, singers
and users of the entire social network Study the social culture Ease the job of advertising to target
customers Connect people who share the same
interests
![Page 40: Commentary-based Video Categorization and Concept Discovery](https://reader036.vdocuments.us/reader036/viewer/2022081515/56815928550346895dc64fe1/html5/thumbnails/40.jpg)
Q & A
Questions?