computer vision at the university of ioannina...2020/09/20 · computer vision at the university of...
TRANSCRIPT
![Page 1: Computer Vision at the University of Ioannina...2020/09/20 · Computer Vision at the University of Ioannina Christophoros Nikou Department of Computer Science and Engineering University](https://reader035.vdocuments.us/reader035/viewer/2022081409/607727a3dba5040a5010a479/html5/thumbnails/1.jpg)
Computer Vision at the
University of Ioannina
Christophoros NikouDepartment of Computer Science and EngineeringUniversity of IoanninaGreece
Overview of research activities
![Page 2: Computer Vision at the University of Ioannina...2020/09/20 · Computer Vision at the University of Ioannina Christophoros Nikou Department of Computer Science and Engineering University](https://reader035.vdocuments.us/reader035/viewer/2022081409/607727a3dba5040a5010a479/html5/thumbnails/2.jpg)
A bit about me
Athens Santorini
![Page 3: Computer Vision at the University of Ioannina...2020/09/20 · Computer Vision at the University of Ioannina Christophoros Nikou Department of Computer Science and Engineering University](https://reader035.vdocuments.us/reader035/viewer/2022081409/607727a3dba5040a5010a479/html5/thumbnails/3.jpg)
A bit about me
My
hometown:Thessaloniki
Electrical Engineering,
1994
PhD in Computer Vision,
1999
University of
Strasbourg,France,
Professor,
University of Ioannina,
Department of CSE
![Page 4: Computer Vision at the University of Ioannina...2020/09/20 · Computer Vision at the University of Ioannina Christophoros Nikou Department of Computer Science and Engineering University](https://reader035.vdocuments.us/reader035/viewer/2022081409/607727a3dba5040a5010a479/html5/thumbnails/4.jpg)
4
University of Ioannina
◼ 27 departments
◼ 530 faculty members
◼ 120 research associates
◼ 200 administrative staff members
◼ 25000 students
![Page 5: Computer Vision at the University of Ioannina...2020/09/20 · Computer Vision at the University of Ioannina Christophoros Nikou Department of Computer Science and Engineering University](https://reader035.vdocuments.us/reader035/viewer/2022081409/607727a3dba5040a5010a479/html5/thumbnails/5.jpg)
5
Department of Computer Science
and Engineering (CSE)
◼ Founded in 1990
◼ Undergraduate program (5 years)
◼ MSc program
◼ PhD program
◼ 26 faculty members
◼ 9 lab instuctors
◼ 5 administrative staff members
◼ 1200 undergraduate students
◼ 70 MSc and PhD students
![Page 6: Computer Vision at the University of Ioannina...2020/09/20 · Computer Vision at the University of Ioannina Christophoros Nikou Department of Computer Science and Engineering University](https://reader035.vdocuments.us/reader035/viewer/2022081409/607727a3dba5040a5010a479/html5/thumbnails/6.jpg)
CSE Research topics◼ Algorithmic Game Theory
◼ Artificial Intelligence
◼ Combinatorial Optimization
◼ Computer Architecture
◼ Computational Geometry
◼ Computational Logic
◼ Computer Graphics
◼ Computer Networks
◼ Computer Vision
◼ Database Systems
◼ Data Mining
◼ Data Structures
6
◼ Design and Testing of VLSI Circuits
◼ Digital Signal and Image Processing
◼ Graph Theory
◼ Information Systems Security
◼ Machine Learning
◼ Medical Informatics
◼ Middleware
◼ Mobile Computer Sytems
◼ Multimedia and Hypermedia
◼ Neural Networks
◼ Operating Systems
◼ Optimization Theory and Software
![Page 7: Computer Vision at the University of Ioannina...2020/09/20 · Computer Vision at the University of Ioannina Christophoros Nikou Department of Computer Science and Engineering University](https://reader035.vdocuments.us/reader035/viewer/2022081409/607727a3dba5040a5010a479/html5/thumbnails/7.jpg)
CSE Research topics (cont.)
◼ Parallel and Distributed Algorithms
◼ Parallel Processing
◼ Pattern Recognition
◼ Programming Languages
◼ Scientific Calculations
◼ Software Engineering
◼ Theory of Computation and Algorithms
7
![Page 8: Computer Vision at the University of Ioannina...2020/09/20 · Computer Vision at the University of Ioannina Christophoros Nikou Department of Computer Science and Engineering University](https://reader035.vdocuments.us/reader035/viewer/2022081409/607727a3dba5040a5010a479/html5/thumbnails/8.jpg)
8
Information Processing and
Analysis Research Group (IPAN) ◼ 6 Faculty members
◼ 3 post-doctoral researchers
◼ 10 PhD students
◼ 15 MSc students
◼ 25 BSc students
◼ Over 35 alumni
◼ Collaborators in many countries
![Page 9: Computer Vision at the University of Ioannina...2020/09/20 · Computer Vision at the University of Ioannina Christophoros Nikou Department of Computer Science and Engineering University](https://reader035.vdocuments.us/reader035/viewer/2022081409/607727a3dba5040a5010a479/html5/thumbnails/9.jpg)
9
Information Processing and
Analysis Research Group (IPAN) ◼ Artificial Intelligence - Intelligent Agents
◼ Bioinformatics
◼ Medical Signal and Image Analysis
◼ Computer Vision
◼ Machine Learning
◼ Data Mining
◼ Multimedia Information Processing, Analysis and Retrieval
◼ Pattern Recognition
◼ Neural Networks Robotics
◼ Signal and Image Processing
◼ Robotics
![Page 10: Computer Vision at the University of Ioannina...2020/09/20 · Computer Vision at the University of Ioannina Christophoros Nikou Department of Computer Science and Engineering University](https://reader035.vdocuments.us/reader035/viewer/2022081409/607727a3dba5040a5010a479/html5/thumbnails/10.jpg)
10
Computer Vision team
(C. Nikou)◼ Since 2004
◼ 3 PhD students
◼ 4 MSc students
◼ 15 undergraduate students
◼ Research
Computer vision
Pattern recognition
Image/video processing
Image/video analysis
Bio-medical image processing and analysis
![Page 11: Computer Vision at the University of Ioannina...2020/09/20 · Computer Vision at the University of Ioannina Christophoros Nikou Department of Computer Science and Engineering University](https://reader035.vdocuments.us/reader035/viewer/2022081409/607727a3dba5040a5010a479/html5/thumbnails/11.jpg)
11
Research activities
◼ Low level computer vision Video colorization
◼ Mid-level computer vision Image segmentation
Image feature analysis
Visual tracking
◼ High level computer vision Handwritten document indexing using word spotting
Human action recognition
Human identification
◼ Biomedical image analysis Cytological image analysis
![Page 12: Computer Vision at the University of Ioannina...2020/09/20 · Computer Vision at the University of Ioannina Christophoros Nikou Department of Computer Science and Engineering University](https://reader035.vdocuments.us/reader035/viewer/2022081409/607727a3dba5040a5010a479/html5/thumbnails/12.jpg)
12
Research activities
◼ Low level computer vision Video colorization
◼ Mid-level computer vision Image segmentation
Image feature analysis
Visual tracking
◼ High level computer vision Handwritten document indexing using word spotting
Human action recognition
Human identification
◼ Biomedical image analysis Cytological image analysis
![Page 13: Computer Vision at the University of Ioannina...2020/09/20 · Computer Vision at the University of Ioannina Christophoros Nikou Department of Computer Science and Engineering University](https://reader035.vdocuments.us/reader035/viewer/2022081409/607727a3dba5040a5010a479/html5/thumbnails/13.jpg)
13
Video colorization
◼ 3D generative adversaial network (GAN)
◼ Color consistency
Standard indices consider differences between individual frames of the estimate and the ground truth
Color consistency considers color differences between consecutive frames also
![Page 14: Computer Vision at the University of Ioannina...2020/09/20 · Computer Vision at the University of Ioannina Christophoros Nikou Department of Computer Science and Engineering University](https://reader035.vdocuments.us/reader035/viewer/2022081409/607727a3dba5040a5010a479/html5/thumbnails/14.jpg)
14
Colorization with no ground truth
A streetcar named desire 12 angry men
![Page 15: Computer Vision at the University of Ioannina...2020/09/20 · Computer Vision at the University of Ioannina Christophoros Nikou Department of Computer Science and Engineering University](https://reader035.vdocuments.us/reader035/viewer/2022081409/607727a3dba5040a5010a479/html5/thumbnails/15.jpg)
15
Research activities
◼ Low level computer vision Video colorization
◼ Mid-level computer vision Image segmentation
Image feature analysis
Visual tracking
◼ High level computer vision Handwritten document indexing using word spotting
Human action recognition
Human identification
◼ Biomedical image analysis Cytological image analysis
![Page 16: Computer Vision at the University of Ioannina...2020/09/20 · Computer Vision at the University of Ioannina Christophoros Nikou Department of Computer Science and Engineering University](https://reader035.vdocuments.us/reader035/viewer/2022081409/607727a3dba5040a5010a479/html5/thumbnails/16.jpg)
16
Image segmentation
◼ Problem Group together similar pixels
Image intensity is not sufficient
High level knowledge is needed for semantic segmentation
Bottom-up or top-down process?
Supervised or unsupervised?
◼ Proposed methods
Spatially varying Gaussian mixture models◼ Gibbs prior
◼ Dirichlet prior
◼ Student’s t prior
![Page 17: Computer Vision at the University of Ioannina...2020/09/20 · Computer Vision at the University of Ioannina Christophoros Nikou Department of Computer Science and Engineering University](https://reader035.vdocuments.us/reader035/viewer/2022081409/607727a3dba5040a5010a479/html5/thumbnails/17.jpg)
17
Spatially varying GMM
◼ Smoothing prior on the mixture weights
◼ Edge preservation
◼ On-line estimation of the number of segments
1
( | , ) Norm( ; )K
n n
j j
j
Pr x x =
=Π Θ
1. C. Nikou, N. Galatsanos and A. Likas. A class-adaptive spatially variant finite
mixture model for image segmentation. IEEE Transactions on Image
Processing, Vol. 16, No 4, pp. 1121-1130, 2007.
2. G. Sfikas, C. Nikou and N. Galatsanos. Edge-preserving spatially varying
mixtures for image segmentation. Proceedings of the IEEE Computer
Society Conference on Computer Vision and Pattern Recognition
(CVPR’08), 23-28 June 2008, Anchorage, Alaska, USA. Oral presentation
(top 3% papers).
3. C. Nikou, A. Likas and N. Galatsanos. A Bayesian framework for image
segmentation with spatially varying mixtures. IEEE Transactions on Image
Processing, Vol. 19, No 9, pp.2278-2289, 2010.
4. . Sfikas, C. Nikou, N. Galatsanos and C. Heinrich. Majorization-minimization
mixture model determination in image segmentation. IEEE Computer Society
Conference on Computer Vision and Pattern Recognition (CVPR’11), pp.
2169-2176, 20-25 June 2011, Colorado Springs, Colorado, USA.
![Page 18: Computer Vision at the University of Ioannina...2020/09/20 · Computer Vision at the University of Ioannina Christophoros Nikou Department of Computer Science and Engineering University](https://reader035.vdocuments.us/reader035/viewer/2022081409/607727a3dba5040a5010a479/html5/thumbnails/18.jpg)
18
Spatially varying GMM (2)Horizontal differences Vertical differences
Sky
Cupola
Building
nk
ju Separation of the j-th class from the remaining classes.
![Page 19: Computer Vision at the University of Ioannina...2020/09/20 · Computer Vision at the University of Ioannina Christophoros Nikou Department of Computer Science and Engineering University](https://reader035.vdocuments.us/reader035/viewer/2022081409/607727a3dba5040a5010a479/html5/thumbnails/19.jpg)
19
Spatially varying GMM (3)ε=10 human
![Page 20: Computer Vision at the University of Ioannina...2020/09/20 · Computer Vision at the University of Ioannina Christophoros Nikou Department of Computer Science and Engineering University](https://reader035.vdocuments.us/reader035/viewer/2022081409/607727a3dba5040a5010a479/html5/thumbnails/20.jpg)
20
Image feature analysis
◼ Problem
Summarize image features by line segments
Applications◼ Robot navigation
◼ Image understanding
◼ Proposed method
Split and merge algorithm◼ Vanishing point detection
◼ Sampling image edges
◼ Elimination of outliers using the Helmholtz principle
![Page 21: Computer Vision at the University of Ioannina...2020/09/20 · Computer Vision at the University of Ioannina Christophoros Nikou Department of Computer Science and Engineering University](https://reader035.vdocuments.us/reader035/viewer/2022081409/607727a3dba5040a5010a479/html5/thumbnails/21.jpg)
21
Split and merge algorithm
D. Gerogiannis, C. Nikou and A. Likas. Modeling sets of unordered points using highly eccentric ellipses. EURASIP
Journal on Advances in Signal Processing, 2014:11, 2014.
Demo
Vanishing point detectionSampling image edges
Our Method
Random Monte-Carlo
Malik
![Page 22: Computer Vision at the University of Ioannina...2020/09/20 · Computer Vision at the University of Ioannina Christophoros Nikou Department of Computer Science and Engineering University](https://reader035.vdocuments.us/reader035/viewer/2022081409/607727a3dba5040a5010a479/html5/thumbnails/22.jpg)
22
Split and merge algorithm (2)
Elimination of outliers from 2D point sets using the Helmholtz principle. IEEE Signal Processing Letters, Vol. 22, No 10,
pp. 1638-1642, 2015.
◼ Elimination of outliers using the Helmoltz principle
Randomness is modeled by a Pareto distribution (a contrario model)
A meaningful observation is characterized by a divergence from randomness
Pareto DistributionLength distribution
![Page 23: Computer Vision at the University of Ioannina...2020/09/20 · Computer Vision at the University of Ioannina Christophoros Nikou Department of Computer Science and Engineering University](https://reader035.vdocuments.us/reader035/viewer/2022081409/607727a3dba5040a5010a479/html5/thumbnails/23.jpg)
23
Visual tracking
◼ Problem
Estimate the location of a target object in video sequences◼ Real time
◼ Varying illumination, color, motion model
◼ Occlusions
◼ Proposed methods
Histogram modeling by mixture models◼ Earth Mover’s distance between Gaussian mixtures
◼ Weighted likelihood maximization (handles changes in appearance)
◼ Von Mises mixtures (handles change in color)
Clustering of motion trajectories
![Page 24: Computer Vision at the University of Ioannina...2020/09/20 · Computer Vision at the University of Ioannina Christophoros Nikou Department of Computer Science and Engineering University](https://reader035.vdocuments.us/reader035/viewer/2022081409/607727a3dba5040a5010a479/html5/thumbnails/24.jpg)
24
Earth Mover’s distance and Weighted
likelihood Representative examples
1. V. Karavasilis, C. Nikou and A. Likas. Visual tracking using the earth mover’s distance between Gaussian mixtures and
Kalman filtering. Image and Vision Computing, Vol. 29, No 5, pp. 295-305, 2011.
2. V. Karavasilis, C. Nikou and A. Likas, Visual tracking using spatially weighted likelihood of Gaussian mixtures, Computer
Vision and Image Understanding, Vol. 140, pp. 43-57, 2015.
![Page 25: Computer Vision at the University of Ioannina...2020/09/20 · Computer Vision at the University of Ioannina Christophoros Nikou Department of Computer Science and Engineering University](https://reader035.vdocuments.us/reader035/viewer/2022081409/607727a3dba5040a5010a479/html5/thumbnails/25.jpg)
25
Weighted likelihood
Representative examples (2)
https://youtu.be/4b_mA4W92R0 https://youtu.be/oqXOIe1iKWY https://youtu.be/FrwHIeCbgcM
https://youtu.be/5HmGfkOp34E https://youtu.be/YBp_9HBIiUc https://youtu.be/tFjP9MzAMyk
1. V. Karavasilis, C. Nikou and A. Likas. Visual tracking using the earth mover’s distance between Gaussian mixtures and
Kalman filtering. Image and Vision Computing, Vol. 29, No 5, pp. 295-305, 2011.
2. V. Karavasilis, C. Nikou and A. Likas, Visual tracking using spatially weighted likelihood of Gaussian mixtures, Computer
Vision and Image Understanding, Vol. 140, pp. 43-57, 2015.
![Page 26: Computer Vision at the University of Ioannina...2020/09/20 · Computer Vision at the University of Ioannina Christophoros Nikou Department of Computer Science and Engineering University](https://reader035.vdocuments.us/reader035/viewer/2022081409/607727a3dba5040a5010a479/html5/thumbnails/26.jpg)
26
Representative examples with
rotation
https://youtu.be/vvcQOJy0Vok https://youtu.be/bD1FZVn6Qws
1. V. Karavasilis, C. Nikou and A. Likas. Visual tracking using the earth mover’s distance between Gaussian mixtures and
Kalman filtering. Image and Vision Computing, Vol. 29, No 5, pp. 295-305, 2011.
2. V. Karavasilis, C. Nikou and A. Likas, Visual tracking using spatially weighted likelihood of Gaussian mixtures, Computer
Vision and Image Understanding, Vol. 140, pp. 43-57, 2015.
![Page 27: Computer Vision at the University of Ioannina...2020/09/20 · Computer Vision at the University of Ioannina Christophoros Nikou Department of Computer Science and Engineering University](https://reader035.vdocuments.us/reader035/viewer/2022081409/607727a3dba5040a5010a479/html5/thumbnails/27.jpg)
27
Parameters update
![Page 28: Computer Vision at the University of Ioannina...2020/09/20 · Computer Vision at the University of Ioannina Christophoros Nikou Department of Computer Science and Engineering University](https://reader035.vdocuments.us/reader035/viewer/2022081409/607727a3dba5040a5010a479/html5/thumbnails/28.jpg)
28
Parameters update (2)
https://youtu.be/VkvkGSItSHA https://youtu.be/K2n7KQ1rVzg
No update Updated parameters
![Page 29: Computer Vision at the University of Ioannina...2020/09/20 · Computer Vision at the University of Ioannina Christophoros Nikou Department of Computer Science and Engineering University](https://reader035.vdocuments.us/reader035/viewer/2022081409/607727a3dba5040a5010a479/html5/thumbnails/29.jpg)
29
Von Mises mixture model
V. Karavasilis, C. Nikou and A. Likas. Real time visual tracking using a spatially weighted von Mises mixture model. Pattern
Recognition Letters, Vol. 90, pp. 50-57, 2017.
![Page 30: Computer Vision at the University of Ioannina...2020/09/20 · Computer Vision at the University of Ioannina Christophoros Nikou Department of Computer Science and Engineering University](https://reader035.vdocuments.us/reader035/viewer/2022081409/607727a3dba5040a5010a479/html5/thumbnails/30.jpg)
30
Von Mises mixture model (2)
https://youtu.be/7gDhMw4FGNc
V. Karavasilis, C. Nikou and A. Likas. Real time visual tracking using a spatially weighted von Mises mixture model. Pattern
Recognition Letters, Vol. 90, pp. 50-57, 2017.
![Page 31: Computer Vision at the University of Ioannina...2020/09/20 · Computer Vision at the University of Ioannina Christophoros Nikou Department of Computer Science and Engineering University](https://reader035.vdocuments.us/reader035/viewer/2022081409/607727a3dba5040a5010a479/html5/thumbnails/31.jpg)
31
Clustering trajectories of motion
curvesLandmarks-
Harris corners
https://youtu.be/-_g-uB9EYTg
V. Karavasilis, K. Blekas and C. Nikou. A novel framework for motion segmentation and tracking by clustering incomplete
trajectories. Computer Vision and Image Understanding, Vol. 116, No 11, pp. 1135-1148, 2012.
![Page 32: Computer Vision at the University of Ioannina...2020/09/20 · Computer Vision at the University of Ioannina Christophoros Nikou Department of Computer Science and Engineering University](https://reader035.vdocuments.us/reader035/viewer/2022081409/607727a3dba5040a5010a479/html5/thumbnails/32.jpg)
32
Clustering trajectories of motion
curves (2)Trajectory creation from landmark tracking
https://youtu.be/n1AY
PJ-7jps
Clustering of curves
V. Karavasilis, K. Blekas and C. Nikou. A novel framework for motion segmentation and tracking by clustering incomplete
trajectories. Computer Vision and Image Understanding, Vol. 116, No 11, pp. 1135-1148, 2012.
![Page 33: Computer Vision at the University of Ioannina...2020/09/20 · Computer Vision at the University of Ioannina Christophoros Nikou Department of Computer Science and Engineering University](https://reader035.vdocuments.us/reader035/viewer/2022081409/607727a3dba5040a5010a479/html5/thumbnails/33.jpg)
33
Research activities
◼ Low level computer vision Video colorization
◼ Mid-level computer vision Image segmentation
Image feature analysis
Visual tracking
◼ High level computer vision Handwritten document indexing using word spotting
Human action recognition
Human identification
◼ Biomedical image analysis Cytological image analysis
![Page 34: Computer Vision at the University of Ioannina...2020/09/20 · Computer Vision at the University of Ioannina Christophoros Nikou Department of Computer Science and Engineering University](https://reader035.vdocuments.us/reader035/viewer/2022081409/607727a3dba5040a5010a479/html5/thumbnails/34.jpg)
34
Handwritten document indexing
◼ Problem
Vast digitized document collections are not efficiently exploited by researchers and scholars.
Optical Character Recognition (OCR) is not feasible in most cases◼ writing style variability, cursive text, open vocabulary, text skew or
slant, bleed through
◼ Proposed methods
Learning-based word spotting using contour-based models.
Learning-free word spotting.
![Page 35: Computer Vision at the University of Ioannina...2020/09/20 · Computer Vision at the University of Ioannina Christophoros Nikou Department of Computer Science and Engineering University](https://reader035.vdocuments.us/reader035/viewer/2022081409/607727a3dba5040a5010a479/html5/thumbnails/35.jpg)
35
Learning-base word spotting
using contour-based model◼ Multi-writer handwritten word spotting.
◼ Query-by-example.
◼ Extract pairs of adjacent segments (PAS).
◼ Create PAS codebook
r
1
(x, y)2
s=|r|
![Page 36: Computer Vision at the University of Ioannina...2020/09/20 · Computer Vision at the University of Ioannina Christophoros Nikou Department of Computer Science and Engineering University](https://reader035.vdocuments.us/reader035/viewer/2022081409/607727a3dba5040a5010a479/html5/thumbnails/36.jpg)
36
Learning-based word spotting
using contour-based model (2)◼ Learn a statistical representatation for a word-class.
◼ Spot previously unseen word instances. Model alignment and non rigid contour-point matching
A. Giotis, D. Gerogiannis and C. Nikou. Word spotting in handwritten text using contour-based models. 14th International Conference
on Frontiers in Handwriting Recognition (ICFHR‘14), 1-4 September 2014, Hersonisos, Crete, Greece, pp. 399-404, 2014.
![Page 37: Computer Vision at the University of Ioannina...2020/09/20 · Computer Vision at the University of Ioannina Christophoros Nikou Department of Computer Science and Engineering University](https://reader035.vdocuments.us/reader035/viewer/2022081409/607727a3dba5040a5010a479/html5/thumbnails/37.jpg)
37
Learning-free word spotting
◼ Learning free version of the previous work.
◼ Represent word images with local contour features.
◼ Word image matching Alignment by Hough voting
![Page 38: Computer Vision at the University of Ioannina...2020/09/20 · Computer Vision at the University of Ioannina Christophoros Nikou Department of Computer Science and Engineering University](https://reader035.vdocuments.us/reader035/viewer/2022081409/607727a3dba5040a5010a479/html5/thumbnails/38.jpg)
38
Learning-free word spotting (2)
◼ Word image matching Deformable non-rigid registration (TPS-RPM)
◼ False positive match
◼ True positive match
Giotis, G. Sfikas, C. Nikou and B. Gatos. Shape-based word spotting in handwritten document images. 13th International
Conference on Document Analysis and Recognition (ICDAR’15), 23-26 August 2015, Nancy, France, pp. 561-565.
![Page 39: Computer Vision at the University of Ioannina...2020/09/20 · Computer Vision at the University of Ioannina Christophoros Nikou Department of Computer Science and Engineering University](https://reader035.vdocuments.us/reader035/viewer/2022081409/607727a3dba5040a5010a479/html5/thumbnails/39.jpg)
39
Human action recognition
◼ Problem
Design accurate visual models for learning and analyzing human movements.◼ Fine differences between and within classes (e.g., running vs.
jogging).
◼ Short duration of human movements in time.
◼ Video annotation is time consuming.
◼ Variations in human poses, viewpoint, lighting conditions and frame resolution.
◼ Cluttered backgrounds.
![Page 40: Computer Vision at the University of Ioannina...2020/09/20 · Computer Vision at the University of Ioannina Christophoros Nikou Department of Computer Science and Engineering University](https://reader035.vdocuments.us/reader035/viewer/2022081409/607727a3dba5040a5010a479/html5/thumbnails/40.jpg)
40
Human action recognition
◼ Proposed methods
Matching mixtures of curves
Hidden conditional random fields (HCRF)
Learning using privileged information (LUPI) and HCRF (HCRF+)
Active learning with privileged information from weakly labeled samples (a-HCRF+)
Facial expression recognition using privileged information
![Page 41: Computer Vision at the University of Ioannina...2020/09/20 · Computer Vision at the University of Ioannina Christophoros Nikou Department of Computer Science and Engineering University](https://reader035.vdocuments.us/reader035/viewer/2022081409/607727a3dba5040a5010a479/html5/thumbnails/41.jpg)
41
Matching mixtures of curves
M. Vrigkas, V. Karavasilis, C. Nikou and I. Kakadiaris. Matching mixtures of curves for human action
recognition. Computer Vision and Image Understanding, Vol. 119, pp. 27-40, 2014.
![Page 42: Computer Vision at the University of Ioannina...2020/09/20 · Computer Vision at the University of Ioannina Christophoros Nikou Department of Computer Science and Engineering University](https://reader035.vdocuments.us/reader035/viewer/2022081409/607727a3dba5040a5010a479/html5/thumbnails/42.jpg)
42
Hidden conditional random fields
Friendly Aggressive Neutral
Parliament dataset
M. Vrigkas, C. Nikou and I. Kakadiaris. Identifying human behaviors using synchronized audio-visual cues.
IEEE Transactions on Affective Computing, Vol. 8, No 1, pp. 54-66, 2017.
◼ Recognize human activities that correspond to a specific emotional state of a person.
◼ Audio-visual data association with canonical correlation analysis (CCA).
![Page 43: Computer Vision at the University of Ioannina...2020/09/20 · Computer Vision at the University of Ioannina Christophoros Nikou Department of Computer Science and Engineering University](https://reader035.vdocuments.us/reader035/viewer/2022081409/607727a3dba5040a5010a479/html5/thumbnails/43.jpg)
43
HCRF+
M. Vrigkas, E. Kazakos, C. Nikou and I.A. Kakadiaris. Inferring human activities using robust privileged probabilistic
learning. 4th Workshop on Transferring and Adapting Source Knowledge in Computer Vision (TASK-CV), in conjunction
with the International Conference on Computer Vision (ICCV'17), Venice, Italy, October 22-29 2017.
◼ HCRF exploiting LUPI
![Page 44: Computer Vision at the University of Ioannina...2020/09/20 · Computer Vision at the University of Ioannina Christophoros Nikou Department of Computer Science and Engineering University](https://reader035.vdocuments.us/reader035/viewer/2022081409/607727a3dba5040a5010a479/html5/thumbnails/44.jpg)
44
a-HCRF+
M. Vrigkas, C. Nikou and I. Kakadiaris. Active privileged learning of human activities from weakly labeled samples.
IEEE International conference on Image Processing (ICIP’16), pp. 3036-3040, 25-28 September 2016, Phoenix,
Arizona, USA.
◼ LUPI and active learning Semi-supervised
Reduces tedious human effort for annotation
![Page 45: Computer Vision at the University of Ioannina...2020/09/20 · Computer Vision at the University of Ioannina Christophoros Nikou Department of Computer Science and Engineering University](https://reader035.vdocuments.us/reader035/viewer/2022081409/607727a3dba5040a5010a479/html5/thumbnails/45.jpg)
45
Facial expression recognition
using privileged information
M. Vrigkas, C. Nikou and I. Kakadiaris. Exploiting privileged information for facial expression recognition. IAPR/IEEE
International Conference on Biometrics (ICB’16), 13-16 June 2016, Halmstad, Sweden. Best paper award.
◼ HCRF and LUPI with knowledge transfer from privileged to observable space
◼ Penalize training in observable space using the weights learnt from the privileged space
![Page 46: Computer Vision at the University of Ioannina...2020/09/20 · Computer Vision at the University of Ioannina Christophoros Nikou Department of Computer Science and Engineering University](https://reader035.vdocuments.us/reader035/viewer/2022081409/607727a3dba5040a5010a479/html5/thumbnails/46.jpg)
46
Human identification
◼ Problem
Predict soft biometrics from still images◼ Gender, height,…
Predict visual attributes from still images◼ “wears red shirt”, “has backpack”,
◼ Proposed methods
Gender and height prediction using privileged information
Classification of visual attributes◼ Multi-task learning
◼ Curriculum learning
![Page 47: Computer Vision at the University of Ioannina...2020/09/20 · Computer Vision at the University of Ioannina Christophoros Nikou Department of Computer Science and Engineering University](https://reader035.vdocuments.us/reader035/viewer/2022081409/607727a3dba5040a5010a479/html5/thumbnails/47.jpg)
47
Predict the gender from images
◼ Learning using privileged information
◼ Ratios of anthropometric measurements
◼ SVM+
I.A. Kakadiaris, N. Sarafianos, C. Nikou, “Show me your body: gender classification from still images,” in Proc. IEEE
International Conference on Image Processing (ICIP’17), Phoenix, AZ, Sep. 25-28, 2016
![Page 48: Computer Vision at the University of Ioannina...2020/09/20 · Computer Vision at the University of Ioannina Christophoros Nikou Department of Computer Science and Engineering University](https://reader035.vdocuments.us/reader035/viewer/2022081409/607727a3dba5040a5010a479/html5/thumbnails/48.jpg)
48
Predict the height from images
◼ Learning using privileged information
◼ Ratios of anthropometric measurements
◼ Predict the privileged information at testing in a regression setup
◼ ε-SVR+
N. Sarafianos, C. Nikou and I.A. Kakadiaris, “Predicting privileged information for height estimation,” in Proc. IAPR
International Conference on Pattern Recognition (ICPR’17), Cancun, Mexico, Dec. 4-8, 2016.
![Page 49: Computer Vision at the University of Ioannina...2020/09/20 · Computer Vision at the University of Ioannina Christophoros Nikou Department of Computer Science and Engineering University](https://reader035.vdocuments.us/reader035/viewer/2022081409/607727a3dba5040a5010a479/html5/thumbnails/49.jpg)
49
Predict visual attributes
N. Sarafianos, Th. Giannakopoulos, C. Nikou and I. A. Kakadiaris. Curriculum learning of visual attribute clusters for
multi-task classification. Pattern Recognition, Vol. 80, pp. 94-108, 2018.
◼ Multi-task learning
◼ Curriculum learning
◼ Split tasks into groups based on their correlation
◼ ConvNet
![Page 50: Computer Vision at the University of Ioannina...2020/09/20 · Computer Vision at the University of Ioannina Christophoros Nikou Department of Computer Science and Engineering University](https://reader035.vdocuments.us/reader035/viewer/2022081409/607727a3dba5040a5010a479/html5/thumbnails/50.jpg)
50
Predict visual attributes (2)
N. Sarafianos, Th. Giannakopoulos, C. Nikou and I. A. Kakadiaris. Curriculum learning of visual attribute clusters for
multi-task classification. Pattern Recognition, Vol. 80, pp. 94-108, 2018.
![Page 51: Computer Vision at the University of Ioannina...2020/09/20 · Computer Vision at the University of Ioannina Christophoros Nikou Department of Computer Science and Engineering University](https://reader035.vdocuments.us/reader035/viewer/2022081409/607727a3dba5040a5010a479/html5/thumbnails/51.jpg)
51
Research activities
◼ Low level computer vision Video colorization
◼ Mid-level computer vision Image segmentation
Image feature analysis
Visual tracking
◼ High level computer vision Handwritten document indexing using word spotting
Human action recognition
Human identification
◼ Biomedical image analysis Cytological image analysis
![Page 52: Computer Vision at the University of Ioannina...2020/09/20 · Computer Vision at the University of Ioannina Christophoros Nikou Department of Computer Science and Engineering University](https://reader035.vdocuments.us/reader035/viewer/2022081409/607727a3dba5040a5010a479/html5/thumbnails/52.jpg)
52
Pap smear image analysis
M. Plissiti, P. Dimitrakopoulos, G. Sfikas, C. Nikou, O. Krikoni and A. Charchanti. SPIKAMED : a new dataset for
feature and image based classification of normal and pathological cervical cells in Pap smear images. 25th IEEE
International Conference on Image Processing (ICIP’2018), Athens, Greece, 7-10 October 2018.
◼ Nuclei detection and segmentation
◼ SPIKAMED
annotated benchmark for cell classification tonormal and pathological
morphometric features
Image features (CNN)
![Page 53: Computer Vision at the University of Ioannina...2020/09/20 · Computer Vision at the University of Ioannina Christophoros Nikou Department of Computer Science and Engineering University](https://reader035.vdocuments.us/reader035/viewer/2022081409/607727a3dba5040a5010a479/html5/thumbnails/53.jpg)
53
Data augmentation with a GAN
P. Dimitrakopoulos, G. Sfikas and C. Nikou. ISING-GAN: Annotated data augmentation with a spatially constrained generative adversarial network. IEEE International Symposium on Biomedical Imaging (ISBI’20), 3-7 April 2020, Iowa City, Iowa, USA.
◼ Synthesize new images from a data set
◼ MRF prior to encourage smoothness
Real images
Artificial images
![Page 54: Computer Vision at the University of Ioannina...2020/09/20 · Computer Vision at the University of Ioannina Christophoros Nikou Department of Computer Science and Engineering University](https://reader035.vdocuments.us/reader035/viewer/2022081409/607727a3dba5040a5010a479/html5/thumbnails/54.jpg)
54
GAN performance evaluation
P. Dimitrakopoulos, G. Sfikas and C. Nikou. WIND: Wasserstein inception distance for evaluating generative adversarial network performance. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP’20), 4-8 May, 2020, Barcelona, Spain.
◼ Wasserstein inception distance (WID) to replace Fréchet inceptioncdistance (FID)
◼ Measure the distance between:
The distribution of the training set
The distribution of the images generated by the GAN
◼ Gaussian mixture models
◼ Wasserstein distance between mixtures
◼ It captures cases that FID evaluates as similar
![Page 55: Computer Vision at the University of Ioannina...2020/09/20 · Computer Vision at the University of Ioannina Christophoros Nikou Department of Computer Science and Engineering University](https://reader035.vdocuments.us/reader035/viewer/2022081409/607727a3dba5040a5010a479/html5/thumbnails/55.jpg)
55
GAN performance evaluation
P. Dimitrakopoulos, G. Sfikas and C. Nikou. WIND: Wasserstein inception distance for evaluating generative adversarial network performance. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP’20), 4-8 May, 2020, Barcelona, Spain.
FID=0 for these distributions