a computer analysis of structures in image sequences...

A computer analysis of structures in image sequences: methods and applications

João Manuel R. S. Tavares

[email protected] www.fe.up.pt/~tavares

Mathematical Aspects of Imaging, Modeling and Visualization in Multiscale BiologyMarch 31 – April 4, 2009, The University of Texas at Austin, USA

Outline

• Presentation• Introduction• Tasks and Applications

– Segmentation– Tracking– Matching, Registration and Morphing– 3D Reconstruction

• Research Projects & PhD Thesis• Scientific Events and Publications• Research Team• Conclusions and Future Work

João Manuel R. S. Tavares 2A computer analysis of structures in image sequences: methods and applications

Presentation

Presentation

• Assistant Professor at Faculty of Engineering of University of Porto (FEUP) / Department of Mechanical Engineering and Industrial Management

• Senior Research and Projects Coordinator of the Optics and Experimental Mechanics Lab (LOME) of the Institute of Mechanical Engineering and Industrial Management (INEGI)

• PhD and MSc degrees in Electrical and Computer Engineering from FEUP in 2001 and 1995, respectively

• BSc degree in Mechanical Engineering from FEUP in 1992• Research Areas: Image Processing and Analysis, Human-

Computer Interaction, Product Development


FEUP: Identity

• With more than 80 years of history, FEUP is the largest school of Universidade do Porto; since 2000, FEUP has moved to a brand new campus, just outside the city centre, in eastern Porto, Portugal

João Manuel R. S. Tavares A computer analysis of structures in image sequences: methods and applications 5

FEUP: PORTO World heritage


FEUP: Some Numbers


Courses– 3 undergraduate courses (1st Cycle), 2 of which are in cooperation– 9 integrated master in engineering (1st + 2nd Cyles)– 8 MSc programmes (2nd Cycle)– 18 PhD programmes (3rd Cycle)

Academic community– full time teaching and research staff 432 FTE – 77% Doctorate– administrative and technical staff 231 staff | 800 other contracts– undergraduate students (1st + 2nd cycles) 5844– master and doctorate students 532 + 503

Scientific production– 1.1 ISI paper / permanent staff / year– 41 active or pending patents and brands– 18 spin-offs

INEGI: Identity

• The Institute of Mechanical Engineering and Industrial Management (INEGI) is an interface Institution between University and Industry, founded in 1986

• INEGI is oriented to activities of Research and Development (R&D), Innovation and Technology Transfer

• INEGI has two R&D units financed and evaluated by the Fundaçãopara a Ciência e a Tecnologia (FCT): the Experimental Mechanics and New Materials Unit (EXPMAT) and the New Technologies and Advanced Production Processes Unit (NOTEPAP)

• The two research units that are part of the Associated Lab for Energy, Transports and Aeronautics (LAETA), founded in 2006 and member of the Portuguese Council of the Associated Labs (CLA)


INEGI: Identity

• The Experimental Mechanics and New Materials Unit (EXPMAT) is supported by a staff of more than 60 researchers (31 of which have a PhD degree)

• EXPMAT is involved in partnership with more than 30 foreign companies, universities and other research organizations in many research projects under European or more World-wide programs

• Recently, EXPMAT has been developing considerable research in Bioengineering, Biomedical and Computational Vision fields


INEGI: Identity


• INEGI is located in the FEUP Campussince 2007

Introduction: Computer Analysis of Structures in

Image Sequences

Computer Analysis of Structures in Image Sequences

• The sensorial vision system has raised importance for numerous living organisms– Making possible the processing of information of the most basic

sort, as to check the existence of obstacles, or of the most complex kind, as the tracking and analysis of objects movement

– Common operations are: identification of objects (segmentation), tracking of movement (tracking and analysis), objects matching and registration (matching and registration), reconstruction of 3D shapes from images (3D reconstruction)


• The researchers of the Computational Vision domain aim the development of computational algorithms to accomplish the operations and tasks performed by the (awfully complex) human’s vision system in a full or semi-automatic manner

João Manuel R. S. Tavares 13A computer analysis of structures in image sequences: methods and applications 13

Original images

Azevedo et al. (2007), 3D Volumetric Reconstruction and Characterization of Objects from Uncalibrated Images, VIIP 2007, Spain, 141-146

Computational 3D model built: voxelized and

poligonized


• Examples of common tasks involved in the computer analysis of structures in images are: noise removal, geometric correction, data compression, segmentation (2D/3D), tracking and analysis of movement (2D/3D), matching and registration (2D/3D) and 3D reconstruction

• Domains where algorithms of computer analysis of structures in images are frequently used: Medicine, Biology, Industry, Engineering, Biomechanics, Virtual Reality, etc.



Tasks and Applications:Structures Segmentation

Structures Segmentation

• It is intended to identify in a full automatically, or semi-automatically, way the structures (2D/3D) presented in an image or along image sequences

• The most common methodologies used are based on statistical, geometric or physical modeling

• It is one of the most usual operations involved in the computer analysis of structures in images, being frequently the first “important” considered task

• Common problems: noise, low resolution, reduced contrast, shapes not previously known, partial occlusions, multiple structures presented, etc.


• Example: contours segmentation in dynamicpedobarography (Otsu method)



Original images After segmentation

Bastos & Tavares (2004), Improvement of Modal Matching Image Objects in Dynamic Pedobarography using Optimization Techniques, Lecture Notes in Computer Science 3179:39-50

camera mirror

contact layer + glass

reflected light glass

pressure opaque layer

lamp

lamp transparent layer


• Example: analysis of the damage in composite materials (binarization and region analysis)


Original image After segmentation

Damage areaobtained

Measures obtained

Marques et al. (in press), Composites Science and Technology, DOI: 10.1016/j.compscitech.2009.01.025


• Example: analysis of materials microstructures (using neural networks)


Original images After segmentation

Albuquerque et al. (2008), A New Solution for Automatic Microstructures Analysis fromImages Based on a Backpropagation Artificial Neural Network, Nondestructive Testing and Evaluation 23(4):273-283

• Example: control of a servomechanism bygestual language (orientation histograms)


Tavares et al. (2005), Control a 2-Axis Servomechanism by Gesture Recognition usinga Generic WebCam, International Journal of Advanced Robotic Systems 2(1):39-44

Running mode


Conversion for 256 gray levels

Orders orientation histogram vectors

initialization (stored in memory)

1 2 3 4

Orders images acquisition (one by one)

Command image

acquisition Conversion for 256

gray levels

Comparison of the images orientation histogram vector

with the stored vectors of the preset orders images

Associated command

order

1 3 4

1 2 4

1 2 3

2 3 4

4

2

1

Learning mode

• Example: detention of tumors in mammography images (Hough transform)


Chagas et al. (2007), An Application of Hough Transform to Identify Breast Cancer in Images, VIPimage 2007, Portugal, 363-368

Original image After segmentation


• Example: image identification (image template matching)


Carvalho & Tavares (2005), Metodologias para identificação de faces em imagens: Introdução e exemplos de resultados, CMNI 2005, España


×fft fft

ift

( )3ift D CC( )2ift D CC

max CC

Original imageTemplate image

• Example: segmentation using deformablegeometric templates


Carvalho & Tavares (2006), Two Methodologies for Iris Detection and Location in Face Images, CompIMAGE 2006, Portugal, 129-134Carvalho & Tavares (2007), Eye detection using a deformable template in static images, VipIMAGE 2007, Portugal, 209-215

Original image and associated energy fields

Segmentation of the iris using adeformable template (a circle)

Segmentation of an eye using an deformable

template


• Example: segmentation of faces in images using statistical models for the skin


Samples used to build the statistical model Original image and

segmentation obtained

Carvalho & Tavares (2005), Metodologias para identificação de faces em imagens: Introdução e exemplos de resultados, CMNI 2005, España


Probably function used


Background subtraction method

Foreground object detection method

• Example: background/foreground scene segmentation using statistical models

Vasconcelos & Tavares (2008), Image Segmentation for Human Motion Analysis: Methods and Applications, WCCM8 / ECCOMAS 2008, Italy



• Example: segmentation using approaches based on point distributions models (Active Shape Models, Active Appearance Models)


Segmentation using Active Shape Models

Vasconcelos & Tavares (2006), Methodologies to Build Automatic Point Distribution Models for Faces Represented in Images, CompIMAGE 2006, Portugal, 435-440


• Example: segmentation using approaches based on point distributions models (Active Shape Models, Active Appearance Models)


Vasconcelos & Tavares (2008), Methods to Automatically Built Point Distribution Models for Objects like Hand Palms and Faces Represented in Images, Computer Modeling in Engineering & Sciences 36(3):213-241

Segmentation using an Active Appearance Model

Original image Resultant image



• Example: segmentation in the analysis of the vocal tract in speech production (Active Shape Models)

Vasconcelos et al. (2009), Analysis of Tongue Shape and Motion in Speech Production using Statistical Modeling, 2nd South-East European Conf. on Computational Mechanics, Greece

Image labeled Original shape Segmentation

• Example: segmentation using deformable physical models (active contours - snakes)


Original image andinitial contour

Final contour

Tavares et al. (in press), Computer Analysis of Objects’ Movement in Image Sequences: Methods and Applications, International Journal for Computational Vision and Biomechanics


• Example: segmentation using deformable physical models (FEM, Lagrange equation)


Original image and initial contour Final contourGonçalves et al. (2008), Segmentation and Simulation of Objects Represented in Images using Physical Principles, Computer Modeling in Engineering & Sciences 32(1):45-55


rubberk = 200N/m14s

• Example: segmentation of medical images using level-sets methods


Original image Initial segmentation Final segmentation

Perdigão et al. (2005), Geração de modelos de malhas de elementos finitos a partir de imagens médicas 2D, Encontro_1_Biomecânica, Portugal, 81-85


• Example: segmentation of medical images using a new computational framework (VC++, OpenCV, ITK)


Interface of our platform

Ma et al. (2008), Segmentation of Structures in Medical Images: Review and a New Computational Framework, CMBBE2008, Portugal


• Example: segmentation of images of the pelvic cavity (using our platform)


Ma et al. (in press), A Review of Algorithms for Medical Image Segmentation and their Applications to the Female Pelvic Cavity, Computer Methods in Biomechanics and Biomedical Engineering


Region Growing Watershed

Malladi’s AlgorithmGeodesic Active Contour

Tasks and Applications:Structures Tracking

Structures Tracking

• It is intended to track the movement (and/or the deformation) of structures along image sequences

• In this area, the methodologies based on block matching and stochastic methods are common

• Usually, it involves the estimation of the movement involved, the management of the entities being tracked, the analysis of the movement tracked as well as its quantification

• Usual problems: non-rigid movement, geometric distortions, non-constant illumination conditions, occlusion, noise, multiple structures, etc.


Structures Tracking

• Example: contours tracking in dynamic pedobarography(FEM, Modal Analysis)


Tavares & Bastos (2005), Improvement of Modal Matching Image Objects in Dynamic Pedobarography using Optimization Techniques, Electronic Letters on Computer Vision and Image Analysis 5(3):1-20

Contours tracking

Structures Tracking


Prediction Uncertainty Area Measurement Correspondence Result

• Example: tracking in gait analysis (Kalman, optimization)

Pinho et al. (2005), Human Movement Tracking and Analysis with Kalman Filtering and Global Optimization Techniques, ICCB 2005, Portugal, 915-926

João Manuel R. S. Tavares A computer analysis of structures in image sequences: methods and applications

Structures Tracking

• Example: tracking in gait analysis withdetention of events (Kalman, optimization)

Sousa et al. (2007), Registration between Data from Visual Sensors and Force Platform in Gait Event Detection, ISHF2007, Portugal, 331-340Sousa et al. (2007), Selecting Biomechanical Variables for Detect Gait Events using Computational Vision, ICCB 2007, Venuzuela, 291-296

38

Structures Tracking

• Example: structures tracking along lengthy image sequences (Kalman, optimization, features management model)


Pinho et al. (2007), Efficient Approximation of the Mahalanobis Distance for Tracking with the Kalman Filter, International Journal of Simulation Modelling 6(2):84-92

(547 frames)

Pinho et al. (2005), A Movement Tracking Management Model with Kalman Filtering, Global Optimization Techniques and Mahalanobis Distance, LSCCS, Vol. 4A:463-466

Structures Tracking

• Our actual goal: Dynamical model(s) - Learning algorithm(s) to estimate the best models parameters


Several objects = One dynamical model

Large image sequences = One dynamical model

Tasks and Applications:Structures Matching,

Registration and Morphing

Structures Matching, Registration and Morphing


• Matching– It is regularly used in the computer analysis of structures in

images, for example, to register (align) structures in images, recognize structures, obtain 3D information, analyze the movement tracked, etc.

– Generally, it is accomplished through the consideration of characteristic invariants, as the curvature, or the displacements in global spaces, as the modal space

– Common problems: occlusion, non-rigid deformations, high shapes variations, etc.



• Registration– It is commonly required to compare structures represented in

images acquired in different time instants or according distinct conditions

– It is essential, for example, in Medicine to follow up the evaluation of patients’ diseases from images

– Generally, it is accomplished through the consideration of structures characteristic features, as maximum curvature points, their matching and the estimation of the involved transformation

– Common problems: key and invariant features not easily identified, occlusion, non-rigid deformations, high shapes variations, etc.



• Morphing (simulation)– It is a very used task in Computer Graphics but also very useful in

the analysis of structures in images, for example, to estimate the deformation involved between two distinct structures or between two configurations of one structure, to estimate the transitions between two shapes acquired with high temporal gap, etc.

– Generally, it is accomplished through the consideration of geometric transformations

– However, when it must be considered the physical behavior of the involved structures, physical methodologies and modeling, as for example FEM, should be employed

• Common difficulties are related with the estimation of the involved forces and with the properties of the adopted (virtual) material

• The adequate matching of the structures involved becomes crucial

Structures Matching

• Example: matching contours and surfaces in dynamic pedobarography (FEM, Modal Analysis, optimization)


Image of dynamic pedobarography

Initial contour Final contour


Matching found


Matchings found betweeniso-contours

Matchings found betweensurfaces

Structures Matching

• Example: matching contours and surfaces in dynamic pedobarography (FEM, Modal Analysis, optimization)



Structures Matching

• Example: matching contours with order restriction (geometrical modeling, optimization, dynamic programming)

Oliveira & Tavares (2008), Algorithm of dynamic programming for optimization of the global matching between two contours defined by ordered points, Computer Modeling in Engineering & Sciences 31(1):1-11

Original images and contoursMatching found without

order restrictionMatching found with order

restriction

Bastos & Tavares (2006), Matching of Objects Nodal Points Improvement using Optimization, Inverse Problems in Science and Engineering 14(5):529-541


Structures Morphing

• Example: morphing contours (FEM, Modal Analysis, optimization)

Matching found Estimated deformations

Gonçalves et al. (2008), Segmentation and Simulation of Objects Represented in Images using Physical Principles, Computer Modeling in Engineering & Sciences 32(1):45-55

Contours matched

Estimateddeformations

Tavares & Pinho (2005), Estimação Temporal da Deformação entre Objectos utilizando uma Metodologia Física, InfoComp 4(1):9-18

Original images


Structures Morphing

• Example: morphing contours (FEM, Modal Analysis, optimization)

Matching found Deformations estimated

Gonçalves et al. (2008), Segmentation and Simulation of Objects Represented in Images using Physical Principles, Computer Modeling in Engineering & Sciences 32(1):45-55

Original images


Structures Registration

• Example: registration of contours in images (geometrical modeling, optimization, dynamic programming)

Original images and contours found

Matched contours before registration

Matched contours after registration

Oliveira & Tavares (in press), Matching Contours in Images using Curvature and Distance to Centroid Information and Dynamic Programming, Computer Modeling in Engineering & Sciences



• Example: registration of images in dynamic pedobarography (geometrical modeling, optimization, dynamic programming)

Original images and contours found Contours and images

before registrationContours and images

after registration



• Example: registration of images in dynamic pedobarography (geometrical modeling, optimization, dynamic programming)

Original images and contours found

Contours matched before registration

Images after registration



• Example: registration of brain imagesin the diagnosis of multiple sclerosislesions

Source image Target image

After rotation

After scale

After translation

Jacob et al. (2009), Algoritmos para Alinhamento de Imagens Médicas: Princípios e Aplicação em Imagens de Esclerose Múltipla, 3º Congresso Nacional de Biomecânica, Portugal, 549-556

Tasks and Applications:Structures 3D Reconstruction

Structures 3D Reconstruction

• It is intended to accomplish the 3D reconstruction of structures from images

• In this area, the following methodologies are common: external shapes – active techniques (with energy projection or relative movement), passive techniques (without energy projection or relative movement) and of space carving; interior shapes – 2Dsegmentation (contours, for example) and interpolation

• Usually, it involves tasks of camera calibration, data segmentation, matching, triangulation and interpolation

• Common problems: geometric distortion, bad or varying illumination conditions, occlusion, noise, multiple structures, complex shapes, etc.



• Example: 3D reconstruction of organs from medical images

João Manuel R. S. Tavares 56Análise de Estruturas em Imagens: Segmentação, Seguimento e Reconstrução 3D

Segmentation done in a slice and 3D reconstruction obtained

3D reconstruction of some structures of the arm

Perdigão et al. (2005), Sobre a Geração de Malhas Tridimensionais para fins Computacionais a partir de Imagens Médicas, CMNI 2005, España



Segmentation donein a slice

Pelvic floor reconstructed Reconstructed structures of the pelvic cavity

Pimenta et al. (2006), Reconstruction of 3D Models from Medical Images: Application to Female Pelvic Organs, CompIMAGE 2006, Coimbra, Portugal, 343-348Alexandre et al. (2007), 3D reconstruction of pelvic floor for numerical simulation purpose, VipIMAGE 2007, Porto, Portugal, 359-362

• Example: 3D reconstruction of organs from medical images

slices


• Example: 3D reconstruction using techniques of active vision


Disparity mapobtained

Original image pair

Azevedo et al. (2006), Development of a Computer Platform for Object 3D Reconstruction using Active Vision Techniques, VISAPP 2006, Portugal, 383-388


• Example: 3D reconstruction using space carving


Azevedo et al. (2008), 3D Object Reconstruction from Uncalibrated Images using an Off-the-Shelf Camera, Advances in Computational Vision and Medical Image Processing: Methods and Applications, 117-136

Original images Computational 3D model built: voxelized and poligonized



Azevedo et al. (2008), 3D Object Reconstruction from Uncalibrated Images using an Off-the-Shelf Camera, Advances in Computational Vision and Medical Image Processing: Methods and Applications, 117-136

Original images Computational 3D model built: voxelized and poligonized

• Example: 3D reconstruction using space carving

Human Perception andData Visualization

Human Perception and Data Visualization

• Recently, some projects involving the visualization and analysis of complex, large and crucial data have been developed based on principles of human perception– visualization and analysis of huge volumes of data– visualization and analysis of temporal critical data



• Example: visualization and analysis of huge volumes of computational data


Gonçalves & Tavares (2008), A GUI for a Software that Analyses a Composite Bolted Joint, e-Minds: International Journal on Human-Computer Interaction I(4):25-41

Computational interface developed to visualize and analyze the data of a composite bolted joint simulator


• Example: visualization and analysis of critical data in air traffic control (ATC)


Sousa & Tavares (submitted), ATCBlur: an Air Traffic Control display filtering tool, International Journal of Human-Computer Studies

Computational interface developed to analyze and evaluate the use of visualization filters based on human perception in tasks of visual searching in ATC displays

Research Projects &PhD Thesis

Research Projects & PhD Thesis

• In the last years, we have been involved in some national and international research projects, as research team members as well as project coordinators

• Currently, we have several PhD thesis undergoing considering important topics of image processing and analysis


See: www.fe.up.pt/~tavares


• Research Projects (in course)– Biomechanical study of the middle ear for auditory

rehabilitationCoordinator: Renato M. Natal Jorge, FEUPInstitution: IDMEC - Polo FEUPFinancing Institution: FCT, 50.000,00 €Duration: 36 months, started in July 2007

– BIOPELVIC - Study of Female Pelvic Floor DisordersCoordinator: Renato M. Natal Jorge, FEUPInstitutions: IDMEC - Polo FEUP, FMUP, INESC-PortoFinancing Institution: FCT, 198.816,00 €Duration: 36 months, started in July 2007




• Research Projects (in course)– Rede Luso-Brasileiro de Bioengenharia

Coordinator: Renato M. Natal Jorge, FEUPFinancing Institution: GRICES/CAPES 2007Duration: 36 months, started in January 2007

– Aberrant Crypt Foci and Human Colorectal Polyps: mathematical modelling and endoscopic image processingCoordinator: Isabel Figueiredo, FCT/UC-PTInstitutions: FCT/UC-PT, FEUP-PT, FM/UC-PT, IT-PT, UTA-USAFinancing Program: UT Austin|Portugal 2008Duration: 48 months, started in April 2009 (?)




• Research Projects (in course)– Cardiovascular Imaging Modeling and Simulation - SIMCARD

Coordinator: Adélia Sequeira, IST-PTInstitutions: IST-PT, FEUP-PT, HSM-PT, SPRM-PT, UTA-USAFinancing Program: UT Austin|Portugal 2008Duration: 24 months, started in April 2009 (?)




• Research Projects (finished)– BioSpine – Development of External Biocompatible

Prosthesis/VestsCoordinator: Manuel Laranjeira Gomes, ICBASInstitutions: INEGI, INEB, ICBASFinancing Institution: ADI, 100.000,00 €Duration: 24 months, started in January 2007

– ACTIDEF – Integrated Computational and Technological Evaluation of the Performance and Functionality of Citizens with Muscular IncapacitiesCoordinator: Jerónimo de Sousa, CRPGInstitutions: CRPG, INEB, INEGIFinancing Program: POS_C, 150.000,00 €Duration: 12 months, started in February 2006




• Research Projects (finished)– Segmentation, Tracking and Motion Analysis of Deformable

(2D/3D) Objects using Physical PrinciplesCoordinator: João Manuel R. S. TavaresParticipant Institutions: INEGI, INEBFinancing Institution: FCT, 63.420,00 €Duration: 36 months, started in May 2005




• PhD Thesis (in course)– Analysis of Movement using Computational Vision

Student: Raquel R. PinhoSupervisor: João Manuel R. S. TavaresCo-supervisor: Miguel V. Correia, FEUPPhD in Engineering Sciences, University of Porto, since April 2003

– Application of Physical Principles in the Analysis of Objects Represented in ImagesStudent: Patrícia C. T. GonçalvesSupervisor: João Manuel R. S. TavaresCo-supervisor: R. M. Natal Jorge, FEUPPhD in Engineering Sciences, University of Porto, since February 2006




• PhD Thesis (in course)– Computational Modeling for the Analysis of the Mechanical

Behavior of Structures represented in Image Sequences: Applications in Medical ImagesStudent: Ilda M. de Sá ReisSupervisor: João Manuel R. S. TavaresCo-supervisors: R. M. Natal Jorge, FEUP, A. F. Leite-Moreira, FMUPPhD in Engineering Sciences, University of Porto, since March 2007

– Unsupervised Geometrical Modelling of the Scoliotic Spine from RadiographsStudent: Daniel C. de MouraSupervisor: Jorge G. Barbosa, FEUPCo-supervisor: João Manuel R. S. TavaresDoctoral Program in Informatics, University of Porto, since September 2006




• PhD Thesis (in course)– Modeling the Shape and Motion of Objects in Image Sequences:

Applications in Biomechanics of the Human BodyStudent: Maria J. M. de Vasconcelos Supervisor: João Manuel R. S. TavaresCo-supervisor: Miguel V. Correia, FEUPPhD in Engineering Sciences, University of Porto, since July 2007

– Reconstruction of 3D Objects Shape using Computational Vision: Applications in the Reconstruction and Characterization of External Anatomical StructuresStudent: Teresa C. S. AzevedoSupervisor: João Manuel R. S. TavaresCo-supervisor: Mário A. P. Vaz, FEUPPhD in Engineering Sciences, University of Porto, since October 2006




• PhD Thesis (in course)– Computational Methodologies for Three-Dimensional

Reconstruction and Recognition of Shape and Motion Patterns in the Analysis of Dynamic Fluid Flow in Automotive Conditional Air SystemsStudent: Elza M. P. F. ChagasSupervisor: Denilson L. Rodrigues, PCU-MG – BrazilCo-supervisor: João Manuel R. S. TavaresPhD Thesis in Mechanical Engineering, PCU-MG – Brazil, since January 2007




• PhD Thesis (in course)– Image Processing and Analysis in the Characterization of Parts

and OperationsStudent: Victor H. C. de AlbuquerqueSupervisor: João Manuel R. S. TavaresCo-supervisor: Luís M. P. Durão, ISEPPhD in Engineering Sciences, University of Porto, since July 2007

– Segmentation, Tracking and 3D Reconstruction of Structures Represented in Images: Applications in Medical ImagesStudent: Zhen MaSupervisor: João Manuel R. S. TavaresCo-supervisor: R. M. Natal Jorge, FEUPPhD in Engineering Sciences, University of Porto, since July 2007




• PhD Thesis (in course)– Matching and Registration of Structures in Computational

Vision: Applications in Medical ImagesStudent: Francisco P. M. de OliveiraSupervisor: João Manuel R. S. TavaresCo-supervisor: Durval C. Costa, HPP-Medicina Molecular, SA.Doctoral Program in Biomedical Engineering, University of Porto, since 2008/2009

– Neuronal Quantitative Evaluation in Tensor Diffusion ImagesStudent: Eduardo F. C. RibeiroSupervisor: João Manuel R. S. TavaresCo-supervisors: A. M. Reis, PHH, Chandrajit Bajaj, UTADoctoral Program in Biomedical Engineering, University of Porto, since 2008/2009




• PhD Thesis (in course)– Methodologies and Active Systems based on Sensorial Fusion

for Movement Analysis – Applications in BiomechanicsStudent: António F. N. GomesSupervisor: João Manuel R. S. TavaresCo-supervisor: Joaquim G. Mendes, FEUPDoctoral Program in Mechanical Engineering, University of Porto, since 2008/2009



Scientific Events & Publications

Scientific Events & Publications

• In the last years, we have been involved in the organization of several scientific events– mini-symposia, workshops and conferences

• and in the editing of several scientific publications– conferences proceedings books, special issue of journals and the

editing of scientific journals



Events & Publications

• Conferences, Symposia and Workshops



Co-chair: National Meeting on Scientific VisualizationPortugal, 17 September 2005

Co-chair: CompIMAGE - Computational Modelling of Objects Represented in Images: Fundamentals, Methods and ApplicationsPortugal, 20-21 October 2006

http://envc2005.multimeios.pt/�http://paginas.fe.up.pt/CompIMAGE/�





Thematic session “Image processing, Visualization and Interfaces”Métodos Numéricos em Engenharia (CMNE) & XXVIII CILAMCE - Congresso IberoLatino-Americano sobre Métodos Computacionais em EngenhariaPortugal, 13-15 June 2007

Co-chair: VIPimage 2007 - I ECCOMAS Thematic Conference on Computational Vision and Medical Image ProcessingPortugal, 17-19 October 2007

http://paginas.fe.up.pt/~cmne2007/�http://paginas.fe.up.pt/~vipimage/�





Mini-Symposium “Computational Methods in Image Analysis”The ninth U.S. National Congress on Computational Mechanics (USNCCM IX)USA, 22-26 July 2007

Mini-symposium “Computational Bioimaging and Visualization”VIII World Conference on Computational Mechanics (WCCM VIII) & V European Congress on Computational Methods in Applied Science and Engineering (ECCOMAS V)Italy, 30 June - 5 July 2008

http://www.me.berkeley.edu/compmat/USACM/main.html�http://www.iacm-eccomascongress2008.org/frontal/�





Co-chair: Workshop: Applications of a Movement Analysis LabPortugal, 23 March 2007

Mini-Symposium “Image Processing and Analysis”ICCES08 - International Conference on Computational and Experimental Engineering & SciencesHawaii, 17-22 March 2008

http://icces.org/cgi-bin/ices08/pages/index�





Mini-Symposium “Image Processing and Visualization”5º Congresso Luso-Moçambicano de EngenhariaMozambique, 2-4 September 2008

Workshop “Medical Imaging Systems”EUROMEDIA 2008 - The Multimedia Applications ConferencePortugal, 9-11 April 2008

http://paginas.fe.up.pt/clme/2008/index.htm�http://www.eurosis.org/cms/index.php?q=taxonomy/term/22�





Co-chair: EUROMEDIA 2008 - The Multimedia Applications ConferencePortugal, 9-11 April 2008

International Workshop on Combinatorial Image Analysis (IWCIA'08)USA, 7-9 April 2008

http://www.eurosis.org/cms/index.php?q=taxonomy/term/22�http://www.cs.fredonia.edu/iwcia08/�





Mini-Symposium “Biomechanical Behavior of Soft Tissues”Mini-Symposium “Computational Methods in Image Processing for Biomechanics”CMBBE2008 - 8th International Symposium on Computer Methods in Biomechanics and Biomedical EngineeringPortugal, 27 February - 1 March 2008

Special Track “Computational Bioimaging and Visualization”ISVC08 - 4th International Symposium on Visual ComputingUSA, 1-3 December 2008

http://www.cmbbe2008.cf.ac.uk/�http://www.cmbbe2008.cf.ac.uk/�http://www.isvc.net/ST7.pdf�





Co-chair: 1st International Conference on Imaging Theory and Applications (IMAGAPP)Portugal, 5-8 February 2009

Mini-Symposium “Image Processing and Analysis”ICCES'09 - Int. Conference on Computational and Experimental Engineering & SciencesThailand, 8-13 April 2009

http://www.imagapp.org/�http://icces.org/cgi-bin/ices09/pages/index�





Mini-Symposium “Image Processing and Data Visualization”SEECCM 2009 - 2nd South-East European Conference on Computational MechanicsGreece, 22-24 June 2009

Thematic Session “Image Processing, Visualization and Interfaces”Congreso de Métodos Numéricos en Ingeniería 2009Spain, 29 June - 2 July 2009

http://www.seeccm2009.org/index.php�http://congress.cimne.upc.es/MetNum09/frontal/default.asp�





Mini-Symposium “Computational Methods in Image Analysis”10th US National Congress of Computational MechanicsUSA, 16-19 July 2009

Mini-Symposium “Image Processing and Visualization in Solid Mechanics Processes”7th EUROMECH Solid Mechanics Conference – ESMC2009Portugal, 7-11 September 2009

http://usnccm-10.eng.ohio-state.edu/�http://www.dem.ist.utl.pt/esmc2009/index.php�





Workshop “Medical Imaging Systems”EUROMEDIA 2009 - The Multimedia Applications ConferenceBelgium, 15-17 April 2009

Mini-Symposium “Visualization and Human-Computer Interaction”IRF’2009 – Integrity - Reliability - Failure Challenges and OpportunitiesPortugal, 20-24 July 2009

http://www.eurosis.org/cms/index.php?q=node/870�http://paginas.fe.up.pt/clme/IRF2009/index.htm�





Co-chair: I International Conference on Biodental EnginneringPortugal, 26-27 June 2009

Co-chair: 6th International Conference on Technology and Medical SciencesPortugal, 21-22 October 2010

http://paginas.fe.up.pt/~biodenta�





Co-chair: VIPimage 2009 - II ECCOMAS Thematic Conference on Computational Vision and Medical Image ProcessingPortugal, 14-16 October 2009

Co-chair: CompIMAGE'2010 Symposium - Computational Modeling of Objects Represented in Images: Fundamentals, Methods and ApplicationsUSA, 5-7 May 2010

Submissions are Opened

http://www.fe.up.pt/~vipimage�


• Books, Journals and special issues



Co-editor-in-chief: Int. Journal for Computational Vision and Biomechanics (IJCV&B)ISSN: 0973-6778, Serials Publications

Co-editor: Proceedings of the ENVC2005 – National Meeting on Scientific VisualizationISBN: 972-99872-0-3, Fundação Navegar, 2005

Co-editor: Proceedings of the International Symposium CompIMAGE 2006ISBN: 9780415433495, Taylor and Francis, 2007

http://paginas.fe.up.pt/~ijcvb/�http://www.taylorandfrancis.co.uk/shopping_cart/products/product_detail.asp?sku=&isbn=9780415433495&parent_id=&pc=/shopping_cart/search/search.asp?�http://www.fe.up.pt/~ijcvb�





Guest-co-editor: Special issue of the International Journal of Simulation Modelling (IJSM) dedicated to CompIMAGE 2006ISSN 1726-4529, Vol. 6, N. 2, June 2007, pp. 61-136

Guest-co-editor: Special issue of Electronic Letters on Computer Vision and Image Analysis (ELCVIA) dedicated to CompIMAGE 2006ISSN 1577-5097, Vol. 7, N. 2, May 2008

http://www.ijsimm.com/�http://elcvia.cvc.uab.es/specialcmopri.php�





Co-editor: Proceedings of the I ECCOMAS Thematic Conference on Computational Vision and Medical Image Processing: VipIMAGE 2007ISBN: 9780415457774, Taylor and Francis, 2007

Co-editor: Advances in Computational Vision and Medical Image Processing: Methods and ApplicationsComputational Methods in Applied Sciences, Vol. 13 ISBN: 1402090854, Springer, 2008

http://www.taylorandfrancis.co.uk/shopping_cart/products/product_detail.asp?sku=&isbn=9780415457774&parent_id=2797&pc=/shopping_cart/categories/categories_products.asp?parent_id=2797&so=1�http://www.springer.com/computer/computer+imaging/book/978-1-4020-9085-1�





Guest-co-editor: Special issue of the International Journal of Tomography & Statistics on Image ProcessingISSN: 0973-7294, Vol. 15, N. W10, Winter 2010 & Vol. 16, N. S10, Summer 2010

Co-editor: Fourteenth Annual Scientific Conference on web Technology, New Media, Communications and Telematics Theory and ApplicationsISBN: 978-90-77381-38-0, EUROSIS-ETI, 2008

http://paginas.fe.up.pt/~tavares/downloads/calls/call_of_papers_ijts.pdf�





Guest-co-editor: Special issue of the EURASIP Journal on Advances in Signal Processing on Image Processing and Analysis in BiomechanicsISSN: 1687-6172, Hindawi Publishing Corporation

Co-editor: Proceedings of the 1st International Conference on Imaging Theory and Applications (IMAGAPP)ISSN: 978-989-8111-68-5, INSTICC Press, 2009

Submission is Opened

http://www.hindawi.com/journals/asp/si/ipab.html�http://www.imagapp.org/�

Research Team(Computational Vision)

Research Team (Computational Vision)

• PhD students (12):– In course: Raquel Pinho, Patrícia Gonçalves, Maria

Vasconcelos, Ilda Reis, Teresa Azevedo, Daniel Moura, Zhen Ma, Elza Chagas, Victor Albuquerque, Francisco Oliveira, Eduardo Ribeiro, Eduardo Gomes

• MSc students (13):– In course: Fernando Carvalho, Mauro Trindade, Lara Quintela,

Frederico Jacob, Verónica Marques– Finished: Daniela Sousa, Francisco Oliveira, Teresa Azevedo,

Maria Vasconcelos, Raquel Pinho, Luísa Bastos, CândidaCoelho, Jorge Gonçalves

• BSc students (2)– Finished: Ricardo Ferreira, Soraia Pimenta



• Collaborators:– Renato Natal Jorge, Joaquim Gabriel, Mário Vaz, Miguel

Velhote, Jorge Barbosa, Francisco Freitas (FEUP)– Nuno Correia (INEGI)– Luís Durão (ISEP)– Emília Mendes (CRPG)– Denilson Rodrigues (PUC - Minas Gerais, Brazil)– Diana Miranda, Georgeta Oliveira, Ricardo Duarte (HPH)– Ana Mafalda Reis, Manuel Laranjeira (ICBAS/INC)– Manuel Paulo (FMDUP)– Adelino Leite-Moreira (FMUP)– Filipa Sousa, João Paulo Vilas-Boas (FCDUP)



• Collaborators:– Durval C. Costa (HPP-Medicina Molecular, SA.)– Luis Metello (ESTSP)– Chandrajit Bajaj (UTA, USA)– Todd Pataky (University of Liverpool, UK)– Chris Constantinou (Stanford University of Medicine, USA)– Isabel Figueiredo (FCT/UC)– Adélia Cerqueira (IST)– …


Conclusions and Future Working Perspectives

Conclusions and Future Working Perspectives

• The computer analysis of structures in images is a very complex task, but of raised importance in many domains

• Numerous complex challenges exist, as for example, structures with topological variations, complex movements, adverse conditions in the image acquisition processes, etc.

• Much work was already developed in this Computational Vision area, but important and complex goals still to be successfully addressed

• Methods and methodologies of other research areas, as of Mathematics, Computational Mechanics and Biology, can contribute significantly for its development

• For that, collaborations are mostly welcome


Mathematical Aspects of Imaging, Modeling and Visualization in Multiscale BiologyMarch 31 – April 4, 2009, The University of Texas at Austin, USA

A computer analysis of structures in image sequences: methods and applications

João Manuel R. S. Tavares

[email protected] www.fe.up.pt/~tavares

Slide Number 1OutlinePresentationPresentationFEUP: IdentityFEUP: PORTO World heritageFEUP: Some NumbersINEGI: IdentityINEGI: IdentityINEGI: IdentityIntroduction: Computer Analysis of Structures in Image SequencesComputer Analysis of Structures in Image SequencesComputer Analysis of Structures in Image SequencesComputer Analysis of Structures in Image SequencesTasks and Applications:�Structures SegmentationStructures SegmentationStructures SegmentationStructures SegmentationStructures SegmentationStructures SegmentationStructures SegmentationStructures SegmentationStructures SegmentationStructures SegmentationStructures SegmentationStructures SegmentationStructures SegmentationStructures SegmentationStructures SegmentationStructures SegmentationStructures SegmentationStructures SegmentationStructures SegmentationTasks and Applications:�Structures TrackingStructures TrackingStructures TrackingStructures TrackingStructures TrackingStructures TrackingStructures TrackingTasks and Applications:�Structures Matching, Registration and MorphingStructures Matching, Registration and MorphingStructures Matching, Registration and MorphingStructures Matching, Registration and MorphingStructures MatchingStructures MatchingStructures MatchingStructures MorphingStructures MorphingStructures RegistrationStructures RegistrationStructures RegistrationStructures RegistrationTasks and Applications:�Structures 3D ReconstructionStructures 3D ReconstructionStructures 3D ReconstructionStructures 3D ReconstructionStructures 3D ReconstructionStructures 3D ReconstructionStructures 3D ReconstructionHuman Perception and�Data VisualizationHuman Perception and Data VisualizationHuman Perception and Data VisualizationHuman Perception and Data VisualizationResearch Projects &�PhD ThesisResearch Projects & PhD ThesisResearch Projects & PhD ThesisResearch Projects & PhD ThesisResearch Projects & PhD ThesisResearch Projects & PhD ThesisResearch Projects & PhD ThesisResearch Projects & PhD ThesisResearch Projects & PhD ThesisResearch Projects & PhD ThesisResearch Projects & PhD ThesisResearch Projects & PhD ThesisResearch Projects & PhD ThesisResearch Projects & PhD ThesisScientific Events & PublicationsScientific Events & PublicationsEvents & PublicationsEvents & PublicationsEvents & PublicationsEvents & PublicationsEvents & PublicationsEvents & PublicationsEvents & PublicationsEvents & PublicationsEvents & PublicationsEvents & PublicationsEvents & PublicationsEvents & PublicationsEvents & PublicationsEvents & PublicationsEvents & PublicationsEvents & PublicationsEvents & PublicationsEvents & PublicationsResearch Team�(Computational Vision)Research Team (Computational Vision)Research Team (Computational Vision)Research Team (Computational Vision)Conclusions and Future Working PerspectivesConclusions and Future Working PerspectivesSlide Number 105

a computer analysis of structures in image sequences...

Documents