LibreOffice Productivity Suite

Computer vision Engineering: applications in industry, medicine & neuroscience

Carmen Alonso Montescalonso@bcamath.org

Work funded by Ministerio de Educación, Cultura y Ciencia, through the Becas de movilidad José Castillejo 2016

Contents

Computer vision: Industrial & Medical applicationsCoded target application for 3D positioning in tool machinesAutomatic retinal vessel tree extraction

Personal authentication

Arteriolar-to-venular ratio estimation

Neuroimage: A window to see & understand the brain “in vivo”The brainImaging modalitiesBrain imaging applications

Linking Structural & Functional connectivity

Searching Alzheimer disease biomarkers

Blood flow simulation: A window for disease evolution

Connecting Computer Vision to FEniCS

Computer vision

Goal: To extract information from images through artificial systems to produce numerical or symbolic information, e.g. in the forms of decisions.

Machine Vision

Image processing within industrial environments is usually called machine visionTypical issues: dust, fat, poor illumination conditions, real-time constraints, physical restrictions within the machinesTypical examples of applications on this field are used in

Laser-based process: e.g. laser cladding, welding inspection, etcNon Destructive Techniques (NDT) for railway inspectionIdentification of Artificial Markers for machining process

Coded-target (CT) for 3D positioning

Usage of CT for 3D positioning respect to the laser head.

Wireless communication among cameras and PC

Issue: Electrical interferences and loss of communication speed.

Real-time processing but adequate them to realistic laser head movements

Issues to physically place the camera

Not all camera models were valid

Fernandez-Fernandez M., Alonso-Montes C., Bertelsen A., Mendikute A. Industrial Non-Intrusive Coded-Target Identification and Decoding Application Pattern Recognition and Image Analysis 7887 (2013) 790-797

Medical Computer Vision

Medical imaging data is used in applications to support clinicians in the diagnosis, treatment and predication of diseases.

What do we have? Texture, shape, contour and prior knowledge along with contextual information from image sequence

Goal: To provide information that helps with better human understanding.

Industrial vs. Medical ApplicationsPrior knowledge of any medical diagnose is far away from any industrial application.

Clinicians in the team are a MUST TO HAVE

Data alone cannot explain everything

Automatic retinal vascular tree extraction

Goal: Automatic extraction of the retinal vascular treeReal-time (We were in 2003! ) through HW-based applicationKeep accuracy at the level of SW-based approaches

Our approach: Active contours strategy to approximate vessel locations from outside (novelty!)

Tailor to SIMD processors to get advantage of the massively parallel nature (CUDA was even not released at that moment!)

Alonso-Montes, C., Vilariño, D. L., Dudek, P. and Penedo, M. G. (2008), Fast retinal vessel tree extraction: A pixel parallel approach. Int. J. Circ. Theor. Appl., 36: 641–651. doi:10.1002/cta.512

Challenges

Retinal Angiography challenges:Different vessel widths, vessel low contrast Retina boundary, optic disk, pathologies...Central reflex causing a complicated intensity cross-section

Hardware restrictionsThe processing elements simultaneously execute identical instructions (addition, inversion, one-neighbour access) on their local data

Active Contours: Flexibility for complex images

Active contour model or snakes, is a framework in computer vision to describe the boundaries of shapes in an image

Flexible to adapt to any shapeRobust to noise

Pixel Level Snakes

Elastic curve

u(s) = (x(s), y (s)), s [0, 1]∈Evolves from its initial shape and position as a result of the combined action of

External forces: Guide the contours towards the features of interest

Internal forces: Control the smoothness of the contour

Balloon Potential: Guide the contours when the external potential is too weak

Main input images: Initial contour & External potential image (guiding information image)

Automatic extraction of Retinal Vascular Tree

Retinal Vascular Tree for Personal Authentication

The whole execution time for the authentication system is 0.44 s.

Alonso-Montes, Carmen, et al. "Pixel parallel vessel tree extraction for a personal authentication system." 2008 IEEE International Symposium on Circuits and Systems. IEEE, 2008.

Retinal Vascular Tree for Disease Diagnosis

Alonso-Montes, C., M. G. Penedo, and D. L. Vilarino. "Arteriolar-to-venular diameter ratio estimation: A pixel-parallel approach." 2008 11th Int. W. Cellular Neural Networks and Their Applications. IEEE, 2008.

Neuroimaging A window to see & understand the brain “in vivo”

Neuroimaging is the use of various techniques to either directly or indirectly image the structure, function of the nervous system.

Useful to understand interactions between the mind, brain, and body New discipline combining medicine, neuroscience, and psychology.

Categories:Structural imaging: Brain structure (intracranial disease, tumors, injury)

computed axial tomography (CAT)

magnetic resonance imaging (MRI)

positron emission tomography (PET)

Functional imaging: Brain activity in vivo over time, useful for studying cognitive and affective processes

positron emission tomography (PET)

functional magnetic resonance imaging (fMRI)

electroencephalography (EEG) & magnetoencephalography (MEG).

Brain imaging basics

A brain image is represented by a 3D matrix of numbers that correspond to spatial locations Each location in the matrix is called a voxel (volumetric pixel) Can be displayed in slices, in 3 orientations: axial, sagittal, coronal

www.everythingzoomer.com

https://sites.google.com/a/wisc.edu/neuroradiology/image-acquisition/the-basics

Structural MRI Types

The most common MRI sequencesT1-weighted images (short TE and TR times) T2-weighted (longer TE and TR times)MR Angiogram (MRA)

Show arteries to evaluate them for stenosis (abnormal narrowing)

aneurysms (vessel wall dilatations, at risk of rupture).

Diffusion MRI (DWI): measures the diffusion of water molecules in biological tissues

diagnoses of conditions (e.g., stroke) or neurological disorders (e.g., multiple sclerosis)

Connectivity of white matter axons in the central nervous system

Diffusion tensor imaging (DTI), has been used extensively to map white matter tractography

Application: measurement of the location, orientation, and anisotropy of the tracts in white matter.

www.urmc.rochester.edu

Diffusion Tensor Imaging (DTI)

Magnetic gradients are applied in different directions to calculate the diffusion tensor at each voxel location.

Maps white matter fiber tracts in the brain by measuring the diffusion of water molecules, along their main direction.

Fractional Anisotropy (FA) – ranges from 0-1 and describes degree of diffusion restriction

Tractography

We can trace streams through the tensor field to reconstruct the structural connections in the brain!

Functional MRI (fMRI)

Functional MRI (fMRI) is used to understand how different parts of the brain respond to external stimuli or passive activity in a resting state.

Blood oxygenation level dependent (BOLD) fMRI measures the hemodynamic response to transient neural activity resulting from a change in the ratio of oxyhemoglobin and deoxyhemoglobin.

Statistical methods are used to build a 3D parametric map of the brain

Structural & Functional & Effective Connectivity

Understanding the precise relation between brain dynamics and structure is still a challenge for neuroscience.Connectivity

StructuralFunctionalEffective (directed)

Resting State Networks (RSNs)

Functionally integrated by structural connections (van den Heuvel and Sporns, 2013)

http://news.stanford.edu/news/2008/july9/gifs/MRI_Alz_600web.jpg

https://upload.wikimedia.org/wikipedia/commons/9/9a/Default_mode_network-WRNMMC.jpg

Understanding brain cognition: Structural Equation Model

Use of SEM is commonly justified in the social sciences because of its ability to impute relationships between unobserved constructs (latent variables) from observable variableGoal: Similarity among SEM & the underlying structural connectivity in RSNs

Lagged and instantaneous dynamical influences related to brain structural connectivity, C. Alonso-Montes et al. 2015 Frontiers in Psychology

http://www.nature.com/articles/ncomms6658

Distinct aspects of frontal lobe structure mediate age-related differences in fluid intelligence and multitasking

Exploratory SEM linked with Structural Connectivity

Study structural and functional networks and mutual relationships. 15 different ROIs from 3 RSNs.same subject two classes of MRI acquisitions (1 structural (DTI), 1 functional (fMRI))

Structural Connectivity - SC ( fiber number connectivity between ROIs)

Functional Connectivity - FC (pairwise C and PC connectivities)

Effective Connectivity - EC (eSEM technique)

Exploratory SEM (eSEM) for the inference of functional integrationAssess influences among ROIs without assuming any implicit modelStudy of the similarity of eSEM to SC, compared with equal-time correlational analysis by calculating C and PC, leading methods to estimate FC.

capable to separate the set of non-connected pairs (NCP) in the structural network from the set of connected pairs (CP)

Valid for fiber pairs with information flowing in one direction.

Lagged and instantaneous dynamical influences related to brain structural connectivity, C. Alonso-Montes et al. 2015 Frontiers in Psychology

Alzheimer's biomarkers using Neuroimaging

The Alzheimer’s Disease Neuroimaging Initiative (ADNI) Largest dataset with patient and control data (1000 enrolled subjects)Data is composed by: imaging data (MRI, PET); Genetics data; Cognitive tests; CSF and blood biomarkers

Subject classification: control (healthy), early mild cognitive impairment (EMCI), late MCI (LMCI) and Alzheimer dementia (AD)

Looking for Alzheimer disease biomarkers

Alzheimer disease evolution among clinical stages(ongoing work)

Unfortunately, this results were biased by left-handed subjectsSome results cannot be concluded only by data, according to neurologistsCannot be reproduced for right-handed datasets

Too much variability & heterogeneity in the EMCI and LMCIOnly 4% of MCI evolved towards AD

New approach: Use of graph theory to analyse group differences

Patient specific blood flow simulation: My work at KTH

The combination of imaging and simulation techniques can be a window to understand disease evolution in patients.

Project goal: Use of PLS to extract a realistic vessel and build up the geometry to be used in conjunction with FEniCS trying to minimize the computational effort in all the stages

1st Prototype for brain images, but technology can be applied to other organs like heart or liver.

FEniCS medical imaging interface

Developed in Qt technology combined with ITK+VTK for medical image proocessingEarly stage development – design and basic functionality Adapted for non-expert users

The goal of the GUI is to provide a common framework for engineers without knowledge in Medical imaging processing to play easily with the images to improve geometry for their simulations

Automatic FEniCS template code generation

FeniCS GUI for medical applications

MSO4SC – FEniCS & Cloud computing

Mathematical Modelling, Simulation and Optimization for Societal Challenges with Scientific Computing (MSO4SC)

H2020 E-Infrastructure2 years project2.5 M€

Project leaders: Johan Hoffman (KTH team) & Johan Jansson (BCAM team) Goals (KTH&BCAM):

Cloud accessibility for FEniCS-HPC Industrial applications in turbulent flow with FEniCS-HPC

Other partners:AtosEu-maths-inCesga….

All text and image content in this document is licensed under the Creative Commons Attribution-Share Alike 3.0 License (unless otherwise specified). "LibreOffice" and "The Document Foundation" are registered trademarks. Their respective logos and icons are subject to international copyright laws. The use of these therefore is subject to the trademark policy.

Any Question?