tin2010-20900-c04-04 upm group annual report 2012

Team Objectives Results Indicators Collaborations

TIN2010-20900-C04-04UPM GROUP – ANNUAL REPORT 2012

Concha Bielza

Computational Intelligence GroupDepartamento de Inteligencia Artificial

Universidad Politecnica de Madridhttp://cig.fi.upm.es

Albacete, February 7-8, 2013

C. Bielza UPM-Madrid

Outline

1 Team

2 Objectives

3 Scientific-technological activities and resultsSupervised classificationUnsupervised classificationApplicationsPopular science papers

4 Results indicators

5 Collaborations within the project

Outline

1 Team

2 Objectives

13 Members and 11 EDPs

2 Full ProfessorsPedro LarranagaConcha Bielza

2 foreign collaborators, Full ProfessorsTom Heskes (Nijmegen, The Netherlands)Qingfu Zhang (Essex, UK)

1 Associate Professor: Juan A. Fernandez del Pozo

3 PostDoc ResearchersRuben Armananzas (Juan de la Cierva researcher)Dinora Morales (Cajal Blue Brain Project)Luis Guerra (Cajal Blue Brain Project) since Jan 1, 2013R. Santana left this project on Feb 2012

5 PhD StudentsAlfonso Ibanez (Consolider)Hossein Karshenas (Consolider)Pedro L. Lopez-Cruz (FPU)Bojan Mihaljevic (Cajal Blue Brain Project) since Oct 1, 2012Laura Anton (Cajal Blue Brain Project) since Nov 1, 2012D. Vidaurre and H. Borchani left on Oct 2012 and Jan 2013, resp.J. Perez left on Oct 2012

Outline

1 Team

2 Objectives

Objectives

1. Joint probability distribution function learningDefinition of new scores and structural algorithms for learning PGMs.Ideas based on self-similarity, regularization, multicriteria, interaction, andwith complex data (noisy, missing, high-dimensional)Learning the parameters (densities) in models with continuous variables

2. Supervised classificationBNs classifiers in problems with an imbalanced classAdvance in the design of well-known BN classifiers (TAN, KDB, AODE,HAODE, WAODE, FBC, multinets...)Development of new methods for multi-dimensional classificationExtensions to massive data sets and data streamsDevelopment of new methods to convert a problem of classification intoregression modelsExtension of PGMs to hybrid domains (discrete and continuous variables)for its application to classification and regressionExtension to credal classifiers (use imprecise probabilities)Algorithms for learning utility-based classifiers

Objectives

3. InferenceApproximate algorithms for MTE hybrid networks, credal networks,probabilistic decision graphs, precise and imprecise influence diagramsand for BNS using fast factorization and recursive treesAlgorithms based on query importance sampling for hybrid Bayesiannetworks

4. ApplicationsTechnological applications: evolutionary computation, mobile robotics,requirements tracing and classificationLife sciences: biomedicine, agriculture, environment, genomicsSocial domains: bibliometry, prediction of arrival times of city buses,detection of credit card frauds

Team Objectives Results Indicators Collaborations Supervised Unsupervised Applications Popular

Outline

1 Team

2 Objectives

Supervised classification

T2.1 BN classifiers, FSS and high dimensional datasetsEnsemble BN in the N � p scenario: bootstrap resampling of the dataset, FSSwith CFS, learn k-DB Bayesian classifiers. Compute frequencies of all the arcsincluded in the models. Retain robust arcs, occurring more than t times(t = reliability threshold) [Armananzas et al., 2008]

Applied to a genomic study (microarrays) of human colorectal cancer andAlzheimer’s disease [Task T4.2.4 Genomics and Task T4.2.1 Biomedicine >Neuroinformatics]

PapersGarcıa-Bilbao A, Armananzas R, Ispizua Z, Calvo B, Alonso-Varona A, Inza I,Larranaga P, Lopez-Vivanco G, Suarez-Merino B, Betanzos M (2012)Identification of a biomarker panel for colorectal cancer diagnosis,BMC Cancer, 12:43 IF: 3.011 (78/196)Also in Proceedings of the 6th International Meeting on Biotechnology(BioSpain-2012), p 77

Armananzas R, Larranaga P, Bielza C (2012) Ensemble transcript interactionnetworks: A case study on Alzheimer’s disease, Computer Methods andPrograms in Biomedicine, 108, 1, 442–450 IF: 1.516 (14/99)

Supervised classificationT2.2 Advances in known Bayesian network classifiers

Method to learn MoP approximations of probability densities from data using alinear combination of B-splines. Also in naive Bayes classifiers

A stagewise version of the selective naive Bayes, which can be considered aregularized version of the naive Bayes model. Discards both irrelevant andredundant predictors. Applied to fMRI data[Task T4.2.1 Biomedicine > Neuroinformatics]

PapersLopez-Cruz PL, Bielza C, Larranaga P (2012) Learning mixtures of polynomialsfrom data using B-spline interpolation, Proceedings of the 6th EuropeanWorkshop on Probabilistic Graphical Models (PGM-2012), pp. 271–278

Vidaurre D, Bielza C, Larranaga P (2012) Forward stagewise naive Bayes,Progress in Artificial Intelligence, 1, 57–69 Not JCR

T2.3 Multi-dimensional classification with PGMs

Multi-dimensional BN classifiers, with generalstructure and constrained-based learningalgorithms

Applied to Parkinson’s disease[Task T4.2.1 Biomedicine > Neuroinformatics]

PapersBorchani H., Bielza C., Martınez-Martın P, Larranaga P (2012) Markovblanket-based approach for learning multi-dimensional Bayesian networkclassifiers: An application to predict the European Quality of Life-5 Dimensions(EQ-5D) from the 39-item Parkinson’s Disease Questionnaire (PDQ-39),Journal of Biomedical Informatics, 45, 6, 1175–1184 IF: 1.792 (30/99)

T2.5 Relationship between classification and regressionLocally weighted regression and FSS via lasso. Closer points should be moreimportant. However, distance calculation involves irrelevant variables→ incorrectFSS (incorrect weighting scheme) and inaccurate predictions ...but weights arecomputed prior to the regression!

Alternate FSS and distance computation until some stopping criterion: relevanceof Xj is given by its regression coefficient βj 6= 0

Use regression for classification, based on the sign of the response. Applied tofMRI data [Task T4.2.1 Biomedicine > Neuroinformatics]

PapersVidaurre D, Bielza C, Larranaga P (2012) Lazy lasso for local regression,Computational Statistics, 27, 3, 531–550 IF: 0.270 (107/116)

Unsupervised classification

Not a proposed taskComparison of standard quality indices in clustering using more realisticdatabases (with outliers and noisy dimensions). 3 clustering methods (k-means,hierarchical, Gaussian mixture). Compare also against random grouping

PapersGuerra L, Robles V, Bielza C, Larranaga P (2012) A comparison of cluster qualityindices using outliers and noise, Intelligent Data Analysis, 16, 4, 703–715IF: 0.448 (95/111)

T4.1 Technological applications

T4.1.1 Evolutionary computation (1/2)

Review on the PGMs used in evolutionary algorithms, specially in EDAs, andhow they are employed for optimization

Relationship between the general multivariate Gaussian distribution and Markovnetworks. GMRF-EDA for continuous optimization (regularized regression forweighting the possible neighborhood and affinity propagation with these weightsfor final non-overlapping cliques, learnt with an MGD)

Papers

Larranaga P, Karshenas H, Bielza C, Santana R (2012)A review on probabilistic graphical models in evolutionary computation,Journal of Heuristics, 18, 5, 795–819 IF: 1.262 (24/99)Karshenas H, Santana R, Bielza C, Larranaga P (2012) Continuous estimationof distribution algorithms based on factorized Gaussian Markov networks. InMarkov Networks in Evolutionary Algorithms, pp. 157–173, Springer

T4.1 Technological applications

T4.1.1 Evolutionary computation (2/2)

Predict in which direction (right-left) 4 subjects were covertly attending, from theirMEG data. Multi-objective optimization of accuracy for 4 subjects with EDAs andGAs; selection of optimal channels (FSS); lasso regularized logistic regression[Task T4.2.1 Biomedicine > Neuroinformatics]

Find cutpoints of CISI-PD severity scale in PD to correlate with Hoehn-Yahrindex. Then predict CISI-PD from different non-motor indices. FSS with EDAs;NB, k-NN, LDA, C4.5 [Task T4.2.1 Biomedicine > Neuroinformatics]

Papers

Santana R, Bielza C, Larranaga P (2012) Regularized logistic regression andmulti-objective variable selection for classifying MEG data,Biological Cybernetics, 106, 6-7, 389–405 IF: 1.586 (4/20)Armananzas R, Bielza C, Larranaga P (2012) Restating clinical impression ofseverity index for Parkinson’s disease using just non-motor criteria, 25thEuropean Conference on Operational Research (EURO XXV), pp. 231-232

T4.2 Life Sciences

T4.2.1 Applications to biomedicine (1/2)Naive Bayes, CFS-selective NB, filter selective NB (and SVM) applied todiscriminate between cognitive intact Parkinson’s disease patients, mild cognitiveimpairment and dementia. Neuroanatomic biomarkers provided by MRI (volumesof subcortical structures and thickness of cortical parcels)

In ionic conductance-based neuron models, learn coregulation mechanisms toproduce a particular electrophysiological behavior. From the Boltzmanndistribution we compute the mutual information to measure the strength ofinteraction between a pair of conductances. Lobster stomatogastric ganglionneurons

PapersMorales DA, Vives-Gilabert Y, Gomez-Anson B, Bengoetxea E, Larranaga P,Bielza C, Pagonabarraga J, Kulisevsky J, Corcuera-Solano I, Delfino M (2012)Predicting dementia development in Parkinson’s disease using Bayesian networkclassifiers, Psychiatry Research: NeuroImaging, in press IF: 2.524 (56/130)Santana R, Bielza C, Larranaga P (2012) Conductance interaction identificationby means of Boltzmann distribution and mutual information analysis inconductance-based neuron models, BMC Neuroscience, 13(Suppl 1):P100IF: 3.042 (109/244)

T4.2 Life SciencesT4.2.1 Applications to biomedicine (2/2)

CDP detection and classification in cats, relevant for functional organization ofthe neural networks involved in the control of sensory information and tocharacterize changes produced by acute nerve and spinal lesions. A novelfeature extraction approach for signal classification: 1) summarize the entiresignal in a few coefficients derived from the amplitude and separation of thepeaks, 2) apply gradient boosting classification trees with these coefficients asvariables

Papers

Vidaurre D, Rodrıguez EE, Bielza C, Larranaga P, Rudomin P (2012) A newfeature extraction method for signal classification applied to cord dorsumpotential detection, Journal of Neural Engineering, 9, 5, 056009 IF: 3.837 (2/72)

T4.3 Social domains

T4.3.1 Applications to bibliometryFor Spanish computer science, relationship among research collaboration(# authors, international/national/institutional), # citations, # documents(journal/conference, JCR subdisciplines, IF quartiles).20,000 publications from 2,004 researchers

Analyse worldwide participation of biomedical informatics researchers in thebest-known conferences/journals (geographical affiliation)

PapersIbanez A, Bielza C, Larranaga P (2012) Relationship among researchcollaboration, number of documents and number of citations. A case study inSpanish computer science production in 2000-2009, Scientometrics in pressIF: 1.966 (23/99)Maojo V, Garcıa-Remesal M, Bielza C, Crespo J, Perez-Rey D, Kulikowski C(2012) Biomedical Informatics publications: A global perspective. Part I:Conferences, Methods of Information in Medicine, 51, 1, 82–90 IF: 1.532(34/135)Maojo V, Garcıa-Remesal M, Bielza C, Crespo J, Perez-Rey D, Kulikowski C(2012) Biomedical Informatics publications: A global perspective. Part II:Journals, Methods of Information in Medicine, 51, 2, 131–137 IF: 1.532 (34/135)

Popular science papers

PapersLarranaga P, Bielza C, DeFelipe J (2012) Alan Turing y la neurociencia,Mente y Cerebro, 57, 49–51

Larranaga P, Bielza C (2012) Alan Turing and Bayesian statistics,Mathware & Soft Computing Magazine, 19, 2, 23–24

Outline

1 Team

2 Objectives

Indicators

PublicationsType TotalJCR PUBLICATIONS 13PROCEEDINGS 3

We had estimated, per group and year, ≥ 4 JCR papers + 5 papers inconference proceedings

Collaborations

Collaborations, mainly as coauthors

EDAsUPV/EHU

NeuroscienceHospital de la Santa Creu i Sant Pau (Barcelona)Instituto Cajal CSICFundacion CIENCenter for Research and Advanced Studies, Mexico

Biomedicine/BioinformaticsHospital de Cruces (Bizkaia)CIEMAT

Projects and contracts

Consolider Project: “Multimodal Interaction in Pattern Recognition and ComputerVision” (CSD2007-00018) during 2007-2012

“Cajal Blue Brain project” (C08 0020-09) during 2009-2019

National Network Atica on Applied Computational Intelligence(TIN2011-14083-E) during 2012

National Network on Data Mining and Machine Learning (TIN2010-09163-E)from 1-10-2010 to 1-10-2012

Conference involvement

Organization of a special session on “Optimization and Data Mining”at EURO XXV (Bielza, Armananzas)

Members of Program Committees at: ICPRAM-2012, ESMDM-2012,IEEE CEC-2012, GECCO-2012, UAI-2012, PAIS-2012, PGM-2012, HPCS-2012,WCCI2012

Other activities

Other activitiesPlenary talk at EVOLVE-2012 (Aug’12): “Multi-objective optimization withestimation of distribution algorithms” (Larranaga)

Plenary talk at PGM-2012 (Sep’12): “Bayesian networks in neuroscience”(Bielza, Larranaga)

Accepted poster at 2012 International Joint Conference on Neural Networks(IJCNN-2012) (Santana et al.) Jun’12

Oral Presentation at 7th International Conference on Stereology, SpatialStatistics and Stochastic Geometry (S4G) (Perez et al.) Jun’12

Talk at Linkoping ”Neocortical neurons classification”. ADIT Workshop (Guerra)Mar’12

Talk at Aalborg “Learning mixtures of polynomials from data using B-splineinterpolation” (Lopez-Cruz) Nov’12

Attendance at MacBrain Symposium: Machine Learning, Brain Banks andHealth, Nijmegen, Sep’12

2 Techn. Reports UPM-FI/DIA/2012-1 and -2 at UPM on EDAs (Karshenas et al.)

Article in a blog of El Paıs, 20-12-2012: Alan Turing y la Estadıstica Bayesianahttp://blogs.elpais.com/turing/2012/12/alan-turing-y-la-estadistica-bayesiana.html(Larranaga, Bielza)

Training

Giving talks/courses

C. Bielza, P. Larranaga: Inteligencia artificial en neurociencia, “Semana Culturalde Informatica”, U. Malaga, Mar’12

C. Bielza, P. Larranaga: Bayesian networks in neuroanatomy andneurodegenerative diseases, Radboud University Nijmegen, Sep’12

P. Larranaga, C. Bielza, R. Armananzas, L. Guerra, P. Lopez-Cruz, H. Borchani:summer school “Advanced data analysis and modelling”, UPM, organizing andgiving courses, Jul’12

J.A. Fernandez del Pozo: Evaluacion de satisfaccion de servicios en centros deatencion a personas con discapacidad intelectual, CAM, Nov’12

2 International PhD theses and 1 master thesis

D. Vidaurre: Regularization for Sparsity in Statistical Analysis and MachineLearning. Jul’12

L. Guerra: Semi-Supervised Subspace Clustering and Applications toNeuroscience. Oct’12

J. Perez: Replicated Spatial Point Processes for Statistical Neuroscience. Jul’12

Training

Giving talks/courses

C. Bielza, P. Larranaga: Inteligencia artificial en neurociencia, “Semana Culturalde Informatica”, U. Malaga, Mar’12

C. Bielza, P. Larranaga: Bayesian networks in neuroanatomy andneurodegenerative diseases, Radboud University Nijmegen, Sep’12

P. Larranaga, C. Bielza, R. Armananzas, L. Guerra, P. Lopez-Cruz, H. Borchani:summer school “Advanced data analysis and modelling”, UPM, organizing andgiving courses, Jul’12

J.A. Fernandez del Pozo: Evaluacion de satisfaccion de servicios en centros deatencion a personas con discapacidad intelectual, CAM, Nov’12

2 International PhD theses and 1 master thesis

D. Vidaurre: Regularization for Sparsity in Statistical Analysis and MachineLearning. Jul’12

L. Guerra: Semi-Supervised Subspace Clustering and Applications toNeuroscience. Oct’12

J. Perez: Replicated Spatial Point Processes for Statistical Neuroscience. Jul’12

Abroad for International thesis

L. Guerra at Linkoping with JM Pena (Feb-Apr’12)

D. Vidaurre at Nijmegen with T Heskes (March-May’12)

H. Borchani at Porto with J Gama (Jan-March’12)

H. Karshenas at Algarve with F Lobo (Oct-Dec’12)

P. Lopez-Cruz at Aalborg with T Nielsen (Nov’12-Jan’13)

Post-doc research stay: R. Armananzas at UC San Diego with L Ohno-Machado(May-Jun’12)

Stays at UPM

Inma Perez (UAL) Apr’12 (one week). Talk given

Stays at UPM

Concha Bielza

Outline

1 Team

2 Objectives

4 groups collaborating

Joint expected activities

4 workshops, one in each city (work carried out, next plans, difficulties found):months 5, 14, 23 and 31 approx.→ Nov-Dic 2013 at UAL

4 project supervisors will celebrate intermediate meetings, at least one persemester

Mobility and exchange of researchers→ Inma Perez

Shared supervision of PhD theses and research works (D.E.A.)

A server to make papers, software and documentation generated accessible toall the participants→ Our papers accessible

Inclusion of procedures in some open software tool (e.g. Elvira, WEKA,MATEDA, ProGraMo) or by making available to the scientific community somespecific routines→ regularized and variables-objective probabilistic models inmulti-objective problems to enhance learning in EDAs (in MATEDA), a new Rpackage on Bayesian classifiers

Applics results extended to conferences and journals non-specific of PGMs; useof results/software by EPOs. Ours are: Atos, Instituto Cajal, Panda Security(Abbott too)

Keywords per UPM member

T. Heskes→ regularization and neurocomputing

Q. Zhang→ evolutionary computation

J.A. Fernandez del Pozo→ decision analysis, evolutionary computation

R. Armananzas→ evolutionary computation, classification, bioinformatics

D. Morales→ evolutionary computation, classification, neuroscience

A. Ibanez→ classification, bibliometry

H. Karshenas→ evolutionary computation

P.L. Lopez-Cruz→ new Bayesian classifiers, neuroscience

L. Guerra→ probabilistic clustering, semi-supervised, multi-output regression

B. Mihaljevic→ Bayesian classifiers, expert opinions

L. Anton→ spatial statistics, clustering, neuroscience

Proposals to collaborate (revisited)

IdeasHeskes X

Zhang X

Program Chairs of CAEPIA’13: Bielza (UPM), Salmeron (UAL) X

PC Members of CAEPIA’13: Moral-Cano-Gomez (UGR), Gamez-Puerta-Nielsen (UCLM),

Rumı (UAL), Larranaga-Armananzas-Guerra (UPM) X

Comparing MoPs (B-splines) with MTEs, kernels...→ Shenoy (UGR), Salmeron (UAL), Nielsen (UCLM)... In progress X

Consensus of BNs, like Sagrado-Moral’03 “Qualitative combination of BNs” atIJIS or Pena’11 “Finding consensus BN network structures” at JAIR→ Moral (UGR), Pena (UCLM)... Still thinking of X

Genome-based prediction→ Abad (UGR)... Still thinking of XLearning IDs from data→ Gomez (UGR)...

Defining hybrid PGMs with imprecise probabilities for solvingmulti-(noisy)objective optimization problems (with EDAs)→ Cano (UGR)...

Interactive learning of BNs and Markov blankets→ Masegosa (UGR)...

Abductive inference for multi-dimensional classification→ Gamez, Puerta (UCLM)...

IdeasHeskes X

Zhang X

IdeasHeskes X

Zhang X

IdeasHeskes X

Zhang X

IdeasHeskes X

Zhang X

IdeasHeskes X

Zhang X

Concha Bielza

tin2010-20900-c04-04 upm group annual report 2012

Documents