Bestprac*ceindata-sharinginneuroimaging…

andmoreThomasNichols

UniversityofWarwickh?p://warwick.ac.uk/tenichols

OHBMCommi?eeonBestprac*ceinDataAnalysis&Sharing(COBIDAS)History

•  CreatedatOHBM2014–  Promptedby"OHBMCouncilStatementonNeuroimagingResearchandDataIntegrity"

•  Chargedwith1.  Iden*fyingbestprac*cesofdataanalysisanddata

sharinginthebrainmappingcommunity2.  Preparingawhitepaperorganisinganddescribing

theseprac*ces3.  SeekinginputfromtheOHBMcommunity4.  Publishingtheserecommenda*ons

COBIDASStatus

•  Membership

•  Willnotprescribeprac*ce(mostly)•  Ratherfocusonwhattoreport

– Tosupportopenandreproducibleresearch

SimonEickhoff AlanEvans MichaelHanke NikosKriegeskorte

MichaleMilham RusselPoldrack Jean-Bap*stePoline ErikaProal

BertrandThirion DavidvanEssen TonyaWhite BTThomasYeo

ThomasNichols

COBIDASStatus•  fMRI

–  Task&rest•  DivisionsoffMRIPrac*ce

–  Experimentaldesignrepor*ng–  Imageacquisi*onrepor*ng–  Preprocessingrepor*ng–  Sta*s*calmodeling–  Resultsrepor*ng–  Datasharing–  Replica*onandreproducibility

•  Producewhitepaper–  Tobecommentedon,andul*matelyapprovedbyOHBMmembers

COBIDASDocumentForm

•  Foreachdivision– Principalsofopenandreproducibleresearch– Whenfeasible,recommenda*onsforprac*ce– Detailedtabularlis*ngofwhattoreport

Example:ExperimentalDesignRepor*ng

COBIDAS:100+repor*ngitems

Scien*ficData’sMetaDataStandard

RawDataFile DataRepositoryDataRecordAccession

DataProcessingMethod DerivedDataFile DerivedDataRepository

DerivedDataRecordAccession

datafilename datarepositoryname accessionordoi deriveddatafilename datarepositoryname accessionordoidatafilename datarepositoryname accessionordoidatafilename datarepositoryname accessionordoi… … …

Subject organism organismpart developmentalstage age sex laterality Addi?onalcolumns…Pa*ent1 Homosapiens brain adult 43male Right …Pa*ent2 Homosapiens brain child 37male Lee …Pa*ent3 Homosapiens brain adult 23female Right …… … … … … … … …

Subject ScanName ScanningMethod Instrumentname Instrumentmanufacturermagne?cfieldstrengthvalue

magne?cfieldstrengthunit coiltype

numberofcoils TypeofMRIassay

Pa*ent1 Scan1 Magne*cResonanceImaging Siemens3TTIMTrio Siemens 3tesla birdcage 12T1-weightedimagePa*ent2 Scan2 Magne*cResonanceImaging GeneralElectricMR7503Tscanner GeneralElectric 3tesla head 8Mul*-EchoFLASHPa*ent3 Scan3 Magne*cResonanceImaging Magnetom Siemens 7tesla head 8BOLDimage… … … … … … … …

Voxelmeasurements

fieldofview fieldofviewunit matrixsize

matrixsizeunit

resolu?onvalue

resolu?onunit

repe??on?me

repe??on?meunit

echo?me

echo?meunit

256x256x192 millimeter 1x1x1 millimeter 2170millisecond 4.33millisecond64x64x24 millimeter 3.125x3.125x6 millimeter 4000millisecond 12millisecond

27squarecen*meter 256x256 millimeter 7.3millisecond 3millisecond… … … … … … … … … …

ISA-Tab:‘Inves*ga*on’(theprojectcontext),‘Study’(aunitofresearch)and‘Assay’(analy*calmeasurement)inTabularformat

Replicability,Reproducibility,Repeatability•  COBIDASreport

–  Peng’s“Reproducibly”:Samedata,samecode,differentresearchers

–  Replica*on:Differentdata,differentresearchers,differentmethods…sameconclusion

•  ISOrepeatability(ISO3534-2:20063.3.5)–  Same“method”,“testormeasuringfacility”,same“operator”&“equipment”on“iden*caltest/measurementitems”“withinshortintervalsof*me”.

•  ISOreproducibility(ISO3534-2:20063.3.10)–  Same“method”on“iden*caltest/measurementitems”indifferent“testormeasurementfacili*es”,different“operatorsusingdifferentequipment”

•  ISOtermsadoptedby…–  RadiologicalSocietyofNorthAmerica(RSNA)Quan*ta*veImaginginBiomarkersAlliance(QIBA)group(h?ps://www.rsna.org/QIBA/)publishedKessleratal.(StatMethodsMedRes,2014)

same-site test-retest

between-site test-retest

COBIASStatus

•  20thOctober– ManuscriptpostedonOHBMwebsite– Openforcommentfor4weeks(un*l17thNov.)

•  EndofNovember– Membercommentsintegrated– Finaldrae,approvedbyCouncil

•  December– Finalizedmanuscriptpostedforup/downvote– Uponapproval,submissionforpublica*on

SomeUsefulTools&Projects

•  NIDM•  Neurosynth&ReverseInference•  NeuroVault

INCF’sNeuroimagingDataModel(NIDM)

•  Collabora*veefforttorepresentallaspectsofneuroimagingexperiments,datacollec*on&analysisinseman*cweb,machine-readableform

NIDMOverviewNIDMExperiment

Seman*cWeb•  PROV-DM

– Seman*cwebmodelforprovenance

•  NIDM– PROV-DMforneuroimaging

NIDMResults

Statistical estimation

Inference

Design of the experiment, pre-processed data...

Effects, standard error and statistical maps...

Thresholded maps, list of peaks and clusters...

Model estimation

Contrast estimation

Inference

Model Data

Param. Est. map

Resid. & Gd mean

Contrast map

Std. err. map

Statistic map

Software

Mask

Mask

Mask

Cluster & Peak Def.

Threshold

Excursion set

cluster 1 cluster n

peak 1 peak m

Search space

Legend

Activity

Entity

Agent

•  Seman*cweb–  “serialized”intoatextfile

coordinate 1 coordinate m

CommonFormatforallsoeware

Automatically created with Neurotrends based on over 16 000 journal articles;

~80%

NIDMExporters•  SPM12done!

– SPM8extensionunderway

•  FSL–inbetates*ng•  AFNI

–  Inplanningstage

•  Formoreseeh?p://nidm.nidash.org

SPM 12 batch system: SPM è Stats è Results report

Neurosynth&ReverseInference

ReverseInference&BrainImaging•  Poli*csstudyfrom2007

–  VotersviewedimagesofDemocra*ccandidates(N=20)

–  SubsetthatdislikedClinton:•  “…exhibitedsignificantac*vityintheanteriorcingulatecortex,anemo*onalcenter”…,ac*vatedwhenone“feelscompelledtoactintwodifferentwaysbutmustchooseone.”

Iacoboni, et al., “This is your brain on politics”. OP-ED, The New York Times, Nov. 11, 2007

ReverseInference&BrainImaging•  Logic

– Emo*onconflictresolu*ontaskèAnteriorCingulateac*va*onknownfromtheliterature

– HillaryClintonèAnteriorCingulateac*va*onobservedinthisexperiment

– Ergoè HillaryClintoninducesemo*onalconflict

è FaultyReverseInference– HighP(A.C.Act.|Emot.Conf.)doesn’timplyhighP(Emot.Conf.|A.C.Act.)!!!

Iacoboni, et al., The New York Times, Nov. 11, 2007

ReverseInference:Correctly!•  BayesRule

– Cogni*veDomainC,Ac*va*onA– P(C=c|A)=P(A|C=c)P(C=c)/Σc*P(A|C=c*)P(C=c*)summa*onoverallcogni;vedomains!

•  Canwefind“P(Emot.Conflict|ACCAct.)”?– Needtorun100’sofexperiments!– Or,usemetaanalysis!– ButbestNeuroimagingMetaAnalysisdatabasesares*lllimited

•  BrainMap.orghas2,757studies(startedin1988)•  Pubmedfinds28,694refs“fMRI”in*tle/abstract

Neurosynth

•  LexicographicalAnalysisofpapers– Harvestsx,y,zcoordinatesfromtables– Recordswordfrequenciesofbodytext

•  NotcuratedlikeBrainMap.org,butautomated– BrainMap–2kstudies20years– Neurosynth–4kstudiesin<12months

•  Usingkeywords,allowsreverseinference

Yarkoni, Poldrack, Nichols, Essen, & Wager (2011). Large-scale automated synthesis of human functional neuroimaging data. Nature Methods, 8(8), 665-670. www.neurosynth.org

4,393 studies (in < 12 months!)

NeurosynthMethods•  17Neuroscience-focusedjournalsused

–  BiologicalPsychiatry,Brain,BrainandCogni;on,BrainandLanguage,BrainResearch,CerebralCortex,Cogni;veBrainResearch,Cortex,EuropeanJournalofNeuroscience,HumanBrainMapping,JournalofNeurophysiology,JournalofNeuroscience,NeuroImage,NeuroLeHers,Neuron,Neuropsychologia,&Pain.

•  Tagging– Eachar*cle‘tagged’withpsychologicalterms– Scoredashighfrequency(>1/1000words)ornot

•  Coordinateharves*ng– Tablesparsedforx,y,zcoordinates

•  Notexhaus*ve,butalreadymassive– 4,400+studies,145,000+foci

WhataboutAnteriorCingulate?

•  It’salwaysthere!

•  Finally,candorealreverseinference…

Probability of activation over all studies

NeuroVault

NeuroVault

•  BrainMap|Neurosynth– Onlycoordinates– Hugelossofinforma*on

•  Coordinatebasedvs.IntensitybasedMetaAnalysis– Substan*alinforma*onloss

Salimi-Khorshidi,G.,Smith,S.M.,Keltner,J.R.,Wager,T.D.,&Nichols,T.E.(2009).Meta-analysisofneuroimagingdata:Acomparisonofimage-basedandcoordinate-basedpoolingofstudies.NeuroImage,45(3),810–823.

NeuroVault

NeuroVault.org

•  s

Thankyou!