meeting users needs with octave a survey of the use of matlab™ and octave at the nih tom holroyd...
DESCRIPTION
Matlab Users at NIH 25 responses to the survey 9 out of 27 institutes NCI, NHGRI, NLM NHLBI, NIMH, NIAMS NIAID, NIDDK, NIDATRANSCRIPT
Meeting Users Needs with OctaveA survey of the use of Matlab™ and Octave at the NIH
Tom HolroydTom Holroyd
NIMH MEG Core FacilityNIMH MEG Core Facility
Presented at thePresented at the
Octave 2006 WorkshopOctave 2006 WorkshopApril 21, 2006April 21, 2006
OutlineSurvey ResultsSurvey Results
Number of usersNumber of users
LicensesLicenses
UsesUses
Other toolsOther tools
Issues for usersIssues for users
Octave use at the NIHOctave use at the NIHMEG data analysisMEG data analysis
Biowulf cluster (2000+ CPUs)Biowulf cluster (2000+ CPUs)
Matlab Users at NIH
25 responses to the survey25 responses to the survey
9 out of 27 institutes9 out of 27 institutesNCI, NHGRI, NLMNCI, NHGRI, NLM
NHLBI, NIMH, NIAMSNHLBI, NIMH, NIAMS
NIAID, NIDDK, NIDANIAID, NIDDK, NIDA
Matlab Users at NIH
35 licenses of various sorts35 licenses of various sorts
Toolboxes:Toolboxes:Statistics 11Statistics 11
Image processing 9Image processing 9
Bioinformatics 6Bioinformatics 6
Signal processing 6Signal processing 6
Neural network 4Neural network 4
Optimization 4Optimization 4
Database 3Database 3
Curve fitting 3Curve fitting 3
Spline 1Spline 1
Symbolic math 1Symbolic math 1
Fuzzy logic 1Fuzzy logic 1
Compiler 1Compiler 1
Excel link 1Excel link 1
Wavelet 1Wavelet 1
Many UsesMicroarray analysis (6 users)Microarray analysis (6 users)
DesignDesign
VisualizationVisualization
Statistical analysisStatistical analysis
DatabaseDatabase
Web siteWeb site
Image processingImage processing
Content-based image retrievalContent-based image retrieval
SegmentationSegmentation
CompressionCompression
Optical and electron microscope Optical and electron microscope image analysis image analysis
MRI data analysisMRI data analysis
Functional mappingFunctional mapping
Information-based mappingInformation-based mapping
Voxel-based morphometryVoxel-based morphometry
MRI modelling, image reconstructionMRI modelling, image reconstruction
Diffusion tensor imagingDiffusion tensor imaging
StatisticsStatistics
General purpose statisticsGeneral purpose statistics
t-tests, ANOVA, regression, etc.t-tests, ANOVA, regression, etc.
Neural networksNeural networks
Pattern recognition in mass spectraPattern recognition in mass spectra
Feature extractionFeature extraction
Many, Many Uses
GeneticsGenetics
Novel motif discovery in Novel motif discovery in regulatory regions of genomes regulatory regions of genomes
Signal processingSignal processing
Mass spectrometryMass spectrometry
InstrumentationInstrumentation
Real time control and data Real time control and data acquisition acquisition
NeurophysiologyNeurophysiology
MatOFF: M-series OFFline MatOFF: M-series OFFline Analysis for NeurophysiologyAnalysis for Neurophysiology
MEG data analysisMEG data analysis
Time-frequency plotsTime-frequency plots
Multitaper spectral analysisMultitaper spectral analysis
Magnetic field simulationsMagnetic field simulations
StatisticsStatistics
Realtime BCIRealtime BCI
Fieldtrip: ported to OctaveFieldtrip: ported to Octave
Curve fittingCurve fitting
Fluorescence recovery after Fluorescence recovery after photobleachingphotobleaching
Simulations of differential equationsSimulations of differential equations
Other Tools
RR
ScilabScilab
matplotlib/pylabmatplotlib/pylab
The competition? Data The competition? Data import/export issues exist.import/export issues exist.
RR
JavaJava
C/C++C/C++
Extension packages written Extension packages written in other languages.in other languages.
Issues for UsersLicensingLicensing
Multi-processor and clusterMulti-processor and cluster
PlatformPlatformOS X compatibilityOS X compatibility
64-bit systems and large memory (> 2GB)64-bit systems and large memory (> 2GB)
OctaveOctaveLack of GUI support (not an IDE)Lack of GUI support (not an IDE)
Compatibility/portability issues with other analysis packagesCompatibility/portability issues with other analysis packages
Speed (we need to convert more .m files to C++)Speed (we need to convert more .m files to C++)
Support, e.g., bioinformatics expertsSupport, e.g., bioinformatics experts
The License Hog
This ran for a week.This ran for a week.
What the note doesn't say is that he What the note doesn't say is that he was also using a license; nobody else was also using a license; nobody else could use Matlab for a week.could use Matlab for a week.
I had a user who ran a long job which I had a user who ran a long job which failed right at the end when it needed failed right at the end when it needed to do a t-test and the stats toolbox to do a t-test and the stats toolbox was busy.was busy.
Multiple Processor / Cluster
As an example, the Biowulf has 11 licenses (22 CPUs) As an example, the Biowulf has 11 licenses (22 CPUs) and various toolboxes costing $96,000. MathWorks can and various toolboxes costing $96,000. MathWorks can sell us a “cluster” license on 128 CPUs for $39,000. sell us a “cluster” license on 128 CPUs for $39,000. Cheaper, but still about 1900 CPUs short. It works out to Cheaper, but still about 1900 CPUs short. It works out to about $273 / CPU or $500,000 for the whole cluster (they about $273 / CPU or $500,000 for the whole cluster (they have never sold a 2000 CPU license).have never sold a 2000 CPU license).
Octave use at the NIH
Magnetoencephalography (MEG) is the recording and Magnetoencephalography (MEG) is the recording and analysis of the magnetic field of the brain. NIH has a 275 analysis of the magnetic field of the brain. NIH has a 275 channel scanner, which produces very large multivariate channel scanner, which produces very large multivariate datasets.datasets.Analysis methods include time-frequency plots, magnetic Analysis methods include time-frequency plots, magnetic field calculations, and source localization via power field calculations, and source localization via power spectrum estimation. All can be done with Octave.spectrum estimation. All can be done with Octave.
Contour Plots
The time-frequency graph The time-frequency graph shown here was calculated shown here was calculated using Octave with a C++ using Octave with a C++ extension, then the dataextension, then the datawere exported to pylab for were exported to pylab for rendering in a manner similar rendering in a manner similar to that used for Gnuplot.to that used for Gnuplot.
Stockwell Transform
MEG Source Localization
Power spectral estimates Power spectral estimates are used to generate maps are used to generate maps of brain activity. We get a of brain activity. We get a timecourse for each voxel.timecourse for each voxel.
Octave is used as a Octave is used as a compute engine here; we compute engine here; we run hundreds of such run hundreds of such datasets on the cluster. datasets on the cluster. Voxels are all independent!Voxels are all independent!
Biowulf Cluster
Recent patches to Octave 2.9.5+ (not in CVS yet!) make it Recent patches to Octave 2.9.5+ (not in CVS yet!) make it possible to run in a very large cluster environment. The issue was possible to run in a very large cluster environment. The issue was that Octave's cache of .m files was broken, and it was hitting the that Octave's cache of .m files was broken, and it was hitting the NFS server every function call. Now it caches both successes and NFS server every function call. Now it caches both successes and failures. The new issue is that it won't see new files, although it will failures. The new issue is that it won't see new files, although it will pick up files whose modification time has changed.pick up files whose modification time has changed.
New issues: how to get 200 running Octaves to communicate New issues: how to get 200 running Octaves to communicate usefully with each other, for highly parallel computations.usefully with each other, for highly parallel computations.
Octave 2006 Workshop
Congratulations to all on a successful workshop!
See you all at Octave 2007