lenore mullin program director cise ccf algorithmic foundations national science foundation
DESCRIPTION
Architecture Aware Tensor-Based Computing Challenges for the Computer Science and Mathematics Communities CISE CCF Algorithmic Foundations: Moore’s Law and Verifiable, Scalable, Portable, and Reproducible Matrix and Tensor Software. Lenore Mullin Program Director - PowerPoint PPT PresentationTRANSCRIPT
CoProD 08 Friday, October 3, 2008
Architecture Aware Tensor-Based Architecture Aware Tensor-Based ComputingComputing
Challenges for the Computer Science and Mathematics Communities
CISE CCF Algorithmic Foundations: Moore’s Law and Verifiable, Scalable, Portable, and Reproducible Matrix and Tensor Software
Lenore MullinLenore MullinProgram DirectorProgram Director
CISE CCF Algorithmic FoundationsCISE CCF Algorithmic FoundationsNational Science FoundationNational Science Foundation
[email protected]@nsf.gov
2CoProD 08 Friday, October 3, 2008
OutlineOutline
• NSF and CISENSF and CISE• CCF: Algorithmic Foundations and CCF: Algorithmic Foundations and
BeyondBeyond• Challenges and Open QuestionsChallenges and Open Questions• ConclusionsConclusions
3CoProD 08 Friday, October 3, 2008
National Science FoundationNational Science Foundation
Administrative Offices
Directorate for BiologicalSciences
Directorate for Computer &Information Science & Engineering
Directorate for Social, Behavioral & Economic Sciences
Directorate for Education& Human Resources
Directorate for Engineering Office of International Science and Engineering
Office of the Director
National ScienceBoard
Office Cyberinfrastructure
Office ofInspector General
Directorate for Mathematical &
Physical Sciences
Directorate for Geosciences Office of Polar Programs
4CoProD 08 Friday, October 3, 2008
CISE GoalsCISE Goals1.1. EnableEnable the United States to remain the United States to remain
competitive in computing, competitive in computing, communications, and information science communications, and information science and engineeringand engineering
2.2. PromotePromote understanding of the principles understanding of the principles and uses of advanced computing, and uses of advanced computing, communications, and information systems communications, and information systems in service to societyin service to society
3.3. ContributeContribute to universal, transparent, and to universal, transparent, and affordable participation in an information-affordable participation in an information-based societybased society
5CoProD 08 Friday, October 3, 2008
Achieving CISE GoalsAchieving CISE Goals• CISE supports investigator initiated CISE supports investigator initiated
research in all areas of computer and research in all areas of computer and information science and engineeringinformation science and engineering
• CISE helps develop and maintain CISE helps develop and maintain cutting-edge national computing and cutting-edge national computing and information infrastructure for research information infrastructure for research and educationand education
• CISE contributes to the education and CISE contributes to the education and training of the next generation of training of the next generation of computer scientists and engineers. computer scientists and engineers.
6CoProD 08 Friday, October 3, 2008
Assistant Director: Jeannette Wing
Deputy Assist Dir: Deborah Crawford
Div Dir:
Taieb
Znati
Div Dir:
Haym Hirsh
CISE OrganizationCISE Organizationhttp://www.nsf.gov/cise/about/org_chart.jsphttp://www.nsf.gov/cise/about/org_chart.jsp
Div Dir:
Sampath
Kannan
7CoProD 08 Friday, October 3, 2008
CCF: Computing andCCF: Computing andCommunication Foundations DivisionCommunication Foundations Division
http://www.nsf.gov/div/index.jsp?div=CCFhttp://www.nsf.gov/div/index.jsp?div=CCF
• Emerging Models and Technologies for Emerging Models and Technologies for ComputationComputation– Computational biology; quantum computing; nano-scale computing; Computational biology; quantum computing; nano-scale computing;
biologically-inspired computingbiologically-inspired computing
• Foundations of Computing Processes Foundations of Computing Processes and Artifactsand Artifacts– Advanced computation research; compilers; computer Advanced computation research; compilers; computer
architecture; design automation (micro/nano); graphics & architecture; design automation (micro/nano); graphics & visualization; software engineering & languagesvisualization; software engineering & languages
• Theoretical/Algorithmic Foundations Theoretical/Algorithmic Foundations – Computer science and communication theory; numeric Computer science and communication theory; numeric
symbolic/graphic computation; theory of computing; symbolic/graphic computation; theory of computing; computational algebra and geometry; signal processingcomputational algebra and geometry; signal processing
8CoProD 08 Friday, October 3, 2008
Theoretical/Algorithmic FoundationsTheoretical/Algorithmic FoundationsNumeric, Symbolic and Algebraic ComputingNumeric, Symbolic and Algebraic Computing
• Investigations into new data structures and algorithms that yield Investigations into new data structures and algorithms that yield optimizations for particular applications are encouraged.optimizations for particular applications are encouraged.
• This includes the design and construction of high quality scientific This includes the design and construction of high quality scientific software ideally adept across numerous scientific domains. software ideally adept across numerous scientific domains. Tensors are pervasive throughout NSF disciplines. Tensors are pervasive throughout NSF disciplines.
• Specific research topics of interest include, but are not limited to, Specific research topics of interest include, but are not limited to, the following: the following: numerical linear and multi-linear algebras, numerical linear and multi-linear algebras, tensor tensor algebras and decompositions used in memory hierarchyalgebras and decompositions used in memory hierarchy mappings;mappings; linear and non-linear optimization; modeling and linear and non-linear optimization; modeling and simulation of complex processes; and numerical solutions of simulation of complex processes; and numerical solutions of differential equations and PDE’s. Research in numerical differential equations and PDE’s. Research in numerical computing and optimization has natural interdisciplinary computing and optimization has natural interdisciplinary applications. In fact, applications. In fact, this program seeks applications in science this program seeks applications in science and engineering whose basic problems actually require the and engineering whose basic problems actually require the development of new numerical and optimization methods.development of new numerical and optimization methods.
9CoProD 08 Friday, October 3, 2008
Theoretical/Theoretical/Algorithmic FoundationsAlgorithmic Foundations
Numeric, Symbolic, and Algebraic ComputingNumeric, Symbolic, and Algebraic Computing• Research focused on finding powerful methods for Research focused on finding powerful methods for
symbolically solving algebraic - numeric systems that combine symbolically solving algebraic - numeric systems that combine differential, integral and polynomial equations is required. differential, integral and polynomial equations is required. Interests include foundational research in algorithms and their Interests include foundational research in algorithms and their efficient execution. efficient execution.
• Basic research topics include: computational algebra and Basic research topics include: computational algebra and analysis, computational number theory and algebraic analysis, computational number theory and algebraic geometry, geometry, integration of numeric and symbolic techniques, integration of numeric and symbolic techniques, symbolic scientific applications and software.symbolic scientific applications and software. Fruitful Fruitful application areas for symbolic computation include the application areas for symbolic computation include the solution of complex equation sets.solution of complex equation sets.
• Symbolic/Numeric manipulation and Tensors: Symbolic/Numeric manipulation and Tensors: – composition of tensor operations(symbolic) and numeric composition of tensor operations(symbolic) and numeric
instantiation: e.g. SAGE, Matlab, Mathematica, Maple, Expression instantiation: e.g. SAGE, Matlab, Mathematica, Maple, Expression Templates, XML, compilers, interpreters, …Templates, XML, compilers, interpreters, …
– Tensors are n-d arraysTensors are n-d arrays
10CoProD 08 Friday, October 3, 2008
CCF: Theoretical/Algorithmic CCF: Theoretical/Algorithmic Foundations (AF)Foundations (AF)
Cluster supports research in the following areas: • Models of computationModels of computation• Computational complexityComputational complexity• Parallel and distributed computationParallel and distributed computation• Random and approximate algorithmsRandom and approximate algorithms• Algorithmic algebra, geometry, topology, and logicAlgorithmic algebra, geometry, topology, and logic• Computational optimizationComputational optimization• Techniques for representing, coding and transmitting informationTechniques for representing, coding and transmitting information• $30M/Year$30M/Year• New TF Program Solicitation New TF Program Solicitation NSF 08-518NSF 08-518
– Due Date March 12, 2008 - March 19, 2008 Due Date March 12, 2008 - March 19, 2008 http://www.nsf.gov/pubs/2008/nsf08518/nsf08518.htmhttp://www.nsf.gov/pubs/2008/nsf08518/nsf08518.htmTR Program Officers:TR Program Officers: John Cozzens, Lenore Mullin, Richard Biegel, John Cozzens, Lenore Mullin, Richard Biegel,
Sirin Tekinay, Robert Grafton, EK ParkSirin Tekinay, Robert Grafton, EK Park
11CoProD 08 Friday, October 3, 2008
Theoretical/Algorithmic Foundations Theoretical/Algorithmic Foundations and BEYOND!!!and BEYOND!!!
• How can we create transformational How can we create transformational science when we can’t verify scientific science when we can’t verify scientific software?software?
• How can domain scientists doing How can domain scientists doing computational experiments achieve computational experiments achieve reproducibility:reproducibility:– Same answer and is that answer correct?Same answer and is that answer correct?– Are the resources used the same?Are the resources used the same?– Can the software scale to today’s and Can the software scale to today’s and
tomorrow’s hardware?tomorrow’s hardware?– Can we produce software that is optimal?Can we produce software that is optimal?
12CoProD 08 Friday, October 3, 2008
Theoretical/Algorithmic Foundations Theoretical/Algorithmic Foundations and BEYOND!!!and BEYOND!!!
• Optimality and Large Data SetsOptimality and Large Data Sets• Optimality and Data Locality across Optimality and Data Locality across
processor/memory hierarchyprocessor/memory hierarchy• Peta-Scale Computing and Beyond: Peta-Scale Computing and Beyond:
scalability and portabilityscalability and portability• Algebra of Arrays to build ANY Tensor Algebra of Arrays to build ANY Tensor
based applicationbased application– Must be a closed algebra without Must be a closed algebra without
anomalies for verificationanomalies for verification– No language today has such an algebraNo language today has such an algebra
13CoProD 08 Friday, October 3, 2008
Moore’s Law:Moore’s Law: Data Density Data Density DoublesDoubles every every 18 Months18 Months
EXCEPTEXCEPT Notice Notice flatteningflattening of slope due to Compilers of slope due to Compilers
1850 1950 20001900 2050
10-6
103
1
10-3
106
109
Babbage Engine
CMOS ICs
TX-2
ENIAC
Differential Analyzer
GeneralArchitecture
Lattice-GasArchitecture
Quantum Dots
Liquid NMR
Conve
ntion
al Com
puter
Roa
dmap
QC
Roa
dmap
MIPS
Year
14CoProD 08 Friday, October 3, 2008
Proebsting’s Law:Proebsting’s Law:CompilerCompiler Advances Advances Double Double Computing PowerComputing Power Every Every 18 Years18 Years
This means that This means that while while hardware hardware computing horsepowercomputing horsepower increases at increases at roughlyroughly 60%/year 60%/year, , compiler optimizations contribute compiler optimizations contribute onlyonly 4%. 4%.
1850 1950 20001900 2050
10-6
103
1
10-3
106
109
Babbage Engine
CMOS ICs
TX-2
ENIAC
Differential Analyzer
GeneralArchitecture
Lattice-GasArchitecture
Quantum Dots
Liquid NMR
Conve
ntion
al Com
puter
Roa
dmap
QC
Roa
dmap
MIPS
Year
15CoProD 08 Friday, October 3, 2008
What is Computational Science and What is Computational Science and Engineering?Engineering?
Mathematics
Physical Sciences and Biological Sciences
Computer Science and Engineering
X
X = The Intersection of Domain Sciences, Mathematics andComputer Science and Engineering
16CoProD 08 Friday, October 3, 2008
What can we do?What can we do?• Recent Award: for mini-symposia at the 2009 SIAM Annual
meeting (Lenore Cowen, Tufts: Uniting Discrete Methods, optimizations and CISE Community with the Community studying Matrix Operations, Tensors, Verifiable Computational Experiments and Scalability) in which Computer Scientists and students will be funded to attend and interact. This was initiated due to numerous tensor sessions at the 2008 SIAM Annual meeting.
– Tensor Decompositions Solving Fundamental Problems in Chemistry
– Tensor Decompositions for Large-Scale Date Applications– A Novel Higher-order Generalized Singular Value Decomposition for
Comparative Analysis of DNA Microarray Data from Different Organisms
– Tensor Algebraic Methods and Their Application to High-Dimensional Multi-Modal Data
– TensorFaces: Multilinear (Tensor) Decomposition of Image Ensembles
– Multilinear (Tensor) Independent Component Analysis– Modeling of Epileptic Seizures using Tensor Analysis– On a Generalization of Sylvester Methods for Symmetric Tensor
Decomposition
17CoProD 08 Friday, October 3, 2008
What can we do?What can we do?• A mini-symposium at the 2008 SIAM Annual meeting (MS3)
entitled Architecture-Aware Scientific Computing. Organizers and Presenters: L. Mullin (NSF) and Padma Raghavan (NSF PI).
• Plans to have an invitation only workshop with Frank Olken (IIS) are planned for spring 2009 to bring together experts in Knowledge Representation, Tensors, Algorithms and other related areas in Computer Science. Charles Van Loan, Cornell
• Recent Award: for a workshop at the Courant Institute to bring together Mathematicians and Computer Scientists to discuss scalable algorithms for PDEs on parallel, distributed, and multi-core algorithms. ODEs and PDEs can be represented as matrix and tensor operations. Numeric and symbolic environments are growing in popularity to combine verification and optimal implementations.
• Career Application 2008: BIO and CCF AF (Orly Alter: Integrative and Comparative Tensor Algebra Models of DNA Microarray Data from Different Studies of the Cell Cycle)
18CoProD 08 Friday, October 3, 2008
What can we do?What can we do?• Milestones in Computer Algebra 2008:
(Invitation only workshop) Systematic Tensor Simplification: a Diagramatic
Approach by A. D. Kennedy and T. Reiter. This workshop illustrated the need for combined numeric and symbolic environments to compute and symbolically prove correctness of designs. Numerous articles from this workshop discussed the need to combine environments, which was validated by an NSF supported workshop report written by E. Kaltofen (one of the organizers) November 2007 at NSF in Arlington. Symbolic/Numeric proposals entered in the 2008 NSG solicitation showed a 100% growth over 2007.
19CoProD 08 Friday, October 3, 2008
What can we do?What can we do?• SCAN 2008: expected outcomesSCAN 2008: expected outcomes
– Hardware and software support for verification toolsHardware and software support for verification tools– Theory, algorithms and arithmetic for verified numerical Theory, algorithms and arithmetic for verified numerical
computationscomputations– Supercomputing and reliabilitySupercomputing and reliability– Dynamical systems and verified numerical computationDynamical systems and verified numerical computation– Global optimization and verified numerical computationGlobal optimization and verified numerical computation– Programming tools for verified numerical computationProgramming tools for verified numerical computation– Computer aided proofsComputer aided proofs– Industrial and scientific applications of verified numerical Industrial and scientific applications of verified numerical
computationscomputations
• CoProD 08: expected outcomesCoProD 08: expected outcomes– Definition of new directions for combining numeric and Definition of new directions for combining numeric and
symbolic approaches in solving constraints and symbolic approaches in solving constraints and optimization problems in particular and in decision making optimization problems in particular and in decision making in general.in general.
20CoProD 08 Friday, October 3, 2008
What can we do?What can we do?• Matrix and Tensor operations are pervasive in science and Matrix and Tensor operations are pervasive in science and
engineeringengineering• Tensors are n-d arrays, but n-d arrays are more generalTensors are n-d arrays, but n-d arrays are more general• Generalized multi-dimensional Inner and Outer productsGeneralized multi-dimensional Inner and Outer products• Summations of multi-dimensional arraysSummations of multi-dimensional arrays• Projection operatorsProjection operators• AX=B like problemsAX=B like problems• Coupled differential and integral equations, eigenvalue Coupled differential and integral equations, eigenvalue
problems: generally translate to matrix problemsproblems: generally translate to matrix problems• Even non-linear operations: iterative solutionsEven non-linear operations: iterative solutions• Linear and Multilinear Algebra is not enough!Linear and Multilinear Algebra is not enough!
– Scalars, anomaliesScalars, anomalies• Existing languages are not enough!Existing languages are not enough!
21CoProD 08 Friday, October 3, 2008
Possible SolutionsPossible Solutions
• Identify a closed algebra that subsumes Identify a closed algebra that subsumes important matrix operationsimportant matrix operations
• Augment existing languages with this Augment existing languages with this algebra: optional usealgebra: optional use
• Solve a few important problems completelySolve a few important problems completely• Use the same algebra to map to processor Use the same algebra to map to processor
memory hierarchiesmemory hierarchies• Use the same algebra to abstract machinesUse the same algebra to abstract machines• These concepts proposed at Sandia These concepts proposed at Sandia
Workshop on Memory Hierarchy Workshop on Memory Hierarchy Optimizations for Scientific Software, Optimizations for Scientific Software, January 2008January 2008
22CoProD 08 Friday, October 3, 2008
Possible SolutionsPossible Solutions• Synergize Mathematicians, Synergize Mathematicians,
Computer Scientists and Domain Computer Scientists and Domain Scientists to collaborateScientists to collaborate
• Create a new community that solves Create a new community that solves these open questionsthese open questions– Revisit and reinventRevisit and reinvent
• Community then creates a research Community then creates a research base for funding agenciesbase for funding agencies
• Workshops and ColloquiaWorkshops and Colloquia– Supplements and/or new grantsSupplements and/or new grants
23CoProD 08 Friday, October 3, 2008
Numeric, Symbolic and Algebraic Numeric, Symbolic and Algebraic Computing Program in TFComputing Program in TF
• These issues appear in last year’s and These issues appear in last year’s and this year’s solicitationsthis year’s solicitations
• MPS and CISE cooperating programsMPS and CISE cooperating programs– Hope to develop new solicitationsHope to develop new solicitations
• Attend SIAM, SC, APS, MRS, etc. Attend SIAM, SC, APS, MRS, etc. – Raise the consciousness of computational Raise the consciousness of computational
scientists in these communitiesscientists in these communities• After solving small number of After solving small number of
algorithms within this algebra, identify algorithms within this algebra, identify what to do next.what to do next.– Can be used in existing programs.Can be used in existing programs.
24CoProD 08 Friday, October 3, 2008
Thank YouThank You
Questions?Questions?
Lenore MullinLenore MullinProgram DirectorProgram Director
National Science FoundationNational Science FoundationComputer & Information Science & Engineering Computer & Information Science & Engineering
DirectorateDirectorateDivision of Computer and Communications Foundations Division of Computer and Communications Foundations
Algorithmic Foundations ClusterAlgorithmic Foundations Cluster
[email protected]@nsf.gov