microsoft research presentation

8/10/2019 Microsoft Research Presentation

1/59


2/59

Advance the field in

the areas we choose todo research

Transfer the best

research results intoMicrosofts products

Ensure the future of

Microsoft and thecomputing field

MSR was born from a 1990 memo from CTO Nathan Myhrvold Since 1991 led by Rick Rashid Peter Lee took over in July 2013

http://research.microsoft.com

/
http://research.microsoft.com/http://research.microsoft.com/http://research.microsoft.com/


3/59

Microsoft Research: Computational Ecology

and Environmental Science Group

http://research.microsoft.com/en-us/groups/ecology/
http://research.microsoft.com/en-us/groups/ecology/http://research.microsoft.com/en-us/groups/ecology/


4/59

The climate dependence of the terrestrial carbon cycle,including parameter and structural uncertainties
http://research.microsoft.com/apps/pubs/default.aspx?id=180603http://research.microsoft.com/apps/pubs/default.aspx?id=180603http://research.microsoft.com/apps/pubs/default.aspx?id=180603


5/59

Future Global Change Predictions


6/59

Future Global Change Predictions


7/59

Hypotheses

Data

Bayesianinference

Model structure

Parameterdistribution

Predictions withuncertainty
http://research.microsoft.com/en-us/um/cambridge/groups/science/tools/filzbach/filzbach.htmhttp://research.microsoft.com/en-us/um/cambridge/groups/science/tools/filzbach/filzbach.htm


8/59

Predicted changes in wheat yields under future warming scenarios (1 to 5

degrees warming in mean global temperature)http://research.microsoft.com/apps/pubs/default.aspx?id=217346
http://research.microsoft.com/apps/pubs/default.aspx?id=217346http://research.microsoft.com/apps/pubs/default.aspx?id=217346http://research.microsoft.com/apps/pubs/default.aspx?id=217346


9/59

Nature, Vol. 493, pp. 295-297http://www.madingleymodel.org/

Emergent Global Patterns of Ecosystem Structure andFunction from a Mechanistic General Ecosystem Model
http://www.madingleymodel.org/http://www.plosbiology.org/article/info%3Adoi%2F10.1371%2Fjournal.pbio.1001841http://www.plosbiology.org/article/info%3Adoi%2F10.1371%2Fjournal.pbio.1001841http://www.plosbiology.org/article/info%3Adoi%2F10.1371%2Fjournal.pbio.1001841http://www.plosbiology.org/article/info%3Adoi%2F10.1371%2Fjournal.pbio.1001841http://www.madingleymodel.org/


10/59

www.technologyfornature.org
http://www.technologyfornature.org/http://www.technologyfornature.org/


11/59


12/59

Data

Acquisition &modelling

Collaboration

andvisualisation

Analysis &data mining

Dissemination& sharing

Archiving andpreserving

fourthparadigm.org

Data-intensive Research
http://fourthparadigm.org/http://fourthparadigm.org/


13/59

X-Info

Data ingest

Managing a petabyte Common schema How to organize it How to reorganize it How to share with others

Query and Vis tools

Building and executing models Integrating data and Literature Documenting experiments Curation and long-term

preservation

The Generic Problems

Experiments &Instruments

Simulations

Literature

Other Archives

facts

facts

facts

facts

Questions

Answers


14/59

Monitoring

Collation

Quality assurance

Aggregation

Analysis

Reporting

Forecasting

Distribution

Done poorly,but a few notablecounter-examples

Done poorly to moderately,not easy to find

Sometimes done well,generally discoverable and available,

but could be improved

Integration

(I. Zaslavsky & CSIRO, BOM, WMO)


15/59


16/59


17/59


18/59

Complex shared detector Simple instrument (if any)

Complex and Heavy process by experts Ad hoc observations and models

KB

GB

TB

PB

Science happens when PBs, TBs, GBs, and KBs can be mashed up simply

Provenance and trust widely variesData acquisition, early processing, and reporting rangesfrom a large government agency to individual scientists.

Smaller data often passed around in email; big datadownloads can take days (if at all)

Data sharing concerns and patterns varyOpen access followed by (non-repeatable and tedious) pre-

processingTrue science ready data set but concerns about misuse,misunderstanding particularly for hard won data.

Computational tools differ.Not everyone can get an account at a supercomputer center

Very large computations require engineering (error

handling)Space and time arent always simple dimensions
https://lpdaac.usgs.gov/https://lpdaac.usgs.gov/http://nsidc.org/daac/index.htmlhttp://nsidc.org/daac/index.html


19/59


20/59


21/59

http://research.microsoft.com/projects/msrceesdm/
http://research.microsoft.com/projects/msrceesdm/http://research.microsoft.com/projects/msrceesdm/


22/59


23/59

Software to facilitate data-constrained modelling

Combines: Fetchclimate and OData (or read in data file) Model specification (directly in F# or indirectly e.g. MATLAB CLR) Auto-detect inference problems Filzbach inference engine to estimate parameters

Propagation of parameter uncertainty in predictions Visualise probabilistic outputs Run arbitrary F# code Provenance recorded, navigable and modifiable Push computation to the cloud Remote and local git source control Multi-platform Push models and predictions to Fetchclimate

... in progress... Incorporating new modelling tools of the ComputationalScience group e.g. SPIM, GEC, Madingley


24/59

Currently being used exclusively on the research of thecomputational science group, with example applications being Global crop modelling (statistical and process based) Global marine plankton dynamics prediction (shown here) Understanding gene expression

How different? Its a more ideal combination Advanced analytical methods (inference on top of process based

models) Application to large scale problems, in terms of large in space (e.g.

whole globe), large in computational demand, large in levels ofdetails in models (e.g. a whole forest model)

Allows co-production and sharing of models and entire research

pipeline Complete provenance always survives allowing future use WITHOUT forcing user into a cumbersome workflow paradigm


25/59

Data

management

& organization

Data Reuse &

Reproducibility

https://dataup.org/

Facilitate

Archiving

Sharing and

Publishing

https://web.dataup.org/
https://dataup.org/https://web.dataup.org/https://web.dataup.org/https://web.dataup.org/https://dataup.org/


26/59

http://www.fetchclimate.org/

Project details: http://research.microsoft.com/en-us/projects/fetchclimate/
http://www.fetchclimate.org/http://www.fetchclimate.org/http://research.microsoft.com/en-us/projects/fetchclimate/http://research.microsoft.com/en-us/projects/fetchclimate/http://research.microsoft.com/en-us/projects/fetchclimate/http://www.fetchclimate.org/


27/59


28/59


29/59


30/59


31/59


32/59


33/59

www.chronozoomproject.org

L
http://www.chronozoomproject.org/http://www.chronozoomproject.org/


34/59

WorldWideTelescope Tours

Layerscape

http://www.layerscape.org/

E l P BI
http://www.worldwidetelescope.org/http://www.worldwidetelescope.org/http://www.layerscape.org/Home/Indexhttp://www.layerscape.org/Home/Indexhttp://www.layerscape.org/Home/Indexhttp://www.worldwidetelescope.org/http://www.microsoft.com/en-us/powerbi/default.aspxhttp://www.microsoft.com/en-us/powerbi/default.aspx


35/59

Excel Power BI

Predictive modelling
http://www.microsoft.com/en-us/powerbi/default.aspxhttp://www.microsoft.com/en-us/powerBI/bi-blog.aspx?blog=91d46819-8472-40ad-a661-2c78acb4018c:10523484#fbid=lhKY88q5K7hhttp://www.microsoft.com/en-us/powerBI/bi-blog.aspx?blog=91d46819-8472-40ad-a661-2c78acb4018c:10523484#fbid=lhKY88q5K7hhttp://www.microsoft.com/en-us/powerbi/default.aspx


36/59


37/59

http://github.com/azure
http://github.com/windowsazurehttp://github.com/windowsazurehttp://github.com/windowsazure


38/59

Weather Forecast

Computation as a Service

ttp://aka.ms/oljnt2

htt : weatherservice.clouda .net
http://aka.ms/oljnt2http://aka.ms/oljnt2http://weatherservice.cloudapp.net/http://weatherservice.cloudapp.net/http://aka.ms/oljnt2


39/59

http://weatherservice.cloudapp.net


40/59


41/59


42/59

Web search:

open we ther

data zure


43/59


44/59


45/59

Water depth map of Newcastle City Council area (~130km2).Storm event of 60 minutes and 100 years return period

http://www.ncl.ac.uk/ceser/researchprogramme/in

formatics/citycaturbanfloodmodel/
http://www.ncl.ac.uk/ceser/researchprogramme/informatics/citycaturbanfloodmodel/http://www.ncl.ac.uk/ceser/researchprogramme/informatics/citycaturbanfloodmodel/http://www.ncl.ac.uk/ceser/researchprogramme/informatics/citycaturbanfloodmodel/http://www.ncl.ac.uk/ceser/researchprogramme/informatics/citycaturbanfloodmodel/


46/59


47/59

Customer Data Center

MODIS Azure:


48/59

Computing Evapotranspiration ET) in the Cloud

A pipeline fordownload,

processing, and

reduction of

diverse NASAMODIS satellite

imagery.

Catharine van Ingen (Microsoft Research), Jie Li, Marty Humphrey (UVA), Youngryel Ryu (UCB), Deb Agarwal (BWC/LBL), Keith Jackson(BL), Jay Borenstein (Stanford) , Team SICT: Vlad Andrei, Klaus Ganser, Samir Selman, Nandita Prabhu (Stanford), Team Nimbus: David Li,Sudarshan Rangarajan, Shantanu Kurhekar, Riddhi Mittal (Stanford)

http://research.microsoft.com/en-us/projects/modisazure/default.aspx

MODIS Azure Service

Source Imagery Download Sites
http://research.microsoft.com/en-us/projects/modisazure/default.aspxhttp://research.microsoft.com/en-us/projects/modisazure/default.aspxhttp://research.microsoft.com/en-us/projects/modisazure/default.aspx


49/59

Reduction #1 Queue

ScientificResultsDownload

Reduction #2 Queue

SourceMetadata

MODIS AzureService Web Role

Portal

RequestQueue

Analysis Reduction Stage

Data Collection Stage

Source Imagery Download Sites

. . .

ReprojectionQueue

Derivation Reduction StageReprojection Stage

Download

Queue

Scientists

Science

results

Catharine van Ingen (Microsoft Research), Jie Li, Marty Humphrey (UVA), Youngryel Ryu (UCB), Deb Agarwal (BWC/LBL), Keith Jackson(BL), Jay Borenstein (Stanford) , Team SICT: Vlad Andrei, Klaus Ganser, Samir Selman, Nandita Prabhu (Stanford), Team Nimbus: David Li,Sudarshan Rangarajan, Shantanu Kurhekar, Riddhi Mittal (Stanford)


50/59


51/59

Scale out Computing byUsing R, Matlab, andPython with MicrosoftAzure on-demandwebinar
http://research.microsoft.com/apps/video/default.aspx?id=217431http://research.microsoft.com/apps/video/default.aspx?id=217431http://research.microsoft.com/apps/video/default.aspx?id=217431http://research.microsoft.com/apps/video/default.aspx?id=217431http://research.microsoft.com/apps/video/default.aspx?id=217431http://research.microsoft.com/apps/video/default.aspx?id=217431


52/59

http://azure.microsoft.com/en-us/campaigns/machine-learning/

Register for webinaron 22nd July 2014

Research Cloud Ecosystem
http://azure.microsoft.com/en-us/campaigns/machine-learning/https://www.eventbuilder.com/microsoft/event_desc.asp?p_event=o5b0a8p4https://www.eventbuilder.com/microsoft/event_desc.asp?p_event=o5b0a8p4https://www.eventbuilder.com/microsoft/event_desc.asp?p_event=o5b0a8p4https://www.eventbuilder.com/microsoft/event_desc.asp?p_event=o5b0a8p4https://www.eventbuilder.com/microsoft/event_desc.asp?p_event=o5b0a8p4http://azure.microsoft.com/en-us/campaigns/machine-learning/


53/59


54/59

www.azure4research.com
http://www.azure4research.com/http://www.azure4research.com/


55/59

Use laptops &desktop computers

Overwhelmed bydata

Finding analysisever more difficult;sharing evenharder

www.azure4research.com

Microsoft Azure for ResearchAccelerate the Speed of Scientific Discovery
http://www.azure4research.com/http://www.azure4research.com/


56/59

Microsoft Azure forResearch Group

@azure4research

www azure4research com

Accelerate the Speed of Scientific Discovery

[email protected]
http://www.linkedin.com/groups/Windows-Azure-Research-6521580?home=&gid=6521580&trk=groups_most_popular-h-logohttp://www.linkedin.com/groups/Windows-Azure-Research-6521580?home=&gid=6521580&trk=groups_most_popular-h-logohttps://twitter.com/Azure4Researchhttp://www.azure4research.com/http://www.azure4research.com/https://twitter.com/Azure4Researchhttp://www.linkedin.com/groups/Windows-Azure-Research-6521580?home=&gid=6521580&trk=groups_most_popular-h-logo


57/59


58/59


59/59

microsoft research presentation

Documents