Discovery of synoptic patterns of climate variability and change using data mining and high performance computing

Climate Wikience: A Summary of Results

DevelopersZapletin E.A., Melnykov A.I.

Donetsk National Technical University

Ukraine

Ideas, Architect, Slides Rodriges Zalipynis R.A.

P.I.Prof. Averin G.V.

SpeakerProf. Bashkov E.A.

The Wikience Concept

Let’s “create” new analysts!

Usually domain experts have a solid background on the subject. They will perform very well without computer scientists, even if equipped with much simpler tools. They know what to seek and how to seek. Many active experts are currently uninvolved, especially operational professionals.

WikienceOriginates from Wiki + Science

Wikience – international pronunciation rather than wiki science

Wiki science – already reserved – is the study of how the phenomenon of wikis, and free, open collaborative works in general (like free software and open source), are growing, changing and adapting.http://en.wikiversity.org/wiki/Wiki_science

Reanalysis archivesInterpolate all available data by statistical and numerical methods on regular latitude-longitude grid

Pioneer, released in 1996

NCEP/NCAR Reanalysis 1

1949 – present

2.5 x 2.5 grid

Applications

From simple warming trends to validation of climate models

Reanalysis archives• NCEP/NCAR 2• ERA-40, ERA-15• ERA-Interim• JRA-25• NOAA-CIRES 20CR• NASA MERRA• NCEP CFSR

Over 9 000 publications reference

Kalnay et al., The NCEP/NCAR 40-year reanalysis project, BAMS, 77, 437-470, 1996.

Visualization is vital for data understanding and perception

Is it?

University of Melbourne (Australia)

Visualization is vital for data understanding and perception

Is it?

National Oceanic and Atmospheric Administration (NOAA, USA)

Visualization is vital for data understanding and perception

Is it?

Improving the Accuracy of Mapping Cyclone Numbers and Frequencies, 2010 by O.Zolina and S. Gulev (Russia)

Wikience

4D isopleths• 1st in the world • real-time• closed polygons (GIS)• all TBs of reanalysis data on-line

KEY GRAPHICS

FOR CLIMATE

RESEARCH

Vast amounts of data (TBs)

1996 NCEP/NCAR ~ 4 TB

2010 NCEP CFSRR ~ 100 TB

Real-time technology for thousands of users

How to maintain all available data on-line?

Scientific formats

Convert data to new representation?

Need to keep metadata intact

Invent new schemas to import metadata into?

3D visualization technology

No open algorithms for isopleths, are they?

Obstacles for wide data accessibility

ChronosServer

Turns vast amounts of existing data into actionable intelligence

Rodriges Zalipynis R.A. ChronosServer: real-time access to “native” multi-terabyte retrospective data warehouse by thousands of concurrent clients. Informatics, cybernetics and computer engineering, pp. 151–161. Vol. 14 (188) – Donetsk: DonNTU — 2011. – 391 p.

Atmospheric major components: cyclones

Areas of low pressure

(simple definition)

Seen as several closed isobars nested in each other

L

Climate is changing along with cyclones

Cyclone importance

• heat transition from tropics to the poles• define local weather• local winds • severe weather events• pollution situation

key drivers of climate change

How to research cyclones? – TrackInitial

In a day

Sharing “active” research data4D Cyclone paths• 1st in the world • real-time

Others can:• verify tracks• interactively analyze• use in own research

Remote Sensing: Terra MODIS• 36 spectral channels• Daily frequency• Global coverage

MODIS L3 product• Global 1º×1º grid• Over 600 datasets

(clouds, water vapor, aerosol properties)

http://www.universetoday.com/84961/terra-satellite/http://modis-atmos.gsfc.nasa.gov/tools_level3browse.html

Terra MODIS: Cloud Optical Thickness

Tropical Cyclones

IBTrACS

International Best Track Archive for Climate Stewardship• 1842 – present• multiple characteristics• the most complete

August 23 – 30, 2005280 km/h highest wind

1,833 fatalities

$108 billion damage

Hurricane Katrina

just a picture

IBTrACS:Hurricane Katrina

NOT just a picture

Hurricane Katrina (IBTrACS) and Cloud Optical Thickness (MODIS L3)

August 28, 2005

Near peak strength

Analysis toolsSimply select & watch

Acknowledgements

This work was supported by Award No. UKM1-2973-DO-09 of the U.S. Civilian Research & Development Foundation (CRDF). Any opinions, findings and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of CRDF.

DNTU Computer Cluster• Hybrid GPU-CPU• 32 CPU cores• 2 688 GPU cores, CUDA• 7 Teraflops• 35 TB HDD • 1 Gbit/sec local network• 1 Gbit/sec Internet band, optic fiber• 10 KW power supply

Donetsk National Technical University

Global Models

Databases in knowledge domains

Wikience

Data mining

Donetsk National Technical University

Development of human potential

Database fusion:

World bank http://data.worldbank.org

UN Development Programhttp://www.hdr.undp.org

Economical Forum http://www.weforum.org

Data mining:Multidimensional scaling

Donetsk National Technical University

Planet Biodiversity

Database fusion:

(Databases of animal and plants)

GIS-models for areals of biological species spread

(150 - 200 thousands of main animal and plant species)

Climate reanalysis archives

Donetsk National Technical University

Data mining: world countries development indicators

Fertility Tuberculosis rate

Children's mortality

Internet usersGovernmental debt service

Population with age before15

Donetsk National Technical University

Data mining: world countries development - energy

CountryDevelopment

indexCountry’s

rateCountry’s rate according to development rate

Norway 63138 2 6

USA 35043 10 16

Germany 18045 22 53

Poland 9155 50 49

Russian 15658 33 22

Ukraine 9108 51 42

China 3500 91 65

India 1385 115 95

Nigeria 305 138 126

Donetsk National Technical University

ЕЕА and DNTU: System for ecological monitoring of Donetsk region – English version

Discovery of synoptic patterns of climate variability and change using data mining and high performance computing

wikience.donntu.edu.ua Contact address: rodriges@csm.donntu.edu.ua

Climate Wikience: A Summary of Results

Please, reference materials taken from this presentation as

Rodriges Zalipynis R.A., Averin G.V., Zapletin E.A., Melnykov A.I., Bashkov E.A., Climate Wikience: A Summary of Results, Minneapolis, USA, 2012. Web: wikience.donntu.edu.ua/rodriges/publications/rodriges_CRDF_2012.pptx