weather and climate visualization software

Dr K C Gouda, ScientistCSIR -4PI, Wind Tunnel RoadBelur Campus, Bangalore-37.

Dr Shakti BanerjeeProfessor and Head of Dept, School of Statistics, Devi Ahilya Viswavidyalaya,Taksashila Parisar, Khandwa Road,Indore:452001

School of Future Studies & Planning,DAVV, Indore (M.P.)

Introduction

• The main aim of the project work entitled“DEVELOPMENT OF VISUALIZATION SOFTWAREFOR WEATHER AND CLIMATE DATA ANALYSIS” isto analyze the input file and determine if the datavalues fall within a certain range and to generatea new file of the same format using the thresholdvalue specified by the user.

• In our project we have read the netCDF files witha utility called ‘ncdump’ and generated a new file.

Organisation ProfileIn the late 1980's, the Council of Scientific and Industrial Research (CSIR) recognized the growingsignificance of Mathematical Modelling and Computer Simulation, as it became evident thatmodelling approaches were critical for illuminating the structure and evolution of complex systemsthat were invading the areas of scientific analysis and technological design.

Computing Facilities at C-MMACSIt has one of the most sophisticated and advanced computing environments in India and certainlythe best in CSIR. The highest performance computing and networking resources in C-MMACSincludes:

• SGI Altix 3700 BX2 Compute Servers• SGI Origin 3900 Compute Server• SGI Altix 350 Compute Servers• SGI Origin 200 Compute Server• High Availability IBM Storage Server• SGI Octane 2 Workstation• SGI Indigo2 Extreme Workstations• SGI O2 Workstations• SGI Indy Workstations• HP Xeon Linux Servers• HP/Compaq D330 Linux Workstations• IBM Net vista A30 Linux/Windows Desktops• IBM PC 300GL Linux/Windows Desktops• C-MMACS Local Area Network

Research Activities• C-MMACS Environment Modelling Program• Monsoon simulation• Tropical Modelling• Hydrological Modelling• Neural Network forecasting• Ocean Atmosphere coupling• Ocean Modelling• Atmospheric modelling• Lithosphere Modelling• Dynamical System Modelling• Industrial Computational MechanicsAreas of Specialization• Aerospace Propulsion• Computational Aerodynamics• Wind Tunnel Testing• Flight mechanics and control• Structural mechanics• Parallel Computing

Organisation Profile

Climate and Weather

Weather

Weather only describe the short-term conditionof the variable in the given region.

Climate

Climate is a measure of the average pattern ofvariation in temperature, humidity, atmosphericpressure, wind, precipitation, atmosphericparticle count and other meteorological variablesin a given region over long period of time.

Description of Related Study

Latitude and LongitudeLatitude is the angular distance, in degrees, minutes and seconds ofa point north and south of the Equator.Longitude is the angular distance, in degrees, minutes and secondsof a point east or west of the Prime (Greenwich) Meridian

ResolutionResolution is represented in the of number of grid points. Increasein resolution means increasing the number of grid points.

GridGrid is a pattern of horizontal and vertical lines on a chart or map,such as those representing latitude and longitude, usually crossingeach other to form squares/rectangles which help determiningabsolute location.

Description of Related Study

Meteorological Weather and climate DataMeteorology is the interdisciplinary scientific study of the atmospherethat focuses on weather process and forecasting.Meteorological phenomena are observable weather events whichilluminate and are explained by the science of meteorology.Those events are bound by the variables that exist in Earth’s atmosphere.They are temperature, air pressure, water vapour and the gradients andinteractions of each variable and how they change in time.Atmosphere and oceanographic data may be archived in several differentcomputer forms: character format, native format, packed binary or in oneof several “standard” scientific data formats. Users of datasets must beaware of how the data are stored.Architecture independent standard formats commonly used foratmosphere and oceanographic datasets include:

• NetCDF (network Common Data Format)• GrADS

Scope of the System

The proposed system aims to analyse data indifferent formats and to perform quality control ifthe data is not within certain standard ranges. Thesoftware is capable of analysing NetCDF, GRIB, ASCIIdata formats. The GUI is incorporated to performthese operations in a user friendly way.

Objective of the Proposed System

• Extracting the data values of the given parameters from the data file.

• Scanning the data and determining the maximum and minimum values in a domain.

• Development of algorithm for Climate data analysis

• Multi-format and Multi-source climate data analysis.

• Development of data display system for User

• Data Mining

• Spatial Interpolation modeling

• Spatial Statistical Analysis.

System Module

My software work as a System and thissoftware system is divided into two part asfollows :

• Perform Statistical Operation on Mined data.

• Perform Interpolation and generate spatialimage Based on mined data.

Data Flow Diagram

User

Weather & Climate Data

Display System

Fill Essential Data

Display Spatial

Maps & Chart

Shapefile(.shp)

Draw Region

Boundary

NetCDF Data

(.nc)

Analysis

the Data

SaveSpatial Image

(.jpeg)

Figure : 0th Level DFD

User

Spatial Analysis

Interpolated

Spatial Map

Fill Essential

Information

Fill Essential

Information

NetCDF Data

(.nc)

Compare the Data

Perform

Interpolation

Shapefile(.shp)Make

Boundary

Statistical Information

& Chart

Interpolated

Map

Figure : 1st Level DFD

Platforms, Languages and Tools

Hardware Configuration

• Processor Intel Pentium Dual Core and Above

• Memory: 512 MB RAM and Above

• Clock Speed: 1.72 GHz and Above

• Hard Disk Capacity: 80GB

Software Configuration

• Operating System: Ubuntu, Fedora Linux, Windows, Mac

• Language: Java

• Complier: Java Compiler (Java Virtual Machine)

• Other tools and utilities: JavaFX, Visual Paradism, many .jar files

Platforms, Languages

• JAVAThe Java platform is the name for a bundle of related programs from Sun thatallow for developing and running programs written in the Java programminglanguage. The platform is not specific to any one processor or operating system,but rather an execution engine (called a virtual machine) and a compiler with a setof libraries that are implemented for various hardware and operating systems sothat Java programs can run identically on all of them.Java Card: A technology that allows small Java-based applications (applets) to berun securely on smart card and similar small-memory devices.Java ME(Micro Edition): Specifies several different sets of libraries (known as

profiles) for devices with limited storage, display, and power capacities. Often usedto develop applications for mobile devices, PDAs, TV set-of-box, and printers.Java SE (Standard Edition): For general-purpose use on desktop PCs, servers andsimilar devices.Java EE (Enterprise Edition): Java SE plus various APIs useful for multi-tier client-server, enterprise application.


Platforms, Languages

JAVA compilerWhen you program for the Java platform, you write source code in.java files and then compile them. The compiler checks your codeagainst the language's syntax rules, then writes out bytecodes in .classfiles. Bytecodes are standard instructions targeted to run on a Javavirtual machine (JVM). In adding this level of abstraction, the Javacompiler differs from other language compilers, which write outinstructions suitable for the CPU chipset the program will run on.

JVMAt run time, the JVM reads and interprets .class files and executes theprogram's instructions on the native hardware platform for which theJVM was written. The JVM interprets the bytecodes just as a CPUwould interpret assembly-language instructions. The difference is thatthe JVM is a piece of software written specifically for a particularplatform. The JVM is the heart of the Java language's "write-once,run-anywhere" principle.


Tools

JavaFX ApplicationsWritten as a Java API,JavaFX applicationcode can referenceAPIs from any Javalibrary.The look and feel ofJavaFX applications canbe customized.Graphic designers caneasily customize theappearance and styleof the applicationthrough the CSS.


EclipsePlatforms, Languages and Tools

Design Pattern

MVC PatternMVC Pattern stands for Model-View-Controller Pattern. Thispattern is used to separateapplication's concerns.

• Model - Model represents an objector JAVA POJO carrying data. It canalso have logic to update controllerif its data changes.

• View - View represents thevisualization of the data that modelcontains.

• Controller - Controller acts on bothModel and view. It controls the dataflow into model object and updatesthe view whenever data changes. Itkeeps View and Model separate.

Design Pattern

Data File Format

.shp(shapefile):• The Esri shapefile, or simply a shapefile, is a

popular geospatial vector data format forgeographic information system software. It isdeveloped and regulated by Esri as a (mostly)open specification for data interoperabilityamong Esri and other GIS software products.Shapefiles spatially describe vector features:points, lines, and polygons, representing, forexample, water wells, rivers, and lakes. Each itemusually has attributes that describe it, such asname or temperature.

• While the term "shapefile" is quite common, a"shapefile" is actually a set of several files. Threeindividual files are mandatory to store the coredata that comprise a shapefile:

• I used shapefilereader.jar. By using this jar we canread any shapefile.

Data File Format

• .shp File Structure (Header)Bytes Type Endianess Usage

0–3 int32 big File code (always hex value 0x0000270a)

4–23 int32 big Unused; five uint32

24–27 int32 big File length (in 16-bit words, including the header)

28–31 int32 little Version

32–35 int32 little Shape type (see reference below)

36–67 double little

Minimum Bounding rectangle (MBR) of all shapes

contained within the shapefile; four doubles in the

following order: min X, min Y, max X, max Y

68–83 double littleRange of Z; two doubles in the following order: min Z, max

Z

84–99 double littleRange of M; two doubles in the following order: min M,

max M

Data File Format

Data File Format• .shp file structure

Value Shape type Fields

0 Null shape None

1 Point X, Y

3 Polyline MBR, Number of parts, Number of points, Parts, Points

5 Polygon MBR, Number of parts, Number of points, Parts, Points

8 MultiPoint MBR, Number of points, Points

11 PointZ X, Y, Z, M

13 PolylineZMandatory: MBR, Number of parts, Number of points, Parts, Points, Z range, Z array

Optional: M range, M array

15 PolygonZMandatory: MBR, Number of parts, Number of points, Parts, Points, Z range, Z array


18 MultiPointZMandatory: MBR, Number of points, Points, Z range, Z array


21 PointM X, Y, M

23 PolylineMMandatory: MBR, Number of parts, Number of points, Parts, Points


25 PolygonMMandatory: MBR, Number of parts, Number of points, Parts, Points


28 MultiPointMMandatory: MBR, Number of points, Points

Optional Fields: M range, M array

31 MultiPatch

Mandatory: MBR, Number of parts, Number of points, Parts, Part types, Points, Z

range, Z array


Data File Format

Data File Format

.nc (NetCDF(Network Common Data Format))A platform independent format for representing multi-dimensional array-orientated scientific data.

• Self Describing - a netCDF file includes information about the data it contains.

• Direct Access - a small subset of a large dataset may be accessed efficiently, without first reading through all the preceding data.

• Sharable - one writer and multiple readers may simultaneously access the same netCDF file.

• NetCDF is new to the GIS community but widely used by scientific communities for around many years

Data File Format

Data File Format

• Multidimentional Data Representation :

XY

TZ

XY

T

Data File Format

Data File Format• Digital Value processed from a satellite data

141 241 341

131 231 331

121 221 321

111 211 311

441

431

421

411

142 242 342

132 232 332

122 222 322

112 212 312

442

432

422

412

143 243 343

133 233 333

123 223 323

113 213 313

443

433

423

413

Y

X

Time

141 241 341

131 231 331

121 221 321

111 211 311

441

431

421

411

142 242 342

132 232 332

122 222 322

112 212 312

442

432

422

412

143 243 343

133 233 333

123 223 323

113 213 313

443

433

423

413

Y

X

Time

Altitude

Data File Format

System Design

Goals of Implementation• Extracting the data values of the given parameters from the

data file.• Scanning the data and perform statistical operation.• Development of algorithm for Climate data analysis• Develop Linear Matrix Interpolation Technique.• Read and Plot shapefile(.shp) ESRI file formate.• NetCDF-format and Multi-source climate data analysis.• Analysis and visualisation of satellite data to study climate

structure• Give the better visualization to GUI it gives give more

accurate result within time.

Functional Requirement

Input RequirementsTo collect the data which may be in netCDF(.nc) and shapefiles(.shp)

• To obtain the threshold value from the user.• Save the extracted data values in the text format and Image formate.Processing Requirements

To read the information from the input file.• To calculate and store statistical result and perform charting in easy way.• Read the shapefile and plot shapefile into GUI for gives country

boundaries.• To generate the pictorial representation of the output.Output Requirements• The user can generate an image from the data values.

26

System Design

Use Case Diagram:System Design

Sequence Diagram for Statistic AnalysisSystem Design

Sequence Diagram for generate Interpolated ImageSystem Design

Package DiagramSystem Design

Class Diagram System Design

Result and DiscussionStarting Screen :

Select Year from Year List for Data Show in Table

Result and Discussion

ncDumpInformation of NetCDF files


Mining the Data from NetCDF FileResult and Discussion

Prepare Charts Based on DataResult and Discussion

Draw Shapefile in PanelResult and Discussion

How To make Shapes in Image File• I relate geo-cordinate system with screen pixel system.• In geo-cordinate system have 180(-90 to 90) Latitude and 360 (-180

to 179)Longitude and I take panel its width=730 and Height=628.Ratio(x) = 730/360 = 2.028Ratio(y) = 628/180 = 3.49

• Choose minimum ratio, if I choose max ratio than some shapes of shapefile will not show in panel.

• So if my co-ordinate is (-180, -90) = Panel Pixel(0, 0)• (int((-180 + 180 ) * 2.028)) = 0 and (int((-90 + 90) * 2.028) )= 0 is

minimum lat and lon point(0, 0).• If my co-ordinate is (40, -20) = (int((40 + 180) * 2.028)) = 446

(int((-20 + 90) * 2.028)) = 141 so pixel co-ordinate is (446, 141).


Perform InterpolationResult and Discussion

Interpolation Method

• 4-Point Matrix Interpolation :


Interpolation MethodCalculate Estimated Temperature


Interpolation Algorithm

• Calculate pixel respect to geo-cordinate.

• Set into as a Matrix.


Zoom Specific AreaResult and Discussion

Advantage

• The big advantage is that Scientist use a software GrADS(Grid Analysisand Display Design). GrADS has a scripting language. But my softwaregives GUI effects. Easy to analysis NetCDF data and perform Systemsoperation. No need to go write console level scripting or generate a Batchfile.

• The software is totally platform independent.• The whole package is easily installable in any type Operating System

and its supportive JAR(Java Application Runtime).• We can perform statistics result easily.• It’s create high resolution picture when I perform interpolation.• It can be widely used by scientist and users from various sectors like

agriculture, health etc to adopt the climate information for the crop, disease analysis and Cyclone Detection.

• It can also be used to train young researchers of various fields regarding multi-source and multi-format weather and climate data analysis and weather informatics etc.


Conclusion

The project “DEVELOPMENT OF VISULIZATION SOFTWAREFOR WEATHER AND CLIMATE ANALYSIS” aims to analyzedata in different formats and to perform quality control ifthe data is not within certain standard ranges. The softwareis capable of analyzing NetCDF data format.Almost all the objectives of the project are fulfilled.

• 1. Displaying the header information of the input file.• 2. Extracting the data values of the given variable in the

input file.• 3. Scanning the data.• 4. Generating a new interpolated image file respect of

extracted NetCDF data.

Conclusion & Future Enhancement

Future Enhancement

Some enhancements can be made in future as follows:

• An enhancement can be done so that the user can choose in which format he/she wants the output file.

• An improvement can be made so that other data formats, such as HDF, Binary etc., can also be given as input files.

• More and more data can be studied to understand the climate change and its impact at the regional scales.

• A GUI can be developed for the easy understanding of the weather and climate informatics.

Conclusion & Future Enhancement

Thanks

weather and climate visualization software

Data & Analytics