david a. yuen minnesota supercomputing institute,university of minnesota, minnesota jessica schmidt...

DAVID A. YUEN Minnesota Supercomputing Institute,University of Minnesota, Minnesota

JESSICA SCHMIDTSaint Scholastica College, Duluth, Minnesota

ERIK O.D. SEVREMinnesota Supercomputing Institute University of Minnesota, Minnesota

NAN ZHANGMedical School, University of Minnesota Minnesota

GRADY B. WRIGHTDept. of Mathematics , Boise State University, Boise, Idaho

Tsunami Modeling with Graphics Processing Unit (GPU) and Radial

Basis Functions (RDF)

JESSICA SCHMIDTSaint Scholastica College, Duluth, Minnesota

CECIL PIRETInstitute of Applied Mathematics for Geosciences, National Center of Atmospheric Research, Boulder, Colorado

SPRING LIUMinnesota Supercomputing Institute University of Minnesota, Minnesota

NATASHA FLYERInstitute of Applied Mathematics for Geosciences, National Center for Atmospheric Research, Boulder, Colorado

Outline

Introduction to Tsunamis and Tsunami Modeling

Visualization of tsunamis with Amira visualization package

Virtues of  Graphics Accelerated Board (GPU)Applications of GPU to Shallow-Water

equationsRadial Basis Functions (RBF) Swirling FlowsApplications of GPU to RBF equationsConcluding Remarks

Background

What is a Tsunami?(soo-NAH-

mee)

Tsunami Definition & Causes

Most of the waves present on the ocean’s surface are wind-generated waves.

Size and type of wind-generated waves are controlled by: Wind velocity, Wind duration, Fetch, and Original state of sea surface.

Wave in the OceanWave types

Tsunamis consist of a series of long-period waves characterized by very long wave length (up to 100 km) and high speed (up to 760 km/hr) in the deep ocean.

Because of their large wave length, tsunamis are shallow-water to intermediate-water waves as they travel across the ocean basin.

They only become DANGEROUS, when reaching coastal areas where wave height can reach 10 m.

Tsunamis originate from earthquakes, volcanic explosions, or submarine landslides.

7-5

Tsunami

Tsunami Source (1)

Tsunami Source (2)

Tsunami Source (3)

Tsunami Sources in the world(2180 events from 1628BC to 2005)

Numerical Tsunami

Modeling

Backgroun

d

Seismic Tsunami

Modelling

Backgroun

d Killer Tsunamis in Historical Times

General Tsunami Modelling

Displacement Field (initial Condition)Propagation (Linear and Nonlinear

model)Run-up

1 Physical Analysis

2 Numerical Simulation

3 Visualization

4 Result Analysis and Digestion

Seismic Tsunami Modelling

Navier-Stokes Equations SystemBoussinesq EquationsShallow Water Equations

1 Analyze the phenomenon

2 Choose Coordinates

3 Choose the equations

5 Boundary and initial conditions

4 solution of grid

7Analysis results

Etopo1, Etopo2, Strm30,

or local bathymetry data

Satellite data or tidal data

(Local and Far-field)

6 Visualization

The initial wave( From

earthquake)

Seismic Tsunami

Modelling

Generation, Propagation, and Run-up of Tsunamis Generation, Propagation, and Run-up of Tsunamis

R u n - u p / d o w nP r o p a g a t i o nG e n e r a t i o n

dispersion effect nonlinear effect

Existing Tsunami Models

Introduction of Amira

Amira is a powerful, multifaceted software platform for visualizing, manipulating, and understanding scientific data coming from a all types of sources and modalities.

Multi purpose - One tool for interdisciplinary work Flexible - Option packages to configure amira to your needs Efficient - Exploits latest graphics cards and processors Easy to use - Intuitive user interface and great documentation Cost effective - Multiple options and flexible license models Handling large data - Very large data sets are easily accessible with specific readers Extensible - C++ coding wizard for technical extension and customization Support - Customer direct support with high level of interaction Innovative - Technology always up dated to the latest innovation

Data Visualization __ Amira

Load Topography Background

Movie Maker

Highlight of Visualization with Amira 3

This figure shows the height field with a scaled height.

Wave Propagation Visualization of Tsunami Modeling- Eastern China Sea

Wave Propagation Visualization of Tsunami Modeling- Solomon Islands

Wave Propagation Comparison of Linear and Nonlinear Modeling

Different Bathymetry Resolution Comparison of Nonlinear Modeling on Shallow Part of the Ocean Part

Grids: 1201*1201 601*601

Conclusion (1)   Visualization promotes a rapid

understanding of the waves'  paths from initial stages ; influences from the initial surroundings

(2)   Visualization Allows us to understand better the subsequent events when the waves are interacting with the coastline and off-shore islands

(3)   Visualization  Helps to teach people about wave propagation for local and regional scenarios

Linear and Nonlinear Model in Yellow Sea Area

TSUNAMI SIMULATION WITH GPU PROGRAMMING

JESSICA SCHMIDT from computer science and mathematics

UNDERGRADUATE SUMMER INTERN

Jessica SchmidtUndergraduate summer

intern

Tsunami Simulation with GPU Programming

Overview

Why we do this project?GPU with CUDA programmingTsunami Simulation with CUDA RBF ( RADIAL BASIS FUNCTIONS ) Summary

What does the future hold?

Viable set-up for real-time tsunami visualization

Earthquake

Tsunami

Real Tsunami Visualization

(Interface Window)

Seismology

Bathymetric Data

Tsunami Simulation with GPU

Programming

By Erik

By Jessica

Tsunami Warning

GPU

Graphics Processing UnitMuch faster than CPU nowGetting more expensive, can easily nowOutstrip the cost of a laptop itselfTakes the load off of the CPU

Computes many complex math problemsFaster graphics processing speedIncreased detailed and complexity without compromising performance

CUDA

Benefits DrawbacksTakes load off

CPUEasy to learn and

implement

Difficult to find video card , MAC is cooler for this .

Compute Unified Device Architecture

Developed by NVIDIABased on C

GPU Specs.GPU GeForce

8600M GTGeForce 8800 Ultra

Core clock (MHz) 540 612

Shader clock (MHz)

1190 1500

Memory clock (MHz)

700 1080

Memory Amount (MB)

256 768

Memory Interface 128-bit 384-bit

Memory bandwidth (GB/s)

22.4 103.7

Texture Fill Rate (billion/sec)

8.64 39.2

There are other GPUs that work with CUDA as well. - NVIDIA GeForce 8000 and above- NVIDIA Quadro, DELUXE MODEL- NVIDIA Tesla

Jessica’s Job This Summer

Covert linear tsunami codesSpring Liu ----second Finite Difference Method

Cecile Piret ---- Radial Basis Function (RBF)

Implement CUDA for Spring’s and Cecile’s linear codes, then see if there is speedup

2-D Shallow Water Equations

Linear Non-Linear

0

0

0

=∂∂

+∂∂

=∂∂

+∂∂

=∂∂

+∂∂

+∂∂

yz

gDtN

xz

gDt

MyN

xM

tz

0)()(

0)()(

0

2

2

=+∂∂

+∂∂

+∂∂

+∂∂

=+∂∂

+∂∂

+∂∂

+∂∂

=∂∂

+∂∂

+∂∂

ρτρτ

y

x

yz

gDDN

yDMN

xtM

xz

gDD

MNyD

Mxt

M

yN

xM

tz

M, N = mass fluxes in horizontal planez = wave heightt = timeh = ocean water depth

D = total water depth, D = z + hρ = densityτx, τy = shear stress along x and y axis

Bathymetric Data: Etopo1

Parameters of Rupture: From HARVARD Database , Miyaki Ishii

Visualization: Amira

Radial Basis Functions (RBF) Method

An Introduction

The RBF method

• 70s Rolland Hardy introduces a new method for scattered data interpolation for geological data, the MQ method, so named for its use as basis of the multi-quadric function. First published in JGR

• 70s-80s The method is generalized to more radial functions. It is renamed the “Radial Basis Functions (or RBF) method”.

• 90s Ed Kansa from UC Davis uses the RBF method to solve partial differential equations.

• Given scattered data

• Define the RBF

interpolant

The RBF method

• Given scattered data

• Define the RBF

interpolant

The RBF method

• Given scattered data

• Define the RBF

interpolant

The RBF method

• Find by solving the system

The RBF methodCoding the RBF method is fast and easy

RBF part of the code

+• Interpolation on scattered data. No grid necessary.• Very easy implementation in N-dimensions.• The basis functions are not orthogonal with each

other, but we are guaranteed a non-singular system for most types of RBFs.

• Spectral accuracy for infinitely smooth radial functions

- High complexity. No fast algorithm.

The RBF method

Radial Basis Functions (RBF)--- CecileInterpolating data takes the form:

Use RBFs to model 2-D linear waves

Cecile Piret wrote simulations using Matlab

Convert to GPU using Jacket – developed by Accelereyes

The comparison of GPU and CPU ------- Linear Tsunami Codes

Grid Size

CPU (laptop) GPU (laptop) Speedup times

30x30 Approx. 315 seconds

Approx. 105 seconds

3 times

Cecile’s linear tsunami code (400 time steps)

Grid Size

CPU (Lilli) GPU (laptop)

Speedup times

601x601

Approx. 240 minutes

Approx. 30 minutes

8 times

Spring’s linear tsunami code (21600 time steps)

Lilli – an opteron-based system with 4 CPUsGPU – nVIDIA 8600M GT graphics cardLaptop – standard MacBook Pro

Results for Tsunami Simulation

beginning of simulation middle of simulation

Simulation Movie

Comparison

Summary

With the comparison of the computing times of tsunami equations that be solved by different numerical methods (Finite Difference Method and RBF) and different hardware surroundings (GPU and CPU), we would provide a ideal computing and visualization method for tsunami simulation, that allow for hazard preparation and timely warning for lands in the masses in the path of tsunami wave.

GPU really speeds up computing time, that is at least 3 times that of CPU for our tsunami code.

What does the future hold?

Complete GPU programming for Nonlinear Shallow Water Code with CUDA ---- Jessica

Development of Visualization Interface Erik Sevre

References

Liu, Y. Numerical tsunami modeling[PowerPoint]. Minneapolis, Minnesota: University of Minnesota.

(2008, June 7). NVIDIA CUDA compute unified device architecture. Retrieved July 29, 2008, from NVIDIA: http://developer.download.nvidia.com/compute/cuda/2.0-Beta2/docs/Programming_Guide_2.0beta2.pdf

Piret, C. (2007). Analytical and numerical advances in radial basis functions, (Doctoral dissertation, University of Colorado at Boulder, 2007). Retrieved from http://amath.colorado.edu/student/piret/thesis.pdf

Sevre, E., Yuen, D. A., & Liu, Y. (2008). Visualization of tsunami waves with Amira package.

Examples of Commonly Used Radial Basis Functions

Comparison of the 1-D Chebychev  Pseudo-Spectral Basis and the Gaussian RBF Basis and Influence of the Shape

Parameter Epsilon

Comparison of the Swirling Flow for Different Initial Conditions

Sample MATLAB Code  for Solid-Body Rotation

ep = 6; % Value of epsilon to usedalpha = pi/2; % Angle of rotation measured

from the equatora = 6.37122e6; % Mean radius of the earth

(meters)u0 = 2*pi*a/12; % Speed of rotation (m/day)-

one full revolution in 12 daysR = a/3; % Width of bell%%% Load Nodes:

http://web.maths.unsw.edu.au/~rsw/Sphere/Energy/index.html%%%

load(‘me1849.dat’); x = me1849(:,1); y = me1849(:,2); z = me1849(:,3);

%%% Compute r2 = (x_j x_k)2+ (y_j y_k)2+(z_j z_k)2 %%%

nodes = [x,y,z];rd2 = zeros(length(nodes),length(nodes));for j = 1:3xd1 = nodes(:,j); xd1 =

xd1(:,ones(length(xd1), 1));xd2 = xd10;rd2 = rd2 + (xd1 xd2).2;end%%% Set-up 2D surface grids in (theta,phi)

for computing B (eqn.(11)) %%%theta = atan2(z,sqrt(x.2+y.2)); phi =

atan2(y,x); % phi = lambda in paper

tn = theta; tn = tn(:,ones(length(xd1), 1)); tc = tn’;

pn = phi; pn = pn(:,ones(length(phi), 1)); pc = pn’;

%%% Compute differentiation matrix D %%%%

B = 2*(cos(alpha).*cos(tn).*cos(tc).*sin(pn-pc) + sin(alpha).*(cos(tn).*cos(pn).*sin(tc) - cos(tc).*cos(pc).*sin(tn)));

B = (u0/a)*B.*(-ep2*exp(-ep2.*rd2));A = exp(-ep2.*rd2);D = B/A;%%% Initial Condition Cosine Bell %%%r = a*acos(cos(theta).*cos(phi)); % initially

located at equator, (0,0)h = 1000/2*(1+cos(pi*r/R)); % height of bell

is 1000 mh(r >= R)=0;%%% Time-Stepping - 4th Order RK %%%dt = 12/288*5/6; % Time-Step for 12 days

revolutionfor nt = 2:(1*288*6/5)d1 = dt*D*h;d2 = dt*D*(h + 0.5*d1);d3 = dt*D*(h + 0.5*d2);d4 = dt*D*(h + d3);end

Sample MATLAB Code  for Solid-Body Rotation with GPU Implementation ( NO BIGGIE )

ep = 6; % Value of epsilon to usedalpha = pi/2; % Angle of rotation measured

from the equatora = 6.37122e6; % Mean radius of the earth

(meters)u0 = 2*pi*a/12; % Speed of rotation (m/day)-

one full revolution in 12 daysR = a/3; % Width of bell%%% Load Nodes:

http://web.maths.unsw.edu.au/~rsw/Sphere/Energy/index.html%%%

load(‘me1849.dat’); x = me1849(:,1); y = me1849(:,2); z = me1849(:,3);

%%% Compute r2 = (x_j x_k)2+ (y_j y_k)2+(z_j z_k)2 %%%

nodes = [x,y,z];rd2 = zeros(length(nodes),length(nodes));for j = 1:3xd1 = nodes(:,j); xd1 =

xd1(:,ones(length(xd1), 1));xd2 = xd10;rd2 = rd2 + (xd1 xd2).2;end%%% Set-up 2D surface grids in (theta,phi)

for computing B (eqn.(11)) %%%theta = atan2(z,sqrt(x.2+y.2)); phi =

atan2(y,x); % phi = lambda in paper

tn = theta; tn = tn(:,ones(length(xd1), 1)); tc = tn’;

pn = phi; pn = pn(:,ones(length(phi), 1)); pc = pn’;

%%% Compute differentiation matrix D %%%%

B = 2*(cos(alpha).*cos(tn).*cos(tc).*sin(pn-pc) + sin(alpha).*(cos(tn).*cos(pn).*sin(tc) - cos(tc).*cos(pc).*sin(tn)));

B = (u0/a)*B.*(-ep2*exp(-ep2.*rd2));A = exp(-ep2.*rd2);D = B/A;%%% Initial Condition Cosine Bell %%%r = a*acos(cos(theta).*cos(phi)); % initially

located at equator, (0,0)h = 1000/2*(1+cos(pi*r/R)); % height of bell is

1000 mh(r >= R)=0;D=gsingle(D); % puts matrix D on the GPUh=gsingle(h); % puts h on the GPU%%% Time-Stepping - 4th Order RK %%%dt = 12/288*5/6; % Time-Step for 12 days

revolutionfor nt = 2:(1*288*6/5)d1 = dt*D*h;d2 = dt*D*(h + 0.5*d1);d3 = dt*D*(h + 0.5*d2);d4 = dt*D*(h + d3);end

Results for Solid Body Rotation

CPU time GPU time Speed up

119 seconds 23 seconds 5.2 times

Notes:The CPU used was an Intel Duo Core ProcessorThe GPU used was an NVIDIA GeForce 8600M GT in a MacBook ProThe times calculated were the RK4 loop, as that was the part on the GPU.GPU implementation was facilitated by the Jacket software package produced by AccelerEyes

Several Thousand RBF Points Laid Out On Spherical Shell

for 3-D thermal convection

Concluding Remarks and Perspectives

GPU can make a difference in speeding up for both single processor on your laptop or clusters of GPU's , they are much cheaper , $10,000 can mean 50 times your present compute power can be put in your office, whither Brutus et TU !!

RBF's is a new method which can have a future because of its algorithmic simplicity. Easy to learn and program. It is still in nascent stage, like finite-elements were in the late 1960’s,

when Paul Tackley was a babe.

Combining GPU with RBF, we can now speed up the nonlinear shallow-water equation by a factor close to 100, for the same physical elapsed time  so a calculation which took 30 minutes can be done in 20 seconds( on single processor armed with GPU). This makes real-time tsunami warning a definite possibility