m u n -march 10, 2005 - phil bording1 computer engineering of wave machines for seismic modeling and...

M U N -March 10, 2005 - Phil Bording

1

Computer Engineering of Wave Machines for

Seismic Modeling and Seismic Migration

R. Phillip Bording

March 10, 2005

0 Max Address

Husky Energy Chair in Oil and Gas ResearchMemorial University of Newfoundland


2

Cache Memory- Three LevelsArchitecture

Address Pointer

MemoryMulti-

Gigabytes

Large and Slow160 X

16XL3 CacheMemory

Cache ControlLogic

L2 CacheMemory

L1 CacheMemory

2X 8X

16 Megabytes128 Kilobytes32 Kilobytes

2 Gigahertz Clock

Featuring Really Non-Deterministic Execution


3

Problem Solving – 3DExample of Array

Addressing

Address = (k-1)*Lx*Ly +(j-1)*Lx+(i-1) + base

Grid Points

i,j,ki-1,j,k i+1,j,k


4

Cache Memory Access Streams

1D Streams – 100%

1D +/-1 100%

2D +/-1 100%

2D +/-N 80%

2D +/-1 +/-N 26%


5

Cache Memory Access Streams

3D +/-1 100%

3D +/-N 80%

3D +/-N*N 28%

3D ALL 7%


6

IEEE 754 Floating Point


7



8



9

SeismicModeling and the

Inverse Problem


10


11

12 Streamers x 5.1 Kilometers Long Data collected for 70 continuous daysOver 2300 Square Km.


12

3D Seismic Modeling1. Large Scale 3D ~200+ Wave Lengths2. Acoustic and Elastic Wave Equations3. In-Homogeneous Earth has widely varying parameters. 4. Complexity limits use of 3D elastic modeling5. Problem Scale

• Nx=Ny=Nz ~ 1000• Ntime ~ 10,000• Work per Grid Point ~ 100• Number of Seismic Shots per Survey ~ 100,000• Single Survey Simulation is 10^20 Operations.


13

The Babbage Difference Engine, circa 1853


14

Wave Equation Difference Engine (WEDE)

for Seismic Modeling

Four Processors

Acoustic Wave Equation

My PhD thesis project at the

University of Tulsa


15

Wave Equation Difference Engine

Finite DifferencesElastic or Acoustic Wave EquationsRegular GridsSponge/One-Way Wave Equation

Boundary ConditionsAny Source/Receiver GeometryExplicit 4th order in Time & 8th order in Space?


16


No Cache MemoryDeterministic ExecutionNot a MIMD or SIMD or Data Flow

Data movement and control matches the algorithmEach grid point has control wordThree levels of parallelism, ( Amount of Parallelism)

Instruction trees, ~ 10-20Multiple Instructions with selection, ~2-3Multiple Grid points, ~Hundreds of Thousands


17

Acoustic, Constant Density

Density is so constant it does not appear in the equation.

2 2 2 2

2 2 2 2 2

1

, ,src t

C

C is the P Wave Velocity.The source energy is in src.Psi is the wave field.


18


Machine Performance

100 operations in pipeline

1,000,000 grid point processors

100 Megahertz Clock

10^16 Operations per second


19


20


21

Application Specific Parallel Computing

•Choose carefully an application which is BIG.•Find an algorithm which is suitable.

Good data locality.Regular structure in data movementHigh memory data transfers

•Map the algorithm into hardware


22

Application Specific Parallel Computing

What it is not!

•Not suitable for just any algorithm•Not general purpose, we will have an efficient but

specific memory subsystem. • Does not match the alphabet soup, SIMD,

MIMD,NUMA, etc


23

What do ASP machines need??

VLSI Design Team, fabless and good?

Clever Architect for the problem.

A very good memory design!


24

What do ASP machines do away with??

Language CompilersOutdated junk in the processor design, x86!Cache memories!Non-deterministic execution!


25

Multiple Bank Memory Systems

Starting + 1 +2 +3Address +N +2N +3NMod 4

Memory Banks

Bank 0 1 2 3

As many as are needed!!!!


26

Pipelined Instruction Trees

Each higher level offers parallel operations

Pipeline assumes all registers are loaded every cycle

Hardwired?? Actually today the instruction trees could be re-configurable using re-programmable cells!!!

r = a+b-x*y


27

Pipelined Instruction Treesa b d y

-*

-

a b x y

*+

Multiple Trees offer the second level of Parallelism

+


28

Three Levels of Parallelism

1. Instruction Trees, Multiple Levels

2. Multiple Results

3. Multiple Grid Point Processors


29

Wave Machine


30

Imaging Machine


31

Wave Equationa) 8th or 10th Order in spaceb) 4th Order in time, tricky but possiblec) Sponge Boundary Conditions,

slowly varying weights along sidesd) Nominal flat topography, new schemes

are building in topographye) Any seismic source location,

any geophone location


32

Elastic Wave Equationa) Grid point work is about 100 operationsb) About 20,000 time steps per shot c) 200 Wavelengths gives about 160,000 geophone locationsd) Traces have 4096 samples, 2 milliseconds, could be 1 ms.


33

Elastic Wave EquationShots are placed at twice the receiver

spacing

Number of shots equals 40,000

Model Frequency is velocity dependent,assume something on the order of 60 hertz.


34

Economics

Up Front Fixed Cost, $5 to $ 10 Million

Each ASP Chip is $5 to 10

A Petaflop for $5 or $10 Million


35

Economics

Seismic Shot takes 0.1 seconds

5 Year life is 50,000 Models

A realistic 3D elastic seismic model would cost $200


36

Comparison

10 Clusters ~ $10 Million10 models per year

One Waves in Linear Motion Analyzer (WILMA) ~$10 Million

10,000 models per year


37

Comparison

Waves in Linear Motion Analyzer

1000X faster

For the same money!.


38

Summary

1000 Megawatts is a good sized power station

Good memory design is worth the money!

Removing the obstacles to efficient computing gives sustainable performance


39

Summary

Slower is better.Less power is better.High Efficiency is better.


40

Conclusions

Deterministic Computing is important for performance………Application Specific Computing is a good fit for the wave equation…..And very cost effective………..


41

Thanks

SEG – Continuing Education

Memorial University of Newfoundland


42

Hamming

“The purpose of computing is insight, not numbers”

m u n -march 10, 2005 - phil bording1 computer engineering of wave machines for seismic modeling and...

Documents