gis and g&g database synchronization · process. we have successfully developed a tool to...
TRANSCRIPT
©2005 ChevronTexaco Corporation - All Rights Reserved
GIS and G&G Database Synchronization
Ananta Dwi Bodhitama*Mursyid Nento*David Stodola+
* Data Management, ChevronTexaco Indonesia Business Unit+ GIS/Remote Sensing, ChevronTexaco Energy Technology Company
2005 ESRI Petroleum User GroupHouston TX, 7-9 March 2005
©2005 ChevronTexaco Corporation - All Rights Reserved2
Abstract
Geological and Geophysical (G&G) master database is a corporate wide master database that stores up-to-date and validated G&G data coming from project databases located in different sites. ChevronTexaco Indonesia Business Unit uses Finder as its G&G master database and ArcSDE as its GIS database.
Finder keeps important subsurface information and we want to have this information available in GIS applications. Having both subsurface and the already existing surface information available in one place is very useful in decision making process.
We have successfully developed a tool to synchronize data from Finder to ArcSDE. The synchronization process takes account important subsurface information: well location, well path, seismic navigation, seismic line and seismic fault. The synchronization process runs very quickly and is set to run periodically on a daily basis to ensure up-to-date consistency with new entry in Finder.
*ArcSDE is a trademark of ESRI, Finder is a trademark of Schlumberger
©2005 ChevronTexaco Corporation - All Rights Reserved3
Introduction
Geographic Information System (GIS) database provides data for various GIS applications.
Geological and Geophysical (G&G) master database is a corporate wide master database that stores up-to-date and validated G&G data.
Indonesia Business Unit (IBU) uses Finder as its G&G master database and Arc Spatial Data Engine (ArcSDE) as its GIS database.
©2005 ChevronTexaco Corporation - All Rights Reserved4
Indonesia Business Unit
Where are we?
IBU,Sumatra
Australia
Pacific Ocean
Atlantic Ocean
©2005 ChevronTexaco Corporation - All Rights Reserved5
IBU Concession Blocks
Where are we? Geographic Regions
Kisaran (New Block)
Balam
Bangko
Bekasap
Duri
Libo
Minas
Petahapan
Petani
Kuantan
25 km
©2005 ChevronTexaco Corporation - All Rights Reserved6
Problem?
ArcSDE: Arc Spatial Data Engine that acts as master spatial database and serves GIS-based applications.
Finder: IBU master database that stores the most recent and up-to-date G&G data.
Need to have all spatial data (surface & subsurface) in ArcSDE better information for decision making.
Gap: No efficient way to store and keep up-to-date G&G data in ArcSDE.
©2005 ChevronTexaco Corporation - All Rights Reserved7
IBU Database Flow Diagram
FINDER
ArcSDE
GISRumbai
OpenWorks
•F93RBIP•MINAS•E_T•RUMBAI
•OW3RBIP•REGI_PRO_RBI•CSBS_PRO_RBI
FINDER
OpenWorks
•F93DRIP•DURI•BEKASAP
•OW3DRIP•DURI_PRO•CSBN_PRO_RBI
OpenWorks
•OW3MNSP•MINA_PRO_MNS
Minas
Duri
SLN HO
SDEsynchInterpretation DataMaster Data
©2005 ChevronTexaco Corporation - All Rights Reserved8
SDEsynch
Custom program built in C. Based on program created by David Stodola of GIS/Remote Sensing Team of ETC. Rewritten to meet specific IBU needs.
Solution for dealing the mentioned gap by providing synchronization of G&G data and store them in the ArcSDE database.
Part of Spatial Data Consolidation Project under UTx Master Data Consolidation.
Synchronized Data: Well Top & Bottom Location, Well Path, Seismic Navigation, Seismic Line, Seismic Fault, Oil & Gas Field Boundary.
©2005 ChevronTexaco Corporation - All Rights Reserved9
8 Types of Data Transferred
FINDER
•Spatial Data•Well Top Location•Well Bottom Location•Well Path•Seismic Navigation•Seismic Line•Seismic Fault•Oil Field Boundary•Gas Field Boundary
•Attribute Data•Unique well identifier (UWI)•Well name & Number•Surface elevation •Well status
ArcSDE
GIS
• Subsurface Data•Well Top Location•Well Bottom Location•Well Path•Seismic Navigation•Seismic Line•Seismic Fault•Oil Field Boundary•Gas Field Boundary
•Surface Data•Land Matter•HCT•PGT•Soil•Facilities
ArcMAP
SynchronizationUsing SDEsynch
©2005 ChevronTexaco Corporation - All Rights Reserved10
Problem: multipoint check?
16001100002
15001000001
………
YaXaWell
Finder
16001100002
15001000001
………
YbXbWell
GIS
?
• Simply compare if Xa=Xb and Ya=Yb
…
1102
1001
Xa2
…
1602
1501
Ya2
…
-
…
…
…
1600
1500
Ya1
…
-
1050
Xan
-1100002
15501000001
………
YanXa1Survey
Finder
• Compare Xa1=Xb1 and Xa2=Xb2 and … Xan=Xbn• Compare Ya1=Yb1 and Ya2=Yb2 and … Yan=Ybn
??
©2005 ChevronTexaco Corporation - All Rights Reserved11
Checksum
Minimize comparison processes during synchronization by concatenating sum of all digits as most significant digits of the sum itself.
Can accurately detect data changes without checking every point.
Helpful in multipoint data like well path where the number of point needed to construct line or polygon is high.
©2005 ChevronTexaco Corporation - All Rights Reserved12
Checksum
Formula:(X + Y)1 + (X + Y)2 … + (X + Y)n = SS1 + S2 … + Sm = CChecksum = C || S
Where X & Y = X/Y pair location of a pointn = number of points needed to construct line or polygonS = sum of all X & Y m = number of digits in SC = sum of all digits of SChecksum = concatenation of C and S
©2005 ChevronTexaco Corporation - All Rights Reserved13
Checksum
Example:Checksum for well path with 5 directional survey points{(0,0), (10,10), (20,20), (30,30), (40,40)}
Where S = (0+0) + (10+10) + (20+20) + (30+30) + (40+40) S = 200C = 2 + 0 + 0 = 2Checksum = C || S
= 2 || 200= 2200
Only 2 values compared instead of many.
©2005 ChevronTexaco Corporation - All Rights Reserved14
Problem: binary storage?
16001100002
15001000001
………
YXWell
Finder
OK002
OK001
……
ShapeWell
GIS
?
• Use ArcSDE API to convert Finder ‘text’ data to shape
bbb.dat002
aaa.dat001
……
Binary FilesOil Field
Finder
• Need to have conversion from binary to text ??
More aaa.dat==================================….brâèbbe'A@0ò@šÑ'A+ø@`Øb@þö@@ë`ús÷@ µ÷@@ë÷@ÀÞ ÷@Ú e÷@qþö@ce÷@æö@`ošÑ'A`0Öõ@ +Ñ'AÀÙõ@`-Ð'AšÝõ@`9Ð'A
©2005 ChevronTexaco Corporation - All Rights Reserved15
Use Nlbrowser
Finder utility to read Nlist files, proprietary of Schlumberger, used to keep most array-type data in Finder like logs and seismic data as well.
There is no Finder API available cannot integrate Nlist related operations into C routine code.
Executable program with parameters where in the routine parameters are taken from Oracle database records.
Invoke by using execve() C function.
©2005 ChevronTexaco Corporation - All Rights Reserved16
SDEsynch Algorithm
Open connection to ArcSDE
Process input file
Loop for every input files record Create database linkCreate copy table from Finder & checksumDelete orphaned recordsDelete updated recordsInsert new records and create shapeDrop database link
Close ArcSDE connection
©2005 ChevronTexaco Corporation - All Rights Reserved17
SDEsynch Process/Data Flow
FINDERInput FileStart SDEsynch
GIS
1 2
3Delete Orphaned Data
Delete Updated Data
Insert Data
Temporary File
Nlbrowser
45
6
6b6a
6c
7
8
End SDEsynch
9
©2005 ChevronTexaco Corporation - All Rights Reserved18
SDEsynch Process/Data Flow
1. Read input file containing project & instance names.2. Create database link to Finder.3. Copy Finder tables to GIS database & checksum.4. Comparing Finder tables and GIS tables by first
deleting orphaned data in GIS database.5. Deleting GIS data that are different from Finder data
(different means need to be updated).6. Insert new data to GIS. Only new data are inserted.
6a. In case data is stored in Nlist (ex: fault)6b. Launch Nlbrowser and keep extracted data in a
temporary file then (6c.) Back to insert routine7. Use SDE API to insert data8. Loop for every input file entry until eof (9.)
©2005 ChevronTexaco Corporation - All Rights Reserved19
ArcSDE Server Setup
Update tnsnames.ora in ArcSDEAdd SDE instance name in /etc/services Install ArcSDE / ArcMap API & librariesInstall C compiler & libraries
Create 8 feature class layers using ArcCatalogWELL_TOP_FINDER (point)WELL_BASE_FINDER (point)WELL_PATH_FINDER (line)SEIS_NAV_FINDER (polygon)SEIS_LINE_FINDER (line)SEIS_FAULT_FINDER (line)FIELD_OIL_FINDER (polygon)FIELD_GAS_FINDER (polygon)
©2005 ChevronTexaco Corporation - All Rights Reserved20
Synchronization Setup
Update input file with projects to synchronize.
Create Oracle user-id that can access listed project in each instance.
Setup script for mail notifications.
Run the SDEsynch programs periodically using crontab.
DONE!
©2005 ChevronTexaco Corporation - All Rights Reserved21
Development Tools
C programming language
ArcSDE C API (Application Programming Interface) version 8.1 for Solaris 8
Gcc C compiler version 2.95 for Solaris 8
Xemacs editing tools (optional)
Makefile (optional)
©2005 ChevronTexaco Corporation - All Rights Reserved22
From Text to Map
SynchronizationUsing SDEsynch
Oracle ‘Textual’ Data ‘Map’ in GIS-based application
©2005 ChevronTexaco Corporation - All Rights Reserved23
From Text to Map
Seismic Navigation
Seismic Line
©2005 ChevronTexaco Corporation - All Rights Reserved24
From Text to Map
Seismic Fault
Well Path
WellLocation
©2005 ChevronTexaco Corporation - All Rights Reserved25
From Text to Map
Oil Field Boundary
Gas Field Boundary
©2005 ChevronTexaco Corporation - All Rights Reserved26
Access From Web Browser
Seismic Fault
Well Path
WellLocation
©2005 ChevronTexaco Corporation - All Rights Reserved27
Data Synchronized (as March 2005)
11364 wells (top & bottom)
985 well deviation surveys
75 seismic navigations
35751 seismic lines
663 seismic faults
8 gas fields
127 oil fields
©2005 ChevronTexaco Corporation - All Rights Reserved28
Results
All IBU Finder projects are synchronized to GIS database on a daily basis.
Able to create shape directly from ArcSDE. Previously, a special application like ArcMap needed to perform this task.
Time saving. The synchronization process runs for less than 5 minutes. Manual update requires much more time, estimated about 2 hours, and any changes can be very difficult to spot.
©2005 ChevronTexaco Corporation - All Rights Reserved29
Acknowledgement
Data Management Team - ChevronTexaco Indonesia Business Unit (CT IBU)
IT - CT IBU
Technology Support - CT IBU
Management - CT IBU
GIS/Remote Sensing Team - Chevron Texaco Energy Technology Company (CT ETC)
©2005 ChevronTexaco Corporation - All Rights Reserved30
References
ArcSDE Developer Help – ArcSDE 8.1 for Client CDROM
Finder 9.1 & 9.3 Data Model – Schlumberger
Oracle 8 : Database Administration – Oracle University
©2005 ChevronTexaco Corporation - All Rights Reserved31
Author Information
Ananta Dwi Bodhitama, Data Management – CT IBURumbai, Sumatra, IndonesiaEmail: [email protected]
Mursyid Nento, Data Management – CT IBURumbai, Sumatra, IndonesiaEmail: [email protected]
David Stodola, GIS/Remote Sensing – CT ETCSan Ramon, CA, USEmail: [email protected]
©2005 ChevronTexaco Corporation - All Rights Reserved32
Thank you
©2005 ChevronTexaco Corporation - All Rights Reserved33
Backup Slides
©2005 ChevronTexaco Corporation - All Rights Reserved34
Well Top Location
Finder Tables:WELL_HDRNODES
SDE Layers:WELL_TOP_FINDER
Source Data Type: Oracle Table
SDE API: SE_shape_generate_point
Routine: Single loop
©2005 ChevronTexaco Corporation - All Rights Reserved35
Well Bottom Location
Finder Tables:WELL_HDRNODES
SDE Layers:WELL_BASE_FINDER
Source Data Type: Oracle Table
SDE API: SE_shape_generate_point
Routine: Single loop
©2005 ChevronTexaco Corporation - All Rights Reserved36
Well Path
Finder Tables:WELL_DIR_SRVY_PTSWELL_HDRNODES
SDE Layers:WELL_PATH_FINDER
Source Data Type: Oracle Table
SDE API: SE_shape_generate_line
Routine: Double loop + use checksum
©2005 ChevronTexaco Corporation - All Rights Reserved37
Seismic Navigation
Finder Tables:SEIS_3D_HDR
SDE Layers:SEIS_NAV_FINDER
Source Data Type: Oracle Table
SDE API: SE_shape_generate_polygon
Routine: Single loop + use checksum
©2005 ChevronTexaco Corporation - All Rights Reserved38
Seismic Line
Finder Tables:SEIS_LINE_HDR
SDE Layers:SEIS_LINE_FINDER
Source Data Type: Oracle Table
SDE API: SE_shape_generate_line
Routine: Single loop + use checksum
©2005 ChevronTexaco Corporation - All Rights Reserved39
Seismic Fault
Finder Tables:SEIS_FAULT_TRACE
SDE Layers:SEIS_FAULT_FINDER
Source Data Type: Nlist File + Oracle Table
SDE API: SE_shape_generate_line
Routine: Nlbrowser + process fork
©2005 ChevronTexaco Corporation - All Rights Reserved40
Oil Field Boundary
Finder Tables:LINESGEM_OBJECTS
SDE Layers:FIELD_OIL_FINDER
Source Data Type: Nlist File + Oracle Table
SDE API: SE_shape_generate_polygon
Routine: Nlbrowser + process fork
©2005 ChevronTexaco Corporation - All Rights Reserved41
Gas Field Boundary
Finder Tables:LINESGEM_OBJECTS
SDE Layers:FIELD_GAS_FINDER
Source Data Type: Nlist File + Oracle Table
SDE API: SE_shape_generate_polygon
Routine: Nlbrowser + process fork
©2005 ChevronTexaco Corporation - All Rights Reserved42
ArcSDE Server Setup
Detail columns of WELL_TOP_FINDER point class layer:
Objectid NumberWell_UWI Varchar(50)Project_Na Varchar(50)Instance_N Varchar(50)Longitude FloatLatitude FloatShape Number
©2005 ChevronTexaco Corporation - All Rights Reserved43
ArcSDE Server Setup
Detail columns of WELL_BASE_FINDER point class layer:
Objectid NumberWell_UWI Varchar(50)Project_Na Varchar(50)Instance_N Varchar(50)Longitude FloatLatitude FloatShape Number
©2005 ChevronTexaco Corporation - All Rights Reserved44
ArcSDE Server Setup
Detail columns of WELL_PATH_FINDER line class layer:
Objectid NumberUWI Varchar(50)Survey_Point NumberChecksum NumberProject_Na Varchar(50)Instance_N Varchar(50)Shape Number
©2005 ChevronTexaco Corporation - All Rights Reserved45
ArcSDE Server Setup
Detail columns of SEIS_NAV_FINDER polygon class layer:
Objectid NumberSurvey Varchar(50)X1 NumberY1 NumberX2 NumberY2 NumberX3 NumberY3 NumberX4 NumberY4 NumberProject_Na Varchar(50)Instance_N Varchar(50)Checksum NumberShape Number
©2005 ChevronTexaco Corporation - All Rights Reserved46
ArcSDE Server Setup
Detail columns of SEIS_LINE_FINDER line class layer:
Objectid NumberSurvey Varchar(50)Lower_Left_X NumberLower_Left_Y NumberUpper_Right_X NumberUpper_Right_Y NumberProject_Na Varchar(50)Instance_N Varchar(50)Checksum NumberLine_Name Varchar(50)Line_Type Varchar(50)Shape Number
©2005 ChevronTexaco Corporation - All Rights Reserved47
ArcSDE Server Setup
Detail columns of SEIS_FAULT_FINDER line class layer:
Objectid NumberHor_Code Varchar(50)Fault_Code Varchar(50)Xmin NumberYmin NumberXmax NumberYmax NumberProject_Na Varchar(50)Instance_N Varchar(50)Checksum NumberShape NumberLast_Update_Finder DateLast_Update_SDE Date
©2005 ChevronTexaco Corporation - All Rights Reserved48
ArcSDE Server Setup
Detail columns of FIELD_OIL_FINDER line class layer:
Objectid NumberObject_Code Varchar(50)Field_Name Varchar(50)Xmin NumberYmin NumberXmax NumberYmax NumberProject_Na Varchar(50)Instance_N Varchar(50)Checksum NumberShape NumberLast_Update_Finder DateLast_Update_SDE Date
©2005 ChevronTexaco Corporation - All Rights Reserved49
ArcSDE Server Setup
Detail columns of FIELD_GAS_FINDER line class layer:
Objectid NumberObject_Code Varchar(50)Field_Name Varchar(50)Xmin NumberYmin NumberXmax NumberYmax NumberProject_Na Varchar(50)Instance_N Varchar(50)Checksum NumberShape NumberLast_Update_Finder DateLast_Update_SDE Date
©2005 ChevronTexaco Corporation - All Rights Reserved50
Program Execution
Command line:SDEsynchFinder <input_file>
Input file format: <project_name> <instance_name>
Environment variables for running SDEsynch:ARCHOME = /rbappl/users/sde/arcexe81SDEHOME = /rbappl/users/sde/sdeexe81 ORACLE_HOME = /rbappl/users/ora8i/app/oracle/product/8.1.7LD_LIBRARY_PATH = $SDEHOME/lib:$ARCHOME/lib:$ORACLE_HOME/lib
©2005 ChevronTexaco Corporation - All Rights Reserved51
Input Files
Current input file for SDE-Finder Synchronization:
MINAS F93RBIPE_T F93RBIPRUMBAI F93RBIPDURI F93DRIPBEKASAP F93DRIP
Current input file for SDE-Finder Synchronization (for Oil & Gas Field Boundary):
RUMBAI F93RBIP
©2005 ChevronTexaco Corporation - All Rights Reserved52
Process Fork
Create a child process using fork() system call.
Fork is needed before calling execve() because execve() does not create new process and overlays the current process with new one.
Current process or parent process then waits for return code from child process before continuing execution.
©2005 ChevronTexaco Corporation - All Rights Reserved53
Author Brief Biography
Ananta Dwi Bodhitama
Title: IT Engineer. Have been working for 5 years for CT IBU. Keen interest in G&G and GIS software development. Have attended various IT and earth science courses as well.
Formal Education:
Bachelor of Computer Science (1998) University of Indonesia, Indonesia.
Master of Applied Information Technology (2002)Universite de Rennes I, France. (scholarship)