1 philip a. adams – llnl/national ignition facility john c. hax – oracle corporation database...
Post on 21-Dec-2015
214 views
TRANSCRIPT
![Page 1: 1 Philip A. Adams – LLNL/National Ignition Facility John C. Hax – Oracle Corporation Database File Systems in Support of eScience](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d605503460f94a409b6/html5/thumbnails/1.jpg)
1
Philip A. Adams – LLNL/National Ignition Facility
John C. Hax – Oracle Corporation
Database File Systems in Support of eScience
![Page 2: 1 Philip A. Adams – LLNL/National Ignition Facility John C. Hax – Oracle Corporation Database File Systems in Support of eScience](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d605503460f94a409b6/html5/thumbnails/2.jpg)
2
Science – A product of data analysis
“Science does not result from the launch of a mission or the collection of data.
Rather, science only occurs through the analysis and understanding of that data.”
- Philosophy of the NASA Science Mission Directorate (SMD)
![Page 3: 1 Philip A. Adams – LLNL/National Ignition Facility John C. Hax – Oracle Corporation Database File Systems in Support of eScience](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d605503460f94a409b6/html5/thumbnails/3.jpg)
3
Questions to Ask
Are we building IT Systems that support Research and Analysis or Infrastructure that supports the collection of data?
![Page 4: 1 Philip A. Adams – LLNL/National Ignition Facility John C. Hax – Oracle Corporation Database File Systems in Support of eScience](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d605503460f94a409b6/html5/thumbnails/4.jpg)
4
Scientific Computing History
Commercial Relational Databases
Scientific (minimal data shared)• Raw Data• Decentralized/Desktop Management• Open source software• Low quality of support/service
• Best Effort• Mission critical operations
• Primarily file based – HDF5,Lustre• Millions of Files• Write once, read many
• Background processing• Pipelines• Computationally intensive applications• Long running transactions• Output of Large Data Sets
• Single application profile
Scientific (minimal data shared)• Raw Data• Decentralized/Desktop Management• Open source software• Low quality of support/service
• Best Effort• Mission critical operations
• Primarily file based – HDF5,Lustre• Millions of Files• Write once, read many
• Background processing• Pipelines• Computationally intensive applications• Long running transactions• Output of Large Data Sets
• Single application profile
Scientific Systems
vs.
Enterprise (all data shared)• Metadata• Centralized management• Industrial strength software• High qualities of support/service
• SLA guarantees• Mission Critical Operations
• Mission critical operations• Databases & files
• Read and Update• Enforced data integrity
• Interactive processing• Interactive workflows• Transactional, intensive applications• Short running transactions (<8 hours)• Output of Individual Rows
• Mixed application profile
![Page 5: 1 Philip A. Adams – LLNL/National Ignition Facility John C. Hax – Oracle Corporation Database File Systems in Support of eScience](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d605503460f94a409b6/html5/thumbnails/5.jpg)
5
Filesystems and Legacy Databases – The Gap
Superior query/search capability over filesystems SQL standard
Easy manipulation of data Functions PL/SQL Java, C, PHP, Perl
Low latency, interactive data access suited for application access
Provides a structured way of storing data and ensuring data integrity Tables/ConstraintsSuperior backup and recovery capabilities RMAN, Redo/Archive logging Block and Point-in-Time Recovery Block Level Corruption DetectionInstitutional Resources
Superior query/search capability over filesystems SQL standard
Easy manipulation of data Functions PL/SQL Java, C, PHP, Perl
Low latency, interactive data access suited for application access
Provides a structured way of storing data and ensuring data integrity Tables/ConstraintsSuperior backup and recovery capabilities RMAN, Redo/Archive logging Block and Point-in-Time Recovery Block Level Corruption DetectionInstitutional Resources
Database Benefits
Provided maximum scalability to meet data volume and ingestion requirements
HDF5GFS (Google Filesystem)Lustre
Ubiquity of accessing filesystemsNumber of protocols NFS, SMB, CIFS and FTP
Able to access the data right from the OS Windows, Mac, Linux, Solaris, HP/UX
Application programming interfaces support native access file open (f_open), file close (f_close) importing the java io package ifstream/ofstream C++ file I/O classes
Provided maximum scalability to meet data volume and ingestion requirements
HDF5GFS (Google Filesystem)Lustre
Ubiquity of accessing filesystemsNumber of protocols NFS, SMB, CIFS and FTP
Able to access the data right from the OS Windows, Mac, Linux, Solaris, HP/UX
Application programming interfaces support native access file open (f_open), file close (f_close) importing the java io package ifstream/ofstream C++ file I/O classes
Filesystem Benefitsvs.
![Page 6: 1 Philip A. Adams – LLNL/National Ignition Facility John C. Hax – Oracle Corporation Database File Systems in Support of eScience](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d605503460f94a409b6/html5/thumbnails/6.jpg)
6
Data Challenges Physical Limitations
• I/O Intensive - limitations on max IOPS• Network speeds - time to ship data to compute nodes
Multiple Data Silos• Governance issues
Pedigree of the data Multiple access policies to get to the data Duplicate data stored in each silo
• Need to scale disparate systems as data grows Increased effort required for Scientists, Developers, Administrators
• Correlating the data across data silos• Coordinated backup and recovery plan• Multiple Data Aggregation Efforts
![Page 7: 1 Philip A. Adams – LLNL/National Ignition Facility John C. Hax – Oracle Corporation Database File Systems in Support of eScience](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d605503460f94a409b6/html5/thumbnails/7.jpg)
7
The Result: The Split Architecture – a step in the wrong direction
These drawbacks include but are not limited to:• Data curation• Security• Availability• Recoverability• Manageability
Because no common database and filesystem access protocol was available, the burden shifted to the application developers and scientific researchers to make sense of the two silos of information
![Page 8: 1 Philip A. Adams – LLNL/National Ignition Facility John C. Hax – Oracle Corporation Database File Systems in Support of eScience](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d605503460f94a409b6/html5/thumbnails/8.jpg)
8
How much of an issue is this?
Level 0 (Raw) data is typically enriched with data from other sources.
What happens when/if a diagnostic is found to have incorrect calibration data?
Without strict relationships, this could be a nightmare. It may be easier to rerun analysis to reproduce the Level 1, 2 and 3 data. However, an unknown quantity of Level 4 content has been generated from this data and is stored on many researchers’ workstations and file shares.
Lack of pedigree in data analysis can result in instrument/machine damage, increased financial costs, or embarrassment to scientific researchers who rely on the data
![Page 9: 1 Philip A. Adams – LLNL/National Ignition Facility John C. Hax – Oracle Corporation Database File Systems in Support of eScience](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d605503460f94a409b6/html5/thumbnails/9.jpg)
9
Future of Scientific Computing and Analysis
Data Intensive
Data Intensive Collaborative Science
Collaborative
+
![Page 10: 1 Philip A. Adams – LLNL/National Ignition Facility John C. Hax – Oracle Corporation Database File Systems in Support of eScience](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d605503460f94a409b6/html5/thumbnails/10.jpg)
10
Data Intensive Collaborative Science
Cost
KnowledgeBase
Complexity
DriversDrivers
The WebThe WebNetwork CapacityNetwork Capacity
Clustering/Grid
Technologies
Clustering/Grid
Technologies
Moores Law
Moores LawStandardsStandards
Collaboration
EnablersEnablers
Interdependence
![Page 11: 1 Philip A. Adams – LLNL/National Ignition Facility John C. Hax – Oracle Corporation Database File Systems in Support of eScience](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d605503460f94a409b6/html5/thumbnails/11.jpg)
11
What’s driving the data volumes?
- Better and more diverse instrumentation
- Flexible optics- Coordinated multi-instrument
observatories- Increased Precision- Genomics
Diverse types of data generated: SQL/Scalar, XML, Image, Monte Carlo simulations, Audio/Video, telemetry, and spectrometers
![Page 12: 1 Philip A. Adams – LLNL/National Ignition Facility John C. Hax – Oracle Corporation Database File Systems in Support of eScience](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d605503460f94a409b6/html5/thumbnails/12.jpg)
12
Database Filesystems
Bridge the Gap between Filesystems and Relational Database Systems
• Maintain Filesystem Performance• Leverage multiple access methods• Single Security Mechanism• Unified Administrative Tools• Data Pedigree• Unified Architecture and Skill sets• Leverage Institutional Resources for IT • Enabling Collaboration around Data• Optimized for Data Access
![Page 13: 1 Philip A. Adams – LLNL/National Ignition Facility John C. Hax – Oracle Corporation Database File Systems in Support of eScience](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d605503460f94a409b6/html5/thumbnails/13.jpg)
1304/18/23 13
Pedigree with a database filesystem
![Page 14: 1 Philip A. Adams – LLNL/National Ignition Facility John C. Hax – Oracle Corporation Database File Systems in Support of eScience](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d605503460f94a409b6/html5/thumbnails/14.jpg)
14
Modern databases have much to offer in the realm of data analysis
RDF/OWL can allow semantic searching of data Predictive Analytics Spatial Data Analysis Text Mining of Unstructured Content
![Page 15: 1 Philip A. Adams – LLNL/National Ignition Facility John C. Hax – Oracle Corporation Database File Systems in Support of eScience](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d605503460f94a409b6/html5/thumbnails/15.jpg)
15
Some of the native data mining techniques and algorithms available
Algorithms
Logistic Regression
Naive Bayes
Support Vector Machine
Decision Tree
Multiple Regression
Minimum Description Length
One-Class Support Vector Machine
Enhanced K-Means
Orthogonal Partitioning Clustering
Apriori
Non-negative Matrix Factorization
Technique
Classification
Regression
Attribute Importance
Anomaly Detection
Clustering
Association
Feature Extraction
![Page 16: 1 Philip A. Adams – LLNL/National Ignition Facility John C. Hax – Oracle Corporation Database File Systems in Support of eScience](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d605503460f94a409b6/html5/thumbnails/16.jpg)
16
Key Components of Secure Files Architecture
•Delta Update•Write Gather Cache• Transformation Management• Inode Management • Space management• I/O Management
Finally the database can accept both structured and non-structured data in an efficient manner
![Page 17: 1 Philip A. Adams – LLNL/National Ignition Facility John C. Hax – Oracle Corporation Database File Systems in Support of eScience](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d605503460f94a409b6/html5/thumbnails/17.jpg)
17
National Ignition Facility
![Page 18: 1 Philip A. Adams – LLNL/National Ignition Facility John C. Hax – Oracle Corporation Database File Systems in Support of eScience](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d605503460f94a409b6/html5/thumbnails/18.jpg)
National Ignition Facility and 11g SecureFiles
NLIT 2009
Philip A. AdamsSr. Systems Architect
National Ignition FacilityLawrence Livermore National Laboratory
June 1-3 2009
This work performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory under Contract DE-AC52-07NA27344
UCRL-PRES-236394
![Page 19: 1 Philip A. Adams – LLNL/National Ignition Facility John C. Hax – Oracle Corporation Database File Systems in Support of eScience](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d605503460f94a409b6/html5/thumbnails/19.jpg)
19NIF-1107-14129.ppt Oracle, 11/12/07 19
Overview of the National Ignition Facility
The National Ignition Facility (NIF) is known as the world’s largest and most energetic laser
When fully operational, its 192 beams will converge 1.8 MJ of laser energy onto a single target to achieve thermonuclear ignition
NIF will enable experiments that produce temperatures and densities like those in the Sun or in an exploding nuclear weapon
![Page 20: 1 Philip A. Adams – LLNL/National Ignition Facility John C. Hax – Oracle Corporation Database File Systems in Support of eScience](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d605503460f94a409b6/html5/thumbnails/20.jpg)
20NIF-1107-14129.ppt Oracle, 11/12/07 20
Overview of the National Ignition Facility
The 192 laser beams of NIF will generate:
• A peak power of 500 trillion watts, 1000 times the electric generating power of the United States
• A pulse energy of 1.8 million joules of ultraviolet light
• A pulse length of three to twenty billionths of a second
![Page 21: 1 Philip A. Adams – LLNL/National Ignition Facility John C. Hax – Oracle Corporation Database File Systems in Support of eScience](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d605503460f94a409b6/html5/thumbnails/21.jpg)
21NIF-1107-14129.ppt Oracle, 11/12/07 21
The Optics make NIF work
Optical components:• 7500 large optics including 3072 laser glass
slabs as well as large lenses, mirrors, and crystals
• More than 15,000 small optical components
Precision optics: • Total area of 33,000 square feet (3/4 of an
acre)
• More than 40 times the total precision optical surface in the world’s largest telescope (Keck Observatory, Hawaii)
![Page 22: 1 Philip A. Adams – LLNL/National Ignition Facility John C. Hax – Oracle Corporation Database File Systems in Support of eScience](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d605503460f94a409b6/html5/thumbnails/22.jpg)
22NIF-1107-14129.ppt Oracle, 11/12/07 22
Example of Optic Damage
2 µm
3 ns
![Page 23: 1 Philip A. Adams – LLNL/National Ignition Facility John C. Hax – Oracle Corporation Database File Systems in Support of eScience](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d605503460f94a409b6/html5/thumbnails/23.jpg)
23NIF-1107-14129.ppt Oracle, 11/12/07 23
On high quality optical surfaces initiated damage sites are very small
![Page 24: 1 Philip A. Adams – LLNL/National Ignition Facility John C. Hax – Oracle Corporation Database File Systems in Support of eScience](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d605503460f94a409b6/html5/thumbnails/24.jpg)
24NIF-1107-14129.ppt Oracle, 11/12/07 24
Performance Gains found in NIF with 11g SecureFiles
Test Environment• Database Server
HP Blade Server w/ 4-way AMD Opteron CPUs RHEL 4 – 32-bit kernel 11g Oracle Database – 32-bit version
Single Instance ASM
Dual Port Fibre Channel Mezzanine Card (2 Gbit)
• Application Server
Dell PowerEdge 2650 w/ 2-way Intel Xeon CPUs RHEL 4 – 32-bit kernel 10g Oracle Application Server 10g Oracle CMSDK (Content Management Software Development
Toolkit)
![Page 25: 1 Philip A. Adams – LLNL/National Ignition Facility John C. Hax – Oracle Corporation Database File Systems in Support of eScience](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d605503460f94a409b6/html5/thumbnails/25.jpg)
25NIF-1107-14129.ppt Oracle, 11/12/07 25
Performance Gains found in NIF with 11g SecureFiles
Test Environment• SAN Storage
3PAR S400
Production Environment• 11g RAC Environment
• 10g CMSDK Clustered Application Server Environment
![Page 26: 1 Philip A. Adams – LLNL/National Ignition Facility John C. Hax – Oracle Corporation Database File Systems in Support of eScience](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d605503460f94a409b6/html5/thumbnails/26.jpg)
26NIF-1107-14129.ppt Oracle, 11/12/07 26
Measure the throughput of the environment
Perform dd tests to the disks to establish the theoretical max:
WRITE
> dd if=/dev/zero of=/dev/raw/raw6 count=10000 bs=1M
READ
> dd if=/dev/raw/raw6 if=/dev/null count=10000 bs=1M
MONITOR
> iostat –xdk 3 100
We saw 180 MB/sec Read/Write throughput to the disks
Warning: Be sure not to perform dd write tests on your ASM configured storage or else you’ll damage it
![Page 27: 1 Philip A. Adams – LLNL/National Ignition Facility John C. Hax – Oracle Corporation Database File Systems in Support of eScience](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d605503460f94a409b6/html5/thumbnails/27.jpg)
27NIF-1107-14129.ppt Oracle, 11/12/07 27
Create a few test tables
Create a test table for BasicFiles and a test table for SecureFiles:
BasicFile Example:
CREATE TABLE "FOO_BASICFILE_TABLE" ( "PKEY" NUMBER(4) NOT NULL , "DOCUMENT" BLOB) TABLESPACE "LOB_DEMO" LOB ("DOCUMENT") STORE AS BASICFILE ( TABLESPACE "LOB_DEMO");
SecureFiles Example:
CREATE TABLE "FOO_SECUREFILE_TABLE" ( "PKEY" NUMBER(4) NOT NULL , "DOCUMENT" BLOB) TABLESPACE "LOB_DEMO" LOB ("DOCUMENT") STORE AS SECUREFILE ( TABLESPACE "LOB_DEMO");
![Page 28: 1 Philip A. Adams – LLNL/National Ignition Facility John C. Hax – Oracle Corporation Database File Systems in Support of eScience](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d605503460f94a409b6/html5/thumbnails/28.jpg)
28
Throughput Results of Table Tests
Speed tests from database server (Oracle 11.1.0 DB, using Oracle jdk 1.5.0_11 in $OH/jdk, using ojdbc5.jar)
Inserting twenty 32MB image files per test
NIF server - server testsBlock Size 64k 1M 32MSecureFiles - jdbc thin 103.29 123.59 161.44SecureFiles - oci 118.34 130.94 139.06BasicFiles - jdbc thin 6.72 48.35 83.7BasicFiles - oci 7.48 45.75 73.86
![Page 29: 1 Philip A. Adams – LLNL/National Ignition Facility John C. Hax – Oracle Corporation Database File Systems in Support of eScience](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d605503460f94a409b6/html5/thumbnails/29.jpg)
29NIF-1107-14129.ppt Oracle, 11/12/07 29
SecureFile vs. BasicFile Server Results
SecureFile Server vs. BasicFile Server Performance
0
20
40
60
80
100
120
140
160
180
64k 1M 32M
Block Size
Th
rou
gh
pu
t (M
B/s
ec)
SecureFiles - jdbc thin
SecureFiles - oci
BasicFiles - jdbc thin
BasicFiles - oci
![Page 30: 1 Philip A. Adams – LLNL/National Ignition Facility John C. Hax – Oracle Corporation Database File Systems in Support of eScience](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d605503460f94a409b6/html5/thumbnails/30.jpg)
30NIF-1107-14129.ppt Oracle, 11/12/07 30
Measure the throughput of the network
Used a tool called iperf available at:
http://sourceforge.net/projects/iperf/
On Server run:
./iperf -s –fM
On Client run:
./iperf -f M -c blackstone
------------------------------------------------------------
Client connecting to blackstone.llnl.gov, TCP port 5001
TCP window size: 0.06 MByte (default)
------------------------------------------------------------
[ 5] local XXX.XXX.XXX.XXX port 58590 connected with XXX.XXX.XXX.XXX port 5001
[ ID] Interval Transfer Bandwidth
[ 5] 0.0-10.0 sec 1120 MBytes 112 MBytes/sec
![Page 31: 1 Philip A. Adams – LLNL/National Ignition Facility John C. Hax – Oracle Corporation Database File Systems in Support of eScience](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d605503460f94a409b6/html5/thumbnails/31.jpg)
31
Throughput Results of Client-Server Tests
NIF client - server testsBlock Size 64k 1M 32MSecureFiles - jdbc thin 59.3 83 103.12SecureFiles - oci 53.69 69.94 83.77BasicFiles - jdbc thin 7.14 46.51 75.08BasicFiles - oci 7.14 41.41 64.5
Speed tests from database server (Oracle 10.1.2 Client, using jdk 1.5.0_11 and ojdbc14.jar)
Inserting twenty 32MB image files per test
![Page 32: 1 Philip A. Adams – LLNL/National Ignition Facility John C. Hax – Oracle Corporation Database File Systems in Support of eScience](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d605503460f94a409b6/html5/thumbnails/32.jpg)
32
SecureFile vs. BasicFile Client Results
SecureFile Client vs. Basic File Client
0
20
40
60
80
100
120
64k 1M 32M
Block Size
Th
rou
gh
pu
t (M
B/s
ec)
SecureFiles - jdbc thin
SecureFiles - oci
BasicFiles - jdbc thin
BasicFiles - oci
![Page 33: 1 Philip A. Adams – LLNL/National Ignition Facility John C. Hax – Oracle Corporation Database File Systems in Support of eScience](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d605503460f94a409b6/html5/thumbnails/33.jpg)
33NIF-1107-14129.ppt Oracle, 11/12/07 33
SecureFile Performance Benefits
During our testing, we’ve seen a 2-20 times increase in performance using SecureFiles over traditional BasicFiles
We’ve seen equivalent or better performance using SecureFiles as we see writing the same file to our NFS mounted NetApp
![Page 34: 1 Philip A. Adams – LLNL/National Ignition Facility John C. Hax – Oracle Corporation Database File Systems in Support of eScience](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d605503460f94a409b6/html5/thumbnails/34.jpg)
34NIF-1107-14129.ppt Oracle, 11/12/07 34
Database Tuning to optimize for SecureFiles
Create a separate tablespace for your LOB data• Use Uniform Extents – 1M seems best
overall• Tried 32M/64M extents with no performance
increase; your mileage may vary
Enable Automatic Segment Space Management on the tablespace
Create large enough redo log files• We used 200M – 1024M to reduce log file
switches during heavy loads
![Page 35: 1 Philip A. Adams – LLNL/National Ignition Facility John C. Hax – Oracle Corporation Database File Systems in Support of eScience](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d605503460f94a409b6/html5/thumbnails/35.jpg)
35NIF-1107-14129.ppt Oracle, 11/12/07 35
Database Tuning to optimize for SecureFiles
Utilize the AWR Snapshots before and after a SecureFile load and note the wait conditions
SQL> EXECUTE dbms_workload_repository.create_snapshot();
PL/SQL procedure successfully completed
Run the AWR report
$ORACLE_HOME/rdbms/admin/awrrpt.sql
![Page 36: 1 Philip A. Adams – LLNL/National Ignition Facility John C. Hax – Oracle Corporation Database File Systems in Support of eScience](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d605503460f94a409b6/html5/thumbnails/36.jpg)
36
Conclusion
The ultimate goal of science is to create new knowledge and new discoveries.
Database Filesystems have a number of features which can benefit the scientific community and ease the burden of pedigree, data management, and analysis
Using a database filesystem will enable data intensive collaborative science.
As new discoveries are made and data volumes increase, it is imperative to have a robust database system that is not only capable of managing the pedigree of that data, but also serve as a knowledge repository for the future.
![Page 37: 1 Philip A. Adams – LLNL/National Ignition Facility John C. Hax – Oracle Corporation Database File Systems in Support of eScience](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d605503460f94a409b6/html5/thumbnails/37.jpg)
37
For More Information
http://search.oracle.com
or
http://www.oracle.com/
SecureFiles