ppt
TRANSCRIPT
![Page 1: ppt](https://reader035.vdocuments.us/reader035/viewer/2022070316/555e0803d8b42a99188b4805/html5/thumbnails/1.jpg)
An easy way to manage Relational Databases
in the Globus Community
Sandro Fiore
ISUFI/ Center for Advanced Computational TechnologiesDirector: prof. Giovanni Aloisio
University of Lecce, Italy
![Page 2: ppt](https://reader035.vdocuments.us/reader035/viewer/2022070316/555e0803d8b42a99188b4805/html5/thumbnails/2.jpg)
ClientDBMS
Client
Client
A simple Scenario
Client
“How can Grid-aware Applications interact with their relational Data Resources in a distributed environment in order to make the most of a computational Grid?”
?
![Page 3: ppt](https://reader035.vdocuments.us/reader035/viewer/2022070316/555e0803d8b42a99188b4805/html5/thumbnails/3.jpg)
A Grid-DBMS is a system which dynamically and
transparently reconfigures components such as Data
Resources at runtime, according to the Grid state, in
order to maintain a desired performance level. It must
offer an efficient, robust, intelligent, transparent,
uniform access to Grid-Databases
Definition of Grid-DBMS
![Page 4: ppt](https://reader035.vdocuments.us/reader035/viewer/2022070316/555e0803d8b42a99188b4805/html5/thumbnails/4.jpg)
Definition of Grid-DataBase
A Grid-DataBase is a collection of one or more
Databases which can also be heterogeneous and
contain replica, accessible through a Grid-DBMS front
end . It represents an extension and a virtualization of
the Database concept in a grid environment."
Grid-DB1 Grid-DB2 Grid-DB4Grid-DB3
![Page 5: ppt](https://reader035.vdocuments.us/reader035/viewer/2022070316/555e0803d8b42a99188b4805/html5/thumbnails/5.jpg)
Dynamic Reconfiguration
What do we mean by Dynamic Reconfiguration?
• Dynamic Database Relocation
• Dynamic Database Replication
• Dynamic Database Partition
![Page 6: ppt](https://reader035.vdocuments.us/reader035/viewer/2022070316/555e0803d8b42a99188b4805/html5/thumbnails/6.jpg)
MySQLOracle PostgreSQL ……. DB2
APP 3APP 1 APP 2 APP 4 APP 5
The Grid-DBMS layer
Grid-DBMS
![Page 7: ppt](https://reader035.vdocuments.us/reader035/viewer/2022070316/555e0803d8b42a99188b4805/html5/thumbnails/7.jpg)
A Grid-DBMS must be:
• Secure
• Transparent
• Easy to manage
• Robust
• Efficient
• Intelligent
Grid-DBMS requirements
…and it must support:
• Different DBMS
• High level functionalities
• High level Grid technologies(e.g. GridFTP)
• Dynamic reconfiguration mechanisms
• Performance Monitoring of the DBMS
![Page 8: ppt](https://reader035.vdocuments.us/reader035/viewer/2022070316/555e0803d8b42a99188b4805/html5/thumbnails/8.jpg)
Introducing the GRelC Project
Grid Relational Catalog is a project that
aims at designing and deploying the first
Grid-DBMS for the globus community
![Page 9: ppt](https://reader035.vdocuments.us/reader035/viewer/2022070316/555e0803d8b42a99188b4805/html5/thumbnails/9.jpg)
First Steps
Connection
• Drivers (basic building blocks)
Interaction
• Queries (core and advanced)
![Page 10: ppt](https://reader035.vdocuments.us/reader035/viewer/2022070316/555e0803d8b42a99188b4805/html5/thumbnails/10.jpg)
GRelC Basic Architecture
GRelC Protocol
RDBMS
Authorizedclient
Oracle
MySQL
PostgreSQL
SQAccess policy
SQLogging
Configurationserver
DTD validMQ packet
MQAccess policy
MQLogging
Single Query
Multi Query
Ser
ver
fro
nt-
end
SQ packets
MQ packets
Result packet
Result packetXML
Globus I/OGSI
SessionSession packet
Result packet
Client App built on top of the GRelC Library
![Page 11: ppt](https://reader035.vdocuments.us/reader035/viewer/2022070316/555e0803d8b42a99188b4805/html5/thumbnails/11.jpg)
Main Features
• Authentication
• Authorization
• Access control policy
• Data Encryption
• Single Query Support
• Multi Query Support
• MultiDBMS Support
• XML Data Validation
• Logging
![Page 12: ppt](https://reader035.vdocuments.us/reader035/viewer/2022070316/555e0803d8b42a99188b4805/html5/thumbnails/12.jpg)
GRelC-Server Configuration
<CONFIGURATION_SERVER><SERVER_PORT>13002</SERVER_PORT><VALIDATION_DATA_STREAM>y</VALIDATION_DATA_STREAM><REPOSITORY_DATA_PACKET>../grelc_repository_data_packet/</REPOSITORY_DATA_PACKET><DATABASES>
<DATABASE GRELC_DBNAME=”Student"><DB_HOST_NAME>gandalf.unile.it</DB_HOST_NAME><DB_NAME>grelcdb</DB_NAME><DB_LOGIN>db-login</DB_LOGIN><DB_PASSWORD>db-pwd</DB_PASSWORD><DB_PORT>5432</DB_PORT><DTD_FILENAME>../grelc_dtd/grelc_schema2.dtd</DTD_FILENAME><AUTHORIZATION_CLIENT>y</AUTHORIZATION_CLIENT><AUTHORIZED_CLIENT>
<DN INSERT="TRUE” DELETE=”FALSE">DN-user1</DN><DN CREATE_DB="TRUE” DROP_DB="TRUE">DN-user2</DN><DN UPDATE="TRUE” GRIDFTPSQ="TRUE">DN-user3</DN><DN MQ="TRUE” INSERT="TRUE">DN-user3</DN>
</AUTHORIZED_CLIENT></DATABASE><DATABASE GRELC_DBNAME=”Library">
...</DATABASE>...
</DATABASES></CONFIGURATION_SERVER>
GRelC-Server General Info
Database Student Configuration
Database Configuration
Database Library Configuration
Database Authorization Policy
Access Control Policy
![Page 13: ppt](https://reader035.vdocuments.us/reader035/viewer/2022070316/555e0803d8b42a99188b4805/html5/thumbnails/13.jpg)
Access Policy If true…
CREATE_DB Allow user to create new databases
DROP_DB Allow user to drop databases
MQ Allow user to do MultiQuery
GRIDFTPMQ Allow user to do MultiQuery Grid FTP
GRIDFTPSQ Allow user to do SingleQuery Grid FTP
TRANSACTION Allow user to do transactions
INSERT Allow user to do Insert Query
UPDATE Allow user to do Update Query
DELETE Allow user to do Delete Query
Access Policy
![Page 14: ppt](https://reader035.vdocuments.us/reader035/viewer/2022070316/555e0803d8b42a99188b4805/html5/thumbnails/14.jpg)
Logging
Connection from /O=Grid/O=Globus/OU=unile.it/CN=Sandro Fiore to grelcdb at 15/07/2003 13:25 [ OK ]
Connection from /O=Grid/O=Globus/OU=unile.it/CN=Daniele Lezzi to grelcdb at 15/07/2003 13:40 [ OK ]
Connection from /O=Grid/O=Globus/OU=unile.it/CN=Marco Polo to grelcdb at 15/07/2003 13:44 [ FAILED ]
/O=Grid/O=Globus/OU=unile.it/CN=Sandro Fiore SINGLE select * from student 15/07/2003 13:25
/O=Grid/O=Globus/OU=unile.it/CN=Sandro Fiore SINGLE select * from seminar 15/07/2003 13:25
/O=Grid/O=Globus/OU=unile.it/CN=Daniele Lezzi SINGLE select title from seminar 15/07/2003 13:40
GRelC_Connection.log
GRelC_server.log_grelcdb
![Page 15: ppt](https://reader035.vdocuments.us/reader035/viewer/2022070316/555e0803d8b42a99188b4805/html5/thumbnails/15.jpg)
<TABLES GRELC_DBNAME="grelcdb"><TABLE NAME="person"> <RECORDS> <RECORD>
<ATTRIBUTES> <ATTRIBUTE NAME=“PID" TYPE="STRING">DTJdfjksdk£23423</ATTRIBUTE> <ATTRIBUTE NAME="name" TYPE="STRING">Sandro Fiore</ATTRIBUTE> <ATTRIBUTE NAME="address" TYPE="STRING">Via Carlo V</ATTRIBUTE> <ATTRIBUTE NAME="fax" TYPE="STRING">+39 0832 297279</ATTRIBUTE>
</ATTRIBUTES> </RECORD> <RECORD>
<ATTRIBUTES> <ATTRIBUTE NAME=“PID" TYPE="STRING">kjgjkgdd£32424</ATTRIBUTE> <ATTRIBUTE NAME="name" TYPE="STRING">Marco Polo</ATTRIBUTE> <ATTRIBUTE NAME="address" TYPE="STRING">Via America</ATTRIBUTE>
<ATTRIBUTE NAME="fax" TYPE="STRING">+39 0832 555777</ATTRIBUTE></ATTRIBUTES>
</RECORD> </RECORDS>
</TABLE></TABLES>
MultiQuery XML file example
Person
PIDnameaddressfax
![Page 16: ppt](https://reader035.vdocuments.us/reader035/viewer/2022070316/555e0803d8b42a99188b4805/html5/thumbnails/16.jpg)
<?xml version="1.0" encoding="UTF-8"?><!ELEMENT TABLES (RELATIONS?,TABLE+)><!ELEMENT RELATIONS (RELATION+)><!ELEMENT RELATION (REFERENCEFIELDS)><!ELEMENT REFERENCEFIELDS (ATTRIBUTE+)><!ELEMENT TABLE (RELATIONS?,RECORDS)><!ELEMENT RECORDS (RECORD+)><!ELEMENT RECORD (ATTRIBUTES?, RELATIONS?)><!ELEMENT ATTRIBUTES (ATTRIBUTE+)><!ELEMENT ATTRIBUTE (#PCDATA)><!ATTLIST TABLES GRELC_DBNAME (grelcdb) #REQUIRED><!ATTLIST TABLE NAME (person) #IMPLIED><!ATTLIST ATTRIBUTE NAME ( name| PID |address | fax ) #IMPLIEDTYPE (INTEGER | FLOAT | DOUBLE | STRING | LONG) #IMPLIED>
Person
PIDnameaddressfax
MultiQuery DTD file example
![Page 17: ppt](https://reader035.vdocuments.us/reader035/viewer/2022070316/555e0803d8b42a99188b4805/html5/thumbnails/17.jpg)
<TABLES GRELC_DBNAME="grelcdb"> <TABLE NAME="person">
<RECORDS> <RECORD>
<RELATIONS> <RELATION FOREIGNKEY=”idcountryref” REFERENCETABLE=”country” REFERENCEKEY=”idcountry”> <REFERENCEFIELDS>
<ATTRIBUTE NAME="countryname" TYPE="STRING">Italy</ATTRIBUTE> </REFERENCEFIELDS>
</RELATION> </RELATIONS> <ATTRIBUTES>
<ATTRIBUTE NAME="name" TYPE="STRING">Sandro Fiore</ATTRIBUTE> <ATTRIBUTE NAME="address" TYPE="STRING">Via Carlo V</ATTRIBUTE> <ATTRIBUTE NAME=“PID" TYPE="STRING">jhdhsfdhj9833</ATTRIBUTE>
</ATTRIBUTES> </RECORD>
</RECORDS> </TABLE></TABLES>
Person CountryBorn
PIDnameaddress
idcountrycountryname
MultiQuery XML file example
N 1
![Page 18: ppt](https://reader035.vdocuments.us/reader035/viewer/2022070316/555e0803d8b42a99188b4805/html5/thumbnails/18.jpg)
<?xml version="1.0" encoding="UTF-8"?><!ELEMENT TABLES (RELATIONS?,TABLE+)><!ELEMENT RELATIONS (RELATION+)><!ELEMENT RELATION (REFERENCEFIELDS)><!ELEMENT REFERENCEFIELDS (ATTRIBUTE+)><!ELEMENT TABLE (RELATIONS?,RECORDS)><!ELEMENT RECORDS (RECORD+)><!ELEMENT RECORD (ATTRIBUTES?, RELATIONS?)><!ELEMENT ATTRIBUTES (ATTRIBUTE+)><!ELEMENT ATTRIBUTE (#PCDATA)><!ATTLIST TABLES GRELC_DBNAME (grelcdb) #REQUIRED><!ATTLIST TABLE NAME (person | country) #IMPLIED><!ATTLIST RELATION FOREIGNKEY (idcountryref) #IMPLIEDREFERENCETABLE (country) #IMPLIED REFERENCEKEY (idcountry) #IMPLIED><!ATTLIST ATTRIBUTENAME ( name | address | PID | countryname | idcountry ) #IMPLIEDTYPE (INTEGER | FLOAT | DOUBLE | STRING | LONG) #IMPLIED>
Person CountryBorn
PIDnameaddressidcountryref
idcountrycountryname
MultiQuery DTD file example
N 1
![Page 19: ppt](https://reader035.vdocuments.us/reader035/viewer/2022070316/555e0803d8b42a99188b4805/html5/thumbnails/19.jpg)
GRelC QUERIES
You can submit several GRelC-Queries to the GRelC-Server:
1) Single Query (SQ)
2) Single Query GridFTP (SQ-GridFTP)
3) Single Query Remote GridFTP (SQR-GridFTP)
4) Multi Query (MQ)
5) Multi Query GridFTP (MQ-GridFTP)
6) Multi Query GridFTP-ThirdParty (MQ-GridFTP-TP)
![Page 20: ppt](https://reader035.vdocuments.us/reader035/viewer/2022070316/555e0803d8b42a99188b4805/html5/thumbnails/20.jpg)
SINGLE QUERY
GRelC-ServerDBMS
Client
Single Query Packet Single Query
Result Packet
SQL
Recordset
GrelCRecordset
GrelC-libAPIs
GrelCRecordset
![Page 21: ppt](https://reader035.vdocuments.us/reader035/viewer/2022070316/555e0803d8b42a99188b4805/html5/thumbnails/21.jpg)
SINGLE QUERY GRIDFTP
GRelC-ServerDBMS
Client
Single Query GridFTP Packet
GridFTPTransfer
SQL
Recordset
GrelCRecordset in XMLformatSingle
Query GridFTP file
Single Query GridFTP Result Packet
GrelCRecordset
GrelCLoad Recordset
GrelCLib APIs
![Page 22: ppt](https://reader035.vdocuments.us/reader035/viewer/2022070316/555e0803d8b42a99188b4805/html5/thumbnails/22.jpg)
SINGLE QUERY REMOTE GRIDFTP
GRelCServer
DBMS
Client
Single Query GridFTP Packet
GridFTPTransfer
SQL
Recordset
GrelCRecordset in XMLformat
SingleQuery GridFTP file
Single Query GridFTP Result Packet
GrelCRecordset
GrelCLoad Recordset
GrelCLib APIs
SingleQuery GridFTP file
![Page 23: ppt](https://reader035.vdocuments.us/reader035/viewer/2022070316/555e0803d8b42a99188b4805/html5/thumbnails/23.jpg)
GRelC-Server
DBMS
Client
Multi Query Packet Multi Query
ACK Packet
MultiQuery File
SQL Queries
MultiQuery File OK
MultiQuery File
Validation Process
MULTI QUERY
![Page 24: ppt](https://reader035.vdocuments.us/reader035/viewer/2022070316/555e0803d8b42a99188b4805/html5/thumbnails/24.jpg)
GRelCServer
DBMS
Client
Multi Query GridFTP Packet
Multi Query GridFTPACK Packet
MultiQuery File
MULTI QUERY GRIDFTP
SQL Queries
GridFTPTransfer
MultiQuery File
Validation Process
MultiQuery File OK
![Page 25: ppt](https://reader035.vdocuments.us/reader035/viewer/2022070316/555e0803d8b42a99188b4805/html5/thumbnails/25.jpg)
GRelCServer
DBMS
Client A
Multi Query Packet
Multi QueryACK Packet
MultiQuery File
MULTI QUERY GRIDFTP THIRD-PARTY
SQL Queries
GridFTPTransfer
MultiQuery File
Validation Process
MultiQuery File OK
Client B
![Page 26: ppt](https://reader035.vdocuments.us/reader035/viewer/2022070316/555e0803d8b42a99188b4805/html5/thumbnails/26.jpg)
The GRelC Library: a new layer
New Layer
Appl. built on top of the GRelC-lib
Grid Applications
GRelC Library
Globus APIs DBMS APIs XML LibraryCore Libraries
![Page 27: ppt](https://reader035.vdocuments.us/reader035/viewer/2022070316/555e0803d8b42a99188b4805/html5/thumbnails/27.jpg)
The GRelC Library: APIs Classification
We can classify the 42 APIs into 5 categories:
1) Connection APIs
2) Data Manipulation APIs
3) Core APIs
4) Administration APIs
5) High level APIs
![Page 28: ppt](https://reader035.vdocuments.us/reader035/viewer/2022070316/555e0803d8b42a99188b4805/html5/thumbnails/28.jpg)
int grelc_select(globus_io_handle_t*, char*, Grelc_Answer* );
int grelc_search_MQ(globus_io_handle_t*, char*, char* );
int grelc_grid_ftp_SQ(globus_io_handle_t*, char*, char*, char*, char* );
int grelc_grid_ftp_MQ(globus_io_handle_t*, char* );
int grelc_unbind(globus_io_attr_t* attr, globus_io_handle_t* handle);
int grelc_bind(globus_result_t result, char* hostname, unsigned short port, globus_io_attr_t* , globus_io_handle_t* );
int grelc_schema(Grelc_Answer* );
int grelc_schema_table(Grelc_Answer* );
int grelc_free_data(Grelc_Answer* );
void grelc_channel_initialization(globus_io_attr_t* attr,
globus_io_secure_authorization_callback_t globus_io_secure_authorization_callback, void *args);
void grelc_channel_initialization_without_callback(globus_io_attr_t* attr);
int grelc_create_database(globus_io_handle_t* handle,char* database);
int grelc_drop_database(globus_io_handle_t* handle,char* database);
int grelc_create_table(globus_io_handle_t* handle,char* query);
int grelc_drop_table(globus_io_handle_t* handle,char* table);
int grelc_open_transaction(globus_io_handle_t* handle);
int grelc_abort_transaction(globus_io_handle_t* handle);
int grelc_rollback_transaction(globus_io_handle_t* handle);
int grelc_commit_transaction(globus_io_handle_t* handle);
GRelC Library v2.0 (1/2)
![Page 29: ppt](https://reader035.vdocuments.us/reader035/viewer/2022070316/555e0803d8b42a99188b4805/html5/thumbnails/29.jpg)
int grelc_insert(globus_io_handle_t* handle,char* query);
int grelc_update(globus_io_handle_t* handle,char* query);
int grelc_delete(globus_io_handle_t* handle,char* query);
int grelc_get_number_records(Grelc_Answer* );
int grelc_get_number_fields(Grelc_Answer* );
int grelc_get_position_record(Grelc_Answer* );
int grelc_find_first(Grelc_Answer* data, char* attribute, char* comp, char* value);
int grelc_find_next(Grelc_Answer* data, char* attribute, char* comp, char* value);
int nomatch(Grelc_Answer* data);
int grelc_move_first(Grelc_Answer* );
int grelc_move_last(Grelc_Answer* );
int grelc_move_next(Grelc_Answer* );
int grelc_move_previous(Grelc_Answer* );
int grelc_move(Grelc_Answer* ,int );
int grelc_eof(Grelc_Answer* );
int grelc_bof(Grelc_Answer* );
int grelc_is_null(char* );
char* grelc_get_field_by_attribute(Grelc_Answer* ,char* );
char* grelc_get_field_by_position(Grelc_Answer* ,int );
char* grelc_get_name_field_by_position(Grelc_Answer* ,int );
GRelC Library v2.0 (2/2)
![Page 30: ppt](https://reader035.vdocuments.us/reader035/viewer/2022070316/555e0803d8b42a99188b4805/html5/thumbnails/30.jpg)
grelc_channel_initialization_without_callback(&attr);
grelc_bind(result,hostname,database_name,port,&attr,&handle);
grelc_select(&handle,query,&data);
grelc_unbind(&attr,&handle);
// Library Usage //
printf("Number of Records %d\n",grelc_get_number_records(&data));
printf("Number of fields %d\n",grelc_get_number_fields(&data));
grelc_move_first(&data);
while(!grelc_eof(&data)){
for (i=1; i<=grelc_get_number_fields(&data) ; i++)
printf("Field: %s -> %s\n",grelc_get_name_field_by_position(&data,i),grelc_get_field_by_position(&data,i));
grelc_move_next(&data);
}
grelc_free_data(&data);
exit(EXIT_SUCCESS);
}
How to use the GRelC Library
Connection
Query
Data Manipulation
Close Connection
![Page 31: ppt](https://reader035.vdocuments.us/reader035/viewer/2022070316/555e0803d8b42a99188b4805/html5/thumbnails/31.jpg)
Releases
Two current releases:1. GRelCv1.02. GRelCv2.0
Differences:• Only 23 APIs in the first version vs 42 in the second
one.• Different Grelc-Server management• New operations for data manipulation• Extended recordset structure• Access control policy• Logging• High-level functionalities supported.
![Page 32: ppt](https://reader035.vdocuments.us/reader035/viewer/2022070316/555e0803d8b42a99188b4805/html5/thumbnails/32.jpg)
SQ, MQ, MQFTP Comparison10000 Query x Client
0
50
100
150
200
250
Time
MQ Overhead 0 35 35
Connection 220,5 174,8 18,5
SQ MQ MQFTP
SQ, MQ, MQFTP Comparison1000 Query x Client
0
50
100
150
200
250
Time
MQ Overhead 0 6 6
Connection 21 15,1 2,8
SQ MQ MQFTP
1 GrelC-Server 10 Cients
020406080
100120140160180200220240
1 10 100 1000 10000Num Queries
Time (Sec)
Single Query Multi Query Multi Query GridFTP
First Tests in our Campus
10 Clients each one submitting n (1:10000) insert queries to a unique GRelC-Server
(10:100.000 total insert queries)
Three different ways to do that.1) 1:10.000 Single Query2) 1 MultiQuery 3) 1 MultiQuery GridFTP
![Page 33: ppt](https://reader035.vdocuments.us/reader035/viewer/2022070316/555e0803d8b42a99188b4805/html5/thumbnails/33.jpg)
• GRelC Library (more performant)
IndustriesReal Applications
• Web/Grid Services (less performant but OGSA compliant)
Academic environment
Research
Two parallel directions
![Page 34: ppt](https://reader035.vdocuments.us/reader035/viewer/2022070316/555e0803d8b42a99188b4805/html5/thumbnails/34.jpg)
• Web/Grid Services Version (a basic version is already deployed and used for internal projects)
• Support for Oracle, MySQL DBMS
• Support for Distributed Query (very hard and interesting challenge)
• Library Extensions (new APIs)
• New Queries that support compression mechanisms
• Scheduling strategies related to replicated and partitioned databases
• XML temporary datasets management
Future Works
![Page 35: ppt](https://reader035.vdocuments.us/reader035/viewer/2022070316/555e0803d8b42a99188b4805/html5/thumbnails/35.jpg)
For any information
Director: Prof. Giovanni Aloisio (giovanni.aloisio @unile.it)Project P. I. : Sandro Fiore ([email protected])
Center for Advanced Computational Technologies - CACT/ISUFI, University of Lecce - ITALY
WebSite : http://gandalf.unile.it/grelc.html