ATLAS Database Access Library Local Area LCG3D Meeting Fermilab, Batavia, USA October 21, 2004 Alexandre Vaniachine (ANL) LCG3D Meeting at Fermilab, October 21, 2004 Alexandre Vaniachine (ANL) ATLAS Database Project LCG3D Meeting at Fermilab, October 21, 2004 Alexandre Vaniachine (ANL) Database Client Library In ATLAS Computing Model various applications require access to the data resident in the relational databases ATLAS Database Project is responsible for ensuring the integration and operation of the full distributed database and data management infrastructure To achieve the integration goal the Distributed Database Services client library serves as a unique layer for enforcing policies, following rules, establish best practices and encode logic to deliver efficient, secure and reliable database connectivity to applications in a heterogeneous distributed database services environment LCG3D Meeting at Fermilab, October 21, 2004 Alexandre Vaniachine (ANL) Priority Requirements In a production environment of ATLAS Data Challenges and Combined Testbeam efforts applications we have learned of various new database connectivity requirements that have to be satisfied in different software domains: Database Connection Management Access Model Uniform across Domains Connection Indirection Layer LCG3D Meeting at Fermilab, October 21, 2004 Alexandre Vaniachine (ANL) Scalability Requirements To provide efficient and robust applications access to database-resident data the client library will provide Connection Pooling Connection Fallout Connections On-demand Connection Timeouts Connection Retries Connection Failover Load Balancing LCG3D Meeting at Fermilab, October 21, 2004 Alexandre Vaniachine (ANL) Other Requirements Client library should be lightweight: no heavier or more complex then necessary, with minimal external dependencies To assure concurrent interoperability of service functions a testing tool suite is required To analyse, monitor and debug the application sequence of database queries a client-side logging, debugging and performance monitoring is required LCG3D Meeting at Fermilab, October 21, 2004 Alexandre Vaniachine (ANL) As a first step in our project we collected these and other requirements in the Client Library Document The feedback received was positive (and was incorporated in the document) Client Library Document LCG3D Meeting at Fermilab, October 21, 2004 Alexandre Vaniachine (ANL) Technology Choices We evaluated many technologies for heterogeneous DB access: JDBC, ODBC, ROOT&RDBC, Qt, POOL Qt has been chosen as a most advanced and versatile to build the first proof-of-the-principle prototype (both with Qt 3.3 and Qt 4.0) to achieve the first Milestone of the project capability to connect to Oracle and MySQL from the same program via plugin libraries Most of the technologies evaluated do not address concerns of the distributed database services A closest match was C-JDBC: Java access to cluster of databases: LCG3D Meeting at Fermilab, October 21, 2004 Alexandre Vaniachine (ANL) Implementation Decision After consultations with ATLAS Database Project management and other considerations minimal dependencies licensing (e.g. Qt) features we need most we decided that instead of wrapping others code we should proceed with our own code base written in C++ Through emphasis on a practical approach, and by avoiding too much abstraction and flexibility at early stage we concentrate on solving actual problems of ATLAS operations workflow LCG3D Meeting at Fermilab, October 21, 2004 Alexandre Vaniachine (ANL) Layer Decomposition External layer provides management of database drivers, database connections and Connections/Servers lists Internal library layer is composed of a number of technology specific database drivers providing support for MySQL and Oracle initially Oracle MySQL ODBC To hide the complexity of heterogeneous database technologies the library is separated in two layers LCG3D Meeting at Fermilab, October 21, 2004 Alexandre Vaniachine (ANL) Layered Architecture Application Interface Connections Pool Manager Singleton Oracle Driver MySQL Driver Policies Connections Pool Drivers Common Interface Server Indirection User Credentials LCG3D Meeting at Fermilab, October 21, 2004 Alexandre Vaniachine (ANL) Indirection Mechanism Based on the POOL catalog indirection architecture MySQL & Oracle Catalogues hold logical/physical mapping - logical server names and its replicas Both catalogues are synchronized change in one causes change in the other Per default time user application updates the XML Catalogue in its local working area from the central mysql/oracle catalogue MySQL Oracle Catalog DBClient A single point-of-control over server indirection LCG3D Meeting at Fermilab, October 21, 2004 Alexandre Vaniachine (ANL) Status and Plans The overall completion milestone is in time for the Data Challenge 3 ATLAS software release with a series of the progressive release dates before that A first Client Library release was completed on schedule The current focus is on testing tool suite development, integration/alignment with ATLAS/LCG software various coding rules, build tools and library documentation updates Then will add the next set of functionalities: new modules for server indirection, logging & debugging and credentials management LCG3D Meeting at Fermilab, October 21, 2004 Alexandre Vaniachine (ANL) We are interested in integration of ATLAS Client Library as a low-level part of 3D reference implemen- tation Integration Proposal