frontier: high performance database access using standard web components in a scalable multi-tier...
Post on 12-Jan-2016
213 Views
Preview:
TRANSCRIPT
FroNtier: FroNtier: High Performance Database Access High Performance Database Access Using Standard Web Components in a Using Standard Web Components in a
Scalable Multi-tier ArchitectureScalable Multi-tier Architecture
Marc PaternoMarc Paterno
FermilabFermilab
CHEP 2004 CHEP 2004 Sept. 27-Oct. 1, 2004 Sept. 27-Oct. 1, 2004
Interlaken SwitzerlandInterlaken Switzerland
2
CreditsCredits
Fermilab, Batavia, IllinoisFermilab, Batavia, IllinoisJim Kowalkowski, Sergey Kosyakov, Dmitri Litvintsev, Lee Jim Kowalkowski, Sergey Kosyakov, Dmitri Litvintsev, Lee
Lueking, Marc Paterno, Stephen WhiteLueking, Marc Paterno, Stephen White
Johns Hopkins University, Baltimore, MarylandJohns Hopkins University, Baltimore, MarylandBarry Blumenfeld, Petar Maksimovic, Mark MathisBarry Blumenfeld, Petar Maksimovic, Mark Mathis
Rovaniemi Polytechnic, Rovaniemi, FinlandRovaniemi Polytechnic, Rovaniemi, FinlandLauri Autio Lauri Autio
(through collaboration with Fermilab(through collaboration with Fermilaband The University of Helsinki)and The University of Helsinki)
3
OutlineOutline
MotivationMotivation Overview Overview TestingTesting DeploymentDeployment Status and PlansStatus and Plans
4
MotivationMotivation
CDF wasCDF was– concerned about the scalability of their current database concerned about the scalability of their current database
access layeraccess layer
– looking for a straightforward path to move to a multi-looking for a straightforward path to move to a multi-tier data delivery architecture using commodity toolstier data delivery architecture using commodity tools
Why a multi-tier approach with caching?Why a multi-tier approach with caching?– Ideal for write once, read many data Ideal for write once, read many data
– Reliability and high performance w/o expensive DB Reliability and high performance w/o expensive DB serversservers
– Readily scalable to serve thousands of clientsReadily scalable to serve thousands of clients
– Decouples client and database development and Decouples client and database development and maintenancemaintenance
5
Transport ProtocolTransport Protocol
Several choices for transport protocol were Several choices for transport protocol were possiblepossible– many projects use home-grown protocols, based on many projects use home-grown protocols, based on
bare socketsbare sockets
The use of HTTP for the server-client protocol The use of HTTP for the server-client protocol provides a starting pointprovides a starting point– Opens the door to many well understood commodity Opens the door to many well understood commodity
tools and techniques used routinely on the internet.tools and techniques used routinely on the internet.
– These tools are easily deployed and have low These tools are easily deployed and have low administrative overheads.administrative overheads.
6
A Multi-tier ArchitectureA Multi-tier Architecture
Database (Oracle, MySQL, etc.)Database (Oracle, MySQL, etc.) Database Access LayerDatabase Access Layer
– Tomcat: servlet management engineTomcat: servlet management engine
– JDBC: Database connection JDBC: Database connection
– Database Connection pool management Database Connection pool management
Caching and proxy layer: SquidCaching and proxy layer: Squid– widely used, highly configurable, widely used, highly configurable,
caching proxy servercaching proxy server
ClientClient– Client API provides simple interface packaged in a Client API provides simple interface packaged in a
compact client librarycompact client library
7
FroNtier OverviewFroNtier Overview
CDF Persistent Object Templates(Java)
FroNtier components in yellow
Client
Caching
FroNtierServer
Database
FroNtier Client API Library
Squid Proxy/Caching Server
FroNtier Servlet running under Tomcat
Database (or other
persistency service)
XML Server Descriptors
DDL for Table Descriptions
C++ Header and Stubs
JDBC
HTTP
HTTP
8
FroNtier Servlet DesignFroNtier Servlet Design
1.1. Client sends request (URI)Client sends request (URI)
2.2. Command ParserCommand Parser translates translates URI into commands + URI into commands + valuesvalues
3.3. Servicer FactoryServicer Factory gets XSD gets XSD (XML Server Description) (XML Server Description) from database and from database and
4.4. Instantiate aInstantiate a Servicer Servicer
5.5. Servicer queries database Servicer queries database andand
6.6. Results sent for encodingResults sent for encoding
7.7. EncoderEncoder marshals marshals (serializes) the data to (serializes) the data to requestor clientrequestor client DatabaseDatabase
CommandCommandParserParser
ServicerServicerFactoryFactory
ServicerServicer
EncoderEncoder
ClientClient
1
2
3
4
5
6
7
9
FroNtier XML Server DescriptorFroNtier XML Server Descriptor
Object name and Object name and version informationversion information
Response Response descriptiondescription
The SQL mapping to The SQL mapping to the databasethe database
– Select statementSelect statement
– From statementFrom statement
– Where clauseWhere clause
– Special modifiers Special modifiers ((order byorder by, etc), etc)
<descriptor type="CalibRunLists“ version="1" xsdversion="1"><attribute position="1“ type="int“ field="calib_run" /> <attribute position="2" type="int“ field="calib_version" /> <attribute position="3" type="string“ field="data_status" /> <select> calib_run, calib_version, data_status </select><from> CalibRunLists </from> <where> <clause> cid = @param </clause> <param position="1" type="int“ key="cid"/> </where><final> </final>
</descriptor>
10
FroNtier use of Squid CacheFroNtier use of Squid Cache
Squid (v 2.5STABLESquid (v 2.5STABLEnn) – Open source) – Open source– Well documented, widespread operational experienceWell documented, widespread operational experience– Easily installed and maintainedEasily installed and maintained– Highly configurable for access control, disk cache tuning, Highly configurable for access control, disk cache tuning,
distributed cache peer relationships,…distributed cache peer relationships,…– Monitoring built in through SNMP-2 interfaceMonitoring built in through SNMP-2 interface
Refreshing the cache (3 options)Refreshing the cache (3 options)– Driven by servlet: expiration time sent in HTTP headerDriven by servlet: expiration time sent in HTTP header– Driven by client: forced object refresh through requestDriven by client: forced object refresh through request– Delete each Squid’s cache files and rebuild the cacheDelete each Squid’s cache files and rebuild the cache
However, the objects being delivered are generally However, the objects being delivered are generally not changing, so a static cache matches the not changing, so a static cache matches the requirements.requirements.
11
FroNtier client API featuresFroNtier client API features
C and C++ interfacesC and C++ interfaces PortablePortable
– 32 and 64 bit systems tested32 and 64 bit systems tested
Transparent object accessTransparent object access– Type conversion detectionType conversion detection
– Preserves data integrityPreserves data integrity
Multi-object requestsMulti-object requests Easy runtime configurationEasy runtime configuration Extensive error reportingExtensive error reporting
– Adjustable logsAdjustable logs
FroNtier Service
User application
FroNtier API
12
General Deployment OverviewGeneral Deployment Overview
Dedicated Squid Node
Tomcat Node 1
Tomcat Node N
Tomcat Node 2
CAF 1Processor Farm
Database
Dedicated Squid Node
CAF NProcessor Farm
Dedicated Squid Node
CAF 2Processor Farm
Squid Node N
SquidNode 2
Cache Layer local to each CAFGbit ethernit
Intermediate Cache Layer
DB Access Layer
SquidNode 1
Load Balancing and Failover
LaunchpadWANWAN
14
Client Side: FNAL/SDSCClient Side: FNAL/SDSC
No CachingCaching at SDSC Squid
1000 Sec 1 Sec
16
FroNtier Status and PlansFroNtier Status and Plans
CDF Production readyCDF Production ready– Client library is included in CDF production code Client library is included in CDF production code
– Production executables built and tested with FroNtier.Production executables built and tested with FroNtier.
– Widespread use will begin as new releases are builtWidespread use will begin as new releases are built
Plan to extend the use in CDF beyond calibration Plan to extend the use in CDF beyond calibration data to trigger and other static data needed offline.data to trigger and other static data needed offline.
Exploring a prototype of a FroNtier “plug-in” for Exploring a prototype of a FroNtier “plug-in” for POOL (LCG). Some interest in CMS to use a POOL (LCG). Some interest in CMS to use a web-based approach for DB access. web-based approach for DB access.
17
SummarySummary
FroNtier is a multi-tier architecture providing FroNtier is a multi-tier architecture providing clients high throughput, low latency, scalable clients high throughput, low latency, scalable access to a persistent store, such as a database. access to a persistent store, such as a database.
The CDF DB access framework has been adapted The CDF DB access framework has been adapted to use the FroNtier approach. Extensive testing to use the FroNtier approach. Extensive testing was performed to confirm the integrity, was performed to confirm the integrity, performance, and reliability of the system.performance, and reliability of the system.
We are in the process of executing a deployment We are in the process of executing a deployment plan and will soon have real-world production plan and will soon have real-world production experience with the system.experience with the system.
18
ReferencesReferences
FroNtier Talks:FroNtier Talks:– http://whcdf03/ntier-wiki/Additional_20Documentationhttp://whcdf03/ntier-wiki/Additional_20Documentation
FroNtier working page:FroNtier working page:– http://whcdf03/ntier-wikihttp://whcdf03/ntier-wiki
top related