digir1 distributed databases and applications john wieczorek museum of vertebrate zoology, uc...
Post on 31-Mar-2015
216 Views
Preview:
TRANSCRIPT
DiGIR 1
Distributed Databases and Applications
John Wieczorek
Museum of Vertebrate Zoology, UC Berkeley
DiGIR 2
Distributed Databases Multiple sources of data …under local control, …with concepts in common …and a desire to deliver data as part of a
community.
DiGIR 3
Distributed Databases The Species Analyst (TSA) The Integrated Taxonomic Information System (ITIS) FishNet The Mammal Networked Information System (MaNIS) HerpNET The Ornithological Information System (ORNIS) …
DiGIR 4
Distributed Databases European Natural History Science Information
Network (ENHSIN) Biological Collection Access for Europe (
BioCASE) Australia Virtual Herbarium (AVH) Red Mundial de Información Sobre
Biodiversidad, Comisión Nacional para el Conocimiento y Uso de la Biodiversidad (REMIB, CONABIO)
…
DiGIR 5
Distributed Databases Mountain and Plains Spatio-Temporal Database-
Informatics (MaPSTeDI) Ocean Biogeographic Information System (OBIS) Pacific Basin Information Node, National Biological
Information Infrastructure (PBIN, NBII) Species Link, Centro de Referência em Informação
Ambiental (Species Link, CRIA) A Virtual Herbarium of the Chicago Region (vPlants) Spatial Analysis of Local Vegetation Inventories Across
Scales (SALVIAS) …
DiGIR 6
Distributed Databases Berkeley Natural History Museums (BNHM) Association of Biological Collections, UC Davis …
DiGIR 7
Distributed Databases LifeMapper Global Biodiversity Information Facility (GBIF)
DiGIR 8
Distributed vs. centralized Multiple sources of data …under local control, …with concepts in common …and a desire to deliver data as part of a
community
DiGIR 9
Distributed vs. centralized
In other words, distribute the headache rather than have one central migraine.
DiGIR 10
DiGIRDistributed Generic Information Retrieval
John Wieczorek, Stan Blum, Dave Vieglais, P.J. Schwartz
DiGIR 11
Project Rationale To avoid multiple incongruous
development efforts To pool resources and create a community
of experts To solve the problem of scalability
DiGIR 12
Project Goals To define a protocol for retrieving
structured data from multiple, heterogeneous databases across the Internet
To build a reference implementation of both provider and portal software using said protocol
DiGIR 13
Design Goals To use open protocols and standards, such
as HTTP and XML To decouple the protocol, software and
semantics To make new data provider installations as
easy as possible To have open source development and
GNU General Public Licensing
DiGIR 14
DiGIR ArchitectureUser InterfaceProtocolPortal EngineProvider
DiGIR 15
DiGIR ArchitectureProvider
DiGIR 16
DiGIR ArchitectureProviderRegistry
DiGIR 17
DiGIR ArchitecturePortal Engine
DiGIR 18
DiGIR ArchitecturePortal EngineRegistry
DiGIR 19
DiGIR ArchitectureUser Interface
DiGIR 20
DiGIR ArchitectureUser InterfaceProtocolPortal Engine
DiGIR 21
DiGIR ArchitectureUser InterfaceProtocolPortal EngineProtocolProvider
DiGIR 22
DiGIR ArchitectureUser InterfaceProtocolPortal EngineProtocolProvider
DiGIR 23
DiGIR ArchitectureUser InterfaceProtocolPortal Engine
DiGIR 24
DiGIR Component Summary
DiGIR 25
DiGIR Protocol Defines request and response message
formats for communication between provider, portal engine, and user interfaces Metadata requests Search requests Inventory requests
Remains unfettered by the structure of the data it transfers
DiGIR 26
Portal Engine The entry point for a “user” Can query a registry for
potential providers Can determine, based on
provider metadata, whether a provider should be queried
Can send requests to multiple providers
Communicates via protocol compliant messaging only
DiGIR 27
Portal Engine, continued Assembles responses
from providers Returns packaged results
to the “user” Logs activity
DiGIR 28
Provider Receives requests Retrieves data from database Sends results to requestor Supplies metadata to describe
data classification and availability
Logs requests
DiGIR 29
Registry Supports provider
“advertising” May be global and open May be private Need not be used at all Example: Universal
Description, Discovery and Integration (UDDI)
DiGIR 30
User Interfaces Must be able to assemble and
send a request document to a portal
Must be able to receive and interpret a response document from the portal
This is where the real fun is!
DiGIR 31
Example Network Configurations
DiGIR 32
BNHM Network Configuration
PHMAWorking
Database
OnlineDatabase
UCBGWorking
Database
DiGIRProvider
BNHMDiGIRPortal
UCJEPSWorking
Database
OnlineDatabase
UCMPWorking
Databases (4)
OnlineDatabase
EssigWorking
Database
OnlineDatabase
OnlineDatabase
BNHMPresentation
Layer
DiGIR 33
MaNIS Network Configuration
WorkingDatabase
OnlineDatabase
WorkingDatabase
DiGIRProvider
MaNISDiGIRPortal
WorkingDatabase
OnlineDatabase
WorkingDatabase
OnlineDatabase
WorkingDatabase
OnlineDatabase
OnlineDatabase
MaNISPresentation
Layer
DiGIRProvider
MaNISDiGIRPortal
MaNISPresentation
Layer
DiGIRProvider
MaNISDiGIRPortal
MaNISPresentation
Layer
DiGIRProvider
DiGIRProvider
DiGIR 34
MaNIS Network Configuration
LACMMS AccessDatabase
OnlineMS AccessDatabase
MVZSybase
Database
MaNISDiGIRPortal
TTUFoxPro
Database
OnlineMS AccessDatabase
UWBM4D-Mac
Database
OnlineMS AccessDatabase
CASSQL ServerDatabase
OnlineSQL ServerDatabase
OnlineMS AccessDatabase
MaNISDiGIRPortal
MaNISDiGIRPortal
MVZ-MaNISPresentation
Layer
LACM-MaNISPresentation
Layer
UWBM-MaNISPresentation
Layer
DiGIRProvider
DiGIRProvider
DiGIRProvider
DiGIRProvider
DiGIRProvider
DiGIR 35
MaNIS Network Configuration
LACMMS AccessDatabase
OnlineMS AccessDatabase
MVZSybase
Database
MaNISDiGIRPortal
TTUFoxPro
Database
OnlineMS AccessDatabase
UWBM4D-Mac
Database
OnlineMS AccessDatabase
CASSQL ServerDatabase
OnlineSQL ServerDatabase
OnlineMS AccessDatabase
MaNISDiGIRPortal
MaNISDiGIRPortal
MVZ-MaNISPresentation
Layer
LACM-MaNISPresentation
Layer
UWBM-MaNISPresentation
Layer
DiGIRProvider
DiGIRProvider
DiGIRProvider
DiGIRProvider
DiGIRProvider
DiGIR 36
MaNIS Network Configuration
LACMMS AccessDatabase
OnlineMS AccessDatabase
MVZSybase
Database
MaNISDiGIRPortal
TTUFoxPro
Database
OnlineMS AccessDatabase
UWBM4D-Mac
Database
OnlineMS AccessDatabase
CASSQL ServerDatabase
OnlineSQL ServerDatabase
OnlineMS AccessDatabase
MaNISDiGIRPortal
MaNISDiGIRPortal
MVZ-MaNISPresentation
Layer
LACM-MaNISPresentation
Layer
UWBM-MaNISPresentation
Layer
DiGIRProvider
DiGIRProvider
DiGIRProvider
DiGIRProvider
DiGIRProvider
DiGIR 37
MaNIS Network Configuration
LACMMS AccessDatabase
OnlineMS AccessDatabase
MVZSybase
Database
MaNISDiGIRPortal
TTUFoxPro
Database
OnlineMS AccessDatabase
UWBM4D-Mac
Database
OnlineMS AccessDatabase
CASSQL ServerDatabase
OnlineSQL ServerDatabase
OnlineMS AccessDatabase
MaNISDiGIRPortal
MaNISDiGIRPortal
MVZ-MaNISPresentation
Layer
LACM-MaNISPresentation
Layer
UWBM-MaNISPresentation
Layer
DiGIRProvider
DiGIRProvider
DiGIRProvider
DiGIRProvider
DiGIRProvider
DiGIR 38
MaNIS Network Configuration
LACMMS AccessDatabase
OnlineMS AccessDatabase
MVZSybase
Database
MaNISDiGIRPortal
TTUFoxPro
Database
OnlineMS AccessDatabase
UWBM4D-Mac
Database
OnlineMS AccessDatabase
CASSQL ServerDatabase
OnlineSQL ServerDatabase
OnlineMS AccessDatabase
MaNISDiGIRPortal
MaNISDiGIRPortal
MVZ-MaNISPresentation
Layer
LACM-MaNISPresentation
Layer
UWBM-MaNISPresentation
Layer
DiGIRProvider
DiGIRProvider
DiGIRProvider
DiGIRProvider
DiGIRProvider
DiGIR 39
Other Network Configurations
WorkingDatabase
OnlineDatabase
WorkingDatabase
DiGIRProvider
DiGIRProvider
DiGIRPortal
WorkingDatabase
OnlineDatabase
DiGIRProvider
WorkingDatabase
OnlineDatabase
DiGIRPortal
WorkingDatabase
OnlineDatabase
DiGIRProvider
DiGIRPortal
DiGIR 40
DiGing a little deeper
DiGIR 41
Provider Installation Web server (Apache, IIS, etc.) PHP: Hypertext Preprocessor
(PHP) Provider software (DiGIR)
Configuration tool Testing scripts Provider scripts Provider manual (DiGIR)
DiGIR 42
Provider Configuration Tool Provider metadata Resources Database connection Establishing table
relationships Concept to column (i.e.,
field, attribute) mapping
DiGIR 43
Portal Configuration Web server (Apache, IIS, etc.) Sun Java 2 (JDK 1.4) Tomcat (Apache) Portal software (DiGIR) Portal installation
documentation (DiGIR)
DiGIR 44
Portal Installation Engine configuration file
(finding providers) Presentation configuration
file (defining the Information Domain)
Presentation customization Engine start and stop scripts Presentation start and stop
scripts
DiGIR 45
Portal Demonstrations
DiGIR 46
DiGIR Project Information The DiGIR project is a collaborative effort DiGIR is currently established as an open
source development project on SourceForge (https://sourceforge.net/projects/digir).
Further documentation is available on the DiGIR web site (http://digir.net).
top related