data mirrors: techniques and issues by sarah ponrathnam [email protected]
TRANSCRIPT
Data mirrors: Data mirrors: Techniques and IssuesTechniques and Issues
Data mirrors: Data mirrors: Techniques and IssuesTechniques and Issues
ByBy
Sarah PonrathnamSarah Ponrathnam
[email protected]@iucaa.ernet.in
http://www.iucaa.ernet.inhttp://www.iucaa.ernet.in//
Overview Introduction…What is Data Mirroring?Techniques of Mirroring…Motives for Mirroring…Mirror site Issues…Popular Mirroring Tools in Unix…Mirroring Experience at IUCAA…
What is data mirroring?
Creating an exact duplicate copy in real-time.
In terms of web sites, sites are often mirrored to reduce the traffic on one server.
Techniques of MirroringReplication is one way to solve
availabilityproblem.
Distributed Servers Cluster Servers Web site Mirrors
Techniques of Mirroring Distributed Servers:Large web destinations such as google,Yahoo have enough capital to set upand to support distributed servers.Eg. www.google.com, www.google.co.in
Techniques of MirroringCluster servers:
Techniques of Mirroring Website Mirrors:Mirrors simply compare and pull the contentsfrom a single master web site at a regular intervals and make identical contentsavailable on another computer, ideally closerto the users making use of it.
Motives for Mirroring Load Balancing - yahoo.co.uk yahoo.co.inHigh Availability - ADS mirrorsMultilingual replication - www.debian.org
Database Sharing – www.desert.net www.tucksonweekly.com
Franchise / Local Versions - quicken.excite.com quicken.com Virtual Hosting - sports.catalogue.com
www.accesports.com
Mirroring IssuesMaintenance of mirror-sites in different
geographical locations. Integrity of the mirrored contents Providing Host-Independent URLS. Initial transfer of data
Optimization Economical constraintsProviding Efficient routing
Popular Mirroring tools on Unix
RSync http://samba.anu.edu.au/rsync/Wget http://www.gnu.org/Mirror http://www.wehlus.de/mirror/download.html
Mirroring ExperienceIUCAA (Inter University CentreFor Astronomy and Astrophysics)works closely with ERNET(Educational and ResearchNETwork) to make this network a content orientednetworkhttp://www.iucaa.ernet.in/
ERNET India ? In 1986, then Department of Electronics
had initiated a project "ERNET" with thefunding from UNDP. The objective was toestablish and operate a nationwideInternet for the Indian academic andresearch community. Now it has become a full pledged ISP known as ERNET India.
ERNET partners
ERNET Network
ERNET NOC at IUCAA
Mirrors available in IUCAA
VizieR provides access to the most complete library of published astronomical catalogues and data tables available on line, organized in a self-documented database. http://urania.iucaa.ernet.in
Mirrors available in IUCAA
The NASA Astrophysics Data System (ADS) maintains four bibliographic databases containing more than 4 million records. http://ads.iucaa.ernet.in
VOI Data Archives at IUCAA
SDSS Sloan Digital Sky Survey – 1 Tb2MASS 2 Micron All Sky Survey – 194 Gb2dfGRS 2 degree field Galaxy Redshift
Survey – 5.5 Gb2QZ 2 Degree field QSO Survey – 630
Mb FIRST Survey Faint Images of Radio Sky
at Twenty centimeters – 226 Gb
VOI Hardware
Mirror-site under construction
Chandra Data Archive (CDA) 424 GB data is available through ftp
service. ftp://cdaftp.iucaa.ernet.in
Incremental update is being done regularly.
Web based CDA service will be provided, once the new release of CXCDS software is made available.
AcknowledgementDr. Francois Ochsenbein - VizieRDr. Guenther Eichhorn and Dr.Alberto
Accomazzi - ADSDr. Ramadurai Padmanabhan - CDA
Questions
or
Comments?