iwmw 1997: www caching
TRANSCRIPT
![Page 2: IWMW 1997: WWW Caching](https://reader036.vdocuments.us/reader036/viewer/2022070509/58a1f1dd1a28ab531e8b6f17/html5/thumbnails/2.jpg)
Overview of Presentation
• Why caching?• Caching Infrastructures.• National Caching.• Caching hardware and software• Implementation of caching• Non-Technical Issues
![Page 3: IWMW 1997: WWW Caching](https://reader036.vdocuments.us/reader036/viewer/2022070509/58a1f1dd1a28ab531e8b6f17/html5/thumbnails/3.jpg)
Why Caching?
• 1000s of users ‘surfing’ the Internet each with their own browser.
• Users and browsers are ‘independant’ resulting in a large amount of replication of information carried over the network.
• Popular Web sites may have many simultaneous connections transmitting identical copies of a single item over the same network trunk routes. This state of affairs is highly undesirable because...
![Page 4: IWMW 1997: WWW Caching](https://reader036.vdocuments.us/reader036/viewer/2022070509/58a1f1dd1a28ab531e8b6f17/html5/thumbnails/4.jpg)
Why Caching?
• Bandwidth - especially international bandwidth - is very expensive, and must be used cost-effectively.
• Web ‘hot-spots’ are created.
• Web object retrieval times are increased.
![Page 5: IWMW 1997: WWW Caching](https://reader036.vdocuments.us/reader036/viewer/2022070509/58a1f1dd1a28ab531e8b6f17/html5/thumbnails/5.jpg)
Why Caching?
• Caching, or Web Caches are an attempt to:
– Minimise bandwidth wastage.
– Decrease object retrieval times.
– reduce number of ‘Hot-Spots’
![Page 6: IWMW 1997: WWW Caching](https://reader036.vdocuments.us/reader036/viewer/2022070509/58a1f1dd1a28ab531e8b6f17/html5/thumbnails/6.jpg)
Caching Infrastructures
• Caches may be implemented:
– Within departments– Within Institutions– Nationally– Internationally
• Caches can co-operate. So we have meshes of caches or caching infrastructures.
![Page 7: IWMW 1997: WWW Caching](https://reader036.vdocuments.us/reader036/viewer/2022070509/58a1f1dd1a28ab531e8b6f17/html5/thumbnails/7.jpg)
Caching Infrastructures
• Caching infrastructures are developing at every level.
– Quite a few departmental caches.– Many Institutions now operate
caches.– Within the UK a National caching
infrastructure is developing.– International infrastructures in
place and developing.
![Page 8: IWMW 1997: WWW Caching](https://reader036.vdocuments.us/reader036/viewer/2022070509/58a1f1dd1a28ab531e8b6f17/html5/thumbnails/8.jpg)
Caching Infrastructures
• Cooperation between caches.– Achieved by the ICP cache
communication protocol in one of two modes:
• Unicast mode - individual connections established to interrogate caches.
• Multicast mode - an ICP multicast packet is ‘multicast’ to a group of cooperating caches.
– Intuitively the multicast approach should be more efficient - reduce bandwidth, etc.
![Page 9: IWMW 1997: WWW Caching](https://reader036.vdocuments.us/reader036/viewer/2022070509/58a1f1dd1a28ab531e8b6f17/html5/thumbnails/9.jpg)
Caching Infrastructures
• For example at Manchester:
– Central campus cache and several departmental caches use it in unicast mode.
– Parent relationships with other caches in the UK, Europe and USA.
![Page 10: IWMW 1997: WWW Caching](https://reader036.vdocuments.us/reader036/viewer/2022070509/58a1f1dd1a28ab531e8b6f17/html5/thumbnails/10.jpg)
National Caching
• HENSA pioneered caching with their Public Caching Proxy Server. Initiated around 1992.– Used Lagoon initially– Then the CERN server– Then Netscape Proxy– And some Squid
• Details described at First International WWW Conference: http://www.hensa.ac.uk/www94
![Page 11: IWMW 1997: WWW Caching](https://reader036.vdocuments.us/reader036/viewer/2022070509/58a1f1dd1a28ab531e8b6f17/html5/thumbnails/11.jpg)
National Caching
• The existing service is hosted by University of Kent at Canterbury and University of Leeds.
• From 1st August 1997 it will be hosted by the University of Manchester and Loughborough University.
• Selection by a recent competitive tendering process.
![Page 12: IWMW 1997: WWW Caching](https://reader036.vdocuments.us/reader036/viewer/2022070509/58a1f1dd1a28ab531e8b6f17/html5/thumbnails/12.jpg)
National Caching
• The situation so far.
– Service still at HENSA and Leeds. We are preparing for the transition.
– Initially exisiting equipment will be used.
– Projection of demand performed and hardware upgrade path budgeted for.
![Page 13: IWMW 1997: WWW Caching](https://reader036.vdocuments.us/reader036/viewer/2022070509/58a1f1dd1a28ab531e8b6f17/html5/thumbnails/13.jpg)
National Caching
• The ‘new’ service will have:– a service ‘arm’– a development ‘arm’
• The National service will be directed by a steering committee and will be, as far as possible, user driven.
• National Caching Web site, regular newsletter, mailing lists, help desk system, fault reporting mechanism, etc, etc.
![Page 14: IWMW 1997: WWW Caching](https://reader036.vdocuments.us/reader036/viewer/2022070509/58a1f1dd1a28ab531e8b6f17/html5/thumbnails/14.jpg)
Benefits of National Caching
• Trans-Atlantic bandwidth and bandwidth to Europe are both very expensive and in great demand. Caching reduces bandwidth consumption. The resulting cost savings can be used to fund other things.
• Faster document retrieval time - in theory!
![Page 15: IWMW 1997: WWW Caching](https://reader036.vdocuments.us/reader036/viewer/2022070509/58a1f1dd1a28ab531e8b6f17/html5/thumbnails/15.jpg)
National Caching - Useful addresses and URLs
• Email addresses:– [email protected]
general mailing list for users.– [email protected] mailing
list for Special Interest Group.– [email protected], National
Cache Joint Team mailing list.
• Some URLS:– http://www.hensa.ac.uk– http://www.net.lboro.ac.uk/caching/– http://www.mcc.ac.uk/Cache/
![Page 16: IWMW 1997: WWW Caching](https://reader036.vdocuments.us/reader036/viewer/2022070509/58a1f1dd1a28ab531e8b6f17/html5/thumbnails/16.jpg)
Caching Hardware
• Any Unix platform• Linux• FreeBSD
![Page 17: IWMW 1997: WWW Caching](https://reader036.vdocuments.us/reader036/viewer/2022070509/58a1f1dd1a28ab531e8b6f17/html5/thumbnails/17.jpg)
Caching Software
• Lagoon• CERN• Netscape• Harvest• Squid
![Page 18: IWMW 1997: WWW Caching](https://reader036.vdocuments.us/reader036/viewer/2022070509/58a1f1dd1a28ab531e8b6f17/html5/thumbnails/18.jpg)
Using Caches
• Users interact with caches directly using their favourite browser.
• Caches interact or co-operate with other caches using ICP.
• Browser - cache interaction is a ‘client-server’ type interaction.
![Page 19: IWMW 1997: WWW Caching](https://reader036.vdocuments.us/reader036/viewer/2022070509/58a1f1dd1a28ab531e8b6f17/html5/thumbnails/19.jpg)
Implementation -Browsers
• Netscape– Manual configuration - Select
network preferences from Options menu...
– Automatic configuration - proxy configuration can be automated with Javascript...
• Others: Lynx, Mosaic, Microsoft Internet Explorer.
![Page 20: IWMW 1997: WWW Caching](https://reader036.vdocuments.us/reader036/viewer/2022070509/58a1f1dd1a28ab531e8b6f17/html5/thumbnails/20.jpg)
Implementation - caches
• With reference to Squid– Installation– Configuration– Operations
• Some problems– disk space– discarding documents
![Page 21: IWMW 1997: WWW Caching](https://reader036.vdocuments.us/reader036/viewer/2022070509/58a1f1dd1a28ab531e8b6f17/html5/thumbnails/21.jpg)
Implementation -Installation
• Retrieve from: – http://squid.nlanr.net/Squid/– Decompress and extract.– configure– compile – install
• Operating Systems– Unix, AIX, FreeBSD, HP-UX,
IRIX, Linux, OSF/1, Solaris, SunOS
![Page 22: IWMW 1997: WWW Caching](https://reader036.vdocuments.us/reader036/viewer/2022070509/58a1f1dd1a28ab531e8b6f17/html5/thumbnails/22.jpg)
Implementation - Configuration
• Configuration file– http_port– icp_poty– mcast_groups– Cache_host– cache_host_domain– cache_swap– cache_swap_low– cache_swap_high– cache_dir– cache_access_log
![Page 23: IWMW 1997: WWW Caching](https://reader036.vdocuments.us/reader036/viewer/2022070509/58a1f1dd1a28ab531e8b6f17/html5/thumbnails/23.jpg)
Implemetation - configuration
• Configuration file continued...– pid_filename– debug_options– ftpget_program– negative_ttl
• Access Control lists– http_access allow– htp_access deny
![Page 24: IWMW 1997: WWW Caching](https://reader036.vdocuments.us/reader036/viewer/2022070509/58a1f1dd1a28ab531e8b6f17/html5/thumbnails/24.jpg)
Implemetation - configuration
• Administration parameters– cache_mgr– cache_announce– logfile_rotate– minimum_direct_hops– and so on...
![Page 25: IWMW 1997: WWW Caching](https://reader036.vdocuments.us/reader036/viewer/2022070509/58a1f1dd1a28ab531e8b6f17/html5/thumbnails/25.jpg)
Operation
• Parent or sibling?• Log files• Statistics• Number of requests per day• Machine loading• Network loading• Improvement in latency?• Reduction in bandwidth usage?
![Page 26: IWMW 1997: WWW Caching](https://reader036.vdocuments.us/reader036/viewer/2022070509/58a1f1dd1a28ab531e8b6f17/html5/thumbnails/26.jpg)
Other Issues
• Copyright• Pornography• Log statistsics• Data protection act.
![Page 27: IWMW 1997: WWW Caching](https://reader036.vdocuments.us/reader036/viewer/2022070509/58a1f1dd1a28ab531e8b6f17/html5/thumbnails/27.jpg)
Should I run a cache?
• Should I run a:– Departmental cache?– Institutional cache?
• Should I link together departmental caches?
• Should I link departmental caches to my Institutional cache?
• Should I link my institutional cache to the National Cache?
![Page 28: IWMW 1997: WWW Caching](https://reader036.vdocuments.us/reader036/viewer/2022070509/58a1f1dd1a28ab531e8b6f17/html5/thumbnails/28.jpg)
Should I run a cache?
• There are no hard and fast rules. Clearly caching saves bandwidth and improves latency, but it is not obvious how best to construct a hierarchy to achieve this.
• We are are at the learning stage. Part of the remit of the National Web Network Caching Service will be to investigate this and produce guidelines and recommendations for individual sites.
![Page 29: IWMW 1997: WWW Caching](https://reader036.vdocuments.us/reader036/viewer/2022070509/58a1f1dd1a28ab531e8b6f17/html5/thumbnails/29.jpg)
Should I run a cache?
• The answer is yes!• Consider
– number of users– Type of work– Local Area Network
• loading• Bottlenecks
– Expected demand• Analyse statistics
![Page 30: IWMW 1997: WWW Caching](https://reader036.vdocuments.us/reader036/viewer/2022070509/58a1f1dd1a28ab531e8b6f17/html5/thumbnails/30.jpg)
Futures
• The National WWW Network Caching Service will be involved in the development of caching in the UK. Will investigate hardware and software. Findings will be published on the National Cache Web site:
URL: http://www.wwwcache.ac.uk