international directory network (idn) scalability, security and interoperability wgiss, 2006 tom...
Post on 03-Jan-2016
227 Views
Preview:
TRANSCRIPT
International Directory Network (IDN)Scalability, Security and Interoperability
WGISS, 2006
Tom NorthcuttSystems Administrator: GCMD
September 13, 2006
0
1000000
2000000
3000000
4000000
5000000
6000000
7000000
8000000
Jan-03
Mar-03
May-03
Jul-03
Sep-03
Nov-03
Jan-04
Mar-04
May-04
Jul-04
Sep-04
Nov-04
Jan-05
Mar-05
May-05
Jul-05
Sep-05
Nov-05
Jan-06
Mar-06
month
#hits
# GCMD/IDN Web Page Hits Since January 2003
Introduction of the new web page
Cache opened to InternetSearch robots
FreetextFreetext (Lucene) Database Index Layer
SpatialSpatial Database Index Layer
Controlled VocabularyControlled Vocabulary Database Layer joinsdelegates
ControllerController
User performs query1)
2)
3)
Middleware Search/Retrieval Component(Integration of spatial, freetext, and controlled queries)
4)
Set of unique IDs Result Result ProcessorProcessor
Multi-Layer Search ComponentMulti-Layer Search Component
5)
Title set information, brief summary, dataset links, etc.
6) Returns to user
SearchSearchResultsResults
Cache7) Refines search
Scalability: Core GCMD/IDN Architecture
Complexity:“it is hard to make things look easy.”
– These are complex queries, with very fast search results. – Another example: data resolution refinement
● difficult to implement● Makes it easier for the user
Scalability
● Conventional clustering approach– Load balancing– High availability– (source: redhat.com)
Scalability: GCMD/IDN ImplementationStateful, Web Proxy Based Clusters
http://gcmd.nasa.gov/Keywords.do?...&lbnode=2
http://gcmd.nasa.gov/DocumentBuilder/...
AcceleratedCaching
Scalability: Extensibility of Stateful Web Proxy Clusters
http://gcmd.nasa.gov/OAI-script? ...
http://gcmd.nasa.gov/ontology.wsdl
http://gcmd.nasa.gov/soap/httphttp://gcmd.nasa.gov/xml-rpc
http://gcmd.nasa.gov/ajax/some.jsp
Harvester
SOAP
XML-RPC
AJAX
Scalability: Stateful Web Proxy Clusters
How we implemented this architecture:– Modified version of Squid proxy server– Custom perl scripts to implement state and redirection– Dynamic query caching done on the server end so each
refinement uses cached results
Scalability:Advantages of Web Proxy Clusters for CEOS Partners● Accelerated Caching● Load balance nodes● Stateful architecture ● Open source
● Multiple uses:– Web services– Browse imagery– Metadata search– Data access and retrieval
Scalability:Google Map
● Utility:– Google map is a form of spreading the load– Utilize third party resources for map generation– Google’s resources are distributed globally
Security:Transparent Bridge Filters
BRIDGE
Firewall Network monitoringPort remappingIntrusion detection
BRIDGE
Internal FirewallingNetwork monitoring
BRIDGEProject SegregationNetwork protection
Security:Transparent Bridge AdvantagesApplicability for CEOS Partners
● Applicable to heterogeneous environments– Unobtrusive
● No changes needed on servers or network controllers● May assist with GRID network security requirements
– Ultra secure: invisible at the IP level ● Implements emerging security policies
– Two factor authentication– Efficient encryption, authentication– Port knocking capabilities
● Open source– Non-proprietary– Universal
Conclusion
● IDN continues to grow in popularity– Users – Earth science partnerships
● The system continues to develop – Scalability– Security– Usability– Interoperability
top related