grid content evolution & management
DESCRIPTION
Grid Content Evolution & Management. Institute for Remote Sensing Technology Innovation Management and Technology Marketing. Grid Content Evolution & Management: Enabling e-Science for the Rest of Us. Thilo Ernst - PowerPoint PPT PresentationTRANSCRIPT
Grid Content Evolution & Management
Institute for Remote Sensing Technology
Innovation Management and Technology Marketing
Grid Content Evolution & Management:
Enabling e-Science for the Rest of Us
Thilo Ernst
Fraunhofer Institute for Computer Architecture and Software Technology ([email protected])
Jochen Wauer
DLR Institute for Remote Sensing Technology
Institute for Remote Sensing Technology
Innovation Management and Technology Marketing
Seite 3Arc
hiv
ieru
ng
san
gab
en
Institute for Remote Sensing Technology
Innovation Management and Technology Marketing
Scientific Software Just A Mouse Click Away (1)
A 1968 vision:
... Back to our hypothetical user. He seats himself at his console [...]. He dials his local computer and 'logs in' by presenting his name, problem number, and password to the monitor program. He calls for either a public program, one of his own programs, or a colleague's program that he has permission to use. The monitor links him to it, and he then communicates with that program...
- J.C.R. Licklider, "The Computer as a Communication Device" (Director, ARPA-Information Processing Techniques Office)
Did he just mean telnet?
Grid Content Evolution & Management - e-Science For The Rest Of Us T.Ernst, J.Wauer
Seite 4Arc
hiv
ieru
ng
san
gab
en
Institute for Remote Sensing Technology
Innovation Management and Technology Marketing
e-Science – a contemporary Definition• „e-Science is about global collaboration in key areas of science and the
next generation of infrastructure that will enable it.“
• „large scale science that will increasingly be carried out through distributed global collaborations enabled by the Internet. Typically, a feature of such collaborative scientific enterprises is that they will require access to very large data collections, very large scale computing resources and high performance visualisation“
• „The Grid is an architecture proposed to bring all these issues together and make a reality of such a vision for e-Science“
• „e-Science will change the dynamic of the way science is undertaken“ (*)
(Source: UK Research Council e-Science Core Programme)
Grid Content Evolution & Management - e-Science For The Rest Of Us T.Ernst, J.Wauer
Seite 5Arc
hiv
ieru
ng
san
gab
en
Institute for Remote Sensing Technology
Innovation Management and Technology Marketing
The entire story?
• Grid computing / e-Science still focus on high-demand, large-scale applications
• Evolution: high performance computing Grid computing e-Science renders an emphasis on such applications understandable
• But when mature Grid & e-Science technologies are available they will likely be attractive to, a wealth of smaller (and computationally less demanding) research efforts - e-Science will become ubiquitous (*)
• But for this to happen, these technologies must develop additional characteristics unrelated to the original HPC focus of the Grid movement
Grid Content Evolution & Management - e-Science For The Rest Of Us T.Ernst, J.Wauer
Seite 6Arc
hiv
ieru
ng
san
gab
en
Institute for Remote Sensing Technology
Innovation Management and Technology Marketing
Scientific Software Just A Mouse Click Away (2)2003: Are we there?
• Internet & WWW are ubiquitious and have revolutionized the exchange of scientific information
• But progress w.r.t. scientific software lacks behind when compared with the ease of accessing scientific documents today
• „Reuse by download“: requires installation or even porting – limiting factor for broad exchange
• Distributed computing technologies exist (remote shells, CORBA, Java-RMI, Web Services) but there is no reliable, ubiquitous infrastructure to which Licklider‘s request could be addressed. Answer: No.
Grid Content Evolution & Management - e-Science For The Rest Of Us T.Ernst, J.Wauer
Seite 7Arc
hiv
ieru
ng
san
gab
en
Institute for Remote Sensing Technology
Innovation Management and Technology Marketing
Scientific Software Just A Mouse Click Away (3)
... New question: Why?
• in much of real life science
• (non-Computer) Scientists, beyound a reasonable limit of extra effort, simply don‘t have time and/or motivation to do our (CS/IT) work:
- build, deploy, maintain e-Science-supporting platforms- integrate scientific applications into these platforms on a regular basis- and learn how to do all that in the first place
Grid Content Evolution & Management - e-Science For The Rest Of Us T.Ernst, J.Wauer
Scientist Computer Scientist
Seite 8Arc
hiv
ieru
ng
san
gab
en
Institute for Remote Sensing Technology
Innovation Management and Technology Marketing
How did the Web succeed as a publishing medium?
Grid Content Evolution & Management - e-Science For The Rest Of Us T.Ernst, J.Wauer
Characteristics Enabling factors, ~time Evolutionary advantage
1. easy to use in read-mode for almost everybody
Browsers ~ 1993Search engines ~1994
more„consumers“
2. easy to write without web developer background
LaTex2HTML, MS-Office web integration ~1996
more„producers“
3. collaboration/community support
(Cross-site links ~1990) community portals ~1999
Mutual reinforcement community / medium
4. support for traditional editorial processes
content management systems ~1999
organize content growth,integrate with print media
5. infrastructure ubiquitous, commoditized, reliable
ISPs/hosters become an industry (~1998)
Specialization / division of work
Seite 9Arc
hiv
ieru
ng
san
gab
en
Institute for Remote Sensing Technology
Innovation Management and Technology Marketing
Towards „Broad e-Science“ – what is missing?
For e-Science/Grid technology to achieve the role of the primary collaboration medium in „broad scientific computing“, it seems desirable to develop similar „platform survival characteristics“:
1. Ease of use for passive users („consumers“) – Grid/Science portals / Desktops
2. Ease of use for „producers“ (authors of services) - ?
3. Collaboration support – at the core of the Grid / e-science movement
4. Support for editorial processes - ?
5. Infrastructure gets ubiquitous & commoditized – Industry interest
Grid Content Evolution & Management - e-Science For The Rest Of Us T.Ernst, J.Wauer
Seite 10Arc
hiv
ieru
ng
san
gab
en
Institute for Remote Sensing Technology
Innovation Management and Technology Marketing
DLR VirtualLab: an e-Science precursor (1)- technology transfer project 2000-02 („50%“)
- make scientific software accessible from every web browser (like scientific documents already are)
- tap unexploited application potential
- promote technology transfer as well as scientific exchange (internally, too!)
- make better use of this part of DLR‘s scientifíc output
Grid Content Evolution & Management - e-Science For The Rest Of Us T.Ernst, J.Wauer
Seite 11Arc
hiv
ieru
ng
san
gab
en
Institute for Remote Sensing Technology
Innovation Management and Technology Marketing
DLR VirtualLab: an e-Science precursor (2)
- online at http://vl.nz.dlr.de
- Pre-Grid (dedicated cluster)
- Zope/Python,OpenPBS, mySQL, OpenLDAP, ...
- a specialized ASP platform for an „interesting“ class of scientific software
- IEEE CiSE, Feb. 2003
- project now continues as a DLR/Fraunhofer collaboration
Grid Content Evolution & Management - e-Science For The Rest Of Us T.Ernst, J.Wauer
Seite 12Arc
hiv
ieru
ng
san
gab
en
Institute for Remote Sensing Technology
Innovation Management and Technology Marketing
DLR VirtualLab: Concepts for broad adoption of e-Science(1)Care about the process
- keep the barrier to entry low both for „consumers“ and for „producers“
- learn from publishing (traditional & web-based)
- use staging !
Administrator Anonymousexternal user
App
lica
tion
Ser
ver
Author
I N
TE
RN
ET
Integrator
Internal user
registeredexternal user
External users
Repository
ScientificPrograms
I N
TR
AN
ET
Grid Content Evolution & Management - e-Science For The Rest Of Us T.Ernst, J.Wauer
Seite 13Arc
hiv
ieru
ng
san
gab
en
Institute for Remote Sensing Technology
Innovation Management and Technology Marketing
DLR VirtualLab: Concepts for broad adoption of e-Science(2)Make life easy for consumers &
producers
- don‘t exclude important classes of users
- „black-box“ integration: no internal modifications whatsoever
- Provide „cheap“ UIs through generation
- strive to re-use existing (G)UIs
- Integration tools, perhaps web-based
Grid Content Evolution & Management - e-Science For The Rest Of Us T.Ernst, J.Wauer
Seite 14Arc
hiv
ieru
ng
san
gab
en
Institute for Remote Sensing Technology
Innovation Management and Technology Marketing
DLR VirtualLab: Concepts for broad adoption of e-Science(2)Make life easy for consumers &
producers
- don‘t exclude important classes of users
- „black-box“ integration: no internal modifications whatsoever
- Provide „cheap“ UIs through generation
- strive to re-use existing (G)UIs
- Integration tools, perhaps web-based
ID(na='Lambda', # Name
pe='_NF and _Lambda_use',# precondition
ty='FloatType', # type
de='0.5', # default value
ce='Lambda > 0.0', # constraint
un='mu_m', # physical unit
an='Input wavelength') # annotation
Grid Content Evolution & Management - e-Science For The Rest Of Us T.Ernst, J.Wauer
Seite 15Arc
hiv
ieru
ng
san
gab
en
Institute for Remote Sensing Technology
Innovation Management and Technology Marketing
DLR VirtualLab: Concepts for broad adoption of e-Science(2)Make life easy for consumers &
producers
- don‘t exclude important classes of users
- „black-box“ integration: no internal modifications whatsoever
- Provide „cheap“ UIs through generation
- strive to re-use existing (G)UIs
- Integration tools, perhaps web-based
Grid Content Evolution & Management - e-Science For The Rest Of Us T.Ernst, J.Wauer
Seite 16Arc
hiv
ieru
ng
san
gab
en
Institute for Remote Sensing Technology
Innovation Management and Technology Marketing
DLR VirtualLab: Concepts for broad adoption of e-Science(3)Integrate human-readable information with executable (software) resources
- online help and background info
- (computational) „service search engines“ for managing the new e-Science „resource deluge“
-Structured metadata are good, but documentation can serve as a good fallback „fingerprint“
Grid Content Evolution & Management - e-Science For The Rest Of Us T.Ernst, J.Wauer
Seite 17Arc
hiv
ieru
ng
san
gab
en
Institute for Remote Sensing Technology
Innovation Management and Technology Marketing
Conclusions• e-Science will likely evolve to broad/ubiquitous e-Science
• support not only HPC users but the broader/general scientifc computing community
• to help that process, make life easy for
A widely deployed Grid-based e-science infrastructure is a publshing medium- support the process by content management, leveraging suitable technologies
Grid Content Evolution & Management - e-Science For The Rest Of Us T.Ernst, J.Wauer
„ Grid content producers“
- strive for allowing black-box integration of apps & (G)UIs - provide integration tools
„ Grid content consumers“
- don‘t exclude important groups - simple browser-based access for occasional users remains valuable