technische universität münchen universitätsbibliothek e-research in germany joint seminar...

Technische Universität MünchenUniversitätsbibliothek

E-Research in Germany

Joint Seminar

November 2009 Dr. Reiner Kallenborn Technische Universität München Library


… the large scale science that will increasingly be carried out through distributed global collaborations enabled by the Internet

UK National e-Science Centre

E-Science / E-Research is …


big computational research

team research

networked research

dynamic

interdisciplinary

heterogeneous

requiring a different set of working tools and methods


… in former times:

E-Science / E-Research is …


… now

grid-infrastructures offer:

direct access to resources themselves, i.e.

CPU power

Storage

Scientific instruments

Applications

web access to “static” information

Data

Sensors

Middleware Services


Grid-Infrastructures have many advantages


Better use of resources

Practically infinite storage and CPU capacity

Flexibility, scalability

Automatic adaptation of complex computation processes

Dynamic and concerted operation of networked resources

Higher quality results through grid supported development

User oriented cost reduction


… what’s up in Germany?

E-Science / E-Research …



• Solitary activities

o UNICORE projects (Uniform Interface to Computing Resources; expired)

o DFN development programme (expired)

o Sporadic consortia and projects of individual persons or institutes

o LHC-Grid at Tier1-Centre Karlsruhe

E-Science in Germany up to 2004



• Grid requirements do not stop at national boundaries

• German scientists are part of international communities

• German scientists depend on access to international resources

(computers, data, experiments)

• No national initiative comparable to UK, USA, NL

• No national effort for a grid infrastructure

• There is no grid without standards

• There is no use for each scientific community reinventing all grid aspects

• Germany can’t afford to throw away opportunities offered by grid computing


E-Science in Germany up to 2004

several workshop initiated by the Research Centre Karlsruhe and the Helmholtz Association of German Research Centres initiating D-Grid


several workshop initiated by the Research Centre Karlsruhe and the Helmholtz Association of German Research Centres initiating D-Grid


E-Science in Germany from 2004 on: D-Grid

D-Grid Strategy Paper and Policy Statement

– discussing status and consequences of the grid technology on scientific research and recommending a long-term strategic grid research and development initiative

– resulting in an announcement of the German e-Science Initiative by the German Ministry for Research and Education (BMBF) in March 2004, together with a call for proposals in the areas of Grid Computing, e-Learning, and Knowledge Management

– Demanding (material) committment of scientific communities

June 15th, 2004: Initiation of a BMBF D-Grid support programme allocating 300 million Euro within 5 years


• Net-based coordinated operation and assignment of computer and storage resources

• Constitution of a German wide IT infrastructure for complex applications of research and industry

• Covering a multitude of applications

• Development of community-specific grid services and working methods

• Integration of a broad spectrum of resources and IT services Hardware: computer and storage systems as well as networking facilities Software: proprietary, freely available or commercial Data: internal und external databases Personnel: counselling, training and support

D-Grid: Objectives



D-Grid: Layered Framework

Physics Medicine Finances

Services /Resources

Operation /Middleware

User

Community

Support, Security …Support, Security …



• D-Grid: Organization



Homogenization and qualification of the existing Grid-infrastructure by prototypical applications

Generic middleware development (toolkit for e-science) IT services for scientists, designed and developed by the ‘early adopters’ of

the computer science and scientific computing communities This global services infrastructure is being tested and used by so-called

Community Grids in the areas of highenergy physics, astrophysics, alternative energy, medicine and life sciences, earth sciences (e.g. climate), engineering sciences, and scientific libraries

Target groups: Science and scientific computing communities

D-Grid 1 (2005 – 2008)



D-Grid 2 (2008 – 2010)

stronger focus on grids with industry participation, with more than 40 companies involved

national Grid-Infrastructure on production level establishing of e-science applications and services in

• Construction industry• Finance• Aerospace• Automotive• Enterprise information• Resource planning systems• Geographical data• General IT services



Gap Projects

Service ProjectsCommercial Applications

Academic Applications

Integration Project

Project Categories in D-Grid


Special Projects

Management of Virtual Organisations

Monitoring of Resources and Services

Short-lived Certificates

Net Attacks Recognition

Interoperability of Scheduling Systems

New Projects



Service Projects

Management Information Systems

Backup und Recovery

Sustainability of Grid Infrastructures

Support for Service Level Agreements

Grid-based Knowledge Structures

New Projects




Astronomy

Climate Research

High Energy Physics

Medicine

Engineering

Humanities



Aviation Industry

Finances

Automobile Industry

Building and Construction Industry

Geographical Information Systems

Generic Engineering Applications

Meteorology for Renewable Power Industry

Cooperative Product Development

Medical Engineering and Services

Commercial Applications



Logistics

Analyses and Use of Multimedia Data

Molecular Simulation

Optimisation via Numerical Simulation

Analyses of Medical Signal and Image Data

Plasma Technology

Support of Dynamic Value Added Networks

New Commercial Applications



2004 2007 20102009 20112008 2012

Special Projects

20062005

Integration Project

Service Projects


CommercialApplications

D-Grid Ltd.

Working Groups Pre-Projects Phase 1

Phase 2

Phase 3

D-Grid History



Sustainability

Forthcoming Endeavour:



Integration Project und Gap Projects• Procedures of resource integration• Procedures for generic grid management of resources• Universally valid security models and operational concepts• Business model DGSE (D-Grid Support Establishment)

Service Layer Projects• Offer of community-overlapping services• Compatibility with standards of the German Computer Science Association • Use of proprietary business models

Community Projects• Offers for users within the community• Recipients of DGSE capacities and of service providers

D-Grid 3 (2009 - ): Structural Concepts of the D-Grid Initiative



Direct contracts between user and provider• In case of only few bilateral contacts

Contracts between community and provider• Delegation of trust to the users within a community

Framework agreements between communities and Gauss Alliance• Gauss Alliance = Community of German Computer Centres

Framework agreements between community representatives and providers• Example: Licences for D-Grid

Special regulations for investment and special grants

D-Grid Ltd. (founded 2008) - Coordination, Use of Resources



Bremerhaven

Köln

Offenbach

Dortmund

Core group

Freiburg

Heidelberg

Tübingen

Mannheim

Ulm

Konstanz

Refional group (bwGRiD)

User group (C3-Grid)

Further resource provider

Marburg

Paderborn

AachenBonn

Berlin

Darmstadt

Siegen

Würzburg

Göttingen

KasselLeipzigWuppertal

Berlin

Stuttgart

Dresden

München

Hannover

Jülich

Karlsruhe

Hamburg

Marburg

Paderborn

AachenBonn

Berlin

Darmstadt

Siegen

Würzburg

Göttingen

KasselLeipzigWuppertal

Oldenburg

D-Grid Resources


Development of an Organisational Structure

VertretungVertretung Representatives?Representatives? VertretungVertretung

Providers of IT Services and Resources

AcademicDomains of Application

D-Grid Ltd.D-Grid Ltd.

IndustrialApplications

Organisational Matters

Legal Matters

Gauss AllianceGauss Alliance WissGridWissGrid


Publicity Matters International

Cooperation


25th NORDUnet Conference, Copenhagen

Dieter Kranzlmüller 26

290 Sites55 Countries144.000 Cores25 PetaBytes Disk>17.000 Users>200 VOs>330.000 Jobs/Day

ArchaeologyAstronomyAstrophysicsCivil ProtectionComp. ChemistryEarth SciencesFinanceFusionGeophysicsHigh Energy PhysicsLife SciencesMultimediaMaterial Sciences…

Status July 2009: http://project.eu-egee.org/index.php?id=417


42 European NGIs

+ Asia, US, Latin America

+ OGF-Europe

+ PRACE

+ …

EGI, European Grid Initiative


… how do libraries fit in?

E-Science / E-Research …



The Evolving World of e-Science: Impact and Implications for Science and Technology Libraries

Purdue University (West Lafayette, IN) and Chicago, IL, United StatesJune 20-24, 2010

http://blogs.lib.purdue.edu/iatul2010/

31st Annual IATUL Conference




Join IATUL!

Thank you!


• http://www.d-grid.de/

• H.-G. Hegering: D-Grid Initiative – Statusbericht

http://www.rv-nrw.de/content/koop/workshops/20040601.dgrid/Hegering_DGrid_Statusbericht_ZKI_Mar04.pdf

• W. Gentzsch: 2007D-Grid and the Integration of New Communities

http://www.ogf.org/OGF20/materials/783/OGF20-D-grid.pdf

• F. Schlie-Roosen et al.: German Steps to e-Science

http://www.pd.infn.it/einfrastructures/eirgSlides/German_View_escience.pdf

• U. Lang: D-Grid – Hype and reality

http://www.dini.de/veranstaltungen/jahrestagungen/2009/

• H. Neuroth, M. Kerzel, W. Gentzsch

http://webdoc.sub.gwdg.de/univerlag/2007/D-Grid_en.pdf

• U. Schwiegelshohn• http://www.d-grid-ggmbh.de/downloads/BroschuereAHM2009.pdf

References



Special Projects






New Projects


IVOM: Integration of Shibboleth into the D-Grid infrastructure and the enhancement of the corresponding VO-Management


Special Projects






New Projects


D-MON: Realization of a Grid wide monitoring architecture across several underlying, heterogeneous systems taking in consideration multiple resource providers and virtual organisations


Special Projects







GIDS: Grid Intrusion Detection Systems


Special Projects







DGSI: conception and development of an interoperability layer for scheduling in service Grids, allowing the users of a community to distribute the workload among resources within the management domain of another community


Service Projects


Backup und Recovery




New Projects


BisGRID: Enabling Grid technologies to be used for the integration of decentralised business information systems. This will be achieved by developing and providing organisational and technical extensions based on the current state of the art in Grid technologies, EAI, and SOA


Service Projects


Backup und Recovery




New Projects


F&L Grid: Establishing a generic service-grid for research and education. The pilot sevice-offering will be Backup- and Archive services, later expanded to Information Lifecycle Management


Service Projects


Backup und Recovery




New Projects


SuGI: Dissemination of the knowledge of grid technology and enhancement of its use, especially the organization of training courses and the multimedia processing of further D-Grid courses for Web based publishing.



Astronomy

Climate Research

High Energy Physics

Medicine

Engineering

Humanities


AstroGrid-D• Joining together astronomical science

institutions into a unified nationwide research infrastructure of collaborative distributed environments using innovative grid technology

• Creation of a grid based infrastructure for astronomical and astrophysical research, based on integration of their computational facilities, improving the efficiency and usability of their hardware resources

• Integration of distributed astronomical data archives and, in an extended time frame, of instruments and experiments into a common research infrastructure

• Support of other research institutes to enable them to share their resources, data and application software within the AstroGrid infrastructure, especially to enable small university workgroups to join the collaboration

• Strengthening the ties of the German astronomy community to the huge developments within the international activities of the astronomy community



Astronomy

Climate Research

High Energy Physics

Medicine

Engineering

Humanities


C3 Grid• Connection of distributed climate data

archives• Uniform access and central metadata

catalogue

Work packages

• User Interface (API/GUI) • Grid - Information Service • Connection of local database-systems /

Metadata Backend-Interface • Dataproviding and Preprocessing • Grid-Data Management • Coordidated Grid-Scheduling • Implementation and Integration of the C3-

Grid • Sustainable Connection with Earth System

Science • Project Management and Co-operation

with the Integration Project



Astronomy

Climate Research

High Energy Physics

Medicine

Engineering

Humanities


HEP Grid work packages

• Scalable storage• Job monitoring• Distributed data analysis and use of grid

resources



Astronomy

Climate Research

High Energy Physics

Medicine

Engineering

Humanities


MediGRID will demonstrate the feasibility and usefulness of grid services in medicine and life sciences by applying these services in biomedical research with multidimensional data and by correlating genotypic and phenotypic data. At the same time generic recommendations for the technical and logical infrastructure are compiled and submitted to the D-GRiD integration project.

The MediGRID project has a modular setup: The four methodological modules (middleware, ontology, resource fusion and eScience) plan to incrementally develop and provide a Grid infrastructure while taking into account the need of the biomedical users. The user communities are represented in three research modules for biomedical informatics, image processing and clinical research. The gained experiences are spread to other communities and flow into the further advancement of the project to create synergies and convergence



Astronomy

Climate Research

High Energy Physics

Medicine

Engineering

Humanities


inGRID, Innovative Grid Developments for Engineering Applications

Support of virtual prototyping by means of • internet-based collaborative methods• mapping of engineering workflows on a grid

environment • integration of specific technical data

sources building of portals for selected engineering scenarios.

Development of a knowledge-based system for the optimization of engineering workflows and support of decision-making processes

Development of co-operation, business, safety and confidence models for scientific engineering applications



Astronomy

Climate Research

High Energy Physics

Medicine

Engineering

Humanities


Text Grid, a Virtual Research Environment for the Humantities

• creating a community grid for the collaborative editing, annotation, analysis, and publication of specialist text resources

• establishes a truly interdisciplinary platform and a virtual workbench for research

• focussed on (annotated) text as a data type


Aviation Industry

Finances

Automobile Industry









AeroGrid: Creating a permanent and practice-oriented grid infrastructure to facilitate co-operation between large-scale research institutions, industry and university research in the field of aeronautics


Aviation Industry

Finances

Automobile Industry









FinGrid: design and development of three Grid-based prototypes

Prototype I is a pricing and billing component for Grid-based services – in cooperation with Deutsche Bank and IBM

Prototype II is a Grid-based customer portfolio performance measurement and management tool – in cooperation with DataSynapse and Dresdner Bank

Prototype III is an asset-backed security factory based on Grid architecture – in cooperation with FinanzIT and PA Consulting


Aviation Industry

Finances

Automobile Industry









Biz2Grid aims at migrating industrial business applications to existing grid-middleware systems by developing a price based scheduler to the D-Grid community:

• provide foundations for an effective application of Grid technologies in enterprises

• design and implementation of the billing and pricing component

• business-models have to be developed that are adequate for an application in the Grid

• distribute and parallelize real world applications and

• conceptualize and implement economic business models at the same time


Aviation Industry

Finances

Automobile Industry









BauVoGrid: a Grid-based Platform for the Virtual Organisation in Construction

• controlled representation of responsibility / authorisation structures

• fast configuration and management of both global VO processes and local company-specific processes in accordance with the set up responsibility VO structure

• fast, flexible and secure access to information from different sources (documents, drawings, photographic material etc.) both from headquarters and from the construction site

• ad-hoc changes in the process flow, using semi-automatic process simulation

• mobile capturing of processes and process/product data on the site, thereby providing a basis for faster and more efficient decision making with participation of all site workers


Aviation Industry

Finances

Automobile Industry









GDI Grid: Efficient mining and processing of spatial data for the simulation of noise dispersion and disaster management


Aviation Industry

Finances

Automobile Industry









PartnerGrid develops a collaboration platform for the co-operation of companies on the basis of D-Grid.

Typical scenarios of different branches:

• Collaboration of the customer and the foundry by the

designing of cast parts

• Collaboration in the metal-processing industry at the planning of forming processes


Aviation Industry

Finances

Automobile Industry









WISENT - a Knowledge Network in Energy Meteorology

• IT-optimised cooperation in research and development of organisations in the area of energy meteorology

• Development of electronic services for analysis and visualization of large datasets in atmopheric research and meteorology

• Development of new collaborative methods for the optimization in the use of renewable energy resources like solar and wind energy


Aviation Industry

Finances

Automobile Industry









ProGrid - Grid technology for collaborative product development - goalsutilize Grid technology for collaborative product development anddemonstrate grid advantages by means of selected use cases

• Process optimization• Optimization in fluid mechanics• Multidisciplinary structural

optimization• Connection between product data

management software and computer aided engineering systems

• Collaboration platform


Aviation Industry

Finances

Automobile Industry









S@M - Services@MediGRID: Service Grid for biomedical network research

lowering the entry threshold for the utilisation of the grid computing infrastructure in the life sciences on the basis of the results of the initial project MediGRID


Logistics





Plasma Technology




MoSGrid makes the D-Grid infrastructure available for high-performance computing in the area of molecular simulations, including the annotation of metadata results and its provision for data mining and knowledge generation


Logistics





Plasma Technology




ValueGrids opens the D-Grid-infrastructure for a fundamentally new community, the users and providers of software-based services

A concept is to be provided allowing for seamless integration of interconnected providers into business processes as well as allows for flexible management of value networks based on service level agreements.

technische universität münchen universitätsbibliothek e-research in germany joint seminar...

Documents

dgrid slide

grid computing

grid infrastructure

grid aspects germany

research centre karlsruhe

dgrid dgrid strategy

institutes o lhcgrid

large scale science