top ten grid questions
DESCRIPTION
Top Ten Grid Questions. Written by : Jennifer M. Schopf Bill Nitzberg Presented by : Allen Lee. Also presenting…. “10 Things We Hate About the Grid” Also by Jennifer M. Schopf Powerpoint Presentation http://www.mcs.anl.gov/~jms/Pubs/TopTenTalk.ppt. Topics Covered. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Top Ten Grid Questions](https://reader035.vdocuments.us/reader035/viewer/2022062323/56815265550346895dc098c5/html5/thumbnails/1.jpg)
Top Ten Grid Questions
Written by:Jennifer M. SchopfBill NitzbergPresented by:Allen Lee
![Page 2: Top Ten Grid Questions](https://reader035.vdocuments.us/reader035/viewer/2022062323/56815265550346895dc098c5/html5/thumbnails/2.jpg)
Also presenting…
“10 Things We Hate About the Grid” Also by Jennifer M. SchopfPowerpoint Presentation http://www.mcs.anl.gov/~jms/Pubs/TopTenTalk.ppt
![Page 3: Top Ten Grid Questions](https://reader035.vdocuments.us/reader035/viewer/2022062323/56815265550346895dc098c5/html5/thumbnails/3.jpg)
Topics Covered
How is the Grid different? Why isn’t the Grid a reality?
![Page 4: Top Ten Grid Questions](https://reader035.vdocuments.us/reader035/viewer/2022062323/56815265550346895dc098c5/html5/thumbnails/4.jpg)
The Grid isn’t a new concept
Using multiple distributed resources to cooperatively work on a single application has been around for decades Networked OS (70s) Distributed OS (80s and 90s) Heterogeneous Computing
http://www.acis.ufl.edu/hcw2006/ Parallel Distributed Computing Metacomputing
http://www.cnds.jhu.edu/research/metacomputing/
![Page 5: Top Ten Grid Questions](https://reader035.vdocuments.us/reader035/viewer/2022062323/56815265550346895dc098c5/html5/thumbnails/5.jpg)
How is a grid different?
Grids focus on site autonomy Grids involve heterogeneity Grids involve more resources than just
computers and networks Grids focus on the user
![Page 6: Top Ten Grid Questions](https://reader035.vdocuments.us/reader035/viewer/2022062323/56815265550346895dc098c5/html5/thumbnails/6.jpg)
Grid Criticisms…
“The Grid is a solution looking for a problem.”
![Page 7: Top Ten Grid Questions](https://reader035.vdocuments.us/reader035/viewer/2022062323/56815265550346895dc098c5/html5/thumbnails/7.jpg)
Grid Criticisms…
“We tried to install Globus and found out that it was too hard to do. So we decided to just write our own...”
![Page 8: Top Ten Grid Questions](https://reader035.vdocuments.us/reader035/viewer/2022062323/56815265550346895dc098c5/html5/thumbnails/8.jpg)
#1 Why don’t Grids have basic functionality yet? Grids aren’t being used for what they were
“meant to be”“Application developers aren’t using multiple
machines at geographically distant sites to coordinate and solve a single application.”
Used to solve embarrassingly parallel applications such as Seti@home and Entropia
![Page 9: Top Ten Grid Questions](https://reader035.vdocuments.us/reader035/viewer/2022062323/56815265550346895dc098c5/html5/thumbnails/9.jpg)
#1 Why don’t Grids have basic functionality yet? What is functionality?
The ability to run an application in a straightforward manner using single commands for a user to just run.
![Page 10: Top Ten Grid Questions](https://reader035.vdocuments.us/reader035/viewer/2022062323/56815265550346895dc098c5/html5/thumbnails/10.jpg)
#1 Why don’t Grids have basic functionality yet? Running a process isn’t simple Steps:
Determine what software is on the machine and install it, then check it
Make sure all security credentials are setup (certificates, keys, accounts, etc.)
Determine availabile resources Submit to a Grid computer (set up environment) Run Cleanup
![Page 11: Top Ten Grid Questions](https://reader035.vdocuments.us/reader035/viewer/2022062323/56815265550346895dc098c5/html5/thumbnails/11.jpg)
#1 Why don’t Grids have basic functionality yet? “The few current users have had to go through
heroic measures to achieve any functionality at all.”
Complicated due to socio-political problems It is difficult setting up and maintaining a piece of
new software on one system let alone many systems
Resistance from users who would have to learn a new environment
![Page 12: Top Ten Grid Questions](https://reader035.vdocuments.us/reader035/viewer/2022062323/56815265550346895dc098c5/html5/thumbnails/12.jpg)
Moral #1: Before we can have a successful Grid, we must have a functional Grid
![Page 13: Top Ten Grid Questions](https://reader035.vdocuments.us/reader035/viewer/2022062323/56815265550346895dc098c5/html5/thumbnails/13.jpg)
Grid Criticisms…
“Cynics reckon that the Grid is merely an excuse by computer scientists to milk the political system for more research grants so they can write yet more lines of useless code” – Economist, June 2001
![Page 14: Top Ten Grid Questions](https://reader035.vdocuments.us/reader035/viewer/2022062323/56815265550346895dc098c5/html5/thumbnails/14.jpg)
#2 Why aren’t there more Grid application developers and users? There is a need for user input to guide research,
but users are hard to find and input is hard to receive despite large communities at NSF supercomputer centers
Groups are running specialized code targeted to specific platforms
Even when they use the Grid, they use it “improperly” they use it to gain access to specific machines, not as the Grid was envisioned.
![Page 15: Top Ten Grid Questions](https://reader035.vdocuments.us/reader035/viewer/2022062323/56815265550346895dc098c5/html5/thumbnails/15.jpg)
#2 Why aren’t there more Grid application developers and users? Making apps Grid enabled is seen by
some as a distraction from getting real science done
Grid work is a buzzword for funding, but takes away from other software development work
![Page 16: Top Ten Grid Questions](https://reader035.vdocuments.us/reader035/viewer/2022062323/56815265550346895dc098c5/html5/thumbnails/16.jpg)
Moral #2: Without users there can be no Grid Better software tools to ease transition to a
new environment Standards for uniform access to Grid
services Deployment of Grid software must be
made easier
![Page 17: Top Ten Grid Questions](https://reader035.vdocuments.us/reader035/viewer/2022062323/56815265550346895dc098c5/html5/thumbnails/17.jpg)
#3 Where are the Grid software tools to aid application developers? Parallel programming was difficult so few
people attempted it until debuggers, fast compilers and other flexible tools were available
There is active work on a set of software tools for the Grid such as Globus, however it has a steep learning curve
![Page 18: Top Ten Grid Questions](https://reader035.vdocuments.us/reader035/viewer/2022062323/56815265550346895dc098c5/html5/thumbnails/18.jpg)
#3 Where are the Grid software tools to aid application developers? This is due to socio-political problems
Most projects are academic Little or no funding funding for hardening code
Chicken and Egg problem To make tools, you need use cases Without tools to use, you don’t know what to ask for Constructing a reasonable use case is difficult, it
needs specific details without too much overwhelming detail, especially if you don’t know what is possible
![Page 19: Top Ten Grid Questions](https://reader035.vdocuments.us/reader035/viewer/2022062323/56815265550346895dc098c5/html5/thumbnails/19.jpg)
#3 Where are the Grid software tools to aid application developers? Different language/communication styles
Computer Science-Centric projectsApplication-Centric
![Page 20: Top Ten Grid Questions](https://reader035.vdocuments.us/reader035/viewer/2022062323/56815265550346895dc098c5/html5/thumbnails/20.jpg)
Moral #3: The Grid needs tools that facilitate the use of a Grid
![Page 21: Top Ten Grid Questions](https://reader035.vdocuments.us/reader035/viewer/2022062323/56815265550346895dc098c5/html5/thumbnails/21.jpg)
#4 How do we make Grids secure?
For many systems, the mandatory “security is important” paragraph is all the depth problem receives
GSI is accepted security infrastructure and is used in things such as CVS, OpenSSH
However, that only resolves Authentication
![Page 22: Top Ten Grid Questions](https://reader035.vdocuments.us/reader035/viewer/2022062323/56815265550346895dc098c5/html5/thumbnails/22.jpg)
#4 How do we make Grids secure?
Local accounts are still needed and the issue of authorization and usage policy is not address uniformly
![Page 23: Top Ten Grid Questions](https://reader035.vdocuments.us/reader035/viewer/2022062323/56815265550346895dc098c5/html5/thumbnails/23.jpg)
Moral #4: Without a security infrastructure(including higher-level tools) users will not take advantage of the Grid
![Page 24: Top Ten Grid Questions](https://reader035.vdocuments.us/reader035/viewer/2022062323/56815265550346895dc098c5/html5/thumbnails/24.jpg)
#5 How can we define standard interfaces and definitions for the Grid?
No simple answer for the following questions:How do I run a job on the Grid?What sort of monitoring is available?Where do I get information about the Grid
resources?How do I make sure this operation is secure?
![Page 25: Top Ten Grid Questions](https://reader035.vdocuments.us/reader035/viewer/2022062323/56815265550346895dc098c5/html5/thumbnails/25.jpg)
#5 How can we define standard interfaces and definitions for the Grid?
A lingua franca needs to be created to promote greater interoperability between current toolsLack of agreement between developers over
simple terms such as: Job Resource
![Page 26: Top Ten Grid Questions](https://reader035.vdocuments.us/reader035/viewer/2022062323/56815265550346895dc098c5/html5/thumbnails/26.jpg)
#5 How can we define standard interfaces and definitions for the Grid?
Example Globus Metacomputing Directory Service (MDS) and
Grid Monitoring Architecture (GMA) MDS concentrates on resource discovery (2 protocols) GMA concentrates on the provision of data (3
implementations) However, rather than interoperate to work off each other’s
strengths, there is little collaboration between these standards, interfaces and protocols
![Page 27: Top Ten Grid Questions](https://reader035.vdocuments.us/reader035/viewer/2022062323/56815265550346895dc098c5/html5/thumbnails/27.jpg)
#5 How can we define standard interfaces and definitions for the Grid?
The Global Grid Forum has standardization as a goal
But so does Peer-to-Peer Working Group and the New Productivity Initiative for example
Who will standardize between the standardization bodies?
![Page 28: Top Ten Grid Questions](https://reader035.vdocuments.us/reader035/viewer/2022062323/56815265550346895dc098c5/html5/thumbnails/28.jpg)
Moral #5: The lack of standards continues to hinder the interoperability needed for the Grid
![Page 29: Top Ten Grid Questions](https://reader035.vdocuments.us/reader035/viewer/2022062323/56815265550346895dc098c5/html5/thumbnails/29.jpg)
#6 How can we manage variance on the Grid? Grid resource behavior is unpredictable
Unsure of who else is using a resource and in what way
Users want not only fast execution times from their applications, but unpredictable behavior, and would be willing to sacrifice performance in order ot have reliable run times.
![Page 30: Top Ten Grid Questions](https://reader035.vdocuments.us/reader035/viewer/2022062323/56815265550346895dc098c5/html5/thumbnails/30.jpg)
Moral #6: Varying behavior is a fact in the Grid and must be addressed Teach users how variance can affect their
performance Scheduling algorithms have to be made to
incorporate variance Information services have to incorporate
the handling of the additional variance information
![Page 31: Top Ten Grid Questions](https://reader035.vdocuments.us/reader035/viewer/2022062323/56815265550346895dc098c5/html5/thumbnails/31.jpg)
#7 How can deployment be made easier? Deploying a test bed is difficult Firewalls are often a major stumbling block Accounting methodology
Accounts have to be made for resources that never had policies for Grid access in mind
![Page 32: Top Ten Grid Questions](https://reader035.vdocuments.us/reader035/viewer/2022062323/56815265550346895dc098c5/html5/thumbnails/32.jpg)
Moral #7: If the deployment and setup of software isn’t seamless, getting users to adapt their systems to be Grid compatible will not happen
Halfway solutions can be worse than no solution
![Page 33: Top Ten Grid Questions](https://reader035.vdocuments.us/reader035/viewer/2022062323/56815265550346895dc098c5/html5/thumbnails/33.jpg)
#8 Where are the benefits to encourage sharing on the Grid? Grid computing has high appeal to the end
user and to the top-level policy makersEnd Users: access to resources spread out
geographically and across several domainsTop-Level: Enterprise-wide organization policy
(NASA could direct 90% of its resources to deflecting near-Earth collision)
![Page 34: Top Ten Grid Questions](https://reader035.vdocuments.us/reader035/viewer/2022062323/56815265550346895dc098c5/html5/thumbnails/34.jpg)
#8 Where are the benefits to encourage sharing on the Grid? Tragedy of the Commons
Users don’t play nice; they will act in their own interests and degrade and overuse any shared resource
Middle-Level resource owners and managers are caught in the middle Users will exploit Grid technology increasingly to consume more
resources Grid accounting can’t provide the accounting or control needed
so ssers end up getting a free ride How can Middle-Level managers prove their value to
policymakers while controlling user consumption?
![Page 35: Top Ten Grid Questions](https://reader035.vdocuments.us/reader035/viewer/2022062323/56815265550346895dc098c5/html5/thumbnails/35.jpg)
Moral #8: Benefits to middle-level resource ownersk must be made clear in order fo the Grids to be deployed
![Page 36: Top Ten Grid Questions](https://reader035.vdocuments.us/reader035/viewer/2022062323/56815265550346895dc098c5/html5/thumbnails/36.jpg)
#9 How can we fund the work needed for a functioning Grid? Work to coordinate sites and resources are seen
as not new enough so it doesn’t get funding Fund groups to work on the same problem but
using different methods. Then fund to combine them. Funding several groups is seen as wasteful
Tensions caused by fight between Best Approach and Standard
![Page 37: Top Ten Grid Questions](https://reader035.vdocuments.us/reader035/viewer/2022062323/56815265550346895dc098c5/html5/thumbnails/37.jpg)
Moral #9: Without support for basic work in functionality, standards and software engineering, the Grid will not live up to its potential
Funding agencies need to be educated on the needs of Grid researchers
Money should be made for mundane tasks to get software beyond beta versions, standardization, setting up test beds, functionality
![Page 38: Top Ten Grid Questions](https://reader035.vdocuments.us/reader035/viewer/2022062323/56815265550346895dc098c5/html5/thumbnails/38.jpg)
#10 Where are the performance metrics for success? Success can not be determined without
metrics to compare a project against
![Page 39: Top Ten Grid Questions](https://reader035.vdocuments.us/reader035/viewer/2022062323/56815265550346895dc098c5/html5/thumbnails/39.jpg)
Moral #10: There isn’t a moral
Proposed Success MetricsNo more “Grid” papers
Instead, papers with the following footnote, “This work was achieved using the Grid”
When supercomputer centers don’t give a user the choice of using their machines or the Grid
They will just use the Grid
![Page 40: Top Ten Grid Questions](https://reader035.vdocuments.us/reader035/viewer/2022062323/56815265550346895dc098c5/html5/thumbnails/40.jpg)
Moral #10: There isn’t a moral
Proposed Success Metrics (Cont.)When a SuperComputing demo can be run
any time of the year The SuperComputing demo runs every year since
1988 in November for one week “For one week every year, it’s the world’s fastest
nework, providing direct connectivity to a multitude of worldwide research networks.” Source: SC|05 http://sc05.supercomputing.org
![Page 41: Top Ten Grid Questions](https://reader035.vdocuments.us/reader035/viewer/2022062323/56815265550346895dc098c5/html5/thumbnails/41.jpg)
Agnostic 1
In the introduction, the authors point out some differences between old distributed systems and today’s Grids. Are these all the major differences? If not, can you suggests others grid distinctive characteristics?
![Page 42: Top Ten Grid Questions](https://reader035.vdocuments.us/reader035/viewer/2022062323/56815265550346895dc098c5/html5/thumbnails/42.jpg)
Agnostic 2 - “1. Why don’t Grids have basic functionality yet?”
According to the authors, grids do not have basic functionality because to run a grid application one has to follow a sequence of intricate steps, like installing software, setting security, finding the appropriate resource and submitting a job, which look daunting and extremely complex. Do you think this is a fair description as for users running applications on the grid? How is the Service Oriented Architecture helping improve that scenario?
![Page 43: Top Ten Grid Questions](https://reader035.vdocuments.us/reader035/viewer/2022062323/56815265550346895dc098c5/html5/thumbnails/43.jpg)
Agnostic 3 - “3. Where are the Grid software tools to aid application developers?”
Do you agree with the statement that the difficulty of developing useful tools for the grid is because there are not well defined use cases? Is it really a chicken and egg problem?
![Page 44: Top Ten Grid Questions](https://reader035.vdocuments.us/reader035/viewer/2022062323/56815265550346895dc098c5/html5/thumbnails/44.jpg)
Agnostic 4 - “4. How do we make Grids secure?” It is mentioned that the authorization issue has not been
fully addressed, and typically imply using a local account to access a particular resource. This would difficult tasks like auditing the usage of a certain resource. Could you provide some information as for the advances in this matter?
Another aspect of grid security is reliance on the grid services. A service can mimic a real application and obtain sensitive information from the users. What work has been done in this respect?
![Page 45: Top Ten Grid Questions](https://reader035.vdocuments.us/reader035/viewer/2022062323/56815265550346895dc098c5/html5/thumbnails/45.jpg)
Agnostic 5 - “5. How can we define standard interfaces and definitions for the Grid?”
It is mentioned during the introduction that none of the standards defined at the time of writing the paper (2002) were standing still. Is this assertion still true?
Could you briefly talk about the current status of the Global Grid Forum and the Enterprise Grid Alliance groups as for grid standards?
![Page 46: Top Ten Grid Questions](https://reader035.vdocuments.us/reader035/viewer/2022062323/56815265550346895dc098c5/html5/thumbnails/46.jpg)
Agnostic 6 -“8. Where are the benefits to encourage sharing on the Grid?”
Could you suggest some ways to enforce the user to “play nicely”? In particular, how to manage multiple user applications that can potentially exhaust the grid resources?
Also, can you think of good reasons to convince managers for sharing resources?
![Page 47: Top Ten Grid Questions](https://reader035.vdocuments.us/reader035/viewer/2022062323/56815265550346895dc098c5/html5/thumbnails/47.jpg)
Agnostic 7 -“10. Where are the performance metrics for success?”
The second success metric says that users are not given a choice of using or not the grid, but just use it. What is your view of this statement? Aren’t the authors assuming that all problems are suitable to get solved in a grid environment?