enabling grids for e- science egee and glite are registered trademarks egee-iii infso-ri-222667...

Enabling Grids for E-sciencE

www.eu-egee.org

EGEE and gLite are registered trademarks EGEE-III INFSO-RI-222667

Analysis of Overhead and waiting times in the EGEE Production Grids

Max BergerThomas ZangerlThomas FahringerUniversity of Innsbruck

Enabling Grids for E-sciencE

EGEE-III INFSO-RI-222667

Overview

• EGEE• Definitions• Scheduling Latency• Information Service Latency• Conclusions

Enabling Grids for E-sciencE

EGEE-III INFSO-RI-222667

EGEE

• EGEE: Enabling Grids for E-SciencE• Largest Grid Infrastructure in the World• 140 Institutions, 300 Sites, 50 Countries,

10.000 users, 80.000 CPU cores• Production Grid Infrastructure• Uses the gLite middleware• Organized in Virtual Organizations (VO)

Enabling Grids for E-sciencE

EGEE-III INFSO-RI-222667

VOCE

• VOCE: VO for Central Europe• Part of the EGEE Project• 18 Sites participate• “Liberal” Usage Policy

– Users must be from the CE Region– Any Research can be done

Enabling Grids for E-sciencE

EGEE-III INFSO-RI-222667

Definitions

• Scheduling LatencyDelay between Job Submission and actual execution in seconds

• Information Service (IS) LatencyDelay between actual occurrence of an event and its notification for the user

Enabling Grids for E-sciencE

EGEE-III INFSO-RI-222667

Experiment Description

• Test jobs where submitted to VOCE VO• Between Aug 08 and Oct 08• Approx. every 30 minutes• Measured status change notifications• Real status changes through callbacks• Jobs where canceled after 45 mins

scheduling time

Enabling Grids for E-sciencE

EGEE-III INFSO-RI-222667

Scheduling Latency / Week

Enabling Grids for E-sciencE

EGEE-III INFSO-RI-222667

Scheduling Latency / Day

Enabling Grids for E-sciencE

EGEE-III INFSO-RI-222667

Scheduling Latency (cont.)

• Mean: 121 seconds• Median: 91 seconds• Most of the time short, but exceptions can take a very

long time• No significant changes over the week

– Suggested “Weekend-Effect” was not provable

• No significant changes over the day– The Grid is in use all the time

• Clustering of values• This value is much lower than values shown in related

work!– Real execution start vs. notified execution start

Enabling Grids for E-sciencE

EGEE-III INFSO-RI-222667

Scheduling Latency Histogram

Enabling Grids for E-sciencE

EGEE-III INFSO-RI-222667

Scheduling Latency / Site

Enabling Grids for E-sciencE

EGEE-III INFSO-RI-222667

Information Service Latency

Enabling Grids for E-sciencE

EGEE-III INFSO-RI-222667

IS Latency (cont.)

• Mean: 208 seconds• Median: 198 seconds• IS is organized in layers• Each layer polls the underlying layer• Polling interval defines time needed

Enabling Grids for E-sciencE

EGEE-III INFSO-RI-222667

IS Latency (cont.)

Enabling Grids for E-sciencE

EGEE-III INFSO-RI-222667

Conclusions

• Production Grid are different from Research Grids!• Scheduling Latency is not predictable

• Depends on the Site

• Additional overhead in the Information Service• IS Overhead > Scheduling Latency• Information is relevant for deciding

• Size of workload• Scheduling of activities in workflows