![Page 1: LI - Cyberinfrastructure Efforts - LONI...UCoMS Closed Loop Demonstration -- SC07 Data Scheduling Separation of CPU & IO Traditional Schedulers Traditional schedulers not aware of](https://reader033.vdocuments.us/reader033/viewer/2022060721/6080f536178b0d12401db37f/html5/thumbnails/1.jpg)
AT LOUISIANA STATE UNIVERSITY
Louisiana InstituteCyberinfrastructure Development Efforts
Tevfik Kosar
Department of Computer Science & CCTLouisiana State University
October 31, 2008
![Page 2: LI - Cyberinfrastructure Efforts - LONI...UCoMS Closed Loop Demonstration -- SC07 Data Scheduling Separation of CPU & IO Traditional Schedulers Traditional schedulers not aware of](https://reader033.vdocuments.us/reader033/viewer/2022060721/6080f536178b0d12401db37f/html5/thumbnails/2.jpg)
Statewide Cyberinfrastructure Efforts
• Scheduling, data• Visualization• Web Services• HPC Interfaces & API
2
• Distributed data management
• Storage, data archival & retrieval
• Sensor Networks• Cyber Security
![Page 3: LI - Cyberinfrastructure Efforts - LONI...UCoMS Closed Loop Demonstration -- SC07 Data Scheduling Separation of CPU & IO Traditional Schedulers Traditional schedulers not aware of](https://reader033.vdocuments.us/reader033/viewer/2022060721/6080f536178b0d12401db37f/html5/thumbnails/3.jpg)
End-to-end Workflow Management
Data SchedulingDistributed Data Storage
Today’s Talk
![Page 4: LI - Cyberinfrastructure Efforts - LONI...UCoMS Closed Loop Demonstration -- SC07 Data Scheduling Separation of CPU & IO Traditional Schedulers Traditional schedulers not aware of](https://reader033.vdocuments.us/reader033/viewer/2022060721/6080f536178b0d12401db37f/html5/thumbnails/4.jpg)
End-to-end Workflows
![Page 5: LI - Cyberinfrastructure Efforts - LONI...UCoMS Closed Loop Demonstration -- SC07 Data Scheduling Separation of CPU & IO Traditional Schedulers Traditional schedulers not aware of](https://reader033.vdocuments.us/reader033/viewer/2022060721/6080f536178b0d12401db37f/html5/thumbnails/5.jpg)
Stage-in
Execute job j
Stage-out
Stage-in
Execute job j
Stage-outRelease input space
Release output space
Allocate space for input & output data
JOB i
JOB k
JOB i
JOB i
JOB k
JOB kIndividual Jobs
JOB j
get
put
Stage-in
Stage-out
Stage-in
Stage-outRelease input space
Release output space
Allocate space for input & output data
Compute Jobs
Data placement Jobs
Release input space
Release output space
Allocate space for input & output data
Data-aware Workflows
![Page 6: LI - Cyberinfrastructure Efforts - LONI...UCoMS Closed Loop Demonstration -- SC07 Data Scheduling Separation of CPU & IO Traditional Schedulers Traditional schedulers not aware of](https://reader033.vdocuments.us/reader033/viewer/2022060721/6080f536178b0d12401db37f/html5/thumbnails/6.jpg)
A Less Simpler Workflow
Credits: Montage Project
![Page 7: LI - Cyberinfrastructure Efforts - LONI...UCoMS Closed Loop Demonstration -- SC07 Data Scheduling Separation of CPU & IO Traditional Schedulers Traditional schedulers not aware of](https://reader033.vdocuments.us/reader033/viewer/2022060721/6080f536178b0d12401db37f/html5/thumbnails/7.jpg)
Separation of CPU & IO
![Page 8: LI - Cyberinfrastructure Efforts - LONI...UCoMS Closed Loop Demonstration -- SC07 Data Scheduling Separation of CPU & IO Traditional Schedulers Traditional schedulers not aware of](https://reader033.vdocuments.us/reader033/viewer/2022060721/6080f536178b0d12401db37f/html5/thumbnails/8.jpg)
UCoMS Workflow
![Page 9: LI - Cyberinfrastructure Efforts - LONI...UCoMS Closed Loop Demonstration -- SC07 Data Scheduling Separation of CPU & IO Traditional Schedulers Traditional schedulers not aware of](https://reader033.vdocuments.us/reader033/viewer/2022060721/6080f536178b0d12401db37f/html5/thumbnails/9.jpg)
Monitoring Workflows via WEB
UCoMS Closed Loop Demonstration -- SC07
![Page 10: LI - Cyberinfrastructure Efforts - LONI...UCoMS Closed Loop Demonstration -- SC07 Data Scheduling Separation of CPU & IO Traditional Schedulers Traditional schedulers not aware of](https://reader033.vdocuments.us/reader033/viewer/2022060721/6080f536178b0d12401db37f/html5/thumbnails/10.jpg)
Data Scheduling
![Page 11: LI - Cyberinfrastructure Efforts - LONI...UCoMS Closed Loop Demonstration -- SC07 Data Scheduling Separation of CPU & IO Traditional Schedulers Traditional schedulers not aware of](https://reader033.vdocuments.us/reader033/viewer/2022060721/6080f536178b0d12401db37f/html5/thumbnails/11.jpg)
Separation of CPU & IO
![Page 12: LI - Cyberinfrastructure Efforts - LONI...UCoMS Closed Loop Demonstration -- SC07 Data Scheduling Separation of CPU & IO Traditional Schedulers Traditional schedulers not aware of](https://reader033.vdocuments.us/reader033/viewer/2022060721/6080f536178b0d12401db37f/html5/thumbnails/12.jpg)
Traditional Schedulers Traditional schedulers not aware of characteristics and semantics of data placement jobs
Executable = genome.exe
Arguments = a b c d
Executable = globus-url-copy Arguments = gsiftp://host1/f1
. gsiftp://host2/f2
Any difference?
![Page 13: LI - Cyberinfrastructure Efforts - LONI...UCoMS Closed Loop Demonstration -- SC07 Data Scheduling Separation of CPU & IO Traditional Schedulers Traditional schedulers not aware of](https://reader033.vdocuments.us/reader033/viewer/2022060721/6080f536178b0d12401db37f/html5/thumbnails/13.jpg)
Data-Aware Schedulers Type of a job?
transfer, allocate, release, locate..
Priority, order?Protocol to use?Second vs Third party?Available storage space?Best concurrency level?Reasons for failure?Best network parameters?
tcp buffer sizeI/O block size# of parallel streams
GridFTP
![Page 14: LI - Cyberinfrastructure Efforts - LONI...UCoMS Closed Loop Demonstration -- SC07 Data Scheduling Separation of CPU & IO Traditional Schedulers Traditional schedulers not aware of](https://reader033.vdocuments.us/reader033/viewer/2022060721/6080f536178b0d12401db37f/html5/thumbnails/14.jpg)
Stork Transfer Methods• regular:
– one connection per file, serial transfer
• multi-connection:– one connection per file,
concurrent transfer
• single-connection:– one connection for all
transfers
• data-fusion:– merge small files into
larger chunks
14
A B
A B
A B
a small file
a small file
many small files
A Ba large file
![Page 15: LI - Cyberinfrastructure Efforts - LONI...UCoMS Closed Loop Demonstration -- SC07 Data Scheduling Separation of CPU & IO Traditional Schedulers Traditional schedulers not aware of](https://reader033.vdocuments.us/reader033/viewer/2022060721/6080f536178b0d12401db37f/html5/thumbnails/15.jpg)
Stork Transfer Results
![Page 16: LI - Cyberinfrastructure Efforts - LONI...UCoMS Closed Loop Demonstration -- SC07 Data Scheduling Separation of CPU & IO Traditional Schedulers Traditional schedulers not aware of](https://reader033.vdocuments.us/reader033/viewer/2022060721/6080f536178b0d12401db37f/html5/thumbnails/16.jpg)
40 35 30 25 20 15 10
5 0
40 35 30 25 20 15 10 5 1
Thro
ughp
ut (M
bps)
number of parallel streams
a) Dinda et al Model Prediction Results
GridFTPPredicted_1_5Predicted_1_7
Predicted_1_10
40 35 30 25 20 15 10
5 0
40 35 30 25 20 15 10 5 1
Thro
ughp
ut (M
bps)
number of parallel streams
b) Averaging Prediction Results
GridFTPPredicted_1_7
Predicted_1_15Arithmetic_1_7_15Geometric_1_7_15Quadratic_1_7_15
40 35 30 25 20 15 10
5 0
40 35 30 25 20 15 10 5 1
Thro
ughp
ut (M
bps)
number of parallel streams
c) Break Function Prediction Results
GridFTPPredicted_1_7_10Predicted_1_5_10
40 35 30 25 20 15 10
5 0
40 35 30 25 20 15 10 5 1Th
roug
hput
(Mbp
s)number of parallel streams
d) Logarithmic Prediction Results
GridFTPPredicted_1_7
40 35 30 25 20 15 10
5 0
40 35 30 25 20 15 10 5 1
Thro
ughp
ut (M
bps)
number of parallel streams
e) Newton’s Method Prediction Results
GridFTPPredicted_1_7_15Predicted_1_7_25Dynamic
Tuning
Based on light-weight prediction
![Page 17: LI - Cyberinfrastructure Efforts - LONI...UCoMS Closed Loop Demonstration -- SC07 Data Scheduling Separation of CPU & IO Traditional Schedulers Traditional schedulers not aware of](https://reader033.vdocuments.us/reader033/viewer/2022060721/6080f536178b0d12401db37f/html5/thumbnails/17.jpg)
Distributed Storage
![Page 18: LI - Cyberinfrastructure Efforts - LONI...UCoMS Closed Loop Demonstration -- SC07 Data Scheduling Separation of CPU & IO Traditional Schedulers Traditional schedulers not aware of](https://reader033.vdocuments.us/reader033/viewer/2022060721/6080f536178b0d12401db37f/html5/thumbnails/18.jpg)
• Goal: enable domain scientists to focus on their primary research problem, assured that the underlying infrastructure will manage the low-level data handling issues.
• Novel approach: treat data storage resources and the tasks related to data access as first class entities just like computational resources and compute tasks.
• Key technologies being developed: data-aware storage systems, data-aware schedulers (i.e. Stork), and cross-domain meta-data scheme.
• Provides and additional 250TB disk, and 400TB tape storage
![Page 19: LI - Cyberinfrastructure Efforts - LONI...UCoMS Closed Loop Demonstration -- SC07 Data Scheduling Separation of CPU & IO Traditional Schedulers Traditional schedulers not aware of](https://reader033.vdocuments.us/reader033/viewer/2022060721/6080f536178b0d12401db37f/html5/thumbnails/19.jpg)
UNO
Tulane
LSU
ULL
LaTech
High Energy PhysicsBiomedical Data Mining
Coastal ModelingPetroleum Engineering
Computational BiologyComputational Fluid Dynamics
Biophysics
Petroleum Engineering
Geology
Participating institutions in the PetaShare project, connected through LONI. Sample research of the participating researchers pictured (i.e. biomechanics by Kodiyalam & Wischusen, tangible interaction by Ullmer, coastal studies by Walker, and molecular biology by Bishop).
Molecular BiologyComputational Electrophysiology
Sensor Networking
Sensor Networking
Numerical Relativity
![Page 20: LI - Cyberinfrastructure Efforts - LONI...UCoMS Closed Loop Demonstration -- SC07 Data Scheduling Separation of CPU & IO Traditional Schedulers Traditional schedulers not aware of](https://reader033.vdocuments.us/reader033/viewer/2022060721/6080f536178b0d12401db37f/html5/thumbnails/20.jpg)
LSULaTech TulaneUNOULL
~ 100 TFLOPS
~8 TB RAM
250 TB Disk
400 TB Tape
SDSC/NCSA
Infrastructure Overview
Managed Data Movement
100s TB
![Page 21: LI - Cyberinfrastructure Efforts - LONI...UCoMS Closed Loop Demonstration -- SC07 Data Scheduling Separation of CPU & IO Traditional Schedulers Traditional schedulers not aware of](https://reader033.vdocuments.us/reader033/viewer/2022060721/6080f536178b0d12401db37f/html5/thumbnails/21.jpg)
HSM
Caching/ Prefetching
Scheduled Data Movement
Replica & Meta Data Management
Data Migration
![Page 22: LI - Cyberinfrastructure Efforts - LONI...UCoMS Closed Loop Demonstration -- SC07 Data Scheduling Separation of CPU & IO Traditional Schedulers Traditional schedulers not aware of](https://reader033.vdocuments.us/reader033/viewer/2022060721/6080f536178b0d12401db37f/html5/thumbnails/22.jpg)
POSIX interface:- NO need to change code - NO relinking- NO recompiling- NO privileged access
petashell
Web interface:
ULL UNO LSU Tulane LaTech
SDSC
PetaShare Core
PetaSearch
user-level virtual FS:- NO need to change code - NO relinking- NO recompiling
petafs
![Page 23: LI - Cyberinfrastructure Efforts - LONI...UCoMS Closed Loop Demonstration -- SC07 Data Scheduling Separation of CPU & IO Traditional Schedulers Traditional schedulers not aware of](https://reader033.vdocuments.us/reader033/viewer/2022060721/6080f536178b0d12401db37f/html5/thumbnails/23.jpg)
• a POSIX compatible shell interface to PetaShare
petashell
$ petashell
psh% cp /tmp/foo.txt /petashare/tulane/tmp/foo.txt
psh% vi /petashare/tulane/tmp/foo.txt
psh% cp /tmp/foo2.dat /petashare/anysite/tmp/foo2.dat
psh% genome_analysis genome_data -->
psh% genome_analysis /petashare/uno/genome_data
psh% exit
$
![Page 24: LI - Cyberinfrastructure Efforts - LONI...UCoMS Closed Loop Demonstration -- SC07 Data Scheduling Separation of CPU & IO Traditional Schedulers Traditional schedulers not aware of](https://reader033.vdocuments.us/reader033/viewer/2022060721/6080f536178b0d12401db37f/html5/thumbnails/24.jpg)
PetaSearch
Search Keywords: Katrina
In Archives: SCOOP UCoMS NumRel Digital Media ....
All Archives
![Page 25: LI - Cyberinfrastructure Efforts - LONI...UCoMS Closed Loop Demonstration -- SC07 Data Scheduling Separation of CPU & IO Traditional Schedulers Traditional schedulers not aware of](https://reader033.vdocuments.us/reader033/viewer/2022060721/6080f536178b0d12401db37f/html5/thumbnails/25.jpg)
1) Title of the Project:2) Short Description of the Project (Approximately 250 words):3) Project Web Page (if any): 4) Project PI and Affiliation:5) Other Senior Personnel: 6) If none of the project participants are associated with PetaShare, please specify a PetaShare contact person:7) Amount of Storage Allocation Asked: - In Short Term (first 6 months): - In Long Term (after 6 months):8) Preferred Storage Site(s):
[ ] LSU (available) [ ] LSUHSC (soon)[ ] UNO (available) [ ] LaTech (after May) [ ] ULL (soon) [ ] Other ___________[ ]Tulane (soon)
9) Check if your application includes any of the following:[ ] MPI jobs [ ] Real-time Visualization[ ] Batch jobs [ ] Data Streaming
10) Please specify from which platforms you will be accessing this storage:
[ ] Linux [ ] Windows[ ] AIX [ ] Other ___________
Accepting Allocation Proposals
Send an email to: [email protected]
![Page 26: LI - Cyberinfrastructure Efforts - LONI...UCoMS Closed Loop Demonstration -- SC07 Data Scheduling Separation of CPU & IO Traditional Schedulers Traditional schedulers not aware of](https://reader033.vdocuments.us/reader033/viewer/2022060721/6080f536178b0d12401db37f/html5/thumbnails/26.jpg)
A system driven by the local needs (in LA), but has potential to be a generic solution for the
broader community!For more information on PetaShare:
http://www.petashare.org
Hmm..