ibp and condor micah beck assoc. prof. & director

IBP and CondorMicah Beck

Assoc. Prof. & Director

Condor Week, Madison May 6, 2003

Funding• Dept. of Energy

SciDAC• National Science

Foundation ANIR• UT Center for Info

Technology Research

Logistical Networking Research at UTK

University of Tennessee

• Micah Beck• James S. Plank• Jack Dongarra

University of California, Santa Barbara

• Rich Wolski

IBP: The Internet Backplane Protocol

• A scalable mechanism for deploying shared storage resources throughout the network

• A general store-and-forward overlay networking infrastructure

• A state management infrastructure for distributed applications and active services

The Network Storage Stack

Applications

Logistical File System

Logistical Tools

L-Bone

IBP

Local Access

Physical

exNode

• Our adaption of the network stack architecture for storage• Like the IP Stack• Each level encapsulates details from the lower levels, while still exposing details to higher levels

IBP: How it Works

• Storage provisioned on community “depots”• Very primitive service (similar to block service, but

more sharable)• Goal is to be a common platform (exposed)• Also part of end-to-end design

• Best effort service – no heroic measures• Availability, reliability, security, performance

• Allocations are time-limited!• Leases are respected, can be renewed• Permanent storage is to strong to share!

The Network Storage Stack

The L-bone:Resource Discovery& Proximity queries

IBP: Allocating and managing networkstorage (like a network malloc)

The exNode:A data structurefor aggregation

LoRS: The Logistical Runtime System:Aggregation tools and methodologies

L-Bone: January 2003

Current Storage Capacity: 13 TB

Multithreaded Transfers

Caching/Staging

Point-to-Multipoint

Heterogeneous Multicast

Relationship to Work of Condor Group

• Wide Area File Management/Access

• Management of Computation State

• As a Storage Allocation Layer for • Kangaroo, NeSt

• DiskRouter

• If routers can have disks, how about disks with processors?

Routers, Depots and the Network Functional Unit

router

inin

out

depot

disk/RAM

inin

out

depot

RAM

inin

out

NFU

send

store

load

execute

Scalable Operations

• IBP Depots with NFU define a State Transformation Substrate

• Processes run at “endpoints” but can use NFUs to transform data in network

• Is it processor-in-storage or active networking?

• All state is exposed

http://loci.cs.utk.edu

Micah Beckmbeck@cs.utk.edu

Logistical Computing and Internetworking