space allocations in nest

Joseph StanleyComputer Sciences DepartmentUniversity of Wisconsin-Madison

[email protected]://www.cs.wisc.edu/condor

Space allocations in NeST

www.cs.wisc.edu/condor

Outline

› Overview of NeST

› Space allocations

› Implementation

› Big picture

› Measurements and Future Work


Overview of NeST

› Network storage server designed for the Grid

› Flexibility Unprivileged, user-level software Multiple protocols

• Chirp, GridFTP, FTP, HTTP, NFS Multiple concurrency models

• Processes, threads, non-blocking Portable


Overview of NeST

› Functionality Exports Unix based file system Space allocation ( Lot )

› Manageability User management – Dynamic

users Security – GSI authentication


Dispatcher

Transfer MgrConcurrencies

Control flow

Data flow

Chirp FTP Grid FTP NFS

Common protocol layer

HTTP

Physical network layer

Storage Mgr

Lot Mgr

Physical storage layer

NeST structure


Why space allocations ?

› Data generation and storage Data intensive applications Need to allocate space for temporary files

› Data migration Many simultaneous data flows

• Competition for storage space• Possible that all flows end in partial transfers

Need to ensure at least one succeeds


Space allocations in NeST

› Lot – abstraction for space allocation

› Create lot for a specified size and duration

› User and group lots

› Guaranteed and best-effort lots

› Hierarchical lots


Motivation for hierarchical lot

› SRMs on top of NeSTs

› Client requests streamlined through SRMs

› SRMs create a large lot

› SRMs distribute sub-lots among clients carved from the large lot


Lot operations

› Create, Delete, Update› MoveFile

Moves files across lots

› AddUser, RemoveUser Lot level access control List of users allowed to request sub-lots

› Attach / Detach Associates a path to a lot


Implementation - two choices

› Rely on the underlying quota system Kernel-assisted lot

› Provide the space guarantee within NeST User-level lot

› A third choice? Modify kernel


Dispatcher

Transfer MgrConcurrencies

Control flow

Data flow



HTTP


Storage Mgr

Lot Mgr


Kernel-assisted lot

Quota system


Kernel-assisted lot› Quota system provides the guarantee› Chirp-like protocols pose no challenge

Aware of transfer size

› GridFTP-like protocols ? Transfer size may be unknown Don’t worry at NeST level Quota system watches disk writes


Dispatcher

Concurrencies

Control flow

Data flow



HTTP



Storage Mgr

Lot Mgr

Transfer Mgr

User-level lot


User-level lot

› NeST provides the guarantee

› Chirp-like protocols pose no challenge

› GridFTP-like protocols ? Iterative data transfer Transfer manager repeatedly

queries lot manager


So which one ?

› Kernel-assisted Pros

Less complexPOSIX access

ConsNeeds root privilegesDepends on quota system – Less portabilityCannot distinguish among different allocations for

the same userDoes not have a hierarchical organization


So which one ?

› User-level Pros

Flexible, PortableHierarchical organization is easyEasy to differentiate among lots for the

same user Cons

Difficult to provide POSIX accessComplexity in the transfer and lot

managers


Big picture

NeST

Job

DAGMan

SRM

NeST

NeST

Remote site

GridFTP/DiskRouter transfer

StorkRequest allocation

Chirp allocations

Activate 3rd party put

Chirp input

Chirp output


Where are the overheads ?

› Lot operations

› Puts

› Recovery


Lot operations


Puts using Chirp

›With increase in transfer size overhead is amortized


Puts using GridFTP

›With increase in transfer size, overheads become comparable


Recovery time

›Compaction reduces lot overhead to a manageable level


Future Work

› Advance reservations

› Combo - Space and disk bandwidth reservation

› Recovery management Transaction based lot operations


Questions ?

› Demo on Wednesday Room 3351, CS building, 9am – noon

› More information available at www.cs.wisc.edu/condor/nest

› People John Bent, Joseph Stanley, Nick LeRoy, Alain

Roy Miron Livny, Andrea Arpaci-Dusseau, Remzi

Arpaci-Dusseau, Arie Shoshani

space allocations in nest

Documents