a multicast transport driver for globus xio
DESCRIPTION
A Multicast Transport Driver for Globus XIO. Karl Jeacle. [email protected]. Rationale. Would like to achieve high-speed bulk data delivery to multiple sites Multicasting would make sense Existing multicast research has focused on sending to a large number of receivers - PowerPoint PPT PresentationTRANSCRIPT
ETNGRID 2005
Rationale
Would like to achieve high-speed bulk data delivery to multiple sites
Multicasting would make sense Existing multicast research has focused on
sending to a large number of receivers But Grid is an applied example where sending to
a moderate number of receivers would be extremely beneficial
ETNGRID 2005
Multicast availability
Deployment is a problem!Protocols have been defined and implementedValid concerns about scalability; much FUD “chicken & egg” means limited coverage
Clouds of native multicastBut can’t reach all destinations via multicastSo applications abandon in favour of unicast
What if we could multicast when possible……but fall back to unicast when necessary?
ETNGRID 2005
Multicast TCP?
TCP “single reliable stream between two hosts”
Multicast TCP “multiple reliable streams from one to n hosts”
May seem a little odd, but there is precedent…TCP-XMO – Liang & CheritonM-TCP – Mysore & VargheseM/TCP – Visoottiviseth et alPRMP – Barcellos et alSCE – Talpade & Ammar
ETNGRID 2005
Building Multicast TCP
Want to test multicast/unicast TCP approachBut new protocol == kernel changeWidespread test deployment difficult
Build new TCP-like engineEncapsulate packets in UDPRun in userspacePerformance is sacrificed……but widespread testing now possible
ETNGRID 2005
TCP/IP/UDP/IP
SendingApplication
TCP
IP
UDP
IP
ReceivingApplication
TCP
IP
UDP
IP
If natively implemented
test deployment
ETNGRID 2005
TCP engine
Where does initial TCP come from? Could use BSD or Linux
Extracting from kernel could be problematic
More compact alternative lwIP = Lightweight IPSmall but fully RFC-compliant TCP/IP stack
lwIP + multicast extensions = “TCP-XM”
ETNGRID 2005
TCP-XM overview
Primarily aimed at push applications Sender initiated – advance knowledge of receivers Opens sessions to n destination hosts simultaneously Unicast is used when multicast not available Options headers used to exchange multicast info API changes
Sender incorporates multiple destination and group addresses Receiver requires no changes
TCP friendly, by definition
ETNGRID 2005
TCP-XM connection
ConnectionUser connects to multiple unicast destinationsMultiple TCP PCBs created Independent 3-way handshakes take placeSSM or random ASM group address allocated
(if not specified in advance by user/application)
Group address sent as TCP optionAbility to multicast depends on TCP option
ETNGRID 2005
TCP-XM transmission
Data transferData replicated/enqueued on all send queuesPCB variables dictate transmission modeData packets are multicast (if possible)Retransmissions are unicastAuto fall back/forward to unicast/multicast
CloseConnections closed as per unicast TCP
ETNGRID 2005
Fall back / fall forward
TCP-XM principle “Multicast if possible, unicast when necessary”
Initial transmission mode is group unicastEnsures successful initial data transfer
Fall forward to multicast on positive feedbackTypically after ~75K unicast data
Fall back to unicast on repeated mcast failure
ETNGRID 2005
TCP-XM reception
ReceiverNo API-level changesNormal TCP listenAuto-IGMP join on TCP-XM connectAccepts data on both unicast/multicast ports tcp_input() accepts:
packets addressed to existing unicast destination……but now also those addressed to multicast group
Tracks how last n segs received (u/m)
ETNGRID 2005
Grid multicast?
How can multicast be used in Grid environment?
TCP-XM is new multicast-capable protocol
Globus is de-facto Grid middleware
Would like TCP-XM support in Globus…
ETNGRID 2005
Globus XIO
eXtensible Input Output libraryAllows “i/o plugins” to Globus
APISingle POSIX-like API / set of semanticsSimple open/close/read/write API
Driver abstractionHides protocol details / Allows for extensibilityStack of 1 transport & n transform driversDrivers can be selected at runtime
ETNGRID 2005
XIO/XM driver specifics
Two important XIO data structures
1. Handle Returned to user when XIO framework ready Used for all open/close/read/write calls lwIP netconn connection structure used
2. Attribute Used to set XIO driver-specific parameters… … and TCP-XM protocol-specific options List of destination addresses
ETNGRID 2005
XIO code example// init stackglobus_xio_stack_init(&stack, NULL);
// load drivers onto stackglobus_xio_driver_load("tcpxm", &txdriver);globus_xio_stack_push_driver(stack, txdriver);
// init attributesglobus_xio_attr_init(&attr);globus_xio_attr_cntl(attr, txdriver, GLOBUS_XIO_TCPXM_SET_REMOTE_HOSTS,
hosts, numhosts);
// create handleglobus_xio_handle_create(&handle, stack);
// send dataglobus_xio_open(&handle, NULL, target);globus_xio_write(handle, "hello\n", 6, 1, &nbytes, NULL);globus_xio_close(handle, NULL);
ETNGRID 2005
One-to-many issues
Stack assumes one-to-one connectionsXIO user interface requires modificationNeeds support for one-to-many protocolsMinimal user API changesFramework changes more significant
GSI is one-to-oneAuthentication with peer on connection setupBut cannot authenticate with n peersNeed some form of “GSI-M”
ETNGRID 2005
Driver availability
Multicast transport driver for Globus XIORequires Globus 3.2 or later
Source code onlineSample clientSample serverDriver installation instructions
http://www.cl.cam.ac.uk/~kj234/xio/
ETNGRID 2005
mcp & mcpd
Multicast file transfer application using TCP-XM
‘mcpd &’ on servers ‘mcp file host1 host2… hostN’ on client
http://www.cl.cam.ac.uk/~kj234/mcp/Full source code onlineFreeBSD, Linux, Solaris