aos lab 12: network communication
TRANSCRIPT
Lab 12: Network CommunicationAdvanced Operating Systems
Zubair Nabi
April 24, 2013
Introduction
• In *nix systems, the networking infrastructure abstracts awaymany network architectures
• Each network architecture consists of• Network-communication protocols• The protocol family• Conventions of naming end-points• The address family or address format• Additional facilities
• Network facilities are accessed through the socket abstraction
Introduction
• In *nix systems, the networking infrastructure abstracts awaymany network architectures
• Each network architecture consists of• Network-communication protocols
• The protocol family• Conventions of naming end-points• The address family or address format• Additional facilities
• Network facilities are accessed through the socket abstraction
Introduction
• In *nix systems, the networking infrastructure abstracts awaymany network architectures
• Each network architecture consists of• Network-communication protocols• The protocol family
• Conventions of naming end-points• The address family or address format• Additional facilities
• Network facilities are accessed through the socket abstraction
Introduction
• In *nix systems, the networking infrastructure abstracts awaymany network architectures
• Each network architecture consists of• Network-communication protocols• The protocol family• Conventions of naming end-points
• The address family or address format• Additional facilities
• Network facilities are accessed through the socket abstraction
Introduction
• In *nix systems, the networking infrastructure abstracts awaymany network architectures
• Each network architecture consists of• Network-communication protocols• The protocol family• Conventions of naming end-points• The address family or address format
• Additional facilities
• Network facilities are accessed through the socket abstraction
Introduction
• In *nix systems, the networking infrastructure abstracts awaymany network architectures
• Each network architecture consists of• Network-communication protocols• The protocol family• Conventions of naming end-points• The address family or address format• Additional facilities
• Network facilities are accessed through the socket abstraction
Introduction
• In *nix systems, the networking infrastructure abstracts awaymany network architectures
• Each network architecture consists of• Network-communication protocols• The protocol family• Conventions of naming end-points• The address family or address format• Additional facilities
• Network facilities are accessed through the socket abstraction
Network Subsystem
Consists of three layers:
1 Transport layer: In charge of sockets-amenable addressingstructure and protocol mechanisms, such as ordering, reliability,etc.
2 Network layer: Responsible for delivery of data across thenetwork (must maintain a routing database)
3 Link layer: Responsible for shipping messages between hostsconnected to a common transmission medium
Network Subsystem
Consists of three layers:
1 Transport layer: In charge of sockets-amenable addressingstructure and protocol mechanisms, such as ordering, reliability,etc.
2 Network layer: Responsible for delivery of data across thenetwork (must maintain a routing database)
3 Link layer: Responsible for shipping messages between hostsconnected to a common transmission medium
Network Subsystem
Consists of three layers:
1 Transport layer: In charge of sockets-amenable addressingstructure and protocol mechanisms, such as ordering, reliability,etc.
2 Network layer: Responsible for delivery of data across thenetwork (must maintain a routing database)
3 Link layer: Responsible for shipping messages between hostsconnected to a common transmission medium
Network Subsystem (2)
• The layering is just logical layering
• The network service itself might choose to use more or fewerlayers based on its requirements
• For instance, raw sockets use a null implementation at one ormore layers
• Similarly, tunneling of one protocol through another requiresadditional implementations of multiple layers
Network Subsystem (2)
• The layering is just logical layering
• The network service itself might choose to use more or fewerlayers based on its requirements
• For instance, raw sockets use a null implementation at one ormore layers
• Similarly, tunneling of one protocol through another requiresadditional implementations of multiple layers
Network Subsystem (2)
• The layering is just logical layering
• The network service itself might choose to use more or fewerlayers based on its requirements
• For instance, raw sockets use a null implementation at one ormore layers
• Similarly, tunneling of one protocol through another requiresadditional implementations of multiple layers
Network Subsystem (2)
• The layering is just logical layering
• The network service itself might choose to use more or fewerlayers based on its requirements
• For instance, raw sockets use a null implementation at one ormore layers
• Similarly, tunneling of one protocol through another requiresadditional implementations of multiple layers
Memory Management
• Memory management for communication protocols is differentthan regular entities as memory is required in widely varying sizes
• A special-purpose memory-management facility exists for IPCand networking systems
• The unit of allocation is an mbuf (skbuff in Linux), which is128 bytes long with 100 or 108 bytes reserved for data
• A chain of mbufs can be linked together (m_next) to hold anarbitrary quantity of data
• For instance, a chain of mbufs is used to represent packets bynetwork protocols
Memory Management
• Memory management for communication protocols is differentthan regular entities as memory is required in widely varying sizes
• A special-purpose memory-management facility exists for IPCand networking systems
• The unit of allocation is an mbuf (skbuff in Linux), which is128 bytes long with 100 or 108 bytes reserved for data
• A chain of mbufs can be linked together (m_next) to hold anarbitrary quantity of data
• For instance, a chain of mbufs is used to represent packets bynetwork protocols
Memory Management
• Memory management for communication protocols is differentthan regular entities as memory is required in widely varying sizes
• A special-purpose memory-management facility exists for IPCand networking systems
• The unit of allocation is an mbuf (skbuff in Linux), which is128 bytes long with 100 or 108 bytes reserved for data
• A chain of mbufs can be linked together (m_next) to hold anarbitrary quantity of data
• For instance, a chain of mbufs is used to represent packets bynetwork protocols
Memory Management
• Memory management for communication protocols is differentthan regular entities as memory is required in widely varying sizes
• A special-purpose memory-management facility exists for IPCand networking systems
• The unit of allocation is an mbuf (skbuff in Linux), which is128 bytes long with 100 or 108 bytes reserved for data
• A chain of mbufs can be linked together (m_next) to hold anarbitrary quantity of data
• For instance, a chain of mbufs is used to represent packets bynetwork protocols
Memory Management
• Memory management for communication protocols is differentthan regular entities as memory is required in widely varying sizes
• A special-purpose memory-management facility exists for IPCand networking systems
• The unit of allocation is an mbuf (skbuff in Linux), which is128 bytes long with 100 or 108 bytes reserved for data
• A chain of mbufs can be linked together (m_next) to hold anarbitrary quantity of data
• For instance, a chain of mbufs is used to represent packets bynetwork protocols
mbuf
Data Flow
• Socket-to-network-subsystem• Calls the transport-layer modules that support the socket
abstraction
• Typically started by system calls
• Network-subsystem-to-socket• Flows up the stack and is placed in the receive queue of the
destination socket• Asynchronously received and added to the per-protocol input
message queue
Data Flow
• Socket-to-network-subsystem• Calls the transport-layer modules that support the socket
abstraction• Typically started by system calls
• Network-subsystem-to-socket• Flows up the stack and is placed in the receive queue of the
destination socket• Asynchronously received and added to the per-protocol input
message queue
Data Flow
• Socket-to-network-subsystem• Calls the transport-layer modules that support the socket
abstraction• Typically started by system calls
• Network-subsystem-to-socket• Flows up the stack and is placed in the receive queue of the
destination socket
• Asynchronously received and added to the per-protocol inputmessage queue
Data Flow
• Socket-to-network-subsystem• Calls the transport-layer modules that support the socket
abstraction• Typically started by system calls
• Network-subsystem-to-socket• Flows up the stack and is placed in the receive queue of the
destination socket• Asynchronously received and added to the per-protocol input
message queue
Upwards Dataflow
Network Protocols
• Defined by a set of conventions, including packet formats, states,and state transitions
• Each communication-protocol module implements a particularprotocol and is made up of a collection of procedures and privatedata structures
• The external interface of a module is described by aprotocol-switch structure
• This interface is used by the socket layer for all interaction• The address of this structure is present within the socket’sso_proto field
• Each time a socket is created the protocol is selected based onthe type of socket (pr_type)
• Also in charge of mbuf storage management
Network Protocols
• Defined by a set of conventions, including packet formats, states,and state transitions
• Each communication-protocol module implements a particularprotocol and is made up of a collection of procedures and privatedata structures
• The external interface of a module is described by aprotocol-switch structure
• This interface is used by the socket layer for all interaction• The address of this structure is present within the socket’sso_proto field
• Each time a socket is created the protocol is selected based onthe type of socket (pr_type)
• Also in charge of mbuf storage management
Network Protocols
• Defined by a set of conventions, including packet formats, states,and state transitions
• Each communication-protocol module implements a particularprotocol and is made up of a collection of procedures and privatedata structures
• The external interface of a module is described by aprotocol-switch structure
• This interface is used by the socket layer for all interaction• The address of this structure is present within the socket’sso_proto field
• Each time a socket is created the protocol is selected based onthe type of socket (pr_type)
• Also in charge of mbuf storage management
Network Protocols
• Defined by a set of conventions, including packet formats, states,and state transitions
• Each communication-protocol module implements a particularprotocol and is made up of a collection of procedures and privatedata structures
• The external interface of a module is described by aprotocol-switch structure
• This interface is used by the socket layer for all interaction
• The address of this structure is present within the socket’sso_proto field
• Each time a socket is created the protocol is selected based onthe type of socket (pr_type)
• Also in charge of mbuf storage management
Network Protocols
• Defined by a set of conventions, including packet formats, states,and state transitions
• Each communication-protocol module implements a particularprotocol and is made up of a collection of procedures and privatedata structures
• The external interface of a module is described by aprotocol-switch structure
• This interface is used by the socket layer for all interaction• The address of this structure is present within the socket’sso_proto field
• Each time a socket is created the protocol is selected based onthe type of socket (pr_type)
• Also in charge of mbuf storage management
Network Protocols
• Defined by a set of conventions, including packet formats, states,and state transitions
• Each communication-protocol module implements a particularprotocol and is made up of a collection of procedures and privatedata structures
• The external interface of a module is described by aprotocol-switch structure
• This interface is used by the socket layer for all interaction• The address of this structure is present within the socket’sso_proto field
• Each time a socket is created the protocol is selected based onthe type of socket (pr_type)
• Also in charge of mbuf storage management
Network Protocols
• Defined by a set of conventions, including packet formats, states,and state transitions
• Each communication-protocol module implements a particularprotocol and is made up of a collection of procedures and privatedata structures
• The external interface of a module is described by aprotocol-switch structure
• This interface is used by the socket layer for all interaction• The address of this structure is present within the socket’sso_proto field
• Each time a socket is created the protocol is selected based onthe type of socket (pr_type)
• Also in charge of mbuf storage management
Network Interfaces
• Each interface defines a link-layer path through which messagescan be sent and received
• Typically, a hardware device is represented by this interface• The loopback interface is in software which is used to route
traffic to local sockets
• Also in charge of encapsulation and decapsulation of link-layerprotocol headers
• Typically implemented as a separate layer that is shared byvarious hardware drivers
• The selection of the interface is taken care of by the network-layerprotocol
• Represented by an ifnet structure
Network Interfaces
• Each interface defines a link-layer path through which messagescan be sent and received
• Typically, a hardware device is represented by this interface
• The loopback interface is in software which is used to routetraffic to local sockets
• Also in charge of encapsulation and decapsulation of link-layerprotocol headers
• Typically implemented as a separate layer that is shared byvarious hardware drivers
• The selection of the interface is taken care of by the network-layerprotocol
• Represented by an ifnet structure
Network Interfaces
• Each interface defines a link-layer path through which messagescan be sent and received
• Typically, a hardware device is represented by this interface• The loopback interface is in software which is used to route
traffic to local sockets
• Also in charge of encapsulation and decapsulation of link-layerprotocol headers
• Typically implemented as a separate layer that is shared byvarious hardware drivers
• The selection of the interface is taken care of by the network-layerprotocol
• Represented by an ifnet structure
Network Interfaces
• Each interface defines a link-layer path through which messagescan be sent and received
• Typically, a hardware device is represented by this interface• The loopback interface is in software which is used to route
traffic to local sockets
• Also in charge of encapsulation and decapsulation of link-layerprotocol headers
• Typically implemented as a separate layer that is shared byvarious hardware drivers
• The selection of the interface is taken care of by the network-layerprotocol
• Represented by an ifnet structure
Network Interfaces
• Each interface defines a link-layer path through which messagescan be sent and received
• Typically, a hardware device is represented by this interface• The loopback interface is in software which is used to route
traffic to local sockets
• Also in charge of encapsulation and decapsulation of link-layerprotocol headers
• Typically implemented as a separate layer that is shared byvarious hardware drivers
• The selection of the interface is taken care of by the network-layerprotocol
• Represented by an ifnet structure
Network Interfaces
• Each interface defines a link-layer path through which messagescan be sent and received
• Typically, a hardware device is represented by this interface• The loopback interface is in software which is used to route
traffic to local sockets
• Also in charge of encapsulation and decapsulation of link-layerprotocol headers
• Typically implemented as a separate layer that is shared byvarious hardware drivers
• The selection of the interface is taken care of by the network-layerprotocol
• Represented by an ifnet structure
Network Interfaces
• Each interface defines a link-layer path through which messagescan be sent and received
• Typically, a hardware device is represented by this interface• The loopback interface is in software which is used to route
traffic to local sockets
• Also in charge of encapsulation and decapsulation of link-layerprotocol headers
• Typically implemented as a separate layer that is shared byvarious hardware drivers
• The selection of the interface is taken care of by the network-layerprotocol
• Represented by an ifnet structure
Socket-to-Protocol Interface
• Enabled by two routines: 1) User request, pr_usrreq() and2) Control output, pr_ctloutput()
• These methods are present in the protocol-switch table for eachprotocol
• Control-output: Implements getsockopt and setsockoptsystem calls
• User-request: Implements all other operations
Socket-to-Protocol Interface
• Enabled by two routines: 1) User request, pr_usrreq() and2) Control output, pr_ctloutput()
• These methods are present in the protocol-switch table for eachprotocol
• Control-output: Implements getsockopt and setsockoptsystem calls
• User-request: Implements all other operations
Socket-to-Protocol Interface
• Enabled by two routines: 1) User request, pr_usrreq() and2) Control output, pr_ctloutput()
• These methods are present in the protocol-switch table for eachprotocol
• Control-output: Implements getsockopt and setsockoptsystem calls
• User-request: Implements all other operations
Socket-to-Protocol Interface
• Enabled by two routines: 1) User request, pr_usrreq() and2) Control output, pr_ctloutput()
• These methods are present in the protocol-switch table for eachprotocol
• Control-output: Implements getsockopt and setsockoptsystem calls
• User-request: Implements all other operations
Protocol-to-Network-Interface Interface
• Lowest layer in the protocol family must interact with one or moreinterfaces to send and receive packets
• Obviously a routing decision must have already chosen theoutgoing interface
Protocol-to-Network-Interface Interface
• Lowest layer in the protocol family must interact with one or moreinterfaces to send and receive packets
• Obviously a routing decision must have already chosen theoutgoing interface
Code: Packet Sending
error = (*ifp->if_output)(ifp, m, dst, rt);
struct ifnet *ifp;struct mbuf *m;struct sockaddr *dst;struct rtentry *rt;
Packet Sending
• Packet m is transmitted to destination dst via interface ifp
• Steps:• Validation of the destination address• Queuing of the packet on the send queue• If the interface is not busy, using an interrupt-driven routine to
transmit the packet
• The link-layer address is chosen by ARP in case of Ethernet
Packet Sending
• Packet m is transmitted to destination dst via interface ifp• Steps:
• Validation of the destination address
• Queuing of the packet on the send queue• If the interface is not busy, using an interrupt-driven routine to
transmit the packet
• The link-layer address is chosen by ARP in case of Ethernet
Packet Sending
• Packet m is transmitted to destination dst via interface ifp• Steps:
• Validation of the destination address• Queuing of the packet on the send queue
• If the interface is not busy, using an interrupt-driven routine totransmit the packet
• The link-layer address is chosen by ARP in case of Ethernet
Packet Sending
• Packet m is transmitted to destination dst via interface ifp• Steps:
• Validation of the destination address• Queuing of the packet on the send queue• If the interface is not busy, using an interrupt-driven routine to
transmit the packet
• The link-layer address is chosen by ARP in case of Ethernet
Packet Sending
• Packet m is transmitted to destination dst via interface ifp• Steps:
• Validation of the destination address• Queuing of the packet on the send queue• If the interface is not busy, using an interrupt-driven routine to
transmit the packet
• The link-layer address is chosen by ARP in case of Ethernet
Packet Receiving
• Incoming packets are queued in the corresponding protocol’sinput packet queue
• A software interrupt is posted to initiate network-layer processing
Packet Receiving
• Incoming packets are queued in the corresponding protocol’sinput packet queue
• A software interrupt is posted to initiate network-layer processing
Code: Packet Receiving
if (IF_QFULL(&ipintrq)) {IF_DROP(&ipintrq);ifp->if_iqdrops++;m_freem(m);
} else {schednetisr(NETISR_IP);IF_ENQUEUE(&ipintrq, m)
}
Routing
• The routing system has two components; one within the kerneland one in user-space
• The routing mechanism is present within the kernel while routingpolicies are defined in user-space
• The routing mechanism involves a table lookup to get a first-hopfor a given destination
• Routing policies include components that help in choosingfirst-hop routes
Routing
• The routing system has two components; one within the kerneland one in user-space
• The routing mechanism is present within the kernel while routingpolicies are defined in user-space
• The routing mechanism involves a table lookup to get a first-hopfor a given destination
• Routing policies include components that help in choosingfirst-hop routes
Routing
• The routing system has two components; one within the kerneland one in user-space
• The routing mechanism is present within the kernel while routingpolicies are defined in user-space
• The routing mechanism involves a table lookup to get a first-hopfor a given destination
• Routing policies include components that help in choosingfirst-hop routes
Routing
• The routing system has two components; one within the kerneland one in user-space
• The routing mechanism is present within the kernel while routingpolicies are defined in user-space
• The routing mechanism involves a table lookup to get a first-hopfor a given destination
• Routing policies include components that help in choosingfirst-hop routes
Kernel Routing Mechanism
• Implements a routing table for first/next hop lookup
• Two distinct portions:1 A data structure with routing entries, one per specific route2 A lookup algorithm to locate the correct route for each possible
destination
• Each destination is represented by a sockaddr structure• Routes are either:
1 Host or network2 Direct or indirect
Kernel Routing Mechanism
• Implements a routing table for first/next hop lookup• Two distinct portions:
1 A data structure with routing entries, one per specific route
2 A lookup algorithm to locate the correct route for each possibledestination
• Each destination is represented by a sockaddr structure• Routes are either:
1 Host or network2 Direct or indirect
Kernel Routing Mechanism
• Implements a routing table for first/next hop lookup• Two distinct portions:
1 A data structure with routing entries, one per specific route2 A lookup algorithm to locate the correct route for each possible
destination
• Each destination is represented by a sockaddr structure• Routes are either:
1 Host or network2 Direct or indirect
Kernel Routing Mechanism
• Implements a routing table for first/next hop lookup• Two distinct portions:
1 A data structure with routing entries, one per specific route2 A lookup algorithm to locate the correct route for each possible
destination
• Each destination is represented by a sockaddr structure
• Routes are either:1 Host or network2 Direct or indirect
Kernel Routing Mechanism
• Implements a routing table for first/next hop lookup• Two distinct portions:
1 A data structure with routing entries, one per specific route2 A lookup algorithm to locate the correct route for each possible
destination
• Each destination is represented by a sockaddr structure• Routes are either:
1 Host or network
2 Direct or indirect
Kernel Routing Mechanism
• Implements a routing table for first/next hop lookup• Two distinct portions:
1 A data structure with routing entries, one per specific route2 A lookup algorithm to locate the correct route for each possible
destination
• Each destination is represented by a sockaddr structure• Routes are either:
1 Host or network2 Direct or indirect
User-space Routing Policies
• Policies add, delete, or modify kernel routing table entries
• A number of routing policies exist, including the RoutingInformation Protocol (RIP)
User-space Routing Policies
• Policies add, delete, or modify kernel routing table entries
• A number of routing policies exist, including the RoutingInformation Protocol (RIP)
Protocol Control Blocks
• For each TCP or UDP socket, an Internet protocol control block(inpcb) is created to hold address, ports, routing information,and pointers to any additional data structures
• TCP in addition creates a TCP control block (tcpcb) to holdimplementation-specific information
• TCP and UDP protocol modules each have a private doublylinked list of inpcbs
• Common routines are used by the modules to manipulate theselists
• Traffic is multiplexed by the IP layer on the basis of the protocolidentifier in the protocol and passed on to the individual transportprotocol
• Each protocol is then responsible for passing a direct message tothe appropriate socket
Protocol Control Blocks
• For each TCP or UDP socket, an Internet protocol control block(inpcb) is created to hold address, ports, routing information,and pointers to any additional data structures
• TCP in addition creates a TCP control block (tcpcb) to holdimplementation-specific information
• TCP and UDP protocol modules each have a private doublylinked list of inpcbs
• Common routines are used by the modules to manipulate theselists
• Traffic is multiplexed by the IP layer on the basis of the protocolidentifier in the protocol and passed on to the individual transportprotocol
• Each protocol is then responsible for passing a direct message tothe appropriate socket
Protocol Control Blocks
• For each TCP or UDP socket, an Internet protocol control block(inpcb) is created to hold address, ports, routing information,and pointers to any additional data structures
• TCP in addition creates a TCP control block (tcpcb) to holdimplementation-specific information
• TCP and UDP protocol modules each have a private doublylinked list of inpcbs
• Common routines are used by the modules to manipulate theselists
• Traffic is multiplexed by the IP layer on the basis of the protocolidentifier in the protocol and passed on to the individual transportprotocol
• Each protocol is then responsible for passing a direct message tothe appropriate socket
Protocol Control Blocks
• For each TCP or UDP socket, an Internet protocol control block(inpcb) is created to hold address, ports, routing information,and pointers to any additional data structures
• TCP in addition creates a TCP control block (tcpcb) to holdimplementation-specific information
• TCP and UDP protocol modules each have a private doublylinked list of inpcbs
• Common routines are used by the modules to manipulate theselists
• Traffic is multiplexed by the IP layer on the basis of the protocolidentifier in the protocol and passed on to the individual transportprotocol
• Each protocol is then responsible for passing a direct message tothe appropriate socket
Protocol Control Blocks
• For each TCP or UDP socket, an Internet protocol control block(inpcb) is created to hold address, ports, routing information,and pointers to any additional data structures
• TCP in addition creates a TCP control block (tcpcb) to holdimplementation-specific information
• TCP and UDP protocol modules each have a private doublylinked list of inpcbs
• Common routines are used by the modules to manipulate theselists
• Traffic is multiplexed by the IP layer on the basis of the protocolidentifier in the protocol and passed on to the individual transportprotocol
• Each protocol is then responsible for passing a direct message tothe appropriate socket
Protocol Control Blocks
• For each TCP or UDP socket, an Internet protocol control block(inpcb) is created to hold address, ports, routing information,and pointers to any additional data structures
• TCP in addition creates a TCP control block (tcpcb) to holdimplementation-specific information
• TCP and UDP protocol modules each have a private doublylinked list of inpcbs
• Common routines are used by the modules to manipulate theselists
• Traffic is multiplexed by the IP layer on the basis of the protocolidentifier in the protocol and passed on to the individual transportprotocol
• Each protocol is then responsible for passing a direct message tothe appropriate socket
Today’s task
• Design a network subsystem for xv6
Reading(s)
• Chapter 12 and 13 from “The Design and Implementation of the4.4BSD Operating System” by Marshall Kirk McKusick, KeithBostic, Michael J. Karels, and John S. Quarterman.