aos lab 12: network communication

Lab 12: Network CommunicationAdvanced Operating Systems

Zubair Nabi

zubair.nabi@itu.edu.pk

April 24, 2013

Introduction

• In *nix systems, the networking infrastructure abstracts awaymany network architectures

• Each network architecture consists of• Network-communication protocols• The protocol family• Conventions of naming end-points• The address family or address format• Additional facilities

• Network facilities are accessed through the socket abstraction

Introduction

• In *nix systems, the networking infrastructure abstracts awaymany network architectures

• Each network architecture consists of• Network-communication protocols

• The protocol family• Conventions of naming end-points• The address family or address format• Additional facilities

• Network facilities are accessed through the socket abstraction

Introduction

• In *nix systems, the networking infrastructure abstracts awaymany network architectures

• Each network architecture consists of• Network-communication protocols• The protocol family

• Conventions of naming end-points• The address family or address format• Additional facilities

• Network facilities are accessed through the socket abstraction

Introduction

• In *nix systems, the networking infrastructure abstracts awaymany network architectures

• Each network architecture consists of• Network-communication protocols• The protocol family• Conventions of naming end-points

• The address family or address format• Additional facilities

• Network facilities are accessed through the socket abstraction

Introduction

• In *nix systems, the networking infrastructure abstracts awaymany network architectures

• Each network architecture consists of• Network-communication protocols• The protocol family• Conventions of naming end-points• The address family or address format

• Additional facilities

• Network facilities are accessed through the socket abstraction

Introduction

• In *nix systems, the networking infrastructure abstracts awaymany network architectures

• Each network architecture consists of• Network-communication protocols• The protocol family• Conventions of naming end-points• The address family or address format• Additional facilities

• Network facilities are accessed through the socket abstraction

Introduction

• In *nix systems, the networking infrastructure abstracts awaymany network architectures

• Each network architecture consists of• Network-communication protocols• The protocol family• Conventions of naming end-points• The address family or address format• Additional facilities

• Network facilities are accessed through the socket abstraction

Network Subsystem

Consists of three layers:

1 Transport layer: In charge of sockets-amenable addressingstructure and protocol mechanisms, such as ordering, reliability,etc.

2 Network layer: Responsible for delivery of data across thenetwork (must maintain a routing database)

3 Link layer: Responsible for shipping messages between hostsconnected to a common transmission medium

Network Subsystem

Consists of three layers:

1 Transport layer: In charge of sockets-amenable addressingstructure and protocol mechanisms, such as ordering, reliability,etc.

2 Network layer: Responsible for delivery of data across thenetwork (must maintain a routing database)

3 Link layer: Responsible for shipping messages between hostsconnected to a common transmission medium

Network Subsystem

Consists of three layers:

1 Transport layer: In charge of sockets-amenable addressingstructure and protocol mechanisms, such as ordering, reliability,etc.

2 Network layer: Responsible for delivery of data across thenetwork (must maintain a routing database)

3 Link layer: Responsible for shipping messages between hostsconnected to a common transmission medium

Network Subsystem (2)

• The layering is just logical layering

• The network service itself might choose to use more or fewerlayers based on its requirements

• For instance, raw sockets use a null implementation at one ormore layers

• Similarly, tunneling of one protocol through another requiresadditional implementations of multiple layers

Network Subsystem (2)

• The layering is just logical layering

• The network service itself might choose to use more or fewerlayers based on its requirements

• For instance, raw sockets use a null implementation at one ormore layers

• Similarly, tunneling of one protocol through another requiresadditional implementations of multiple layers

Network Subsystem (2)

• The layering is just logical layering

• The network service itself might choose to use more or fewerlayers based on its requirements

• For instance, raw sockets use a null implementation at one ormore layers

• Similarly, tunneling of one protocol through another requiresadditional implementations of multiple layers

Network Subsystem (2)

• The layering is just logical layering

• The network service itself might choose to use more or fewerlayers based on its requirements

• For instance, raw sockets use a null implementation at one ormore layers

• Similarly, tunneling of one protocol through another requiresadditional implementations of multiple layers

Memory Management

• Memory management for communication protocols is differentthan regular entities as memory is required in widely varying sizes

• A special-purpose memory-management facility exists for IPCand networking systems

• The unit of allocation is an mbuf (skbuff in Linux), which is128 bytes long with 100 or 108 bytes reserved for data

• A chain of mbufs can be linked together (m_next) to hold anarbitrary quantity of data

• For instance, a chain of mbufs is used to represent packets bynetwork protocols

Memory Management

• Memory management for communication protocols is differentthan regular entities as memory is required in widely varying sizes

• A special-purpose memory-management facility exists for IPCand networking systems

• The unit of allocation is an mbuf (skbuff in Linux), which is128 bytes long with 100 or 108 bytes reserved for data

• A chain of mbufs can be linked together (m_next) to hold anarbitrary quantity of data

• For instance, a chain of mbufs is used to represent packets bynetwork protocols

Memory Management

• Memory management for communication protocols is differentthan regular entities as memory is required in widely varying sizes

• A special-purpose memory-management facility exists for IPCand networking systems

• The unit of allocation is an mbuf (skbuff in Linux), which is128 bytes long with 100 or 108 bytes reserved for data

• A chain of mbufs can be linked together (m_next) to hold anarbitrary quantity of data

• For instance, a chain of mbufs is used to represent packets bynetwork protocols

Memory Management

• Memory management for communication protocols is differentthan regular entities as memory is required in widely varying sizes

• A special-purpose memory-management facility exists for IPCand networking systems

• The unit of allocation is an mbuf (skbuff in Linux), which is128 bytes long with 100 or 108 bytes reserved for data

• A chain of mbufs can be linked together (m_next) to hold anarbitrary quantity of data

• For instance, a chain of mbufs is used to represent packets bynetwork protocols

Memory Management

• Memory management for communication protocols is differentthan regular entities as memory is required in widely varying sizes

• A special-purpose memory-management facility exists for IPCand networking systems

• The unit of allocation is an mbuf (skbuff in Linux), which is128 bytes long with 100 or 108 bytes reserved for data

• A chain of mbufs can be linked together (m_next) to hold anarbitrary quantity of data

• For instance, a chain of mbufs is used to represent packets bynetwork protocols

mbuf

Data Flow

• Socket-to-network-subsystem• Calls the transport-layer modules that support the socket

abstraction

• Typically started by system calls

• Network-subsystem-to-socket• Flows up the stack and is placed in the receive queue of the

destination socket• Asynchronously received and added to the per-protocol input

message queue

Data Flow

• Socket-to-network-subsystem• Calls the transport-layer modules that support the socket

abstraction• Typically started by system calls

• Network-subsystem-to-socket• Flows up the stack and is placed in the receive queue of the

destination socket• Asynchronously received and added to the per-protocol input

message queue

Data Flow

• Socket-to-network-subsystem• Calls the transport-layer modules that support the socket

abstraction• Typically started by system calls

• Network-subsystem-to-socket• Flows up the stack and is placed in the receive queue of the

destination socket

• Asynchronously received and added to the per-protocol inputmessage queue

Data Flow

• Socket-to-network-subsystem• Calls the transport-layer modules that support the socket

abstraction• Typically started by system calls

• Network-subsystem-to-socket• Flows up the stack and is placed in the receive queue of the

destination socket• Asynchronously received and added to the per-protocol input

message queue

Upwards Dataflow

Network Protocols

• Defined by a set of conventions, including packet formats, states,and state transitions

• Each communication-protocol module implements a particularprotocol and is made up of a collection of procedures and privatedata structures

• The external interface of a module is described by aprotocol-switch structure

• This interface is used by the socket layer for all interaction• The address of this structure is present within the socket’sso_proto field

• Each time a socket is created the protocol is selected based onthe type of socket (pr_type)

• Also in charge of mbuf storage management

Network Protocols

• Defined by a set of conventions, including packet formats, states,and state transitions

• Each communication-protocol module implements a particularprotocol and is made up of a collection of procedures and privatedata structures

• The external interface of a module is described by aprotocol-switch structure

• This interface is used by the socket layer for all interaction• The address of this structure is present within the socket’sso_proto field

• Each time a socket is created the protocol is selected based onthe type of socket (pr_type)

• Also in charge of mbuf storage management

Network Protocols

• Defined by a set of conventions, including packet formats, states,and state transitions

• Each communication-protocol module implements a particularprotocol and is made up of a collection of procedures and privatedata structures

• The external interface of a module is described by aprotocol-switch structure

• This interface is used by the socket layer for all interaction• The address of this structure is present within the socket’sso_proto field

• Each time a socket is created the protocol is selected based onthe type of socket (pr_type)

• Also in charge of mbuf storage management

Network Protocols

• Defined by a set of conventions, including packet formats, states,and state transitions

• Each communication-protocol module implements a particularprotocol and is made up of a collection of procedures and privatedata structures

• The external interface of a module is described by aprotocol-switch structure

• This interface is used by the socket layer for all interaction

• The address of this structure is present within the socket’sso_proto field

• Each time a socket is created the protocol is selected based onthe type of socket (pr_type)

• Also in charge of mbuf storage management

Network Protocols

• Defined by a set of conventions, including packet formats, states,and state transitions

• Each communication-protocol module implements a particularprotocol and is made up of a collection of procedures and privatedata structures

• The external interface of a module is described by aprotocol-switch structure

• This interface is used by the socket layer for all interaction• The address of this structure is present within the socket’sso_proto field

• Each time a socket is created the protocol is selected based onthe type of socket (pr_type)

• Also in charge of mbuf storage management

Network Protocols

• Defined by a set of conventions, including packet formats, states,and state transitions

• Each communication-protocol module implements a particularprotocol and is made up of a collection of procedures and privatedata structures

• The external interface of a module is described by aprotocol-switch structure

• This interface is used by the socket layer for all interaction• The address of this structure is present within the socket’sso_proto field

• Each time a socket is created the protocol is selected based onthe type of socket (pr_type)

• Also in charge of mbuf storage management

Network Protocols

• Defined by a set of conventions, including packet formats, states,and state transitions

• Each communication-protocol module implements a particularprotocol and is made up of a collection of procedures and privatedata structures

• The external interface of a module is described by aprotocol-switch structure

• This interface is used by the socket layer for all interaction• The address of this structure is present within the socket’sso_proto field

• Each time a socket is created the protocol is selected based onthe type of socket (pr_type)

• Also in charge of mbuf storage management

Network Interfaces

• Each interface defines a link-layer path through which messagescan be sent and received

• Typically, a hardware device is represented by this interface• The loopback interface is in software which is used to route

traffic to local sockets

• Also in charge of encapsulation and decapsulation of link-layerprotocol headers

• Typically implemented as a separate layer that is shared byvarious hardware drivers

• The selection of the interface is taken care of by the network-layerprotocol

• Represented by an ifnet structure

Network Interfaces

• Each interface defines a link-layer path through which messagescan be sent and received

• Typically, a hardware device is represented by this interface

• The loopback interface is in software which is used to routetraffic to local sockets

• Also in charge of encapsulation and decapsulation of link-layerprotocol headers

• Typically implemented as a separate layer that is shared byvarious hardware drivers

• The selection of the interface is taken care of by the network-layerprotocol

• Represented by an ifnet structure

Network Interfaces

• Each interface defines a link-layer path through which messagescan be sent and received

• Typically, a hardware device is represented by this interface• The loopback interface is in software which is used to route

traffic to local sockets

• Also in charge of encapsulation and decapsulation of link-layerprotocol headers

• Typically implemented as a separate layer that is shared byvarious hardware drivers

• The selection of the interface is taken care of by the network-layerprotocol

• Represented by an ifnet structure

Network Interfaces

• Each interface defines a link-layer path through which messagescan be sent and received

• Typically, a hardware device is represented by this interface• The loopback interface is in software which is used to route

traffic to local sockets

• Also in charge of encapsulation and decapsulation of link-layerprotocol headers

• Typically implemented as a separate layer that is shared byvarious hardware drivers

• The selection of the interface is taken care of by the network-layerprotocol

• Represented by an ifnet structure

Network Interfaces

• Each interface defines a link-layer path through which messagescan be sent and received

• Typically, a hardware device is represented by this interface• The loopback interface is in software which is used to route

traffic to local sockets

• Also in charge of encapsulation and decapsulation of link-layerprotocol headers

• Typically implemented as a separate layer that is shared byvarious hardware drivers

• The selection of the interface is taken care of by the network-layerprotocol

• Represented by an ifnet structure

Network Interfaces

• Each interface defines a link-layer path through which messagescan be sent and received

• Typically, a hardware device is represented by this interface• The loopback interface is in software which is used to route

traffic to local sockets

• Also in charge of encapsulation and decapsulation of link-layerprotocol headers

• Typically implemented as a separate layer that is shared byvarious hardware drivers

• The selection of the interface is taken care of by the network-layerprotocol

• Represented by an ifnet structure

Network Interfaces

• Each interface defines a link-layer path through which messagescan be sent and received

• Typically, a hardware device is represented by this interface• The loopback interface is in software which is used to route

traffic to local sockets

• Also in charge of encapsulation and decapsulation of link-layerprotocol headers

• Typically implemented as a separate layer that is shared byvarious hardware drivers

• The selection of the interface is taken care of by the network-layerprotocol

• Represented by an ifnet structure

Socket-to-Protocol Interface

• Enabled by two routines: 1) User request, pr_usrreq() and2) Control output, pr_ctloutput()

• These methods are present in the protocol-switch table for eachprotocol

• Control-output: Implements getsockopt and setsockoptsystem calls

• User-request: Implements all other operations

Socket-to-Protocol Interface

• Enabled by two routines: 1) User request, pr_usrreq() and2) Control output, pr_ctloutput()

• These methods are present in the protocol-switch table for eachprotocol

• Control-output: Implements getsockopt and setsockoptsystem calls

• User-request: Implements all other operations

Socket-to-Protocol Interface

• Enabled by two routines: 1) User request, pr_usrreq() and2) Control output, pr_ctloutput()

• These methods are present in the protocol-switch table for eachprotocol

• Control-output: Implements getsockopt and setsockoptsystem calls

• User-request: Implements all other operations

Socket-to-Protocol Interface

• Enabled by two routines: 1) User request, pr_usrreq() and2) Control output, pr_ctloutput()

• These methods are present in the protocol-switch table for eachprotocol

• Control-output: Implements getsockopt and setsockoptsystem calls

• User-request: Implements all other operations

Protocol-to-Network-Interface Interface

• Lowest layer in the protocol family must interact with one or moreinterfaces to send and receive packets

• Obviously a routing decision must have already chosen theoutgoing interface

Protocol-to-Network-Interface Interface

• Lowest layer in the protocol family must interact with one or moreinterfaces to send and receive packets

• Obviously a routing decision must have already chosen theoutgoing interface

Code: Packet Sending

error = (*ifp->if_output)(ifp, m, dst, rt);

struct ifnet *ifp;struct mbuf *m;struct sockaddr *dst;struct rtentry *rt;

Packet Sending

• Packet m is transmitted to destination dst via interface ifp

• Steps:• Validation of the destination address• Queuing of the packet on the send queue• If the interface is not busy, using an interrupt-driven routine to

transmit the packet

• The link-layer address is chosen by ARP in case of Ethernet

Packet Sending

• Packet m is transmitted to destination dst via interface ifp• Steps:

• Validation of the destination address

• Queuing of the packet on the send queue• If the interface is not busy, using an interrupt-driven routine to

transmit the packet

• The link-layer address is chosen by ARP in case of Ethernet

Packet Sending

• Packet m is transmitted to destination dst via interface ifp• Steps:

• Validation of the destination address• Queuing of the packet on the send queue

• If the interface is not busy, using an interrupt-driven routine totransmit the packet

• The link-layer address is chosen by ARP in case of Ethernet

Packet Sending

• Packet m is transmitted to destination dst via interface ifp• Steps:

• Validation of the destination address• Queuing of the packet on the send queue• If the interface is not busy, using an interrupt-driven routine to

transmit the packet

• The link-layer address is chosen by ARP in case of Ethernet

Packet Sending

• Packet m is transmitted to destination dst via interface ifp• Steps:

• Validation of the destination address• Queuing of the packet on the send queue• If the interface is not busy, using an interrupt-driven routine to

transmit the packet

• The link-layer address is chosen by ARP in case of Ethernet

Packet Receiving

• Incoming packets are queued in the corresponding protocol’sinput packet queue

• A software interrupt is posted to initiate network-layer processing

Packet Receiving

• Incoming packets are queued in the corresponding protocol’sinput packet queue

• A software interrupt is posted to initiate network-layer processing

Code: Packet Receiving

if (IF_QFULL(&ipintrq)) {IF_DROP(&ipintrq);ifp->if_iqdrops++;m_freem(m);

} else {schednetisr(NETISR_IP);IF_ENQUEUE(&ipintrq, m)

}

Routing

• The routing system has two components; one within the kerneland one in user-space

• The routing mechanism is present within the kernel while routingpolicies are defined in user-space

• The routing mechanism involves a table lookup to get a first-hopfor a given destination

• Routing policies include components that help in choosingfirst-hop routes

Routing

• The routing system has two components; one within the kerneland one in user-space

• The routing mechanism is present within the kernel while routingpolicies are defined in user-space

• The routing mechanism involves a table lookup to get a first-hopfor a given destination

• Routing policies include components that help in choosingfirst-hop routes

Routing

• The routing system has two components; one within the kerneland one in user-space

• The routing mechanism is present within the kernel while routingpolicies are defined in user-space

• The routing mechanism involves a table lookup to get a first-hopfor a given destination

• Routing policies include components that help in choosingfirst-hop routes

Routing

• The routing system has two components; one within the kerneland one in user-space

• The routing mechanism is present within the kernel while routingpolicies are defined in user-space

• The routing mechanism involves a table lookup to get a first-hopfor a given destination

• Routing policies include components that help in choosingfirst-hop routes

Kernel Routing Mechanism

• Implements a routing table for first/next hop lookup

• Two distinct portions:1 A data structure with routing entries, one per specific route2 A lookup algorithm to locate the correct route for each possible

destination

• Each destination is represented by a sockaddr structure• Routes are either:

1 Host or network2 Direct or indirect

Kernel Routing Mechanism

• Implements a routing table for first/next hop lookup• Two distinct portions:

1 A data structure with routing entries, one per specific route

2 A lookup algorithm to locate the correct route for each possibledestination

• Each destination is represented by a sockaddr structure• Routes are either:

1 Host or network2 Direct or indirect

Kernel Routing Mechanism

• Implements a routing table for first/next hop lookup• Two distinct portions:

1 A data structure with routing entries, one per specific route2 A lookup algorithm to locate the correct route for each possible

destination

• Each destination is represented by a sockaddr structure• Routes are either:

1 Host or network2 Direct or indirect

Kernel Routing Mechanism

• Implements a routing table for first/next hop lookup• Two distinct portions:

1 A data structure with routing entries, one per specific route2 A lookup algorithm to locate the correct route for each possible

destination

• Each destination is represented by a sockaddr structure

• Routes are either:1 Host or network2 Direct or indirect

Kernel Routing Mechanism

• Implements a routing table for first/next hop lookup• Two distinct portions:

1 A data structure with routing entries, one per specific route2 A lookup algorithm to locate the correct route for each possible

destination

• Each destination is represented by a sockaddr structure• Routes are either:

1 Host or network

2 Direct or indirect

Kernel Routing Mechanism

• Implements a routing table for first/next hop lookup• Two distinct portions:

1 A data structure with routing entries, one per specific route2 A lookup algorithm to locate the correct route for each possible

destination

• Each destination is represented by a sockaddr structure• Routes are either:

1 Host or network2 Direct or indirect

User-space Routing Policies

• Policies add, delete, or modify kernel routing table entries

• A number of routing policies exist, including the RoutingInformation Protocol (RIP)

User-space Routing Policies

• Policies add, delete, or modify kernel routing table entries

• A number of routing policies exist, including the RoutingInformation Protocol (RIP)

Protocol Control Blocks

• For each TCP or UDP socket, an Internet protocol control block(inpcb) is created to hold address, ports, routing information,and pointers to any additional data structures

• TCP in addition creates a TCP control block (tcpcb) to holdimplementation-specific information

• TCP and UDP protocol modules each have a private doublylinked list of inpcbs

• Common routines are used by the modules to manipulate theselists

• Traffic is multiplexed by the IP layer on the basis of the protocolidentifier in the protocol and passed on to the individual transportprotocol

• Each protocol is then responsible for passing a direct message tothe appropriate socket

Protocol Control Blocks

• For each TCP or UDP socket, an Internet protocol control block(inpcb) is created to hold address, ports, routing information,and pointers to any additional data structures

• TCP in addition creates a TCP control block (tcpcb) to holdimplementation-specific information

• TCP and UDP protocol modules each have a private doublylinked list of inpcbs

• Common routines are used by the modules to manipulate theselists

• Traffic is multiplexed by the IP layer on the basis of the protocolidentifier in the protocol and passed on to the individual transportprotocol

• Each protocol is then responsible for passing a direct message tothe appropriate socket

Protocol Control Blocks

• For each TCP or UDP socket, an Internet protocol control block(inpcb) is created to hold address, ports, routing information,and pointers to any additional data structures

• TCP in addition creates a TCP control block (tcpcb) to holdimplementation-specific information

• TCP and UDP protocol modules each have a private doublylinked list of inpcbs

• Common routines are used by the modules to manipulate theselists

• Traffic is multiplexed by the IP layer on the basis of the protocolidentifier in the protocol and passed on to the individual transportprotocol

• Each protocol is then responsible for passing a direct message tothe appropriate socket

Protocol Control Blocks

• For each TCP or UDP socket, an Internet protocol control block(inpcb) is created to hold address, ports, routing information,and pointers to any additional data structures

• TCP in addition creates a TCP control block (tcpcb) to holdimplementation-specific information

• TCP and UDP protocol modules each have a private doublylinked list of inpcbs

• Common routines are used by the modules to manipulate theselists

• Traffic is multiplexed by the IP layer on the basis of the protocolidentifier in the protocol and passed on to the individual transportprotocol

• Each protocol is then responsible for passing a direct message tothe appropriate socket

Protocol Control Blocks

• For each TCP or UDP socket, an Internet protocol control block(inpcb) is created to hold address, ports, routing information,and pointers to any additional data structures

• TCP in addition creates a TCP control block (tcpcb) to holdimplementation-specific information

• TCP and UDP protocol modules each have a private doublylinked list of inpcbs

• Common routines are used by the modules to manipulate theselists

• Traffic is multiplexed by the IP layer on the basis of the protocolidentifier in the protocol and passed on to the individual transportprotocol

• Each protocol is then responsible for passing a direct message tothe appropriate socket

Protocol Control Blocks

• For each TCP or UDP socket, an Internet protocol control block(inpcb) is created to hold address, ports, routing information,and pointers to any additional data structures

• TCP in addition creates a TCP control block (tcpcb) to holdimplementation-specific information

• TCP and UDP protocol modules each have a private doublylinked list of inpcbs

• Common routines are used by the modules to manipulate theselists

• Traffic is multiplexed by the IP layer on the basis of the protocolidentifier in the protocol and passed on to the individual transportprotocol

• Each protocol is then responsible for passing a direct message tothe appropriate socket

Today’s task

• Design a network subsystem for xv6

Reading(s)

• Chapter 12 and 13 from “The Design and Implementation of the4.4BSD Operating System” by Marshall Kirk McKusick, KeithBostic, Michael J. Karels, and John S. Quarterman.