application layer protocols - kth | v¤lkommen till kth

Internetworking

Application Layer Protocols

Literature:Forouzan, TCP/IP Protocol Suite: Ch 18-22, 25

The application layer

● Networked applications give content to the Internet

● 70s-80s

– Text email, remote login, file transfers, newsgroups

● Killer app 90s:

– World-wide-web

● 2000s:

– Instant messaging, peer-to-peer sharing of MP3s

– Streaming video, Internet telephony, Distributed gaming

IP

Ethernet PPP

CSMA async SDH

Copper Fiber Radio

email www phone

SMTP HTTP RTP

TCP UDP

Clients, servers, peers

Computers connected to the Internet are end-systems or hosts (they "host" application programs running on them). Hosts are traditionally divided into clients and servers - the difference nowadays unclear.

But from a program point of view, it is easier:

– Client program - requests a service.

– Server program - provides a service.

– Peer - bot a client and a server program.

Application process, sockets and underlying transport

● Typical networked application: processes communicating over a network

● The processes send messages via a socket interface to ”the network”– API – Application Programming Interface)

● Underlying transport layer takes care of end-to-end communication between two hosts.

host/server

Process

TransportOperatingsystem

Userspace

host/server

Process

Transport

Process

Socket

Internet

SocketMessages Messages

Client-server model

● Client (local application) requests service from server (remote application)

● Many clients use one server● Client runs only when needed● Server is always running● NATs are traversed by initiating all traffic from the client● Peer-to-peer: Every client is also a server

clients server

Concurrent TCP server● Used by most application-servers based on TCP

● Child servers use well-known port and client’s port & IP address. Parent server uses wild-card for client’s port & IP address.

well-known port

well-known port

Application layer protocolsApplication protocols are a vital subset of a networked application

The TCP/IP application layer contains protocols that enable applications to communicate.

The TCP/IP application layer roughly maps to three OSI layers:

– Session: session establishment, dialog control, synchronization

– Presentation: syntax and semantics of data: higher level data structures

– Application: application-specific information and protocols

Some application-protocols exists as RFCs, most nowadays in other forums.

Many are not open or documented

Datatype classification

● Binary format

– Mainly used for ”low level” protocols

– Efficient, but need byte-swapping

– Eg: Ethernet, IP, DNS

● 7-bit ASCII (NVT)

– Primitive control encoding

– Eg: TELNET, FTP, SMTP, ...

● BNF “RFC 822”

– Structured ASCII data

– Eg: HTTP, SIP

● Tree-based

– Hierarchical/recursive structure

– TLV – Type Length Value● Eg: IP options, DHCP

– ASN.1● Eg. SNMP

– XML● Eg: HTML

Binary fixed field: example

The DNS header, taken from RFC 1035.

1 1 1 1 1 1 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ | ID | +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ |QR| Opcode |AA|TC|RD|RA| Z | RCODE | +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ | QDCOUNT | +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ | ANCOUNT | +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ | NSCOUNT | +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ | ARCOUNT | +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+

Fixed binary format

Most common in the underlying layers of the TCP/IP stack.Predefines exactly what information is to be where in the message.The semantics is hard-coded into the application.

Requires common alignment (ie on 16, 32 or 64 -bit boundaries)Requires byte-swapping: How the CPU loads its registers from memory.

Little endian (eg Intel): LSB (Least Significant Byte) first

Big endian (eg Motorola): MSB (Most Significant Byte) first

Network byte order is big endian --> You need to byte-swap on i386 PCs.

Pros and cons

When you feel the urge to design a [...] complex binary application protocol, it is generally wise to lie down until the feeling passes

Eric Raymond: ``The Art of UNIX Programming''

Pros: ● Compact: Efficient computer processing● Fixed syntax and simple semantics

Cons:

● Not extendable● Not human readable.● Byte order, alignment problems

Augmented BNF: example

● In RFC 2048, the HTTP URL is defined as:– URL = “http:” “//” host [ “:” port ] [abs\_path ]

– host = A legal Internet host domain name or IP address (in dotted-decimal form) as defined by Section 2.1 of RFC 1123

– port = *DIGIT

– abs_path = “/” rel_path

– rel_path = [ path ] [ “;” params ] [ “?”query ]

● Classical Internet format defined by Backus-Naur Form (BNF) – derived from context-free grammars

● RFC822 is “syntax-heavy”: keywords are introduced for parsing, requires specific parsers.

Type, Length, Value: TLV

● Binary format usually used as an extensible part of a protocol.

– Type: contains a predefined code, indicating what kind of data the value field contains.

– Length: Contains the size (in bytes) of the value field.

– Value: Contains the payload.

● Examples: IS-IS and OSPFv3, DHCP, and IP options.

● TLVs can be recursive (value field contains new TLVs).

Example: DHCP router option (RFC 2132)

Code Len Address 1 Address 2 +-----+-----+-----+-----+-----+-----+-----+-----+-- | 3 | n | a1 | a2 | a3 | a4 | a1 | a2 | ... +-----+-----+-----+-----+-----+-----+-----+-----+--

Abstract Syntax Notation #1

● A general way to define data types - ASN.1 is as powerful as a typed programming language.

● In ASN.1 the type information is inherent in the data - no external specification necessary.

● Used frequently in ISO protocols, but also to a certain extent in TCP/IP protocols.

– Examples: SNMP, UMTS, LDAP, NFSv4 and many security protocols.

PDU ::= SEQUENCE { request-id Integer32, error-status INTEGER (

noError(0),tooBig(1),noSuchName(2),badValue(3) ),

error-index INTEGER (0..max-bindings), variable-bindings VarBindList}

XML

● Plain-text markup language: simple syntax, easy to parse.

● Definition declared externally by XML Schema or DTD.

● Well suited for complex data formats with recursive and nested structures.

● Cons mainly its textual nature: parsing can be inefficient.

<?xml version="1.0" encoding="ISO-8859-1"?><!DOCTYPE note SYSTEM "InternalNote.dtd"><note> <to>Eva</to> <from>Phil</from> <heading>Reminder</heading> <body>Remember to go to the store!</body></note>

Specific applications in Forouzan

Protocol Chapter Transp Datatype

Telnet 18 TCP NVT

FTP File Transfer Protocol 19 TCP NVT

TFTP Trivial File Transfer Protocol 19 UDP Binary

SMTP Simple Mail Transfer Protocol 20 TCP BNF

SNMP Simple Network Mngmnt Prot 21 TCP ASN.1

HTTP Hypertext Transfer Protocol 22 TCP BNF

RTP Real Time Protocol 25 UDP Binary

SIP Session Initiation Protocol 25 TCP BNF

TELNET – TErminal NETwork● Virtual Terminal – local terminal appears to be a terminal on a

remote system– Nice tool to test other text-based protocols (FTP, HTTP, SMTP, etc)

● Client/Server using TCP, port 23● Good example of interactive TCP application

– Silly window syndrome, (Nagle, Delayed ack,...)

● TELNET uses out-of-band signaling

– eg TCP URG for ^C

● TELNET is text-based and sends data according to NVT

– Shares data and control in same character stream

– Network Virtual Terminal – simple encoding

● ”Security challenged”: use TELNET with Kerberos or use SSH

TELNET session

NVT – Network Virtual Terminal

0/1

● For simple information encoding ● First bit: 0 – data, 1 – control● Bits 1-7: 7-bit ASCII

Some NVT control characters

Char Decimal Description

EOF 236 End of file

IP 244 Interrupt process

AO 245 Abort output

EC 247 Erase character

GA 249 Go ahead

WILL 251 Agreement to enable option

DO 253 Approval to option request

IAC 255 Interpret next character as control

TELNET Examplehunerik> telnettelnet> toggle optionsWill show option processing.telnet> open 127.0.0.1Trying 127.0.0.1...Connected to 127.0.0.1.Escape character is '^]'.SENT DO ENCRYPTSENT WILL ENCRYPTSENT DO SUPPRESS GO AHEADSENT WILL TERMINAL TYPE...RCVD DO ECHOSENT WONT ECHORCVD WILL ECHOSENT DO ECHO...RCVD WILL ENCRYPTSENT IAC SB ENCRYPT REQUEST-STARTSENT IAC SB ENCRYPT SUPPORT DES_CFB64 DES_OFB64 RCVD DO ENCRYPTRCVD IAC SB ENCRYPT SUPPORT DES_CFB64 DES_OFB64 RCVD WILL SUPPRESS GO AHEAD

OpenBSD/i386 (hunerik) (ttyp4)

User not authenticated. Using plaintext username and passwordlogin: olofPassword:

FTP – File Transfer Protocol

● Standard Internet file transfer protocol● FTP uses two TCP connections

– One for control (port 21)– One for data (port 20)

● Datformat: Textual NVT over the control channel● Client defines which file data type to transfer

– ASCII/binary, stream/block mode, file/record structure,..● Control messages

– Clients generate commands: USER/LIST/RETR/...– Response: 3-digits (+ text): 200 (OK)

● FTP runs in two modes:

– Active: Server starts data TCP connection

– Passive: Client creates TCP data connection. Works if client is behind NAT.

Some FTP Commands

Control commands sent on control channel

CWD <arg> Change working directoryRMD <arg> Remove directoryPWD Print working directoryTYPE [I|A|E|L <arg>] Set the data transfer typeRETR <arg> Download a file.STOR <arg> Upload a file.LIST Download the current working directory's content list.

FTP Status codesStatus codes are sent as replies to commands: a number and an explaining text (Also in HTTP and SIP)

1xx Positive Preliminary reply The requested action is being initiated; expect another reply before proceeding with a new command.

2xx Positive Completion replyThe requested action has been successfully completed. A new request may be initiated. (Example: 200 Command OK)

3xx Positive Intermediate replyThe command has been accepted but the requested action is waiting for further information before being completed.

4xx Transient Negative Completion reply The command was not accepted and the requested action did not take place, but the error condition is temporary and the action may be requested again. (Example: 450 File not available)

5xx Permanent Negative Completion replyThe command was not accepted and the requested action did not take place. (Example: 500 Syntax error; unrecognized command)

Example: Login and file transfer

TFTP – Trivial File Transfer Protocol

● Much simpler than FTP● UDP port 69● Datatypes: Fixed Binary ● Stop and Go protocol

– Send data, wait for ack.

● Small implementations

– typically: implemented in boot prom for boot loading for diskless clients

● Five message types

– RRQ – Read ReQuest

– WRQ – Write ReQuest

– DATA

– ACK

– ERROR

SMTP – Simple Mail Transfer Protocol

● The Internet Email protocol● Client/server - TCP port 25● BNF data format● Protocol used to transfer email from hosts to mail servers and

between mail servers.

– Several steps: relaying– Spooling on sender-site– Mailbox and mail-access protocol on receiver

● User Agent (UA) – end hosts● Mail Transfer Agent (MTA) – mail servers● Addressing:

– <mailbox>@<domain name>

– Results ina DNS MX request giving name of MTA.

SMTP – Simple Mail Transfer Protocol (2)

● Mail Access Protocols– From MTA to UA– POP or IMAP

● Messages: – HELO/ELHO – sender host– MAIL FROM – sender user– RCPT TO – intended receiver– DATA – mail body– Status codes similar to FTP

Email system

SM TP

SM TP

M AILACCESS

PROTOCOL

SMTP Example

MIME Multipurpose Internet Mail Exchange

Classical email messages must be written in 7-bit US-ASCII.

MIME extends this with:

– Textual message bodies in other character sets

– Extensible set of different formats for non-textual message bodies,

– Multi-part message bodies,

– Textual header information in other character sets

Related header fields:

– Content-Type - what kind of data the content carries. Examples: text/plain, text/html, audio, video, application/pdf, extension-token, multipart.

– Content-Transfer-Encoding - how data is encoded. Examples: 7bit, 8bit, binary, quoted-printable, base64,...

Internet Network Management

● It is complex to build internetworks and we need to manage them.– Monitoring– Debugging– Control routers and other network devices

● SNMP - Internet management– No special control messages – use TCP/IP itself – Management is on TCP/IP application level

+ Same protocol is used for all managed devices- If IP does not work correctly,...

- SNMP uses TCP/IP- Datatypes: Abstract Syntax Notation ASN.1- Powerful datatype definition language

SNMP Architecture

● Client/server architecture● Client software on

manager’s workstation● Server software on system

being managed

– Agent

● Clients query agents● Agents respond to clients● Agents send traps to

clients

Router/Device being m anaged

Agent

Agent

Agent

Client

Manager’sHost

query

resp

onse

t rap

HTTP - Hypertext Transfer Protocol

● TCP port 80, RFC 2616, BNF dataformat

● HTTP is the main protocol used to download resources from the world wide web.

● Simplest form: a requestor establishes a TCP connection to the web server on port 80 and sends a string describing what resource it wants, and receives the resource in reply.

● The most modern version today is HTTP/1.1.

Some HTTP commands

● GET http_url: Download an http resource.

● POST http_url: Upload data to an http resource.

● PUT http_url: Write an http resource.

● DELETE http_url: Delete an http resource.

HTTP – example

GET /stuff/blah.html HTTP/1.1Host: zipf.pilsnet.sunet.seUser-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.5) Gecko/20031214 Firebird/0.7Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,[...]Accept-Language: en-us,en;q=0.5Accept-Encoding: gzip,deflateAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7Keep-Alive: 300Connection: keep-alive

HTTP/1.1 200 OKDate: Tue, 27 Jan 2004 20:18:28 GMTServer: Apache/1.3.27 (Unix) (Gentoo/Linux) PHP/4.3.4Last-Modified: Tue, 27 Jan 2004 19:53:47 GMTETag: "bb4047-2c-4016c1cb"Accept-Ranges: bytesContent-Length: 44Keep-Alive: timeout=15, max=100Connection: Keep-AliveContent-Type: text/html

<html><b> Hello there </b></html>

HTTP 1/1 persistent connections

● In HTTP 1/0, all HTTP requests generated a new TCP connection.

● But most html documents contain sub-parts --> one TCP connection for each sub-request.

● But TCP congestion control is made for longer connections --> they can adapt to congestion in the network.

● When http traffic grew when the web exploded, these small flows were said to kill the Internet!

● HTTP 1/1 supports persistent connections: keep the TCP connection during the complete session: send all requests on the same TCP connection.

● Now, these longer TCP connection can perform congestion control algorithm in a proper way.

Real-time multimedia

● Time-sensitive, interactive applications (eg telephony, conferencing): use real-time protocol.

– You cannot make retransmissions

– RTP Real-Time Protocol over UDP.● Limited time-sensitivity (eg Video-on-demand,

Radio): use streaming protocols.

– You can buffer at receiver

– RSTP Real-Time Streaming Protocol over TCP● Alternative:

– IPTV can use MPEG over UDP/IP multicast.

Delay Jitter

What happens if the packets arrive with different delays?● There is a gap between first and second packet● This phenomenon is called jitter

RTP: Real-time Transport Protocol

● Designed to carry out variety of real-time data: e.g., audio and video.

● Sequence number for receiver to detect out-of-order delivery

● Timestamp allowing receiver to control playback● Typically run on top of UDP, ● No mechanisms to ensure timely delivery

– Just provides the mechanisms to build a real-time service

SIP Session Initiation Protocol

TCP or UDP port 5060, ABNF

Terminology is similar to SMTP, but is a synchronous protocol (no delays).

SIP uses URI's (Uniform Resource Identifiers) as addresses:

<sip:6534@kth.se>

<sip:bob@biloxi.com>

SIP uses transactions, usually three-way (as TCP connections).

Example:

INVITE --> 200 OK --> ACK

SIP Example

From RFC 3261: softphone proxy proxy SIP Phone

| | | | | INVITE F1 | | | |--------------->| INVITE F2 | | | 100 Trying F3 |--------------->| INVITE F4 | |<---------------| 100 Trying F5 |--------------->| | |<-------------- | 180 Ringing F6 | | | 180 Ringing F7 |<---------------| | 180 Ringing F8 |<---------------| 200 OK F9 | |<---------------| 200 OK F10 |<---------------| | 200 OK F11 |<---------------| | |<---------------| | | | ACK F12 | |------------------------------------------------->| | Media Session | |<================================================>| | BYE F13 | |<-------------------------------------------------| | 200 OK F14 | |------------------------------------------------->|

SIP message example

INVITE sip:000730631661@kth.se SIP/2.0Via: SIP/2.0/UDP 192.36.125.167:5060;branch=z9hG4bK0e4415eaFrom: "6534" <sip:6534@kth.se>;tag=000e38a3b7e8001d597d1d53-1bfa7620To: <sip:000730631661@kth.se>Call-ID: 000e38a3-b7e8001e-34c94c48-72c83866@192.36.125.167Date: Mon, 03 Jan 2005 14:16:06 GMTCSeq: 101 INVITEUser-Agent: CSCO/6Contact: <sip:6534@192.36.125.167:5060>Expires: 180Content-Type: application/sdpContent-Length: 251Accept: application/sdp

Implementing it: The socket interface

The socket interface is used for programming applications with a network component.

Sometimes called BSD sockets - it was first implemented in C in BSD.

Variants exist for most programming languages.

Winsock is almost the same but not quite!

Other programming interfaces include:– Corba, Streams, Remote Procedure Calls (RPC), etc.

The sockets API is a de facto standard for network programming.

Socket function calls

● socket() – create a socket.● bind() – bind the socket to a local address.● connect() – connect the socket to a remote address.● listen() – ready to accept incoming connections.● accept() – accept an incoming connectiong.● sendto()/write() – send a message on a socket.● recvfrom()/read() – receive a message on a socket.● select() – input multiplexing: wait for input on several

sockets/files or timeout.● close() – close a socket.

UNIX Networking code is organized into 3 layers; socket, protocol, and interface layer

● Socket layer – protocol independent interface to the protocol dependent layer

● Protocol layer – TCP/IP implementation

● Interface layer - device driver

Queues● Socket queues – 1 per socket● Interface queues – 1 per IF● Protocol queue – 1 single IP

queue

Typical IP stack implementation

process

socket layer

protocol layer(TCP, UDP, IP, ICMP, IGMP)

interface layer

protocol queue(IP input queue)

socket queues

system calls

interfacequeues

funct ioncall

startoutput

Summary

● Applications vs Application-layer protocols

● Clients, Servers, Peers

● Processes, Sockets, Messages

● Datatypes: Binary, NVT, BNF, ASN.1, XML, TLV

● The specific applications,...