1-1

TCP/IP Protocols and Client-Server Applications

Objectives: TCP, UDP, HTTP protocols HTTP clients and servers Electronic mail Handling TCP message boundary Handling structured data Socket Interface Synchronous and Asynchronous socket

calls

1-2

Service

Mechanism for computers to interact (application layer)

Term refers to overall solution Usually associated with IP port number Differs from protocol which describes

the details of how interaction works Ex) HTTP service builds on TCP/IP

RFC used to define service standard

1-3

Applications

Traditional PC applications Everything done locally Fast but sharing is difficult Word, Excel

Client/server applications Client local and responsive Client provides interface Server centralizes resources Server performs some work

1-4

TCP/IP As A Protocol Suite

Application Telnet, FTP, Web, e-mail, etc.

Transport TCP, UDP

Network IP, ICMP, IGMP

Data Link +Physical device driver and interface card

1-5

TCP Connection Management

Recall: TCP sender, receiver establish “connection” before exchanging data segments

initialize TCP variables: seq. #s buffers, flow control info

(e.g. RcvWindow) client: connection initiator Socket clientSocket = new

Socket("hostname","port

number"); server: contacted by client Socket connectionSocket =

welcomeSocket.accept();

Three way handshake:

Step 1: client host sends TCP SYN segment to server specifies initial seq # no data

Step 2: server host receives SYN, replies with SYNACK segment

server allocates buffers specifies server initial

seq. #Step 3: client receives SYNACK,

replies with ACK segment, which may contain data

1-6

TCP Connection Establishment

client

SYN, seq=x

server

SYN+ACK, seq=y, ack=x+1

ACK, ack=y+1

CLOSED

LISTEN

SYN_SENTSYN_RCVD

Established

Passive open

SYN/SYN+ACK

ACK

Active open;SYN

SYN+ACK/ACK

Solid line for client

Dashed line for server

1-7

TCP Connection Termination

client

FIN

server

ACK

ACK

FIN

closing

tim

ed w

ait

FIN_WAIT1

FIN_WAIT2

CLOSE_WAIT

LAST_ACK

CLOSED

TIME_WAIT

CLOSED

1-8

Multiplexing/Demultiplexing

Use same communication channel between hosts for several logical communication processes

How does Mux/DeMux work? Sockets: doors between process & host UDP socket: (dest. IP, dest. Port) TCP socket: (src. IP, src. port, dest. IP, dest. Port)

TransportLayer

NetworkLayer

TransportLayer

NetworkLayer

HTTP

FTP

Telnet

1-9

Connectionless demux

UDP socket identified by two-tuple: (dest IP address, dest port number)

When host receives UDP segment: checks destination port number in segment directs UDP segment to socket with that port number

IP datagrams with different source IP addresses and/or source port numbers are directed to the same socket

1-10

Connection-oriented demux

TCP socket identified by 4-tuple: source IP address source port number dest IP address dest port number

recv host uses all four values to direct segment to appropriate socket

Server host may support many simultaneous TCP sockets: each socket identified

by its own 4-tuple

Web servers have different sockets for each connecting client non-persistent HTTP will

have different socket for each request

1-11

TCP Communication

In a simple TCP client/server application a byte array of 102 may be as the buffer size for the Send and Receive method calls This worked fine The program is usually run in a controlled

environment The server and client know that the

message size will not be more than this size What about in real world scenarios???

1-12

Problems in TCP Communication Too Small Buffer Size:

There are situations, where we may not determine the size of the data

We must deal with such situations• For example, if more data arrives than the buffer size

Message Boundary Problem Due to TCP’s connection-oriented nature,

messages are considered to form a continuous stream of bytes

This is implemented using TCP internal buffers, which are used to store messages until they are Received/Sent by applications

1-13

Message Boundary Problem

1-14

Solutions

Solution # 1: For Text messages only

• Use ReadLine and WriteLine methods of the StreamReader and StreamWriter classes respectively

• We are basically inserting end-of-line markers in the message

• The problem of too small buffer disappears

1-15

Solutions: Cont. Solution #2:

Send the size of the message before sending the message• Applicable for any type of data where Send and Receive

methods are used for sending and Receiving• The receiver will read the data in a loop until the entire size is

read A loop similar to the following is used

int total = int.Parse(reader.ReadLine());byte [] buffer = new byte[1024];int recv = 0;int sofar = 0;while (sofar < total) {

recv = s.Receive(buffer);process(buffer, recv);sofar += recv;

}

1-16

UDP: User Datagram Protocol [RFC 768]

“bare bones” Internet transport protocol “best effort” service, UDP segments may be:

lost delivered out of order to app

Why use UDP? No connection establishment cost (critical for

some applications, e.g., DNS) No connection state Small segment headers (only 8 bytes) Finer application control over data

transmission

1-17

UDP Segment Structure

often used for streaming multimedia apps loss tolerant rate sensitive

Other appl. Protocols using UDP DNS SNMP

reliable transfer over UDP: add reliability at application layer application-specific

error recovery!

source port # dest port #

32 bits

Applicationdata

(message)

UDP segment format

length checksumLength, in

bytes of UDPsegment,including

header

1-18

Contact a Daemon Using Telnetsaga10:~> telnet cslibrary.standford.edu 80Trying 64.233.167.104...Connected to www.google.akadns.net (64.233.167.104).Escape character is '^]'.GET /test.html HTTP/1.0

HTTP/1.0 200 OKCache-Control: privateContent-Type: text/htmlSet-Cookie: PREF=ID=72459b575402fb39:TM=1089165164:LM=1089165164:S=U8m_gb0hxi2SVKLp; expires=Sun, 17-Jan-2038 19:14:07 GMT; path=/; domain=.google.comServer: GWS/2.1Content-Length: 2096Date: Wed, 07 Jul 2004 01:52:44 GMTConnection: Keep-Alive

<html><head> …

1-19

Example Standardized Services

DNSFTP (your Assignment #1)

SCPPing

FingerTelnet, SSH

SMTPPOP IMAP

HTTP (your assignment #3)

1-20

Web and HTTP

First some jargon Web page consists of objects Object can be HTML file, JPEG image, Java applet, audio

file,… Web page consists of base HTML-file which includes

several referenced objects Each object is addressable by a URL Example URL:

www.someschool.edu/someDept/pic.gif

host name path name

1-21

HTTP overview

HTTP: hypertext transfer protocol

Web’s application layer protocol client/server model

client: browser that requests, receives, “displays” Web objects

server: Web server sends objects in response to requests

HTTP 1.0: RFC 1945 HTTP 1.1: RFC 2068

PC runningExplorer

Server running

Apache Webserver

Mac runningNavigator

HTTP request

HTTP request

HTTP response

HTTP response

1-22

Steps For Fetching URLs

The browser makes up an absolute URL if URL is relative The browser examines the protocol part and the host

part Browser uses DNS to resolve host name to IP address For HTTP protocol, the browser makes a TCP connection

to IP address and port 80 The port maybe :portno following the hostname in URL

The browser sends a GET request as in: GET /path/somefile.htm HTTP/1.0

The server sends the file and closes the connection. Browser renders file. If it is HTML file then any image

referenced by some <img> tag, then go back to step 4

1-23

HTTP overview (continued)

Uses TCP: client initiates TCP

connection (creates socket) to server, port 80

server accepts TCP connection from client

HTTP messages (application-layer protocol messages) exchanged between browser (HTTP client) and Web server (HTTP server)

TCP connection closed

HTTP is “stateless” server maintains no

information about past client requests

Protocols that maintain “state” are complex!

past history (state) must be maintained

if server/client crashes, their views of “state” may be inconsistent, must be reconciled

1-24

HTTP connections

Nonpersistent HTTP At most one object is

sent over a TCP connection.

HTTP/1.0 uses nonpersistent HTTP

Persistent HTTP Multiple objects can

be sent over single TCP connection between client and server.

HTTP/1.1 uses persistent connections in default mode

1-25

Nonpersistent HTTPSuppose user enters URL www.someSchool.edu/someDepartment/home.index

1a. HTTP client initiates TCP connection to HTTP server (process) at www.someSchool.edu on port 80

2. HTTP client sends HTTP request message (containing URL) into TCP connection socket. Message indicates that client wants object someDepartment/home.index

1b. HTTP server at host www.someSchool.edu waiting for TCP connection at port 80, “accepts” connection and notifies the client

3. HTTP server receives request message, forms response message containing requested object, and sends message into its socket

time

(contains text, references to 10

jpeg images)

1-26

Nonpersistent HTTP (cont.)

5. HTTP client receives response message containing html file, displays html. Parsing html file, finds 10 referenced jpeg objects

6. Steps 1-5 repeated for each of 10 jpeg objects

4. HTTP server closes TCP connection.

time

1-27

Response time modeling

Definition of RTT: time to send a small packet to travel from client to server and back.

Response time: one RTT to initiate TCP

connection one RTT for HTTP request and

first few bytes of HTTP response to return

file transmission timetotal = 2RTT+transmit time

time to transmit file

initiate TCPconnection

RTT

requestfile

RTT

filereceived

time time

1-28

Persistent HTTP

Nonpersistent HTTP issues: requires 2 RTTs per object OS must work and allocate

host resources for each TCP connection

but browsers often open parallel TCP connections to fetch referenced objects

Persistent HTTP server leaves connection

open after sending response

subsequent HTTP messages between same client/server are sent over the same connection

Persistent without pipelining: client issues new request

only when previous response has been received

one RTT for each referenced object

Persistent with pipelining: default in HTTP/1.1 client sends requests as

soon as it encounters a referenced object

as little as one RTT for all the referenced objects

1-29

HTTP request message

Two types of HTTP messages: request, response HTTP request message:

ASCII (human-readable format)

GET /somedir/page.html HTTP/1.1Host: www.someschool.edu User-agent: Mozilla/4.0Connection: close Accept-language:fr

(extra carriage return, line feed)

request line(GET, POST,

HEAD commands)

header lines

Carriage return, line feed

indicates end of message

1-30

HTTP request message: general format

1-31

Uploading form input

Post method: Web page often

includes form input Input is uploaded to

server in entity body

URL method: Uses GET method Input is uploaded in

URL field of request line:

1-32

Method types

HTTP/1.0 GET POST HEAD

asks server to leave requested object out of response

usually used for debugging

HTTP/1.1 GET, POST, HEAD PUT

uploads file in entity body to path specified in URL field

DELETE deletes file specified

in the URL field Allows an application

to delete an object on the server

1-33

HTTP response message

HTTP/1.1 200 OK Connection closeDate: Thu, 06 Aug 1998 12:00:15 GMT Server: Apache/1.3.0 (Unix) Last-Modified: Mon, 22 Jun 1998 …... Content-Length: 6821 Content-Type: text/html data data data data data ...

status line(protocol

status codestatus phrase)

header lines

data, e.g., requestedHTML file

1-34

HTTP response status codes

200 OK request succeeded, requested object later in this message

301 Moved Permanently requested object moved, new location specified later in this

message (Location:)

400 Bad Request request message not understood by server

404 Not Found requested document not found on this server

505 HTTP Version Not Supported

In first line in server client response message.A few sample codes:

1-35

User-server state: cookies

Many major Web sites use cookies

Four components:1) cookie header line in

the HTTP response message

2) cookie header line in HTTP request message

3) cookie file kept on user’s host and managed by user’s browser

4) back-end database at Web site

1-36

Cookies: keeping “state” (cont.)

client server

usual http request msgusual http response

+Set-cookie: 1678

usual http request msg

cookie: 1678usual http response

msg

usual http request msg

cookie: 1678usual http response msg

cookie-specificaction

cookie-spectificaction

servercreates ID

1678 for user

entry in backend

database

access

acce

ss

Cookie file

amazon: 1678ebay: 8734

Cookie file

ebay: 8734

Cookie file

amazon: 1678ebay: 8734

one week later:

1-37

Cookies (continued)

What cookies can bring:

authorization shopping carts recommendations one-click shopping user session state

(Web e-mail)

Cookies and privacy: cookies permit sites to

learn a lot about you you may supply name

and e-mail to sites search engines use

redirection & cookies to learn yet more

advertising companies obtain info across sites

aside

1-38

Conditional GET

Goal: don’t send object if cache has up-to-date cached version

cache: specify date of cached copy in HTTP requestIf-modified-since:

<date> server: response contains

no object if cached copy is up-to-date: HTTP/1.0 304 Not

Modified

cache server

HTTP request msgIf-modified-since:

<date>

HTTP responseHTTP/1.0

304 Not Modified

object not

modified

HTTP request msgIf-modified-since:

<date>

HTTP responseHTTP/1.0 200 OK

<data>

object modified

1-39

Synchronous and Asynchronous Socket Calls

In human communications, for example, we use both Asynchronous, and synchronous messaging

If we need information before taking the next step in our daily tasks Performing a Google search, purchasing something

online — we usually choose synchronous messaging

For similar reasons, client/server applications require both synchronous and asynchronous communications

1-40

Synchronous and Asynchronous Socket Calls Asynchronous and synchronous

messaging each have their places in client/server systems, and neither can completely replace the other

1-41

Synchronous and Asynchronous Socket Calls A socket-based client/server application that

will allow two-way asynchronous communication between a server and multiple client applications May uses asynchronous methods May use threads

1-42

Asynchronous Client-Server Applications

Synchronous Methods Asynchronous MethodsConnect() BeginConnect()

EndConnect()

Receive() BeginReceive()EndReceive()

the C# and .NET frameworks provide a rich set of functionalities to do asynchronous communications without introducing the complexity of threading

1-43

Asynchronous Client-Server Applications

Synchronous Methods Asynchronous MethodsConnect() BeginConnect()

EndConnect()

Receive() BeginReceive()EndReceive()

the C# and .NET frameworks provide a rich set of functionalities to do asynchronous communications without introducing the complexity of threading

1-44

Synchronous and Asynchronous Socket Calls A socket-based client/server application that

will allow two-way asynchronous communication between a server and multiple client applications May uses asynchronous methods May use threads

1-45

Asynchronous methods of the Socket & Stream class

The Socket class can be limited to the blocking methods, namely, Accept, Connect, Send and Receive

Similarly, the FileStream and NetworkStream classes (both concrete subclasses of the abstract Stream class), can be limited to the blocking methods, Read and Write

Each of these methods blocks the execution of a program until its operation is completed

How can we solve the problem of blocking???

1-46

Asynchronous methods of the Socket & Stream class

The following are the asynchronous methods of the Socket and Stream classes tabulated based on the tasks they perform

Socket Class:

Tasks Started by Task Task ended by

BeginAccept(…) To accept an incoming connection

EndAccept(…)

BeginConnect(…) To connect to a remote host

EndConnect(…)

BeginReceive(…) To receive data from a socket

EndReceive(…)

BeginReceiveFrom(…) To receive data from a host (UDP)

EndReceiveFrom(…)

BeginSend(…) To send data to a socket

EndSend(…)

BeginSendTo(…) To send data to a host (UDP)

EndSendTo(…)

1-47

Synchronous and Asynchronous Socket Calls The following fragment shows how the BeginAccept and

the EndAccept methods are used:

Socket server = new Socket(AddressFamily.InterNetwok,

SocketType.Stream, ProtocolType.Tcp);IPEndPoint localEP = new IPEndPoint(IPAddress.Any, 9050);server.Bind(localEP);server.Listen(10);server.BeginAccept(new AsyncCallback(CallAccept), server);…private static void CallAccept(IAsyncResult result) {

Socket server = (Socket) result.AsyncState;Socket client = server.EndAccept(result);…

}

1-48

Electronic Mail

Three major components: user agents mail servers simple mail transfer protocol:

SMTP

User Agent a.k.a. “mail reader” composing, editing, reading mail

messages e.g., Eudora, Outlook, elm,

Netscape Messenger, Thunderbird

outgoing, incoming messages stored on server

user mailbox

outgoing message queue

mailserver

useragent

useragent

useragent

mailserver

useragent

useragent

mailserver

useragent

SMTP

SMTP

SMTP

1-49

Email using SMTP/IMAP/POP Email consists of two components

Simple Mail Transfer Protocol (SMTP) for email clients to send out emails (e.g. smtp.nd.edu, port 25)

Internet Message Access Protocol (IMAP) or Post Office Protocol (POP)

• email clients to receive your emails (e.g. imap.nd.edu, port 143)

You can use telnet to “talk” to these servers directly

E.g. type ‘telnet smtp.nd.edu 25’ and then type ‘help’

1-50

Email using SMTP/IMAP/POP For SMTP, your client (say Outlook), connects

to smtp.nd.edu and then delivers an email destined for friend@aol.com

Smtp.nd.edu then locates the SMTP server responsible for AOL. These servers may delegate to other SMTP servers. Eventually it reaches friend@aol.com

Friend will use IMAP or POP to retrieve this email

1-51

Electronic Mail: SMTP [RFC 2821]

uses TCP to reliably transfer email message from client to server, port 25

direct transfer: sending server to receiving server no middle server in-between!

three phases of transfer handshaking (greeting) transfer of messages closure

command/response interaction commands: ASCII text response: status code and phrase

messages must be in 7-bit ASCII (mostly)

1-52

Mail message format

SMTP: protocol for exchanging email messages

RFC 2822: standard for text message format:

header lines, e.g., To: From: Subject:different from SMTP

commands! body

the “message”, ASCII characters only (mostly)

header

body

blankline

1-53

Message format: multimedia extensions

MIME: multimedia mail extension, RFC 2045, 2056 additional lines in msg header declare MIME content

type

From: alice@crepes.fr To: bob@hamburger.edu Subject: Picture of yummy crepe. MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Type: image/jpeg

base64 encoded data ..... ......................... ......base64 encoded data

multimedia datatype, subtype,

parameter declaration

method usedto encode data

MIME version

encoded data

1-54

Mail access protocols

SMTP: delivery/storage to receiver’s server Mail access protocol: retrieval from server

POP: Post Office Protocol [RFC 1939]• authorization (agent <-->server) and download

IMAP: Internet Mail Access Protocol [RFC 1730]• more features (more complex)• manipulation of stored msgs on server

HTTP: Hotmail , Yahoo! Mail, etc.

useragent

sender’s mail server

useragent

SMTP SMTP accessprotocol

receiver’s mail server

1-55

POP3 protocol

authorization phase client commands:

user: declare username pass: password

server responses +OK -ERR

transaction phase, client: list: list message numbers retr: retrieve message by

number dele: delete quit

C: list S: 1 498 S: 2 912 S: . C: retr 1 S: <message 1 contents> S: . C: dele 1 C: retr 2 S: <message 1 contents> S: . C: dele 2 C: quit S: +OK POP3 server signing off

S: +OK POP3 server ready C: user bob S: +OK C: pass hungry S: +OK user successfully logged on

1-56

Email

Mailreader

Maildaemon

SMTP/TCP

Mail gateway

Maildaemon

SMTP/TCP

Mailreader

Maildaemon

1-57

SMTP versus HTTP

SMTP uses persistent connections

SMTP requires message (header & body) to be in 7-bit ASCII

SMTP server uses CRLF.CRLF to determine end of message

Comparison with HTTP: HTTP: pull SMTP: push

both have ASCII command/response interaction, status codes

HTTP: each object encapsulated in its own response msg

SMTP: multiple objects sent in multipart msg