chapter 7 end-to-end data

Apr 20, 2023 CPSC558: Advanced Computer Networks

Chapter 7 End-to-End Data

– Data Manipulating Functions (Affecting Throughputs)• How to encode the message into different formats in

presentation format? (MPEG, JPEG, GIF…)• How to perform data compression in the message

(more redundancy vs. more simplify)Application

data

Presentationencoding

Applicationdata

Presentationdecoding

Message Message Message■ ■ ■

Fig. 7.1 – Presentation formatting involves encoding and decoding application data.


Chapter 7 End-to-End Data

• Argument Marshalling (encoding) and Un-marshalling (decoding) is difficult because

– Computer represents same data in different ways (e.g. Big/Little-endian format.)

– Application programs are written in different languages (Compiler/Machine structure may be different.)

So, what do we consider when dealing with encoding/decoding process? (next slide please…)


Facts to consider about data encoding/decoding

• Data Types– Base types : integer, floating-point, characters– Flat types : structures and arrays– Complex types : for examples, trees (has Pointers)

• Serializing– A network must be able to serialize the complex types

(transfer complex types data) via the network. (Or ‘flatten’ the complex type data). Because complex type of data contains pointers (address of data) which may not the same in different machines.


Data Encoding (Argument Marchalling Strategies

• A. Canonical Intermediate Form– Sender translates message from internal representation to

an external representation for each type.

• B. Receive-makes-right– Sender does not convert the base types but transfer data

in its own internal types to receiver. The receiver is responsible for translating the data from sender into receiver’s own local format. (But use this strategy, every host must be ready to do the translate work – N-by-N solution.)

– NOTE: if both sender and receiver are the same type of machine, then really there is no need to do the translation.


Tags (Encoding issues)

• How the receiver knows what kind of data is contained in the message it receives?– Use tagged data approach

• Type tag : data is a integer, floating point or character• Length tag : no. of integer in an array or size of an

integer• Architecture tag

– Use untagged data approach• In a Remote Procedure call, the message is

programmed to send to receiver and receiver already knows/expect certain format or kind of data will be transmitted from sender.


Stubs (Encoding issues)

• We use stubs (a piece of code) that perform argument marshalling (data encoding).

• Stubs are typically used to support RPC (remote procedure calls)

• On client side : stubs marshalling (encode) the procedure arguments into a message that can be transmitted by means of RPC protocol.

• On server side : stubs un-marshalling (decode) the message back into a set of variables that can be used as arguments to call remote procedure.


Stubs (Encoding issues)

Call P

Clientstub

RPC

Arguments

Marshalledarguments

Interfacedescriptor forprocedure P

Stubcompiler

Message

Specification

P

Serverstub

RPC

Arguments

Marshalledarguments

Code Code

Figure 7.5


Multimedia

OutlineCompression

RTP

Scheduling


Compression Overview

• Encoding and Compression– Huffman codes

• Lossless – data received = data sent– used for executables, text files, numeric data

• Lossy– data received does not != data sent– used for images, video, audio


Lossless Algorithms

• Run Length Encoding (RLE)– example: AAABBCDDDD encoding as 3A2B1C4D – good for scanned text (8-to-1 compression ratio) – can increase size for data with variation (e.g., some

images)

• Differential Pulse Code Modulation (DPCM)– example AAABBCDDDD encoding as A001123333– change reference symbol if delta becomes too large– works better than RLE for many digital images (1.5-to-1)


Dictionary-Based Methods

• Build dictionary of common terms – variable length strings

• Transmit index into dictionary for each term • Lempel-Ziv (LZ) is the best-known example• Commonly achieve 2-to-1 ration on text• Variation of LZ used to compress GIF images

– first reduce 24-bit color to 8-bit color – treat common sequence of pixels as terms in

dictionary– not uncommon to achieve 10-to-1 compression (x3)


Image Compression

• JPEG: Joint Photographic Expert Group (ISO/ITU)• Lossy still-image compression • Three phase process

– process in 8x8 block chunks (macro-block)– grayscale: each pixel is three values (YUV)– DCT: transforms signal from spatial domain into and equivalent

signal in the frequency domain (loss-less)– apply a quantization to the results (lossy) – RLE-like encoding (loss-less)

Sourceimage

JPEG compression

DCT Quantization EncodingCompressed

image


Quantization and Encoding

• Quantization Table3 5 7 9 11 13 15 17

5 7 9 11 13 15 17 19

7 9 11 13 15 17 19 21

9 11 13 15 17 19 21 23

11 13 15 17 19 21 23 25

13 15 17 19 21 23 25 27

15 17 19 21 23 25 27 29

17 19 21 23 25 27 29 31

• Encoding Pattern


MPEG

• Motion Picture Expert Group• Lossy compression of video • First approximation: JPEG on each frame• Also remove inter-frame redundancy


MPEG (cont)

• Frame types– I frames: intrapicture – P frames: predicted picture– B frames: bidirectional predicted picture

• Example sequence transmitted as I P B B I B B

Frame 1 Frame 2 Frame 3 Frame 4 Frame 5 Frame 6 Frame 7

I frame B frame B frame P frame B frame B frame I frame

MPEGcompression

Forwardprediction

Bidirectionalprediction

Compressedstream

Inputstream


MPEG (cont)

• B and P frames– coordinate for the macroblock in the frame– motion vector relative to previous reference frame (B, P)– motion vector relative to subsequent reference frame (B)– delta for each pixel in the macro block

• Effectiveness– typically 90-to-1– as high as 150-to-1– 30-to-1 for I frames– P and B frames get another 3 to 5x


MP3

• CD Quality– 44.1 kHz sampling rate– 2 x 44.1 x 1000 x 16 = 1.41 Mbps– 49/16 x 1.41 Mbps = 4.32 Mbps

• Strategy– split into some number of frequency bands– divide each subband into a sequence of blocks– encode each block using DCT + Quantization +

Huffman– trick: how many bits assigned to each subband


RTP

• Application-Level Framing• Data Packets

– sequence number– timestamp (app defines “tick”)

• Control Packets (send periodically)– loss rate (fraction of packets received since last report)– measured jitter


Transmitting MPEG

• Adapt the encoding– resolution– frame rate– quantization table– GOP mix

• Packetization• Dealing with loss• GOP-induced latency


Layered Video

• Layered encoding– e.g., wavelet encoded

• Receiver Layered Multicast (RLM)– transmit each layer to a different group address– receivers subscribe to the groups they can “afford”– Probe to learn if you can afford next higher group/layer

• Smart Packet Dropper (multicast or unicast)– select layers to send/drop based on observed

congestion– observe directly or use RTP feedback


Real-Time Scheduling

• Priority • Earliest Deadline First (EDF)• Rate Monotonic (RM)• Proportional Share

– with feedback– with adjustments for deadlines

chapter 7 end-to-end data

Documents