SOURCE CODING THEOREM

SOURCE CODING THEOREM

The theorem described thus far establish fundamental limits on error-free communication over both reliable and unreliable channels.In this section we turn to the case in which the channel is error free but the communication process itself is lossy.Under these circumstances, the principal function of the communication system is “information compression”.

The average error introduced by the compression is constrained to some maximum allowable level D.

We want to determine the smallest rate, at which information about the source can be conveyed to the user.

This problem is specifically addressed by a branch of information theory known as rate distortion theory.

----------------------------------------------------------------Communication System

Let the information source and decoder output be defined by the finite ensembles (A, z) and (B, z), respectively.

Information Source Channel Information

User

Encoder Decoder

The assumption now is that the channel of the figure is error free.

So a channel matrix Q, which relates z to v in accordance with v=Qz can be thought of as modeling the encoding-decoding process alone.

Because the encoding-decoding process is deterministic,

where Q determines an artificial zero-memory channel that models the effect of the compression and decompression.

Each time the source produces source symbol , it is represented by a code symbol that is then decode to yield output symbol with probability

Addressing the problem of encoding the source so that the average distoration is less than D.

A non-negative cost function ρ(, ), called a distoration measure, can be used to define the penalty associated with reproducing source output with decoder output .

The output of the source is random, so the distoration also is a random variable whose average value denoted d(Q), is

The notation d(Q) emphasizes that the average distoration is a function of the encoding-decoding procedure.

={|d(Q)≤D}Rate distoration finction will be R(D)=If D=0, then R(D) ≤ H(z).

We simply minimize I(z,v) by appropriate choice of Q (or ) subject to the constraints

d(Q)=D.The above equations are fundamental

properties of channel matrix Q.

THANK YOU