comparisons of fec and codec robustness on voip quality and bandwidth efficiency wenyu jiang henning...

Comparisons of FEC and Codec Comparisons of FEC and Codec Robustness on VoIP Quality and Robustness on VoIP Quality and Bandwidth EfficiencyBandwidth Efficiency

Wenyu JiangHenning Schulzrinne

Columbia UniversityICN 2002, Atlanta, GA

Aug 29, 2002

Introduction to VoIPIntroduction to VoIP The Internet is still best-effort

Subject to packet loss and delay jitter Options for repairing packet loss

Forward error correction (FEC) Low complexity; bit-exact recovery

Packet loss concealment (PLC) Receiver-only; no extra BW overhead

More robust (error resilient) codec better PLC quality, and higher bit-rate

Question: use FEC or a more robust codec?

Metric of VoIP Quality Metric of VoIP Quality Mean Opinion Score (MOS) [ITU

P.830] Obtained via human-based listening

tests Listening (MOS) vs. conversational

(MOSc)

Grade

Quality

5 Excellent

4 Good

3 Fair

2 Poor

1 Bad 1.5

2

2.5

3

3.5

4

0 0.03 0.06 0.09 0.12 0.15

MO

S

average loss probability

iLBC 14kb/sG.729 8kb/s

G.723.1 6.3kb/s

FEC and IP Header FEC and IP Header Overhead Overhead An (n,k) FEC code has (n-k)/k

overhead Typical IP/UDP/RTP header is 40

bytescodec media pkt

size (T=30ms)

rmedia rIP

iLBC(4,2) FEC

54 bytes 14.4 kb/s

25.1 kb/s

108 bytes 28.8 kb/s

39.5 kb/s

G.729(4,2) FEC

30 bytes 8 kb/s 18.7 kb/s

60 bytes 16 kb/s 26.7 kb/s

G.723.1(4,2) FEC

24 bytes 6.4 kb/s 17.1 kb/s

48 bytes 12.8 kb/s

23.5 kb/s

Predicting MOS in VoIPPredicting MOS in VoIP The E-model: an alternative to

human-based MOS estimation Do need a first-time calibration from an

existing human MOS-loss curve In VoIP, the E-model simplifies to two

main factors: loss (Ie) and delay (Id) A gross score R is computed and

translated to MOS. Loss-to-Ie mapping is codec-

dependent and calibrated

Predicting MOS in VoIP, Predicting MOS in VoIP, contdcontd

Example mappings From loss and

delay to their impairment scores and to MOS

10

15

20

25

30

35

40

45

50

0 0.03 0.06 0.09 0.12 0.15 0.18

Ie (l

oss

impa

irmen

t)


G.729 T=20ms random loss

0

5

10

15

20

25

30

35

0 50 100 150 200 250 300 350 400

Id (d

elay

impa

irmen

t)

delay (ms)

E-model Id

0.5

1

1.5

2

2.5

3

3.5

4

4.5

20 40 60 80 100

MO

S

R value

R to MOS mapping

Predicting MOS under FECPredicting MOS under FEC Compute final loss probability pf after

FEC [Frossard 2001] Bursty loss reduces FEC performance Increasing the packet interval T makes

FEC more efficient under bursty loss [Jiang 2002]

Plug pf into the calibrated loss-to-Ie mapping

FEC delay is n*T for an (n,k) code Compute R value and translate to

MOS

Quality Evaluation of FEC Quality Evaluation of FEC vs. Codec Robustnessvs. Codec Robustness

Codecs under evaluation iLBC: a recent loss-robust codec proposed

at IETF; frame-independent coding G.729: a near toll quality ITU codec G.723.1: an ITU codec with even lower bit-

rate, but also slightly lower quality. Utilize MOS curves from IETF

presentations for FEC MOS estimation Assume some loss burstiness

(conditional loss probability of 30%) Default packet interval T = 30ms

G.729+(5,3) FEC vs. iLBCG.729+(5,3) FEC vs. iLBC Ignoring delay effect, a larger T improves

FEC efficiency and its quality When considering delay, however, using

a 60ms interval is overkill, due to higher FEC delay (5*60 = 300ms)

2.4

2.6

2.8

3

3.2

3.4

3.6

3.8

4

0 0.03 0.06 0.09 0.12 0.15

MO

S


G.729+(5,3)G.729+(5,3),T=60ms

iLBC,no FEC 2.4

2.6

2.8

3

3.2

3.4

3.6

3.8

4

0 0.03 0.06 0.09 0.12 0.15

MO

S_c


G.729+(5,3)G.729+(5,3),T=60ms

iLBC, no FEC

G.729+(5,2) vs. G.729+(5,2) vs. iLBC+(3,2)iLBC+(3,2)

When iLBC also uses FEC, and still keeping similar gross bit-rate G.729 still prevails, except for low loss

conditions when considering delay

2.4

2.6

2.8

3

3.2

3.4

3.6

3.8

4

0 0.03 0.06 0.09 0.12 0.15

MO

S


G.729+(5,2)G.729+(5,2),T=60ms

iLBC+(3,2)2.4

2.6

2.8

3

3.2

3.4

3.6

3.8

4

0 0.03 0.06 0.09 0.12 0.15

MO

S_c


G.729+(5,2)G.729+(5,2),T=60ms

iLBC+(3,2) FEC

G.729+(7,2) vs. G.729+(7,2) vs. iLBC+(4,2)iLBC+(4,2)

Too much FEC redundancy (e.g., for G.729) very long FEC block and delay not always a good idea

iLBC wins in this case, when considering delay

2.4

2.6

2.8

3

3.2

3.4

3.6

3.8

4

0 0.03 0.06 0.09 0.12 0.15

MO

S


G.729+(7,2)iLBC+(4,2)

2.4

2.6

2.8

3

3.2

3.4

3.6

3.8

4

0 0.03 0.06 0.09 0.12 0.15

MO

S_c


G.729+(7,2)iLBC+(4,2)

G.729+(3,1) vs. G.729+(3,1) vs. iLBC+(4,2)iLBC+(4,2)

Using less FEC redundancy may actually help, if the FEC block is shorter

Now G.729 performs similar to iLBC

2.4

2.6

2.8

3

3.2

3.4

3.6

3.8

4

0 0.03 0.06 0.09 0.12 0.15

MO

S


G.729+(3,1)iLBC+(4,2)

2.4

2.6

2.8

3

3.2

3.4

3.6

3.8

4

0 0.03 0.06 0.09 0.12 0.15

MO

S_c


G.729+(3,1)iLBC+(4,2)

Comparison with G.723.1Comparison with G.723.1 MOS(G.723.1) < MOS(iLBC) at zero loss

iLBC dominates more low loss areas compared with G.729, whether delay is considered or not

2.4

2.6

2.8

3

3.2

3.4

3.6

3.8

4

0 0.03 0.06 0.09 0.12 0.15

MO

S


G.723.1+(2,1)G.723.1+(2,1),T=60ms

iLBC, no FEC

2.4

2.6

2.8

3

3.2

3.4

3.6

3.8

4

0 0.03 0.06 0.09 0.12 0.15

MO

S_c


G.723.1+(2,1)G.723.1+(2,1),T=60ms

iLBC,no FEC

G.723.1+(3,1) vs. G.723.1+(3,1) vs. iLBC+(3,2)iLBC+(3,2)

iLBC is still better for low loss G.723.1 wins for higher loss

2.4

2.6

2.8

3

3.2

3.4

3.6

3.8

4

0 0.03 0.06 0.09 0.12 0.15

MO

S


G.723.1+(3,1)G.723.1+(3,1),T=60ms

iLBC+(3,2)2.4

2.6

2.8

3

3.2

3.4

3.6

3.8

4

0 0.03 0.06 0.09 0.12 0.15

MO

S_c


G.723.1+(3,1)G.723.1+(3,1),T=60ms

iLBC+(3,2)

G.723.1+(4,1) vs. G.723.1+(4,1) vs. iLBC+(4,2)iLBC+(4,2) iLBC dominates in this case whether delay is

considered or not, (4,2) code already suffices for iLBC (4,1) code’s performance essentially “saturates”

2.4

2.6

2.8

3

3.2

3.4

3.6

3.8

4

0 0.03 0.06 0.09 0.12 0.15

MO

S


G.723.1+(4,1)G.723.1+(4,1),T=60ms

iLBC+(4,2)2.4

2.6

2.8

3

3.2

3.4

3.6

3.8

4

0 0.03 0.06 0.09 0.12 0.15

MO

S_c


G.723.1+(4,1)G.723.1+(4,1),T=60ms

iLBC+(4,2)

The Best of Both WorldsThe Best of Both Worlds Observations, when considering delay:

iLBC is usually preferred in low loss conditions G.729 or G.723.1 + FEC better for high loss

Example: max bandwidth 14 kb/s Consider delay impairment (use MOSc)

G.729+(5,3)

G.723.1+(2,1),T=60ms

iLBC

33.23.43.63.8

4

0 0.03 0.06 0.09 0.12 0.15

MO

S_c


Max BW: 14 kb/s

2.82.62.42.4

2.6

2.8

3

3.2

3.4

3.6

3.8

4

0 0.03 0.06 0.09 0.12 0.15

MO

S_c


iLBC,no FECG.729+(5,3)

G.723.1+(2,1),T=60ms

Max Bandwidth: 21-28 Max Bandwidth: 21-28 kb/skb/s

iLBC

G.729+(5,2)

2.83

3.23.43.63.8

4

0 0.03 0.06 0.09 0.12 0.15

MO

S_c


Max BW: 21 kb/s

2.62.4

G.729+(3,1)G.729+(5,2)

iLBC

33.23.43.63.8

4

0 0.03 0.06 0.09 0.12 0.15

MO

S_c


Max BW: 28 kb/s

2.82.62.42.4

2.6

2.8

3

3.2

3.4

3.6

3.8

4

0 0.03 0.06 0.09 0.12 0.15

MO

S_c


iLBC, no FECG.729+(3,1)G.729+(5,2)

2.4

2.6

2.8

3

3.2

3.4

3.6

3.8

4

0 0.03 0.06 0.09 0.12 0.15

MO

S_c


iLBC, no FECG.729+(5,2)

Effect of Max Bandwidth Effect of Max Bandwidth on Achievable Qualityon Achievable Quality

14 to 21 kb/s: significant improvement in MOSc

From 21 to 28 kb/s: marginal change due to increasing delay impairment by FEC

2.4

2.6

2.8

3

3.2

3.4

3.6

3.8

4

0 0.03 0.06 0.09 0.12 0.15

MO

S_c


Max BW: 14 kb/sMax BW: 21 kb/sMax BW: 28 kb/s

ConclusionsConclusions Compared listening and conversational

MOS achieved by conventional vs. robust codecs, with same BW constraint

iLBC is better under low loss conditions Conventional codec + FEC is better

under high loss, but Usefulness of FEC redundancy saturates

beyond a certain point considering delay At roughly a max BW of 21 kb/s Reveals max achievable quality with current

FEC mechanism

Future WorkFuture Work

Implement the MOS prediction and optimization procedure in software

Investigate the effect of jitter on conventional vs. robust codecs FEC cannot reduce jitter unless there

are many out-of-order packets PLC in a robust codec like iLBC incurs

a much lower delay, thus may be preferable to FEC

ReferencesReferences W. Jiang and H. Schulzrinne, Comparison

and optimization of packet loss repair methods on VoIP perceived quality under bursty loss, NOSSDAV 2002

P. Frossard, FEC performance in multimedia streaming, IEEE Comm Letter 3/2001

ITU-T, Subjective performance assessment of telephone-band and wideband digital codecs, Recommendation P.830 2/1996

comparisons of fec and codec robustness on voip quality and bandwidth efficiency wenyu jiang henning...

Documents