e0262 - multimedia communications anandi giridharan electrical communication engineering, indian...

E0262 - Multimedia Communications

Anandi GiridharanElectrical Communication Engineering,

Indian Institute of Science, Bangalore – 560012, India

Multimedia Communications


Reflection on Multimedia Communication● High-Density File Transfers: This traffic can include everything

from low- resolution, cartoon-like images up to full-color, high-resolution photographs, as well as recorded streams of audio and/or video signals.

● High-density file transfers include three categories: graphics, audio, and video transfers.

● They share common network performance requirements: guaranteed integrity of packetized data upon delivery, without regard for the sequence or timing of that delivery. In fact, failure to retransmit damaged or dropped packets will likely result in noticeable degradation of files.

● They require the transport protocol to provide error detection and correction, as well as resequencing of the received packets. Once this is accomplished, the transport protocol hands the packets to the appropriate application for storage.


Graphics File Transfers●Graphics files vary in size based upon the compression algorithm used, that is, the file's format, physical size, and pixel and color density. ●Graphics have become a readily accepted form of computing since the advent of graphical user interfaces. This integration has been so successful that most users do not consider graphics to be "multimedia." ● Graphics file transmissions do not require timeliness of delivery. Damaged packets can be retransmitted and resequenced without adversely affecting the application.


this could be the World Wide Web (WWW). Web pages are a notorious example

of the quantitative challenge of adding a new, bandwidth-intensive, multimedia

application to an existing network.

This type of traffic is likely to contain in-line graphics embedded in text files, and

users are likely to be waiting for the graphics to be delivered. Large, complex

graphics and/or animation may require multiple retransmissions before being

successfully reassembled by the user's browser. This may take more time than the

user is willing to wait. Though not technically a time-sensitive application, this

example demonstrates the difficulty of constructing categories to define a

spectrum of uses. Fortunately, this type of multimedia communication is readily

abetted by carefully selecting graphics formats for their effectiveness in

compressing the file.


Audio File TransfersPre-recorded audio files can be encoded in several different formats and contain speech,

music, sounds, and so on. A transferred file can be stored either on disk or in memory.

Once received in its entirety, the file can be played back.

Audio transfers, by virtue of having been pre-recorded, can effectively utilize transmission

error detection and correction mechanisms. Temporarily storing them to disk or memory

upon receipt provides the user with a more error-free file and smoother sound quality than

is possible if the file were played back immediately upon receipt.

This temporary storage provides the time needed for damaged or dropped packets to be

identified, and re-sent. This implies that this method can create a greater volume of overall

network traffic than a streaming audio transmission would.


Video File Transfers

Recorded video files can also be encoded in a variety of formats. They may or may not

contain supplemental audio tracks. Those with accompanying audio will be marginally

larger than a similarly sized and same duration video-only file, but they require

synchronization of audio and video. Any errors in the received file will be more obvious

due to the temporary disruption in the continuity between image and sound.

Prerecorded video files, like their audio-only counterparts, can take advantage of

transmission error detection and correction mechanisms. Storing them to disk (memory is

probably not a viable storage option, given the likely sizes of even "small" video files)

upon receipt provides the user with a more accurate replication of the original file than

would have been possible if the file were being viewed immediately.

Temporarily storing, or buffering, the file provides the time needed for damaged or

dropped packets to be re-sent and resequenced.


Audio Communication

Audio communication can take three distinct forms, each with a slightly different set of network performance requirements. Specific application categories include: computer-based telephony, audio conferencing, and audio transmission.


Computer-Based Telephony

Computer-based telephony uses PCs and LANs/WANs to integrate voice

telephony into a data network. The client PC buffers inbound transmissions

and plays them using its sound card and speakers. This buffering can "smooth

out" the sound quality of the transmission somewhat, despite its having

traversed contention-based LANs using error-correcting protocols.

Audio communication is not bandwidth intensive. Audio can be delivered

over dialup facilities as slow as 14.4Kbps. This form of communication,

however, is extremely susceptible to corruption from packets delivered late or

out of sequence. Any such packets are discarded because, by the time a

successful retransmission can be made, the stream being played back will

likely have progressed beyond the point at which that packet was needed.

Thus, re-inserting it late creates a second disturbance that is readily

detectable by the user.


Computer-Based Audio ConferencingAudio conferencing differs from computer-based telephony only in that it is used in

other than point-to-point sessions. Conferences tend to be multipoint-to-multipoint in

the case of a collaborative conference of peers, or point-to-multipoint for broadcasts of

major events.

Given the half-duplex nature of this technology set, point-to-multipoint unidirectional

broadcasts may be the best use of this technology. The network must have some

mechanism for this form of multicasting. Multicasting is the transmission of a single

stream of packetized data with an address that is recognized by more than one

workstation. This is far more bandwidth- efficient than transmitting multiple

simultaneous streams, each destined for a single, specific end-point. End-points that

belong to a multicast group "listen" for both their unique Internet address, and the

address of their group.


Streaming AudioStreaming audio transmissions are unidirectional transmissions of a stream of audio data.

It uses a host that either records audio in real-time, or uses prerecorded audio media. In

either case, packets stream out onto the network as soon as they are generated.

Recipients listen to them as they arrive, generally without buffering them. Dropped or

damaged packets are usually left out of the playback session; some of the newer

streaming audio products on the market, however, offer the opportunity to attempt a

retransmission.

Streaming audio, like most audio-only multimedia applications, is relatively easy to

support in a LAN/WAN environment. It is low bandwidth, and benefits from but does

not require low network latency. Perhaps the most important attribute of streaming audio

technology is that it does not pretend to be a bi-directional audio transmission

technology. Therefore, its practical applications are much more readily determined.

Streaming audio can be used to distribute, on demand, either a feed from a live speech or

copies of recorded speeches, Question and Answer sessions, or even the latest disk from

your favorite group.


Video Communication

Video communication is the acid test of any IT platform. It requires a fairly high-

powered computer and can also be extremely bandwidth intensive. It also

benefits greatly from low-latency network components. At a minimum, it should

use network protocols that can reserve bandwidth at the time of call setup.

Video communication can occur at surprisingly low levels of throughput, given

the right compromise of picture size, quality, and refresh rate. The ideal rate of

refresh is 30 frames per second. At this rate, known as "full motion," the picture

appears smooth, and movement is smooth, not jerky. Unfortunately, even using a

small picture size, like 288 pixels by 352 pixels, the uncompressed stream is

approximately 500Kbps! This represents a generous portion of a T-1's available

bandwidth. It is also a sizeable portion of the useable bandwidth on most LANs.


Dropping the refresh rate to 15 frames per second dramatically reduces bandwidth

consumption, but only marginally degrades perceivable performance. The use of

sophisticated video engines, like those that offer compression "on-the-fly," can further

reduce the bandwidth consumption, albeit at an increase in CPU cycle consumption.

Decreasing the number of colors recognized can also greatly reduce the size of the

transmission stream. Similarly, reducing the size of the picture, as measured either in

inches, pixels, or fractions of a screen (for example, full-screen, quarter-screen, eighth-

screen, and so on) can also greatly reduce the transmission stream.

Unfortunately, compromises in refresh rate, color density, pixel density, and so on,

effectively translate into degraded video picture quality. An overly aggressive bandwidth

conservation effort can easily result in a jerky, tiny, ``talking head" of a video image. This

defeats the very purpose of video conferencing by failing to capture the body language and

other subtle non-verbal communication that can be gleaned from face-to-face interactions.

The trick is to understand the limitations of the networking that will support the video

transmissions, and strive for an optimal combination of tunable parameters that doesn't

have adverse affects on the network.


Video ConferencingReal-time, bi-directional transmissions between two or more points is known as video conferencing.

The accompanying audio can be handled "in-band" or "out-of-band." In-band audio transmissions

bundle the audio signals with the video signals in the same bit stream. This requires the video

conferencing system to have its own speakers and microphone or to interface with those already

installed in the computer.

Out-of-band audio relieves software from having to capture and play back the audio signals. It also

means the video conferencing system doesn't have to synchronize the audio and video. Rather, the

video conferencing system ignores the audio signals and requires conferees to establish a second

communication link over conventional telephony.

Because a video conference entails a bi-directional transmission, the network interconnecting the

conferenced end-points must be capable of supporting two separate video streams. If both conferees

opt for full-screen, full-motion video and use 17-inch monitors set for 1024 x 768 dots per inch (dpi)

resolution, most networks will be extremely hard pressed to deliver the desired level of service.


Because a video conference entails a bi-directional transmission, the network

interconnecting the conferenced end-points must be capable of supporting two separate

video streams. If both conferees opt for full-screen, full-motion video and use 17-inch

monitors set for 1024 x 768 dots per inch (dpi) resolution, most networks will be extremely

hard pressed to deliver the desired level of service.

Proprietary video conferencing hardware and software bundles have been available for

quite some time. Numerous transmission technologies are supported by these bundles,

including ISDN at 128Kbps. Ostensibly, the controlling software is smart enough to

understand the limits of the selected network, and throttles maximum quality settings

accordingly.


Streaming Video

Streaming video differs from video conferencing in that it is not bi-directional. Nor does it

have to be live. The streams can be either live or prerecorded, but are transmitted,

multicast, or broadcast uni-directionally.

As is the case with streaming audio, streaming video does not benefit from a network

protocol's ability to detect and correct errors, or resequence received packets. Rather, the

packets are played back immediately upon receipt. Compared to video conferencing

systems, streaming video transmission systems are fairly crude.

Although it lacks the functionality and sophistication of a video conferencing system,

streaming video may be even more useful just by virtue of being more useable. Video

libraries can be maintained of public events, speeches, meetings, and so on, and played

back on demand. Assuming the picture size and quality were carefully managed, the

streams can be supported on as little as 128Kbps, although a vastly superior video would

result from connections of 384Kbps or higher.


It is inevitable that the currently separate, and redundant, voice, data, and video communication infrastructure will

eventually integrate into a single, broadband, multimedia communication infrastructure, with a common user

interface and vehicle. This integration is only just beginning, and will take years to complete. It is already clear

that the LAN is capable of incrementally growing into the role of multimedia communication network. In the

interim, it is being used to support fledgling attempts at this degree of integration: today's multimedia applications.

Some of these applications require levels of performance that are difficult to achieve in a conventional

LAN/WAN environment. For example, any of the live, interactive applications like computer and video telephony,

or video conferencing, require that packets be delivered on time, in sequence, and intact! Packets that are damaged

are simply discarded without generating a request to the sender for a retransmission. Similarly, packets that are

delivered intact, but are delivered late or out of sequence are also discarded without requesting retransmission.

Today's LANs, WANs, and their protocols, are not well suited to transporting the time-sensitive data of many

multimedia communication technologies. Contemporary networks and protocols are better suited to the transport

of traditional data. They can guarantee the integrity of each packet's payload upon delivery, and are more efficient

at transporting bulk data than they are at guaranteeing timeliness of delivery.


Even if the LANs have been engineered to support high-bandwidth and/or low-latency

applications, WAN links can be the kiss of death. They use high-latency routers, which

function at the relatively slow Layer 3 and in conjunction with relatively low-bandwidth

transmission facilities (DS 1 or less). This can be a vexing problem as it is widely accepted

that the business value of multimedia applications, like video conferencing, is directly

proportional to the geographic distances separating the end-points. In other words, today's

networks are least capable of delivering time-sensitive applications in their highest-value

scenarios.

Similar evolution of application software and hardware will complete this evolution. Until

true multiple-media communication technologies are available, remember the benefits of

open standards. They apply universally throughout the information technologies.

Proprietary, single-function "multimedia" products, regardless of how well they perform,

are likely to impede interoperability of multimedia applications.


●

One possible exception to this could be the World Wide Web (WWW). Web pages are a

notorious example of the quantitative challenge of adding a new, bandwidth-intensive,

multimedia application to an existing network.

This type of traffic is likely to contain in-line graphics embedded in text files, and users

are likely to be waiting for the graphics to be delivered. Large, complex graphics and/or

animation may require multiple retransmissions before being successfully reassembled

by the user's browser. This may take more time than the user is willing to wait. Though not technically a time-sensitive application, this example demonstrates the difficulty of

constructing categories to define a spectrum of uses. Fortunately, this type of multimedia

communication is readily abetted by


RTP Sessions

●An RTP session consists of a number of applications communicating with RTP, and is identified by a network address and two ports. ●One port is used for media data and the other for RTCP (real time transfer control protocol) control data.● Session participants can send, receive, or do both. ●Each media type is transmitted in a separate session, enabling participants to choose which media types they want to receive. ●For example, a user may just want the audio portion of a streaming music video.


●RTP enables identification of the type of data being transmitted, and

using sequence numbers, what order the packets should be presented

in. ●Since correct packet order is not guaranteed in transmission, it is

important for the receiver to be able to order packets and detect packet

losses.● Timestamping allows synchronization of media streams from

different sources, and smooths out possible jitter and noise when real

time applications place received packets into a playback buffer. ●RTP is paired with RTCP (Real Time Transport Control Protocol)

which examines the quality of the connection and sends feedback to

the sender and receiver, allowing both to negotiate the best

transmission scheme. RTCP also provides control and identification

mechanisms for RTP transmissions.


RTP Application Example: A

Videoconference●In a video conference, both ends of the link need to act as RTP clients and RTP servers.● On the client end, the conferencing application needs to receive the media stream from an RTP session, order the packets, and render it onto the console. ●A conference with both audio and video would require two RTP sessions, one to handle each media type. ●On the server end, a media stream might need to be captured with a video camera, and the audio and video components would each be transmitted on a RTP session. ●If the video needed to be encoded in multiple formats for different receiver applications, several RTP sessions might be used.

e0262 - multimedia communications anandi giridharan electrical communication engineering, indian...

Documents

multimedia application

graphics file transmissions

graphics formats

audio transfers

line graphics

transferred file

errorfree file

video transfers