multimedia systems and applications

34
MULTIMEDIA SYSTEMS AND APPLICATIONS By S. Baskaran & T. Salim M.Sc Computer Science, Department of Computer Science, Pondicherry University.

Upload: karuna-moorthi

Post on 17-May-2015

1.312 views

Category:

Technology


3 download

TRANSCRIPT

Page 1: Multimedia systems and applications

MULTIMEDIA SYSTEMS AND APPLICATIONS

By S. Baskaran & T. Salim

M.Sc Computer Science,

Department of Computer Science,

Pondicherry University.

Page 2: Multimedia systems and applications

Data and File Format

• RTF • TIFF• RIFF• MIDI• JPEG• AVI video file formats• MPEG standards.

Page 3: Multimedia systems and applications

RTF

• The Rich Text Format (RTF) Specification provides a method of transferring encoding formatted text and graphics between different output devices, OS and applications.

• RTF uses the American National Standards Institute (ANSI), PC-8, Macintosh, or IBM PC character set to control the representation and formatting of a document, both on the screen and in print.

Page 4: Multimedia systems and applications

RTF(cont.)

• RTF files begins with control word "\rtfN"; where N is the major version number. An entire RTF file is consider a group and must be enclosed in braces. Current version is 1.7, hence the file will begin with

• {\rtf1\ansi\ansicpg1252\....}..

Page 5: Multimedia systems and applications

TIFF

• TIFF: stands for Tagged Image File Format.

• The support for attachment of additional information (referred to as “tags”) provides a great deal of flexibility.

1. The most important tag is a format signifier: what type of compression etc. is in use in the stored image.

2. TIFF can store many different types of image: 1-bit, grayscale, 8-bit color, 24-bit RGB, etc.

3. TIFF was originally a lossless format but now a new JPEG tag allows one to opt for JPEG compression.

4. The TIFF format was developed by the Aldus Corporation in the 1980's and was later supported by Microsoft.

Page 6: Multimedia systems and applications

Resource Interchange File Format (RIFF)

• This is a file format for multimedia data on PCs. It can contain bit-mapped graphics, animation, digital audio and MIDI data.

• The WAV file format is the RIFF format for storing sound data.

• RIFF (similarly to TIFF - Tagged Image File Format) is a tagged file format. Tags allow applications capable of reading RIFF files to read RIFF files by another application, hence the word interchange in RIFF.

• Other Formats/Players - RealPlayer 7 (Windows NT) with RealAudio, MP3 (MPEG Audio Layer 3) audio, Midi players; MP3 players (MP3.com)

Page 7: Multimedia systems and applications

MP3

• Its full title is MPEG-1/2 Layer 3. It is a format for compressing sounds which uses a lossy technique that does not seriously degrade the quality of the sound because it filters out aspects of the original sound that the human ear cannot detect. After filtering it then applies further compression techniques. A form of coding called Huffman encoding is used to compress the data once it has been captured.

• One minute of music takes up around 1 MS of space. MP3 allows compression of CD quality audio files by a factor of 12 with little loss in quality. This explains why it is such a widely used format.

Page 8: Multimedia systems and applications

MIDI

• The Musical Instrument Digital Interface (MIDI) is a standard interface used by musical instruments like keyboards, synthesisers and drum machines which enables notes played on an instrument to be saved on a computer system, edited and played back through a MIDI device.

• The information about the sound is stored in a MIDI file which the computer can then use to tell the instrument which notes to play.

• http://www.cool-midi.com/

Page 9: Multimedia systems and applications

MIDI (Cont.)

• When a MIDI sound is stored in a computer system the following attributes or properties of the sound are stored:

• Instrument, Pitch, Volume, Duration

• Defines the instrument being played. Each built-in sound on a MIDI keyboard has an instrument number assigned to it.

When selected the instrument number is saved by the computer so that, on playback, the notes in the musical piece are played with the sound of that specific instrument.• This sets the musical tone of a note which is determined by the frequency.

Page 10: Multimedia systems and applications

MIDI (Cont.)

• This controls the loudness or amplitude of the note.• This determines the length of a note (the number of

beats).• Tempo Rate is the speed at which the piece of music

is set.

Page 11: Multimedia systems and applications

Advantages of MIDI

• Allows musical pieces or messages to be exchanged and edited on different computers.

• It is an easily manipulated form of data. Changing the tempo is a straightforward matter of changing one of the attributes.

• A musician can store the messages generated by many instruments in one file. This enables a musician to put together and edit a piece of music generated on different midi instruments with complete control over each note of each instrument.

Page 12: Multimedia systems and applications

Advantages of MIDI(cont.)

• Produces much smaller file sizes than other sound formats.

• Because it is digital it is easy to interface instruments, such as keyboards, to computers. The musician can store music on the computer and the computer can then play the music back on the instrument.

• Disadvantage• Browsers require separate plug-ins to play MIDI files.

Page 13: Multimedia systems and applications

EXIF• EXIF (Exchange Image File) is an image format for digital cameras:

1. Compressed EXIF files use the baseline JPEG format.

2. A variety of tags (many more than in TIFF) are available to facilitate higher quality printing, since information about the camera and picture-taking conditions (flash, exposure, light source, white balance, type of scene, etc.) can be stored and used by printers for possible color correction algorithms.

3. The EXIF standard also includes specification of file format for audio that accompanies digital images. As well, it also supports tags for information needed for conversion to FlashPix (initially developed by Kodak).

Page 14: Multimedia systems and applications

JPEG

• JPEG: The most important current standard for image compression.

• The human vision system has some specific limitations and JPEG takes advantage of these to achieve high rates of compression.

• JPEG allows the user to set a desired level of quality, or compression ratio (input divided by output).

• As an example, Fig. 3.17 shows our forestfire image, with a quality factor Q=10%.

- This image is a mere 1.5% of the original size. In comparison, a JPEG image with Q=75% yields an image size 5.6% of the original, whereas a GIF version of this image compresses down to 23.0% of uncompressed image size.

Page 15: Multimedia systems and applications

JPEG image with low quality specified by user

Page 16: Multimedia systems and applications

JPEG I• The Joint Photographic Experts Group is a standards body

that developed what is now known as the JPEG image-encoding format

• Like GIFs, JPEG images are platform independent and specially compressed for high-speed transfer via digital communication technologies

• Unlike GIF, JPEG supports tens of thousands of colors for more detailed, photorealistic digital images

• JPEG uses special algorithms that yield much higher data-compression ratios. It is not uncommon, for example, for a 200 kilobyte GIF image (which uses lossless compression) to be reduced to a 30kilobyte JPEG image

Page 17: Multimedia systems and applications

JPEG II• To achieve that amazing compression, JPEG does lose some

image data• However, you can adjust the degree of ``lossiness'' with

special JPEG tools, so that although the uncompressed image may not exactly match the original, it will be close enough that most people cannot tell the difference

• The JPEG format is nearly universally understood by today's graphical browsers. Some, most notably Netscape, have a built-in JPEG decoder

• Others, like Mosaic, invoke an external viewing tool (helper application) for decoding and displaying JPEG files, which invariably are stored with a .jpg (or .JPG) filename suffix

Page 18: Multimedia systems and applications

Comparison of JPEG and GIF (Cont.)

• You can expect progressive loss of quality each time you do that, especially with different JPEG quality settings

• If you must edit a photographic image, work with it in TIFF or PNG format until it is ready for publication, then convert it to JPEG for the web

• Go straight from a lossless 24-bit format supported by your scanner, such as TIFF or PNG, to JPEG

• If a given image cannot tolerate being reduced to 8 bits for GIF or losing precise accuracy for JPEG, TIFF and PNG are the best options.

Page 19: Multimedia systems and applications

The Progressive JPEG

• Progressive JPEG is a new variation on the JPEG image format

• It is like interlaced GIF in that it fade in gradually instead of being drawn from top to bottom

• This enables the viewer to understand what the image is about very quickly

• Progressive JPEG is much smoother than progressive GIF

Page 20: Multimedia systems and applications

The Video Format

The file format of the video determines: – Which programs can open and play it.

– How much storage space it occupies.

– How fast it travels over an Internet connection.

Page 21: Multimedia systems and applications

Video File Formats

• AVI (Audio Video Interleave)– Windows format, plays in Windows Media Player– Very good quality, even at smaller resolutions– Large file size – not recommended for delivering video over

the Internet.– Popular format for videos stored on a computer.

• MOV (Movie)– Apple format, plays in the QuickTime Player– Very good quality– Popular format for videos downloaded from the Internet.

Page 22: Multimedia systems and applications

Video File Formats

• MPEG (Moving Pictures Expert Group) – The standard for compression and storage of audio and motion

video for use on the World Wide Web.• Creates video small file sizes.• Popular format for videos downloaded from the Internet.

– Its biggest advantage is that It will play in many different media players.

• RM (RealMedia)– Plays in the RealPlayer player.– Typically contains a movie clip.– Popular format for streaming video viewed over the Internet.– Real Player is generally supported by many different computers

and operating systems.

Page 23: Multimedia systems and applications

Video File Formats– WMV (Windows Media Video)

• Proprietary video format developed by Microsoft.• Plays in Windows Media Player.• Popular format for streaming video viewed over the Internet.

– FLV (Flash Video)• New file format widely used on the Internet.• Plays in Adobe Flash Player.• Very small file size.• Popular format for streaming video viewed over the Internet

Page 24: Multimedia systems and applications

MPEG Audio Layer 1

• MPEG (1 and 2) audio allows sampling rate at 44.1 48, 32, 22.05, 24 and 16KHz.

• MPEG filters the input audio into 32 bands.

FilteringAnd

downsampling

Audio384 samples

12 samples

12 samples

12 samples

Perceptualcoder

NormalizeBy scalefactor

Page 25: Multimedia systems and applications

MPEG Audio Layer 2

• Layer 2 is very similar to Layer 1, but groups 3 12-samples together in coding.

• It also improves the scaling factor quantization and also groups 3 audio samples together in bit assignment.

FilteringAnd

downsampling

Audio3x384 samples

36 samples36 samples

36 samples

Perceptualcoder

NormalizeBy scalefactor

Page 26: Multimedia systems and applications

File Format

Header CRC Bit Allocation

Scalefactors Subband Data Header CRC Bit

AllocationScale

factors Subband Data

Mpeg audio puts header in each of the frame, so that they can be decoded separately.

Frame 1 Frame 2

Page 27: Multimedia systems and applications

Other Audio Coding Standards

• MPEG 2 and MPEG 4 ACC (advanced audio coding)– Not backward compatible– Use MDCT without bandpass filtering

• Dolby AC3– MDCT based codec– Similar to MPEG ACC but uses a different

quantization and coding scheme– A de-facto standard for DVD and Digital audio in

Movie.

Page 28: Multimedia systems and applications

MPEG (Moving Pictures Expert Group)

• Committee of experts that develops video encoding standards

• Until recently, was the only game in town (still the most popular, by far)

• Suitable for wide range of videos– Low resolution to high resolution– Slow movement to fast action

• Can be implemented either in software or hardware

Page 29: Multimedia systems and applications

Evolution of MPEG

• MPEG-1– Initial audio/video compression standard– Used by VCD’s–MP3 = MPEG-1 audio layer 3– Target of 1.5 Mb/s bitrate at 352x240 resolution– Only supports progressive pictures

Page 30: Multimedia systems and applications

Evolution of MPEG

• MPEG-2– Current de facto standard, widely used in DVD

and Digital TV– Ubiquity in hardware implies that it will be here

for a long time• Transition to HDTV has taken over 10 years

and is not finished yet– Different profiles and levels allow for quality

control

Page 31: Multimedia systems and applications

Evolution of MPEG

• MPEG-3– Originally developed for HDTV, but abandoned

when MPEG-2 was determined to be sufficient• MPEG-4– Includes support for AV “objects”, 3D content, low

bitrate encoding, and DRM– In practice, provides equal quality to MPEG-2 at a

lower bitrate, but often fails to deliver outright better quality

–MPEG-4 Part 10 is H.264, which is used in HD-DVD and Blu-Ray

Page 32: Multimedia systems and applications

MPEG Container Format

• Container format is a file format that can contain data compressed by standard codecs

• 2 types for MPEG– Program Stream (PS) – Designed for reasonably

reliable media, such as disks– Transport Stream (TS) – Designed for lossy links,

such as networks or broadcast antennas

Page 33: Multimedia systems and applications

AV Synchronization

• Want audio and video streams to be played back in sync with each other

• Video stream contains “presentation timestamps”• MPEG-2 clock runs at 90 kHz– Good for both 25 and 30 fps

• PCR (Program Clock Reference) timestamps are sent with data by sender

• Receiver uses PLL (Phase Lock Loop) to synchronize clocks

Page 34: Multimedia systems and applications

THANK YOU