2000 prentice hall, inc. all rights reserved. chapter 16 - multimedia: images, animation, audio and...

2000 Prentice Hall, Inc. All rights reserved.

Chapter 16 - Multimedia: Images, Animation, Audio and Video

Outline16.1 Introduction16.2 Downloading the Java Media Framework16.3 Loading, Displaying and Scaling Images16.4 Loading and Playing Audio Clips16.5 The Java Media Player16.6 Animating a Series of Images16.7 Animation Issues16.8 Customizing Applets via the HTML param Tag16.9 Image Maps16.10 Java Plug-In16.11 Internet and World Wide Web Resources


16.1 Introduction

• Revolution in computer industry– Before, computers used for high-speed calculations

– Now, data manipulation important

• Multimedia– "Sizzle" of Java - images, sound, video

– CDs, DVDs, video cards

– Demands extraordinary computing power• Fast processors making multimedia possible

• Java– Built-in multimedia capabilities

• Most programming languages do not

– Develop powerful multimedia applications


16.2Downloading the Java Media Framework

• Java media framework (JMF)– Extension to Java API

– Enhanced processing of images, audio, and video

– Constantly evolving

– Some examples required the JMF, not included in J2SDK

http://java.sun.com/products/java-media/jmf/1.1/

• JMF Downloads for Windows and Solaris

– At time of writing, early versions of JMF 2.0 availablehttp://java.sun.com/products/java-media/jmf/index.html


16.2Loading, Displaying and Scaling Images

• Java Multimedia– Discuss graphics, images, animations, sounds, and video

– Begin with images

• Class Image (java.awt)– Abstract class, cannot create an object directly

• Request Image be loaded and returned

– getImage( imageLocation, filename ); • Of class Applet (superclass of JApplet)• imageLocation

– getDocumentBase() is URL (address) of HTML file• filename - .gif or .jpg (.jpeg)

10 private Image logo1;

16 logo1 = getImage( getDocumentBase(), "logo.gif" );



• Displaying Images– Method drawImage (class Graphics)

– Many overloaded versions

g.drawImage( myImage, x, y, ImageObserver );• myImage - Image object• x,y - coordinates to display image• ImageObserver - object on which image is displayed

– Use "this" to indicate the applet

– Can be any object that implements ImageObserver interface

24 g.drawImage( logo1, 0, 0, this );



• Displaying Images– g.drawImage( myImage, x, y, width, height,

ImageObserver );• width and height

– Dimensions of image (automatically scaled)• getWidth(), getHeight()

– Returns dimensions of applet

28 g.drawImage( logo1, 0, 120,

29 getWidth(), getHeight() - 120, this );



• Class ImageIcon– Concrete class (can create objects)

– Example constructor and initialization

• Displaying Icons – Method paintIcon (class Graphics)myIcon.paintIcon( Component, Graphics, x, y )

• Component - Component object on which to display image (this)

• Graphics - Graphics object used to render image (g)• x, y - coordinates of Icon

11 private ImageIcon logo2;

17 logo2 = new ImageIcon( "logo.gif" );



• Usage– ImageIcons simpler than Images

• Create objects directly

• No need for ImageObserver reference

– All components are ImageObservers

– Cannot scale ImageIcons

• Scaling– Use ImageIcon method getImage

• Returns Image reference

• This can be used with drawImage and scaled


Outline

1. Declare objects

2. init()

2.1 Initialize objects

3. paint()

3.1 drawImage

1// Fig. 16.1: LoadImageAndScale.java

2// Load an image and display it in its original size

3// and scale it to twice its original width and height.

4// Load and display the same image as an ImageIcon.

5import java.applet.Applet;

6import java.awt.*;

7import javax.swing.*;

8

9public class LoadImageAndScale extends JApplet {

10 private Image logo1;

11 private ImageIcon logo2;

12

13 // load the image when the applet is loaded

14 public void init()

15 {

1616 logo1 = getImage( getDocumentBase(), "logo.gif" );

17 logo2 = new ImageIcon( "logo.gif" );

18 }

19

20 // display the image

21 public void paint( Graphics g )

22 {

23 // draw the original image

2424 g.drawImage( logo1, 0, 0, this );

25

26 // draw the image scaled to fit the width of the applet

27 // and the height of the applet minus 120 pixels

28 g.drawImage( logo1, 0, 120,

29 getWidth(), getHeight() - 120, this );

Notice the parameters to the constructors. Image objects cannot be created directly.

drawImage calls (second call has scaling). this means the image will be shown on the applet.


Outline

3.2 paintIcon

Program Output

3232 logo2.paintIcon( this, g, 180, 0 );

33 }

34 }

30

31 // draw the icon using its paintIcon method

Call to paintIcon, displays icon on this applet (a Component).


16.4Loading and Playing Audio Clips

• Audio clips– Require speakers and sound board

– Sound engine - plays audio clips• Supports .au, .wav, .aif, .mid• Java Media Framework supports additional formats



• Playing audio clips– play method in Applet– Plays clip once, marked for garbage collection when finished

play( location, soundFileName );location - getDocumentBase (URL of HTML file)

play( soundURL ); soundURL - URL that contains location and filename

of clip



• Playing audio clips– Method play from AudioClip interface

• More flexible than Applet method play• Audio stored in program, can be reused

– getAudioClip• Returns reference to an AudioClip• Same format as Applet method play

– getAudioClip( location, filename )– getAudioClip( soundURL )

46 sound1 = getAudioClip(

47 getDocumentBase(), "welcome.wav" );



• Playing audio clips– Once AudioClip loaded, use methods

• play - plays audio once• loop - continuously loops audio in background• stop - terminates clip that is currently playing

• Upcoming program– Use a JComboBox to select between two sounds

66 currentSound.loop();

68 currentSound.stop();

64 currentSound.play();


Outline

1. Declare objects

1.1 GUI

1.2 Event handler

1.2.1 stop

1// Fig. 16.2: LoadAudioAndPlay.java

2// Load an audio clip and play it.

3import java.applet.*;

4import java.awt.*;

5import java.awt.event.*;


7

8public class LoadAudioAndPlay extends JApplet {

9 private AudioClip sound1, sound2, currentSound;

10 private JButton playSound, loopSound, stopSound;

11 private JComboBox chooseSound;

12

13 // load the image when the applet begins executing


15 {

16 Container c = getContentPane();

17 c.setLayout( new FlowLayout() );

18

19 String choices[] = { "Welcome", "Hi" };

20 chooseSound = new JComboBox( choices );

21 chooseSound.addItemListener(

22 new ItemListener() {

23 public void itemStateChanged( ItemEvent e )

24 {


26

27 currentSound =

28 chooseSound.getSelectedIndex() == 0 ?

29 sound1 : sound2;

Once the selection changes, stop currentSound and set it equal to either sound1 or sound2.


Outline

1.3 Register event handlers

2. Initialize AudioClips

3. Method stop

30 }31 }32 );33 c.add( chooseSound );3435 ButtonHandler handler = new ButtonHandler();36 playSound = new JButton( "Play" );37 playSound.addActionListener( handler );38 c.add( playSound );39 loopSound = new JButton( "Loop" );40 loopSound.addActionListener( handler );41 c.add( loopSound );42 stopSound = new JButton( "Stop" );43 stopSound.addActionListener( handler );44 c.add( stopSound );45

4646 sound1 = getAudioClip(47 getDocumentBase(), "welcome.wav" );48 sound2 = getAudioClip(49 getDocumentBase(), "hi.au" );50 currentSound = sound1;51 }5253 // stop the sound when the user switches Web pages54 // (i.e., be polite to the user)

5555 public void stop()56 {57 currentSound.stop();58 }59

Notice how AudioClips are initialized.

Applet's stop method called when user switches web pages. Ensures clip stop playing (else could be annoying to user).


Outline

4. Event handler

4.1 play, loop, stop

6060 private class ButtonHandler implements ActionListener {61 public void actionPerformed( ActionEvent e )62 {63 if ( e.getSource() == playSound ) 64 currentSound.play();65 else if ( e.getSource() == loopSound )

66 currentSound.loop();

67 else if ( e.getSource() == stopSound )


69 }

70 }

71 }

Call methods play, loop, and stop as appropriate.


16.5 The Java Media Player

• Java Media Player– Comes with JMF

– Can play• Audio: .au, .wav, .aiff, .mid• Video: .avi, .gsm, .mpg

(.mpeg), .mov, .rmf, .rtp, .viv

Play/pause

Position indicator

MuteVolume

Information



• JFileChooser (javax.swing)– Allows you to select a file



• JFileChooser (javax.swing)

– Create JFileChooser

– Open only files (not directories)

– Returns an integer indicating user's action (selected file or cancel)

– If cancel, set file to null

35 JFileChooser fileChooser = new JFileChooser();

37 fileChooser.setFileSelectionMode(38 JFileChooser.FILES_ONLY );

39 int result = fileChooser.showOpenDialog( this );

42 if ( result == JFileChooser.CANCEL_OPTION )43 file = null;



– Otherwise, call getSelectedFile and assign result to file

• Class File discussed next chapter

– Returns object that implements Player interface (javax.media)

• If file of correct format, returns object

• Method toURL converts file name and location into URL format

44 else

45 file = fileChooser.getSelectedFile();

10 private Player player;

57 player = Manager.createPlayer( file.toURL() );



– Registers event handler• Interface ControllerListener, must define controllerUpdate

– Takes ControllerEvent objects

– Many subclasses, we use RealizeCompleteEvent subclass

– Adds visual aspect of media clips• Audio clips return null

58 player.addControllerListener( new EventHandler() );

59 player.start(); // start player

108 Component visualComponent =109 player.getVisualComponent();111 if ( visualComponent != null )112 c.add( visualComponent, BorderLayout.CENTER );



– Display controls

– Update content pane

– Programmer-defined method removePreviousPlayer

• Close current player

114 Component controlsComponent =

115 player.getControlPanelComponent();

117 if ( controlsComponent != null )

118 c.add( controlsComponent, BorderLayout.SOUTH );

120 c.doLayout();

73 player.close();



– Get references to visual and control components

– Container method remove• Remove Components from interface

75 Component visual = player.getVisualComponent();

76 Component control = player.getControlPanelComponent();

80 if ( visual != null )

81 c.remove( visual );

83 if ( control != null )

84 c.remove( control );


Outline

1. Declarations

1.1 Constructor

1 // Fig. 16.3: MediaPlayerDemo.java2 // Uses a Java Media Player to play media files.3 import java.awt.*;4 import java.awt.event.*;5 import java.io.*;6 import javax.swing.*;7 import javax.media.*;89 public class MediaPlayerDemo extends JFrame {10 private Player player;11 private File file;1213 public MediaPlayerDemo()14 {15 super( "Demonstrating the Java Media Player" );1617 JButton openFile = new JButton( "Open file to play" );18 openFile.addActionListener(19 new ActionListener() {20 public void actionPerformed( ActionEvent e )21 {22 openFile();23 createPlayer();24 }25 }26 );27 getContentPane().add( openFile, BorderLayout.NORTH );2829 setSize( 300, 300 );30 show();31 }


Outline

2. Method openFile

2.1 JFileChooser

2.2 file

3. Method createPlayer

3.1 removePrevious Player

3.2 createPlayer

3.3 Event handler

3.4 start

3233 private void openFile()34 { 3535 JFileChooser fileChooser = new JFileChooser();3637 fileChooser.setFileSelectionMode(38 JFileChooser.FILES_ONLY );39 int result = fileChooser.showOpenDialog( this );4041 // user clicked Cancel button on dialog

4242 if ( result == JFileChooser.CANCEL_OPTION )

43 file = null;

44 else

45 file = fileChooser.getSelectedFile();

46 }

47

48 private void createPlayer()

49 {

50 if ( file == null )

51 return;

52

53 removePreviousPlayer();

54

55 try {

56 // create a new player and add listener

5757 player = Manager.createPlayer( file.toURL() );

58 player.addControllerListener( new EventHandler() );

59 player.start(); // start player

60 }

Create a JFileChooser, use it to select a file.

Test for cancel option. If not, assign file to file.

Assign player result of createPlayer. Add event handler (named inner class), call start method to being playing clip.


Outline

4. removePrevious Player

4.1 close

4.2 remove

5. main

61 catch ( Exception e ){62 JOptionPane.showMessageDialog( this,63 "Invalid file or location", "Error loading file",64 JOptionPane.ERROR_MESSAGE );65 }66 }6768 private void removePreviousPlayer()69 {70 if ( player == null )71 return;72

7373 player.close();7475 Component visual = player.getVisualComponent();76 Component control = player.getControlPanelComponent();7778 Container c = getContentPane();79

8080 if ( visual != null )81 c.remove( visual );8283 if ( control != null )84 c.remove( control );85 }8687 public static void main(String args[])88 {89 MediaPlayerDemo app = new MediaPlayerDemo();90

Close the current player

Get references to Components and remove them from content pane.


Outline

6. Event handler

6.1 getVisualComponent

6.2 getControlPanel Component

91 app.addWindowListener(

92 new WindowAdapter() {

93 public void windowClosing( WindowEvent e )

94 {

95 System.exit(0);

96 }

97 }

98 );

99 }

100

101 // inner class to handler events from media player

102 private class EventHandler implements ControllerListener {

103 public void controllerUpdate( ControllerEvent e ) {

104 if ( e instanceof RealizeCompleteEvent ) {

105 Container c = getContentPane();

106

107 // load Visual and Control components if they exist

108108 Component visualComponent =

109 player.getVisualComponent();

110

111 if ( visualComponent != null )

112 c.add( visualComponent, BorderLayout.CENTER );

113

114 Component controlsComponent =

115 player.getControlPanelComponent();

116

117 if ( controlsComponent != null )

118 c.add( controlsComponent, BorderLayout.SOUTH );

Get Components and attach to content pane.


Outline

6.3 doLayout

Program Output

119

120 c.doLayout();

121 }

122 }

123 }

124}


Outline

Program Output


16.6Animating a Series of Images

• Class Timer– Generates ActionEvents at a fixed interval in

milliseconds

– Constructor:Timer ( animationDelay, actionListener );

actionListener - ActionListener that will respond to ActionEvents

– Methods• start• stop• restart• isRunning



• Method repaint– Calls update, which calls paintComponent

• Subclasses of JComponent should use paintComponent• Call superclass paintComponent first to display

components properly

30 super.paintComponent( g );



• View area– Width and height specify entire window, not just client area

• Client area - where components can be displayed

– Dimension objects• Constructor

– Dimension( width, height )• Contains width and height instance variables

– Method setSize( width, height )• Class JFrame• Sets size of window



• getImageLoadStatus – ImageIcon method

• Determines if image is completely loaded into memory

• Only complete images should be displayed (smooth animation)

– If loaded, returns MediaTracker.COMPLETE

– MediaTracker• Can determine when images are loaded, or force program to

wait • ImageIcon creates our MediaTracker for us

32 if ( images[ currentImage ].getImageLoadStatus() ==

33 MediaTracker.COMPLETE ) {



• Following example– Use a series of images stored in an array

– Use Timer to generate ActionEvents• Change image displayed

• Update content pane


Outline

1.1 Class LogoAnimator ( implements ActionListener)

1.2 Declare objects

1.3 Constructor

1.4 Initialize ImageIcon array

2. paintComponent

2.1 super.paint Component

1// Fig. 16.3: LogoAnimator.java2// Animation a series of images3import java.awt.*;4import java.awt.event.*;5import javax.swing.*;67public class LogoAnimator extends JPanel8 implements ActionListener {9 protected ImageIcon images[];10 protected int totalImages = 30,11 currentImage = 0,12 animationDelay = 50; // 50 millisecond delay13 protected Timer animationTimer;1415 public LogoAnimator()16 {17 setSize( getPreferredSize() );1819 images = new ImageIcon[ totalImages ];2021 for ( int i = 0; i < images.length; ++i ) 22 images[ i ] =

2323 new ImageIcon( "images/deitel" + i + ".gif" );2425 startAnimation();26 }2728 public void paintComponent( Graphics g )29 {

3030 super.paintComponent( g );31

Load array of ImageIcons to use for the animation.

Call to superclass paintComponent ensures proper display.


Outline

2.2 getImage LoadStatus

2.3 Increment currentImage

3. actionPerformed

3.1 startAnimation

3.2 stopAnimation

32 if ( images[ currentImage ].getImageLoadStatus() ==

33 MediaTracker.COMPLETE ) {

34 images[ currentImage ].paintIcon( this, g, 0, 0 );

3535 currentImage = ( currentImage + 1 ) % totalImages;

36 }

37 }

38

39 public void actionPerformed( ActionEvent e )

40 {

41 repaint();

42 }

43

44 public void startAnimation()

45 {

46 if ( animationTimer == null ) {

47 currentImage = 0;

4848 animationTimer = new Timer( animationDelay, this );

49 animationTimer.start();

50 }

51 else // continue from last image displayed

5252 if ( ! animationTimer.isRunning() )

53 animationTimer.restart();

54 }

55

56 public void stopAnimation()

57 {

58 animationTimer.stop();

59 }

60

Check if the image has loaded, and if so, increment currentImage.

Create new Timer, start. this applet listed as event handler.

If animationTimer is not running (i.e. user switched web pages and returned), restart it.


Outline

3.3 getMinimumSize

3.4. getPreferredSize

4. main

61 public Dimension getMinimumSize()62 { 63 return getPreferredSize(); 64 }65

66 public Dimension getPreferredSize()

67 {

68 return new Dimension( 160, 80 );

69 }

70

71 public static void main( String args[] )

72 {

73 LogoAnimator anim = new LogoAnimator();

74

75 JFrame app = new JFrame( "Animator test" );

76 app.getContentPane().add( anim, fBorderLayout.CENTER );

77

78 app.addWindowListener(

79 new WindowAdapter() {

80 public void windowClosing( WindowEvent e )

81 {

82 System.exit( 0 );

83 }

84 }

85 );

86


Outline

Program Output

87 // The constants 10 and 30 are used below to size the

88 // window 10 pixels wider than the animation and

89 // 30 pixels taller than the animation.

90 app.setSize( anim.getPreferredSize().width + 10,

91 anim.getPreferredSize().height + 30 );

92 app.show();

93 }

94}


16.7Animation Issues

• Storing images– Interlaced/non-interlaced formats

• Specifies order in which pixels are stored

• Non-interlaced - pixels stored in order they appear on screen

– Image appears in chunks from top to bottom as it is loaded

• Interlaced - pixels stored in rows, but out of order

– Image appears to fade in and become more clear



• Animation flickers– Due to update being called in response to repaint– In AWT GUI components

• Draws filled rectangle in background color where image was

• Draw image, sleep, clear background (flicker), draw next image...

– Swing's JPanel overrides update to avoid this



• Double buffering– Used to smooth animations

– Program renders one image on screen • Builds next image in off-screen buffer

– When time to display next image, done smoothly• Partial images user would have seen (while image loads) are

hidden

• All pixels for next image displayed at once

– Space/Time tradeoff• Reduces flicker, but can slow animation speed and uses more

memory

– Used by Swing GUI components by default


16.8Customizing Applets via the HTML param Tag

• Applets– Customize through parameters in HTML file that invokes

it<html><applet code="LogoApplet.class" width=400

height=400><param name="totalimages" value="30"><param name="imagename" value="deitel"><param name="animationdelay" value="200"></applet></html>

– Invokes applet LogoApplet– param tags

• Each has a name and a value• Use Applet method getParameter (returns a String)

parameter = getParameter( "animationdelay" );


16.8Customizing Applets via the HTML param Tag

• Following example– Use the LogoAnimator class as before, but modified

slightly• Include constructor to specify delay, name, number of images

– Create Applet LogoApplet• Takes parameters

– totalimages– imagename– animationdelay

– Create LogoAnimator object using parameters (use new constructor)

– Play animation


Outline

1. LogoAnimator class as before, but with new constructor---------------------

1.1 getParameter

1.2 LogoAnimator

7 public class LogoAnimator extends JPanel

8 implements ActionListener {2222 public LogoAnimator( int num, int delay, String name )

127import java.awt.*;


129

130public class LogoApplet extends JApplet{


132 {

133 String parameter;

134

135135 parameter = getParameter( "animationdelay" );

136 int animationDelay = ( parameter == null ? 50 :

137 Integer.parseInt( parameter ) );

138

139 String imageName = getParameter( "imagename" );

140

141 parameter = getParameter( "totalimages" );

142 int totalImages = ( parameter == null ? 0 :

143 Integer.parseInt( parameter ) );

144

145 // Create an instance of LogoAnimator

146 LogoAnimator animator;

147

148 if ( imageName == null || totalImages == 0 )

149 animator = new LogoAnimator();

150 else

151151 animator = new LogoAnimator( totalImages,

152 animationDelay, imageName );

The LogoAnimator class now has a new constructor.

Get parameters from HTML file, convert to integers.

Create new LogoAnimator, use custom constructor if applicable.


Outline

2. setSize

2.3 startAnimation

Program Output

153

154 setSize( animator.getPreferredSize().width,

155 animator.getPreferredSize().height );

156 getContentPane().add( animator, BorderLayout.CENTER );

157

158158 animator.startAnimation();

159 }

160} Start the animation.


16.9Image Maps

• Image map– Image that has hot areas

• User can click to accomplish a task

– Bubble help• When mouse over particular point in screen, small message

displayed in status bar

• In the following example– Load image containing several icons

– Use event handler mouseMoved to find x-coordinate

– Based on the x-coordinate, display a message


Outline

1. Declare objects

2. init

2.1 addMouseListener

1 // Fig. 16.7: ImageMap.java

2 // Demonstrating an image map.

3 import java.awt.*;

4 import java.awt.event.*;

5 import javax.swing.*;

6

7 public class ImageMap extends JApplet {

8 private ImageIcon mapImage;

9 private int width, height;

10


12 {

13 addMouseListener(

14 new MouseAdapter() {

15 public void mouseExited( MouseEvent e )

16 {

17 showStatus( "Pointer outside applet" );

18 }

19 }

20 );

21

22 addMouseMotionListener(

23 new MouseMotionAdapter() {

24 public void mouseMoved( MouseEvent e )

25 {

2626 showStatus( translateLocation( e.getX() ) );

27 }

28 }

29 );

30

Pass the x coordinate to translateLocation. This returns a String, which is passed to showStatus.


Outline

2.2 Initialize ImageIcon

3. paint

4. translateLocation

31 mapImage = new ImageIcon( "icons2.gif" ); 3232 width = mapImage.getIconWidth();33 height = mapImage.getIconHeight();34 setSize( width, height );35 }3637 public void paint( Graphics g )38 {39 mapImage.paintIcon( this, g, 0, 0 );40 }4142 public String translateLocation( int x )

43 {

44 // determine width of each icon (there are 6)

4545 int iconWidth = width / 6;

46

47 if ( x >= 0 && x <= iconWidth )

48 return "Common Programming Error";

49 else if ( x > iconWidth && x <= iconWidth * 2 )

50 return "Good Programming Practice";

51 else if ( x > iconWidth * 2 && x <= iconWidth * 3 )

52 return "Performance Tip";


54 return "Portability Tip";


56 return "Software Engineering Observation";


58 return "Testing and Debugging Tip";

Determine width of mapimage.

Determine width of each icon in image. Based on this, display a message.


Outline

Program Output

59

60 return "";

61 }

62 }


16.10 Java Plug-In

• Web browsers– Many different browser versions

– Most support Java 1.0 and 1.1, but few support Java 2• Inconsistent support of 1.1

– To use Java 2 in an applet, use the Java Plug-in• Bypasses browser's Java support

• Installs complete version of Java Runtime Environment on user's computer

• Large file - ideal for corporate intranets and high-speed networks


16.10 Java Plug-In

• Using the Plug in with applets– The <applet>, <param> and </applet> tags must be

converted• Must indicate that Plug-in should be used to execute applet

– Java Plug-in 1.2 HTML Converter• Performs conversion for us• http://java.sun.com/products/plugin/

– Execution• In install directory type java HTMLConverter• Select file to convert and type of conversion


16.11 Internet and World Wide Web Resources

• Internet and web resources – Java Media Framework

– Download and documentation:http://java.sun.com/products/java-media/jmf/

– javax.media API descriptions:http://java.sun.com/products/java-media/ jmf/forDevelopers/


16.11 Internet and World Wide Web Resources

• Downloadable image, audio, and video galleries– Test your multimedia programs

http://www.nasa.gov/gallery/index.html

http://sunsite.sut.ac.jp/multimed/

2000 prentice hall, inc. all rights reserved. chapter 16 - multimedia: images, animation, audio and...

Documents