introducing computer programming with multimedia barb ericson institute for computing education...

Introducing Computer Introducing Computer Programming with Programming with

MultimediaMultimediaBarb EricsonBarb Ericson

Institute for Computing EducationInstitute for Computing Education

Georgia Institute of TechnologyGeorgia Institute of [email protected]@cc.gatech.edu

Computer Science CrisisComputer Science Crisis

• 40% drop in majors since 2001 Nationwide– Up to 60% drop in some places

• Percentage of women is down to about 12%– From a high of 30% in the 1980s

• Never attracted many African Americans, Hispanics, Native Americans

Why the Decline?Why the Decline?

• Dot com crash in 2001– Loss of jobs

• AP Exam switched from C++ to Java– Many schools quit teaching AP Computer Science

• Visual Basic changed– In .net to be object-oriented

• Perceptions of Computer Science– boring, antisocial, not creative, and just programming

Looming ShortageLooming Shortage

• Only a few jobs have gone overseas– About 2-3%

• We have recovered from the dot com crash– There are now more jobs in this area than in

2001

• Computing jobs are some of the fastest growing– US Dept of Labor predicts 45.93% growth

Reversing the TrendReversing the Trend

• We need to get more students interested in computer science– Not drive them away

• We need to appeal to a diverse group of students– Not just geeks

• All students need to have some understanding of computer science– Our society relies on technology

Our SolutionOur Solution

• Use multimedia projects to introduce computing concepts– Pictures, sounds, movies, text

• Use open-ended projects to encourage creativity– Create an image collage, create a sound

collage, create a movie

• Make computing social– Encourage students to show off their work

Choice of LanguagesChoice of Languages

• Python– Used by many companies: IL&M, Google, Nextel, etc.– Easier syntax than many languages– Can use to teach functional, procedural, or object-

oriented programming– Used as a scripting or extension language for many

applications– Growing in popularity

• Java– Popular language used for AP CS

Digitizing PicturesDigitizing Pictures

• We digitize pictures into lots of little dots

• Enough dots and it looks like a continuous whole to our eye

• Each picture element is referred to as a pixel– Each pixel knows its color– It also knows where it is in

its picture

Encoding Color: RGBEncoding Color: RGB

• In RGB, each color has three component colors:– Red, Green, and Blue

• In most computer-based models of RGB, a single byte (8 bits) is used for each– So a complete RGB color

is 24 bits, 8 bits of each– 0,0,0 is black– 255,255,255 is white– 255,0,0 is red

Make and Show a PictureMake and Show a Picture

Use pickAFile function To select a picture file(ends in .jpg)

Use makePicture(file) to create the picture

Use show(picture) todisplay the picture

Use MediaToolsUse MediaTools

• Click on MediaTools• Select the picture to

explore• Move the cursor while

holding down the left mouse button– To see x,y,r,g,b

• Click on zoom– To zoom in or out

Creating a Python FunctionCreating a Python Function

Define a function

Load the function

Try the function

Rules for Python FunctionsRules for Python Functions

• Use the keyword def to define a function– Followed by the

function name

• Use parentheses after the name– Name parameters

• End with a colon• Indent the body of the

function

def pickAndShow(): myFile = pickAFile() myPicture =

makePicture(myFile) show(myPicture)

Using a LoopUsing a Loop

def decreaseRed(picture):

for p in getPixels(picture):

value = getRed(p)

setRed(p,value * 0.5)

More Picture FunctionsMore Picture Functionsdef greyscale(picture): for p in getPixels(picture): redness=getRed(p) greenness=getGreen(p) blueness=getBlue(p) luminance=(redness+blueness+greenness)/3 setColor(p,

makeColor(luminance,luminance,luminance))

def negate(picture): for px in getPixels(picture): red=getRed(px) green=getGreen(px) blue=getBlue(px) negColor=makeColor(255-red,255-green,255-

blue) setColor(px,negColor)

How Sound Works:How Sound Works:

• Sounds are waves of air pressure– Sound comes in cycles– The frequency of a

wave is the number of cycles per second (cps), or Hertz

– The amplitude is the maximum height of the wave

Volume and PitchVolume and Pitch

• Our perception of volume is related (logarithmically) to changes in amplitude– If the amplitude doubles, it’s about a 3 decibel (dB)

change.

• Our perception of pitch is related (logarithmically) to changes in frequency– Higher frequencies are perceived as higher pitches– We can hear between 5 Hz and 20,000 Hz (20 kHz)– A above middle C is 440 Hz

Digitizing SoundDigitizing Sound

• We can estimate the sound curve– Analog-to-digital

conversion (ADC) will give us the amplitude at an instant as a number: a sample

– How many samples do we need?

• 2 x max frequency• CD quality is 44,100

samples per second

Digitizing sound in the computerDigitizing sound in the computer

• Each sample is stored as a number – 16 bits, 216 = 65,536 combinations– But we want both positive and negative values

• To indicate compressions and rarefactions.

• Each sample can be between -32,768 and 32,767

Exploring a SoundExploring a Sound

Use MediaTools to explore a Sound

Working with SoundsWorking with Sounds

• We’ll use getSamples to get all the sample objects out of a sound– Use getSample to get the value from a Sample obj

• We can also get the value at any index with getSampleValueAt

• Sounds also know their length (getLength) and their sampling rate (getSamplingRate)

• Save a sound to a file using writeSoundTo(sound,”file.wav”)

Decreasing the volumeDecreasing the volume

def decreaseVolume(sound): for sample in getSamples(sound): value = getSample(sample) setSample(sample,value * 0.5)

This works just like decreaseRed, but decreasing the amplitude (volume) instead of the red color.

Multimedia Java CS1Multimedia Java CS1

• We use DrJava– A free and easy development environment– With an interactions pane

• We have created Java classes– For the students to use and modify

• World• Turtle • Picture• Sound• FileChooser

• Introduction to Java– Math operators, printing results, data types, casting,

relational operators, Strings, variables

• Introduction to Programming– Creating and naming objects

• Using a turtle and a world

– Creating new Turtle methods• Draw simple shapes• Using parameters

Java SyllabusJava Syllabus

Syllabus - ContinuedSyllabus - Continued

• Modifying Pictures using Loops– One-dimensional

arrays– Use for-each, while,

and for loops to– Increase/decrease

colors, fake a sunset, lighten and darken, create a negative, and grayscale

Create and Show a PictureCreate and Show a Picture

Java Negate MethodJava Negate Method

/** * Method to negate the picture */public void negate(){ Pixel[] pixelArray = this.getPixels(); Pixel pixelObj = null; int redValue, blueValue, greenValue = 0;

// loop through all the pixels for (int i = 0; i < pixelArray.length; i++)

Java Negate Method - ContJava Negate Method - Cont

{ // get the current pixel pixelObj = pixelArray[i];

// get the values redValue = pixelObj.getRed(); greenValue = pixelObj.getGreen(); blueValue = pixelObj.getBlue();

// set the pixel's color pixelObj.setColor( new Color(255 - redValue, 255 - greenValue, 255 - blueValue)); } }

Testing NegateTesting Negate


• Modifying Pixels in a Matrix– Two-dimensional

arrays– Nested loops– Copying, mirroring,

blending, rotation, scaling


• Conditionally Modifying Pixels– Replacing a color,

reducing red-eye, edge detection, sepia-toned, posterize, highlight extremes, blurring, background subtractions, chromakey


• Drawing on Pictures– Using existing Java

classes • Inheritance• Interfaces

– Drawing simple shapes, drawing text, general copy, general scale, shearing, gradient paint, general blending, clipping


• Modifying all Samples in a Sound– 1D arrays– Loops– Conditional execution– Change volume,

normalizing a sound (make it as loud as possible), force to extremes


• Modifying Samples using Ranges– Loops– Clipping, splicing,

reversing, mirroring


• Combining and Creating Sounds– Class and private

methods– Composing

sounds, blending sounds, changing frequencies, and creating echoes

– MP3 and MIDI


• Creating Classes– Defining a class– Overloading constructors– Creating and initializing an

array– Creating getters and setters– Creating a main method– Javadoc comments– Using Inheritance

ConfusedTurtleTurtle

Syllabus - ContinuedSyllabus - Continued• Creating and Modifying

Text– Reading from and writing

to files• Handling Exceptions

– Modifying programs– Getting text from networks– Creating random

sentences– Using text to shift between

media

Syllabus - ContinuedSyllabus - Continued• Making Text for the

Web– Throwing exceptions,

HashMap, Generics, and Iterators

– Create a web page from a directory

– Create a web page from other web pages

– Creating a web page from a database


• Encoding, Manipulating, and Creating Movies– Frame-based

animations with simple shapes and text

– Special effects – fade out, fake sunset, and chromakey


• Speed– What makes programs

fast?• Compilers and Interpreters• Searching• Algorithms that can’t be

written

– What makes computers fast?

• Clock rates, Storage, Display


• Javascript– Syntax– User Interfaces– Multimedia

Multimedia CS2 in JavaMultimedia CS2 in Java

• Driving question: “How did the wildebeests stampede in The Lion King?”

CS2 IdeasCS2 Ideas• Teach data structures using multimedia

– Stacks and queues for simulations– Linked lists of pictures

• Order in linked list is order in scene• Order in linked list is the layer the picture is in• Circular linked list for animation

– Linked lists of sounds• Weave and repeat musical phrases

– Trees of pictures• Scene graph

– Trees of sounds• Different traversal gives different result

Examples of Student WorkExamples of Student Work

Soup-Audio Collage

Canon- LinkedList of (MIDI) Music

ResultsResults

• Python CS1 is used for non-majors at Georgia Tech– Business students went from 49% to 88% pass rate– Similar results at Gainesville and U.Ill-Chicago

• Java CS1 is used in Georgia High Schools– Both in an introductory course and in Advanced

Placement courses– Also used at Dennison and Duke

• Java CS2 is used at Georgia Tech– Students are transferring into computer science

Why Media Computation?Why Media Computation?• Works on nearly all computers

– Unlike Alice which requires newer computers

• Low cost– Unlike LEGO Mindstorms or other robots

• Students can see (and hear) if their programs work• Computer Science concepts are introduced to solve

problems– Looping instead of changing thousands of pixels by hand

• Open-ended assignments allow students to be creative– Students can use their own media

• Students can publicly share their media artifacts with others

What’s on the CDWhat’s on the CD

• CS1-Python materials– Chapters from the Media Computation book in Python– Course slides– Free Jython Environment for Students (JES)– Instructor Resources

• CS1-Java materials– Chapters from the Media Computation book in Java– Course Slides– Java Classes

What’s on the CD - ContinuedWhat’s on the CD - Continued

• CS2-Java materials– Course notes– Course slides– Java Classes

• DrJava– Free Development environment for Java

• Mediasources– Royalty free pictures and sounds

Additional ResourcesAdditional Resources

• Web Site for Media Computation– http://coweb.cc.gatech.edu/mediaComp-plan

• CS1 Python Book– Introduction to Computing and Programming in

Python, A Multimedia Approach by Mark Guzdial, ISBN: 0131176552

• CS1 Java Book– Introduction to Computing and Programming with

Java, A Multimedia Approach by Mark Guzdial and Barbara Ericson, ISBN: 0131496980

• CS2 Java Book– Coming in 2007

Funding SourcesFunding Sources

• National Science Foundation

• Georgia Tech's College of Computing

• Georgia’s Department of Education

• Al West Fund

• Toyota Foundation

introducing computer programming with multimedia barb ericson institute for computing education...

Documents

computing jobs

computer scienceboring

computer programming

computer sciencenot

computerbased models

pictureuse pickafile

computing conceptspictures

picture fileends