Graphics

CS 3540

Graphics in games• None• ASCII art (so called...)• Images

– Bitmaps– Vector

• 2D animation• Video• 3D animation

3D GraphicsCommonly used libraries:• OpenGL

www.opengl.org• Microsoft DirectX

http://msdn.microsoft.com/directx

Many other libraries, wrappers, and game engines are built on one or both of these libraries.

OpenGL vs. DirectX• Open standard• Provides interface to

hardware acceleration• Portable• Widely used in

commercial and non-commercial games

• No native image or 3D model file format

• GLUT

• Proprietary• Provides interface to

hardware acceleration• Only on Microsoft

platforms• Widely used in

commercial and non-commercial games

• Native .x 3D model format, loaders for common bitmap formats

• Windows API• Direct Play, Direct X

Audio, etc.

"OpenGL is the only truly open, vendor-neutral, multiplatform graphics standard."

http://www.opengl.org/about/overview/

Fun with 3D libraries...Both OpenGL and DirectX are complicated libraries and learning how to use them well is not easy.

DirectX, in particular, has gone through major changes that obsolete many earlier books and web sites. Microsoft's documentation can also change in confusing ways.

Making a game with3D graphics

Suppose that I want to make a 3D display for UV Bots (which I did...)

How do I do that?

We'll look at how do that with some references to Unity and other game engines and libraries.

Nine easy stepsNine moderately difficult steps

1. Find or make 3D models

2. Initialize software and hardware

3. Load models4. Add models to world5. Lights6. Camera7. Action8. Interaction9. This, that, and the

other

Nine frustrating steps that will make you wish you were using Klingon pain sticks instead and will make you want to go into something simple like neurosurgery...

Unity makes these steps much easier, but it’s good to know at least a little bit about what’s happening behind the scenes.

1. Find or make 3D modelsFinding models• Google (what else?)• Call in favors from your artistic friends

(you have lots of those, right?)• Borrow them from other games (but

only if it's ethical)

Making models• Blender 3D

blender.org• Milkshape 3D• Wings 3D

www.wings3d.com• Various commercial packages, most of

which are very expensive

3D model formats• Bad news:

There are no "standard" formats for 3D model files that are comparable to JPEG or PNG.

• Worse news:Converting between 3D file formats is a pain and the model will probably not look the same after conversion.

3D model formats• Text or binary?• What documentation is available?

– Is a file format specification available?• How complicated is the file format?• What software can read/write the

format?– Modeling software– Conversion software– Loaders and game engines

• Does it include animation?

3D model formats• .3ds, .max - 3DS max format• .x - Direct X• .dxf - created for AutoCAD by AutoDesk• .obj - Wavefront (an ASCII format)• .mdl – Quake• .fbx – Used by AutoDesk and others• .dae – COLLADA

www.khronos.org/collada/plus many morefileformat.info

.x DirectX• Both text and binary formats• DirectX SDK has loaders• Includes animation• Specification is available but is not for

the faint-hearted• Various importers, exporters,

converters, but many are flaky

Note: It’s been a long time since I’ve used or read about .x files.

.obj Wavefront• Wavefront (which merged with Alias,

which was bought by AutoDesk)• Text format• Relatively simple• No animation• Blender can read/write, Unity can load

.ms3d Milkshape• Binary format• Includes animation• Reading and writing:

– Chumbalum's web site (http://chumbalum.swissquake.ch/) has C++ code to load .ms3d files

– Blender cannot read or write

COLLADA

www.khronos.org/collada/

•COLLADA transports 3D assets between applications Enables binding of diverse DCC and 3D processing tools into a production pipeline

• COLLADA is a XML database schema for 3D assets COLLADA is an intermediate language - not a delivery format and not a scene graph

• COLLADA can be lossless Retains all information - even multiple versions of the same asset

• COLLADA has comprehensive encoding of visual scenes Including geometry, animation, shaders and physics

• COLLADA is an open, archive-grade format that retains meta information When your DCC tool upgrades, you keep your assets

From the Khronos Group web site:

File formats in Unity• Can read .FBX, .dae, .3DS, .dxf,

and .obj files• Natively imports Maya files• “Natively” imports Blender files (via

FBX)

http://unity3d.com/support/documentation/Manual/HOWTO-importObject.html

2. Initialize software and hardwareDirectX

• One book has eight fun-filled pages on how to initialize DirectX, MSDN has a lot more than that...

• Many options, choosing the wrong options can leave you with a blank screen

• Beware of COM• Make sure you deallocate, close,

destroy everything you allocate, open, or create if you want to avoid memory leaks

2. Initialize software and hardwareOpenGL

• Setup is relatively easy if you use GLU and GLUT, but can still be tricky

Encapsulate!• Set-up code can be messy, so hide the

details.

• Encapsulation makes it easier to port to another OS or another graphics library.

3. Load modelsChoices:• Write your own loader

– Not easy since 3D file formats are complicated and often not well documented

• Use someone else's source code– Hard to say how well it will work

• Use a game engine– Only an option if the engine supports

your format

Components of 3D models• Vertices - points in 3D space• Edges - connect vertices• Faces/Polygons

– often triangles, sometimes quads• Meshes

• Normals - Which way is up? (or out?)– Used for shading

More components of models

• Hierarchies - parent/child• Colors• Textures

– Combinations of other materials– Images - often stored in separate file– Must be mapped to mesh

• Bones• Animation - key frames

4. Add model to the world

• Left-handed or Right-handed?

• Location, size, and orientation

• Parent-child hierarchy

Left-handed vs. Right-handed

x

y

z

y

x

z

left-handed

right-handed

"The right-handed system is universally accepted as the standard in mathematics and the physical sciences."

wikipedia

but... 3DS max vs. DirectXUnity is left-handed

Lost in Space(s)• Model space (or local coordinates)

– Coordinates are relative to local origin– Used to create model

• World space– Coordinates are relative to global origin– Used to place models within the world

• Screen space– Projected into 2D plane

y

x

z

Local vs. global coordinates

y

xz

y

xz

The front turrets have the same local coordinates for both instances of the model. The front turrets will have

different global coordinates for each instance of the model.

How do the x, y, and z global coordinates compare for these two instances of the bot model?

Parent/Child hierarchyWhen the bot turns, the turret should turn also, but sometimes the turret should turn when the bot doesn't.

• Frames• Scene graph• Submeshes

Adding models in UnitySome links to the Unity User Guide• Asset Import and Creation• Prefabs

Lights! Camera! Action!

Some material in this section comes fromIntroduction to Game Programming, edited by Steve Rabin

LIGHTING

Questions

• How much light is incident on a surface?

• From which direction does the light come?

• How is the light absorbed, reemitted, and reflected?

• Which outgoing rays reach the eye of the viewer?

Answers (or at least ways to find the answers)

• Forward tracing– trace every photon emitted by a light

source and figure out where it goes– photon mapping: high quality but

extremely slow• Backward tracing

– trace a hypothetical photon that hit the eye to see which object it came from

– raytracing: practical, but not for real time rendering http://www.povray.org

Answers (or at least ways to find the answers)

• "Middle-out" evaluation– "Given this bit of surface, how much light

came from these sources of light and ended up hitting the eye?" *

– Looks at triangle currently being drawn– Real time rendering has to compromise

between speed and quality

• p. 471, Introduction to Game Programming,• edited by Steve Rabin

Kinds of lightingWe'll look at how different kinds of lighting affect this scene.

Note: These pictures were made in Blender, not with real-time rendering.

Ambient light• Light that is not attributed to any

particular source• Can be a reasonable approximation

for some multi-light situations, especially indoors

• Unity: Edit->Render SettingsWhat happens if your scene has too much ambient light?

Too much ambient light

Diffuse lighting

Specular lighting

Hemisphere lighting

Sun + sky + ground

Useful approximation for some outdoor scenes

direct sunlightdiffracted sunlightreflected sunlight

Environment map• Approximates reflections• Cube map:

– Put cube representing environment around the object being rendered.

– Trace vector from eye to object, reflect from object, and see where it strikes the cube.

• Same idea as hemisphere lighting but can contain images or more varied colors

Shadows• Can be done with ray-tracing.• Some games have shadows.

Types of lights

Point

Directional

Spot

Properties of lights

• Type: point, spot, directional• Position: as for any object• Color: color of light• Range: how far the light shines

Properties of spotlights

Same as point lights, plus:• Direction• Falloff• Attenuation• Radius of inner and outer cones

Lights in Unity• Point, directional, spot• Shadows (only in the Pro version)• Ambient light: Edit->Render Settings

http://unity3d.com/support/documentation/Components/class-Light.html

Lights! Camera! Action!

POINT OF VIEWand

RENDERING

From local to output coordinates

Programming Role-Playing Games with DirectX,by Jim Adams

UntransformedVertex

WorldTransformation

MatrixView

TransformationMatrix

ProjectionMatrixOutput

TransformedVertex

local coordinates

place object in world

choose point of view

viewing frustum

The Matrix(actually, lots of matrices)

Matrices are used for transformations:• Translate

– move from one place to another• Rotate

– center of rotation is important• Scale

– center of scaling is important

Using a matrix to transform a vertex

• Vertex is represented as three coordinates– vector from origin to location of vertex– 1x3 matrix

• Multiply vertex matrix by transformation matrix to get new vertex matrixx'

y'z'

=xyz

2 0 00 2 00 0 2

What transformation does this represent?

Combining transformations• Transformations can be combined by

multiplying matrices together• Combine matrices, then multiply each

vertex in object by combined matrix.– More efficient than multiplying each vertex by

each of the transformation matrices.

Note: In order to represent translation with matrix multiplication you have to add an extra coordinate. You will sometimes see vertices represented by 4-element vectors, and 4x4 transformation matrices.

From local to world coordinates

UntransformedVertex

WorldTransformation

MatrixView

TransformationMatrix

ProjectionMatrixOutput

TransformedVertex

local coordinates

place object in world

Placing an object in the world

• Models usually have their center at the origin (0, 0, 0) in local coordinates.

• To place a model in the world, the coordinates of all vertices in the model have to be transformed (translated, rotated, and/or scaled).

• Usually the world transformation is kept in the (C++) object for the model and then applied each time the model is rendered.

Animation using a model's world transformation

• By changing the world transformation each frame, you can animate a model:move, rotate, scale, or some combination

• Since the world transformation applies to the whole model, this kind of animation affects the whole model also.

DirectX

• Call SetTransform before drawing each model• DirectX will apply pMatrix to each vertex in the

model• pMatrix will usually be some combination of

translation, rotation and scaling• Game engines (e.g. Irrlicht) will provide easier

ways of setting the location, orientation, and size of models.

HRESULT IDirect3DDevice8::SetTransform(D3DTRANSFORMSTATETYPE State, // D3DTSWORLDCONST D3DMATRIX *pMatrix) // World matrix to set

Programming Role-Playing Games with DirectX, by Jim Adams

From world to view coordinates

UntransformedVertex

WorldTransformation

MatrixView

TransformationMatrix

ProjectionMatrixOutput

TransformedVertex

place object in world

choose point of view

Choosing a point of view

DirectX

• Call SetTransform with D3DTS_VIEW parameter.

• Only has to be done once (until view is changed).

• How do you make the view matrix (pMatrix)?

HRESULT IDirect3DDevice8::SetTransform(D3DTRANSFORMSTATETYPE State, // D3DTS_VIEWCONST D3DMATRIX *pMatrix) // View matrix to set

Programming Role-Playing Games with DirectX, by Jim Adams

DirectX

Making a view matrix

• vecVP is the coordinates of the viewpoint– Location of the viewer's eye or a camera

• vecTP is the coordinates of the target– what the camera is pointing at

• vecUp tells the orientation of the camera

Programming Role-Playing Games with DirectX, by Jim Adams

D3DXMATRIX matView;D3DXVECTOR3 vecVP, vecTP, vecUp(0.0f, 1.0f, 0.0f);vecVP.x = XPos;vecVP.y = YPos;vecVP.z = ZPos;vecTP.x = vecTP.y = vecTP.z = 0.0f;D3DXMatrixLookatLH(&matView, &vecVP, &vecTP, &vecUp);

OpenGLvoid draw(void){ glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);

glMatrixMode(GL_MODELVIEW); glLoadIdentity(); gluLookAt(xO, yO, zO, xref, yref, zref, Vx, Vy, Vz); gModel.draw(); glFlush();}

glMatrixMode is similar to SetTransform in DirectX

xO, yO, zO specify the location of the viewpoint (camera)

xref, yref, zref specify the location of the target (look-at point)Vx, Vy, Vz specify the orientation of the camera

Cameras

• Game engines usually provide a camera class with methods to change its location and orientation.

• Our (former) textbook defines a camera class (cCamera) as part of the Graphics Core.

From view to projection

UntransformedVertex

WorldTransformation

MatrixView

TransformationMatrix

ProjectionMatrixOutput

TransformedVertex

choose point of view

viewing frustum

The viewing frustum

near clipping plane

far clippingplane

viewing frustum

Objects that are outside the viewing frustum will not be rendered.Unity: You can set the clipping planes for a

camera in the Inspector.

From view to output

UntransformedVertex

WorldTransformation

MatrixView

TransformationMatrix

ProjectionMatrixOutput

TransformedVertex

output to screen

viewing frustum

Output• DirectX provides various functions for

drawing primitives and meshes.• These functions require some set up,

and use the current settings for transformations.

• DirectX primitives:D3DPT_POINTLIST, D3DPT_LINELIST, D3DPT_LINESTRIP, D3DPT_TRIANGLELIST, D3DPT_TRIANGLESTRIP, D3DPT_TRIANGLEFAN

Lights! Camera! Action!

ANIMATION

Remember... The Matrix?The same matrices that are used to place objects in the world can be used to move and resize objects:

• Translate– move from one place to another

• Rotate– center of rotation is important

• Scale– center of scaling is important

Character Animation

• Morphing Animation• Interpolation• Skeletal Animation• Kinematics and Inverse Kinematics

Morphing animation• Quake is the classic example of a

game that uses morphed animation.

MD2 file format:• A set of named animations• Each animation contains a sequence

of frames• Each frame contains a mesh

http://tfc.duke.free.fr/old/models/md2.htm

Question

If I am writing a game that uses the MD2 file format, and I want a high frame rate, does that mean that my MD2 file is going to be huge?

Keyframes• Only store the most important ("key")

frames in the MD2 file.• When animating the model, figure out

the intermediate frames by comparing the positions of corresponding vertices.

• This is called interpolation

InterpolationFilling in between keyframes

For each coordinate of each vertex:xs = starting coordinate value

xf = final coordinate value

How do we calculate an intermediate value?

picture from www.w3.org

Interpolating character poses

picture from www.sarge-sdk.com

Skeletal animationModel contains:• bones connected to

make a skeletal structure

• skin: textured mesh• mapping from skin to

bones:vertex weightingIf you move a bone, what skin moves with it?

picture from Blender documentation

Skeletal animation• Game program animates character by

moving bones--skin will move accordingly

• File formats:• Milkshape (.ms3d), DirectX (.x)

picture from Blender documentation

Skeletal animation

Skeletal animation can also make use of key frames and interpolation, but they will be specified in terms of bones rather than in terms of vertices in a mesh.