3d graphics demystified - nvidiadeveloper.download.nvidia.com/assets/gamedev/docs/3d...3d graphics...

Demystifying 3D Graphics

NVIDIA Corporation

3D Graphics Demystified


NVIDIA Corporation

Creating WorldsAlmost all computer systems shipped today –from the humblest sub-$500 PC to the mostsophisticated workstation – have some form of3D graphics. But most of the marketing andadvertisements never really describe what 3Dgraphics are or why 3D graphics are useful.There's often a vague association with computergames, but little else.

The Purpose of 3D Graphics3D graphics in general are used to create realisticlooking environments that are visible through the2D screen of a computer monitor. Theseenvironments are designed to be fully immersive– meaning the user can "see" or "move" in anydirection, at will. The most sophisticated ofthese environments are highly interactive,allowing users to move or change the appearanceof objects.

The Challenge of InteractivityYou may have seen the stunning visual imagesthat exist in the computer game Myst. In fact,Myst is not interactive at all, but a series of pre-rendered pictures that are brought up each timeyou take a "step" in the world. You cannot turnin any direction, only in fixed increments. Youcan't take a virtual swim in the water, nor canyou "climb" on top of the many buildings yousee. While the authors used 3D graphics tools tocreate the images, they are simply a series ofstatic images.

A movie is not interactive, either. Movies like ABug's Life or Jurassic Park make heavy use ofcomputer graphics, but those are all pre-generated – you can’t manipulate the objects onthe screen.

Creating a 3D world that's visible on yourcomputer screen is a mathematically challengingapplication. Allowing a user to manipulate orinteract with the virtual environment at will, inreal time, is the goal. Even the fastest Pentium®III processor cannot do an adequate job ofgenerating graphics which look realistic andallow fluid, realistic movement through thatworld.

The Applications ConundrumThe most prevalent 3D graphics applications forpersonal computers have been games. However,as CPUs and graphics processors become morepowerful, other applications are emerging. Someare high-end, supercomputer-class applicationsmigrating down to the PC, such as thegeophysical visualization applications used byoil companies. But other types of applicationsare emerging as well. The business world, longmired in the static, 2D spreadsheet, is slowlydiscovering that 3D graphics can allow a fastergrasp of complex business data. Virtualshopping or real estate browsing is now possibleas well.

Enter 3D Graphics HardwareYou can think of the graphics chip on a 3Dgraphics card as a special CPU that can performspecific types of calculations and move certaintypes of data around very, very fast – muchfaster than the Pentium III or other processors.In fact, 3D graphics processors such as theGeForce 256™ from NVIDIA are referred to asGPUs – graphics processing units. The job ofhandling the various parts of an application issplit up. Think of it as an assembly line, with theCPU as the foreman and other processors – likethe graphics chips – as assembly line workersthat can do specific jobs very well. In fact, evenwithin the graphics chip, there's an assembly lineof sorts. We'll talk about that in the next fewpages.

When looking at reality, we don't payattention to the things we can't see.


NVIDIA Corporation

The Real WorldImagine you're standing at the kitchen windowof a house. The sink is in front of you and youcan see clearly out the window above the sink.Outside, you see a driveway with a car. Awoman walks into view, gets in the car anddrives away.

This is a pretty prosaic scene, but think about itfor a moment. You can see out the window. Butwhat's to the left and right of the window frame?Where did the woman come from? How did thecar appear as it drove into the distance?

Now imagine that you can walk around while theabove events are taking place. You turn aroundand notice what's behind you – not a façade (ason a movie set), but the cabinets, refrigerator andother typical items that exist in a kitchen. Youturn around, open the window, and lean out,noticing a tree to the left of the window that washidden by the wall.

Imagine looking out the "window" in avirtual world. What can you see?

Images courtesy of Viewpoint.

The Synthetic WorldWe normally take what we see for granted. Butif we want to recreate the same scene so that itcan be viewed on a computer, all the aboveissues come into play. This is especially true ifthe content has a high degree of interactivity, asin a game. The graphics programmer needs toworry about the things that can be seen, but alsothose items that aren't immediately visible,because as the user interacts with the computer,they may alter the view so that items that werehidden come into view.

Think of the computer screen as our kitchenwindow, above. Peering through our now-virtual

window, we see the woman walk out to the car.A moment ago, she was hidden, but then movedinto view. The car leaves the driveway, headsinto the distance and disappears. If we can"lean" forward in our view, we may see the treeto the left of the window. The graphicsprogrammer – and the graphics software – needsto keep track of things like the woman whomoves into view, or the tree we can'timmediately see. There's a tremendous amountof detail that must be considered. When the cardrives away, how far will it be visible? Howdetailed is the computer model of the car? Howrealistic are the woman's movements andappearance?

The graphics hardware and software mustconsider the unseen as well as the seen.

All motion picture and television is really anillusion. A series of pictures – "frames" areshown on the screen at a relatively high speed –24 frames each second for film, 30 frames eachsecond for television. It's like the flipbooks weplayed with as a kid, but all done in living color.But those images are really a series of staticimages being displayed quickly – they are notgenerated on the fly, and are not interactive.

When a computer creates a realistic animation, itis simply displaying frames on your computerscreen at a high rate of speed – sometimes inexcess of 60 frames each second. This requires agreat deal of mathematical computation. Ithappens to be the kind of mathematicalcomputation that CPUs do moderately well. Onthe other hand, it's possible to design dedicatedsilicon chips (GPUs) that can perform the typesof math needed for three-dimensional graphicsand animation very, very well. This creates thepossibility of virtual environments that arevisually stunning and fully interactive.


NVIDIA Corporation

Breaking It DownThe problem of creating a virtual world withthree-dimensional attributes on your computermonitor has been studied for many years. Likemost problems, it can be broken down intocomponent parts. Think of the process ofgenerating 3D graphics as an assembly line.Like the factory assembly line, there arespecialists that can handle specific jobs well:

• What the application itself – the computerprogram – does. For example, if you havean animation of someone shooting abasketball into a basket, the applicationtakes care of how the player moves, themoment the basketball hits the backboard,the movement of the net when the ball fallsthrough it, and the behavior of the ball whenit bounces on the ground.

• The overall management of what the useractually sees is known as the scene. In ourvirtual basketball game, each individualframe of animation represents a scene. Thescene contains information about what'svisible to the user. As a computer basketballplayer fires off a shot, a number of questionshave to be answered. Can you see theopposing goal (the one on the other end ofthe court)? Is the ball reflected in the glossysurface of the floor? Can you read thenumber on the player's jersey?

At this point, the problem becomes purely agraphics challenge. Our assembly line for 3Dgraphics kicks in. In the jargon of the industry,this assembly line for graphics is called the 3Dgraphics pipeline. There are four main steps –transform, lighting, setup and clipping, andrendering.


NVIDIA Corporation

Transforms: The Art of ChangeTransformation is the task of handling how ascene changes from one frame to the next. Eachframe changes just a little bit. Imagine a seriesof pictures of a horse running. Each picturecaptures slightly different positions of the legsmoving and the head bobbing. Play back thepictures fast enough, and you "see" the horsemoving. However, when creating the animationusing 3D graphics technology, the changes thattake place from one frame to the next mustactually be mathematically calculated. Thiscalculation process is known as transforms.

Some transforms are obvious. When our car inthe previous example moves along the road, it's areally obvious example of change. But there'smore than one type of transform. In our virtualkitchen, we may stick our virtual "head" out thewindow. In this case, the viewpoint is changing– the only object that's moving is the one wecan't see – but we can see the change that occurs.

Simply moving an object is one type oftransform...

Movement is known as translation.

Another way to look at it is to think of the zoomlens on a camcorder. When you zoom into ascene – kids playing in a sandbox – then thepicture in your viewfinder changes. When youzoom in, the child's face suddenly seems to growhuge.

Moving our point of view is another type oftransform.

Here, our virtual "camera" zooms in on partof the car.

If you were trying to simulate this on acomputer, each frame would show more andmore of the objects being zoomed into, butsurrounding objects would leave the scene. Soeach frame is different from the one before it.On top of that, you want this virtual object tolook bigger, just like the child's head in theviewfinder. (When an object size is changed, it’scalled scaling.)


NVIDIA Corporation

3D graphics transforms allow us to do thingsthat can't be done in real life...

Such as make a car much larger...

Or exercise our whimsy by making the firehydrant bigger. Changing an object's size iscalled scaling.

Another type of transform is rotation. If youtake a car and rotate it 90 degrees, that's atransform. It's obvious, then, that in a computeranimation, many changes are occurring from oneframe to the next, from objects moving, to ourviewpoint changing, to the size of objects

Rotation is another type of transform.

We can rotate through several different axes;these are calculated independently.

changing as they move toward or away from ourviewpoint.

It turns out that the mathematics needed tohandle transforms is relatively straightforward –but there's a lot of this type of calculation thatmust happen to create a scene. Remember, we'retrying to simulate a realistic environment on ourcomputer screen. In the real world, you canmove left, right, forward, backwards and, insome cases, up and down. When a ballet dancerpirouettes, she rotates, and parts of her body aremoving forward, backward, left and rightsimultaneously. If we were trying to simulate aballet dancer on the computer, we'd have to keeptrack of all those directions (called axes)simultaneously.


NVIDIA Corporation

We can combine all the different transforms –rotation, scaling, and translation – into onefluid scene.


NVIDIA Corporation

Let There Be LightThe task of creating the illusion of lights,whether from the sun, indoor lighting, spotlightsor other lights is the next step in our assemblyline. After all, you need light to be able to seewhat happens. Creating synthetic lights tobrighten up a computer image is really acomplex task involving an understanding of howlight is emitted from a light source – say a lightbulb – and spreads out over the area,illuminating various objects. Not surprisingly,this step is known as lighting.

Light is such a part of our existence, that theabsence of light fools the eye. The supportingcolumns are not really part of the wall.

Adding a single light source allows us to seethe columns as discrete objects.

If you look around you, you're likely to see anumber of light sources. The sun streams inthrough a window. There's a lamp on your desk.Your computer display glows with its own light.There are several fluorescent light tubes glowing.

On top of that, light is reflected from varioussurfaces in the room. Shiny surfaces, such asmirrors or brass doorknobs are obviously

reflecting light, but even dark or dull coloredobjects reflect some light.

Accurately simulating lights in a 3D graphicsprogram also takes a huge amount of math"under the hood." Because light bounces willy-nilly off hundreds of surfaces in a typical room,trying to keep track of the path of a light beam asit bounces, refracts into multiple beams andeventually becomes absorbed takes a massivenumber of complex calculations. If the graphicsprogram wants to create multiple light sources,the challenge becomes greater. For example, ifyou shine a flashlight on a brass doorknob, howmuch of the light you see is reflected from theflashlight, and how much from the fluorescentlights on the ceiling?

Adding a few realistic light sources makes ourwall look much more realistic.

Many computer games “cheat” when it comes tolighting. Instead of actually tracing the path oflight beams, games often use “light maps.”These are simply bitmaps – pictures stored onthe computer – that fake the eye into thinkingthere are lights present.

Look at it this way. If you stare at a photographof a brightly lit room, there are no real lightsshining – it's a picture. Brightly lit areas aresimply lighter colored. In a similar vein, lightmaps simply change the color of the object,making it appear as though a light is shining onit. Graphics programmers would like theirscenes to do a better job of using realisticlighting. But that takes some serious computerhorsepower – it takes a CPU 1-2 hours tocalculate one frame or a realistically animatedmovie scene.


NVIDIA Corporation

Render Unto CaesarIt's a SetupThe next step is taking the information createdand converting it to something the computerdisplay can begin to understand. It begins withbreaking each object into component parts. Most3D graphics programs create objects out of manysmall triangles. If you take our basketballexample, a basketball can be created with dozensor hundreds of small triangles. Each triangleconsists of edges and corners. The graphics cardis sent the information on these edges andcorners (the corner of a triangle – one point – iscalled a vertex). This data must be set up intopixels because the computer monitor doesn'tunderstand triangles. The process for going fromvertex information to pixel information is calledsetup and clipping. The part of the 3D graphicschip that handles this is the setup engine. (Youmay also see this called triangle setup andclipping.)

Each individual triangle is examined by the setupengine and broken up into individualcomponents suitable for the next step.

At this point, we've completed the task of settingup the geometry of the 3D environment toinformation suitable for the rendering engine.

The Rendering StepRendering is the process of displaying the imagewith the right visual characteristics (colors andshading) is known as rendering.

Computer screens are flat, 2D surfaces. Theydisplay images by using thousands of tiny dots(pixels) to create an image. Take a look at aphotograph from a newspaper. From a distance,it looks like a picture. If you hold it up to yournose, you'll see that the picture really consists ofmany small dots. If you get close enough tosquint at a color newspaper picture, you find outthat it's made up of many small, colored dots. Infact, that's exactly what a computer image is:many small dots that, from a distance, make up apicture. The greater the density of dots, thehigher in quality the resulting image will be.

Converting the geometry of a true 3D world toan image that can be displayed on our pixilatedcomputer screen is yet another seriouscomputational challenge.

At this point in our 3D pipeline, we have a lot ofinformation that has now been converted to scanline data for the computer monitor, along withsome information as to the brightness of thepixels. Now we need to take care of the actualcolors in the scene. It’s at this step that thecorrect colors are created for the screen and thedirection and intensity of the lighting isconverted to various intensities of color.

In addition, modern 3D graphics makes heavyuse of texture maps, which are simply picturesapplied to the object to make it look morerealistic. A picture stored in a computer istypically stored as a bitmap. A bitmap is simplya computer version of our newspaper picture. Ifyou've ever scanned a picture into a computer,you may realize that once in the system thepicture is now a bunch of pixels that can bechanged at will.

If we return to our basketball, a picture of a realbasketball may be used to make our rough objectlook more realistic. Another common exampleis a road surface. A rectangle has a bitmap ofgray asphalt with two yellow lines applied to it.

But it's not quite that simple. Remember, we'retrying to represent a real, 3D world on a flat, 2Dscreen.

If you look out your window again and see aroad, you'll see that the road tapers off in thedistance. The two yellow lines begin to blurtogether. You're witnessing the effect ofperspective. If our computer road was just arectangle, it wouldn't look at all realistic. So thecomputer tapers the rectangle, addingperspective to the image, to make it look morereal. Now we can apply our texture map(picture) of the road surface. But wait – the twoyellow lines seem to go on forever. We need toadd perspective to the texture map as well, so theroad looks correct. This is called,unsurprisingly, perspective correction.

Another step that occurs is filtering. Theproblem with texture maps is that they are fixedsizes, and consist of a fixed number of pixels.Let's say we apply a picture of a stone wall to alarge rectangle. Now we have something thatlooks like a wall. Now let's move our virtualhead so that it's six inches from the wall. Whatyou see are huge, square blobs. Each of theseblobs represents one pixel of the texture. Buteach square blob is actually made up of many


NVIDIA Corporation

screen pixels. If you move rapidly away fromthe wall, you may see the edge of the wall orparts of the texture "swim" in a distractingmanner. This is called aliasing.

A filter is simply a bit of math which, whenapplied to a bitmap, blends together colors ofadjacent pixels on the screen such a way as toremove the sharp edges created as we movecloser to the wall or other surface. Other typesof filtering take care of making the image lookgood as we mover further away. There are a hostof filtering techniques that come into play.

Here's our wall again. Even with somelighting, it looks, well, flat.

Adding some texture mapping and severalmore light sources makes it look better.

As we pull back, we can see the effect of moretexture mapping, as well as perspectivecorrection and filtering.


NVIDIA Corporation

3D for the Real WorldYesterday's HardwareDespite the rapid changes in 3D graphicshardware, today's traditional processors are stilllimited. To understand why, let's return to our3D graphics assembly line. If you recall, thefour stages are transform (what changes fromframe to frame), lighting, setup and clipping, andrendering. The problem is that today's traditional3D processors only handle those last two steps inthe 3D pipeline: setup and rendering.

Game developers – and other content creators –often think in terms of CPU budgets. In the erabefore today's GPU, up to 80% of the computer'shorsepower would be used for rendering. Sonaturally, rendering was one of the first tasks thatwas offloaded from the CPU to the graphicscard. Traditional 3D graphics processors simplycalculate the math necessary for setup andrendering.

As time went by, games became moredemanding. Users wanted higher resolutions,more colors, more detail and better performance.So the 3D chips got faster and added morerendering features. But today, the renderingengines are so fast that they are often waiting forthe CPU to complete its tasks.

As users demanded more realism and betteranimation, the next logical step was to buildhardware that could accelerate the transform andlighting – the tasks the CPU was struggling with– to create a complete GPU.

Today's HardwareIt was inevitable that the next generation ofgraphics accelerators – GPUs – would take onthe chore of transform and lighting. Themathematics is straightforward – the only cost isin designing the chip to take on greaterfunctionality. NVIDIA’s GeForce 256 GPU hasin excess of 22 million transistors. If youcontrast that with today's Pentium III processor,which has only 9 million transistors, you can seethat it is possible to create extremely powerfulprocessors capable of completely handling thegraphics processing with extreme speed.

Since the GeForce 256 GPU handles thetransform and lighting, the CPU can concentrateon other tasks.

The GeForce 256 offloads transform andlighting from the host CPU, allowing the CPUto process more game play elements.

In games, there's a lot left for the CPU to do.There's an insatiable demand for more realisticphysics in games. Flight simulators nowcalculate how the air flows over the wing, ratherthan using a discrete table of information on howa plane should behave. The number of objects inscenes is increasing as well. For example,Falcon 4.0 is a simulation of the Air Force's F-16jet fighter. In the game, an entire ground war issimulated in real time. But if you look closely atthe ground war, you'll see tanks passing througheach other and very little infantry. The task ofdeciding when a virtual object would bump intoanother one is called "collision detection." Butwith Falcon 4.0, which has some of the bestgraphics in a flight simulator, there just wasn'tenough CPU horsepower to perform collisiondetection on hundreds of vehicles, worry aboutthe air war and handle transform and lighting.

Another issue is artificial intelligence (AI).Despite the popularity of Internet gaming, mostgames are still played by one person, with theopponent chores handled by the PC. Program-ming a computer to be a competent opponent is atough challenge, but with more CPU timeavailable, better artificial opponents can becreated.


NVIDIA Corporation

With GeForce 256, the visual appearance ofreal-time, 3D computer graphics reaches anunprecedented level

Offloading graphics from the CPU to the GPUleaves more CPU cycles for physics, collisiondetection and AI, which makes for a richergaming experience.

Today's ApplicationsIn the past three years, the hardware in your PCthat handles the graphics chores went through arevolution. Three years ago, finding a graphicsadapter that could accelerate any part of ourfour-step 3D pipeline wasn't easy. Today, youcannot buy a computer that doesn't have someflavor of 3D graphics hardware already installed.

It was inevitable that game designers wouldadopt the new generations of 3D processors.Simulators such as flying and driving games, andaction games like Quake, all try to create avirtual world that allows the user to move aroundin it at will in any direction – unlike older-generation games.

However, other types of applications are takingadvantage of 3D.

Imagine, for example, real estate shopping on theweb. You'll be able to "walk through" housesthat are still being built in real time yet lookcomplete on your screen. You could look out thewindow and check out the view. You could evenget measurements of rooms to figure outfurniture arrangement.

Another possible application is businessvisualization. One of the hot trends in businessdata mining which involves huge repositories of

data. The traditional 2D spreadsheet – or even arelational database – can't present the data in aneasy-to-digest form. But the data can beconverted to a 3D format in order to visuallyrepresent potential relationships.

Future user interfaces may take advantage of 3Dhorsepower. Xerox and Silicon Graphicspioneered 3D user interfaces for exoticworkstations in the 1980’s, but that capabilitycan now reside on any user’s desktop. Even theWorld Wide Web can be presented as a "galaxy"of interrelated links. All of this is made possibleby the advances in 3D graphics hardware.

© 1999 NVIDIA CorporationNVIDIA, the NVIDIA logo, RIVA TNT, RIVA TNT2 and NVIDIAVanta are trademarks of NVIDIA Corporation. RIVA, RIVA128 and RIVA 128ZX are trademarks of NVIDIA Corporationand STMicroelectronics. Other company and productnames may be trademarks or registered trademarks of therespective companies with which they are associated.

3d graphics demystified - nvidiadeveloper.download.nvidia.com/assets/gamedev/docs/3d...3d graphics...

Documents