What can we Learn from Testing in the Games Industry?

EvolutionProblemsSolutions

Evolution

Where did we come from and…where are we heading to?

Where did all this start?

In September 1977 the first proper home console was released the Atari 2600

Not quite the first there was Pong and…

1954 A game on a scientific computer displayed on an oscillator display and

it was a simple tennis simulation

A bit earlier there was…

Over the years we moved from 8 bit to now 256 bit

At the beginning we had cartridges,Code to type in inside magazines,

audio tapes and Floppy disks.Then we moved on to CD’s/DVD’s

And ended up on Blue-Ray, Cartridge for Handhelds and Digital Downloads.

The games industry is a constantly changing and evolving beast and you need to keep up with the newest developments at any given moment,

with all the multitude of types of games, genres, hardware and business models this industry is one of the most complex and tightly knitted

community.The games changed from simple Jump'n'Runs to complex virtual worlds

where players are free to explore and interact which also change the development requirement radically.

The Industry has changed

At the beginning there were pretty linear titles like Pacman but also first glimpses of a bold future like

Pitfall which offered branched play and free exploring of the world, in this game Pitfall 2 it was a

page flipping the world.

How did gaming evolve?

Then the big growth of the games business happened with the Super Nintendo and Sega Mega Drive.

The sales of these consoles and titles drove a industry to boldly go beyond its know capacities and the title budgets and entertainment values grew beyond

anyone's dreams and then the switch to 32 bit and 3D occurred.This brought more open worlds which were free to explore albeit limited at the

beginning of the 3D games.Testing got a whole lot more complex too.

Then the big steps to conquer the movie industry happened, games became far more realistic with PS3 and Xbox 360 and the whole industry started to sell more than the movie industry per

year.The games became far more complex like GTA 5 a title where you can do whatever pleases you.The world reacts to what you decide and how you perform not anymore that you have to react

to a fixed world this was the big game changer along the visual candy.Testing got severely complex and it became impossible to test everything just by sheer volume

of content and this mixed with the sheer infinite possibilities what players could do a new testing approach dawned.

And then two revolutions occurred

Problems

How the gaming sector success became its own worst nightmare which needed to be conquered

At the beginning videogames where straight forward linear games, sometimes they had an extra room to find but that was the pinnacle of complexity. These projects

where similar to test like ATM’s where the customer has a similar linear experience.During the years the complexity rose and games became free roaming

Complexity rising

Complexity rising

A huge problem for testing is based on the free-roaming in games. Here you face the reality that you need to check every inch of the game against problems in 3D, which is in itself pretty improbable due to sheer

size.Additionally Free-Roaming introduced the problem that the player can choose what he wants to do in

whatever way it pleases him which turned testing these titles into a true nightmare for any QA professional.

Free-Roaming

Games are an art form, they are created by teams sometimes exceeding 400 people and due their creative nature and

complexity face lots of design changes when different elements of the games start to work together for the first time and do not

work out as intended this accounts for roughly 60%.Also Focus group testing enables us to re-align the project better

to the users taste and improve the chances of success.One factor which is attributing to at least 40% of all changes in a

project are Stakeholders like Marketing and Sales Departments at the Publishers or investors from sensible requests to could you

add an technical impossible part to a game or add a Barbie into a Medieval Fantasy game for 16+ males.

Stakeholders and Creativity

Solutions

Project Management changed in all areas of the games development.At the beginning all games were developed in a waterfall environment and had clear

separate phases, while today's projects are often started with a vertical slice to please investors and publishers then the content and more gameplay is added on top of those

this required a completely different approach.Several project management types are merged and mended to fit the needs of the project and teams which work on these titles that you don't have a clear agile or

waterfall you face anything in between with a huge focus on flexibility and adaptability.These two criteria's are also the QA’s biggest challenge, the games are so complex that

they aren’t fully testable in the old sense anymore new approaches had to be taken.Procedural games are a new type of game in which are every time different for each player, which makes this type of product not 100% testable only to a certain degree

which requires different testing strategiesToday’s testing is based on automation testing for everything which can automated

and human testing for localisation and gameplay, while the human force testing for the gameplay is focussed on the most probable gaming paths player might take not on all as some of the more modern projects would take centuries to test with thousands of

testers in the old way.

Localization got more complex as it is not only language but full internationalization is required to adhere to cultural sensitivities and law

regulations therefore companies you specialised QA companies who check for these problems during their language specific testing.

The sheer size of the modern games requires armies of testers to test them and this is only possible with flexible outsourcing, but this need to be integrated into

the whole strategy for the development from day one.The QA and its testing capacities, experience and strategies became a vital part of

the teams and project management a far step from its beginning as dungeon inhabitants.

Additionally Free 2 Play started to grow and this involves a different whole strategy in itself as the development creates only a base game and the interaction

of the players and the resulting data decides which direction the development heads as content is developed in accordance to players preferences due to the

business model relying on players buying extra content not the whole product like in the traditional model.

Thank you!

If you have question please do ask now!