testing for fun · 2009-09-17 · testing for fun a small overview about quality assurance in video...
TRANSCRIPT
Testing for Fun
A small overview about quality
assurance in video games
About me
John-Philip Johansson
+ various projects
Games and gamers
Feelings
Organisation
Testing tasks
Lessons learned
Recap
Questions
Project flow
PrototypeStep
0
• Alfa
• Beta
• Final
Step 1
• Package
• Ship
Step 2
• Live
• Support
Step 3
Game development pipeline
Create Compile Test
Decide
Retry
ScrapPC to consoles
Games and gamers
Feelings
Organisation
Testing tasks
Lessons learned
Recap
Questions
”Every block of stone has a statue inside it
and it is the task of the sculptor to discover it.”
Michelangelo
By us, for us
”My only regret with Shadow Complex is I know
where everything is, so I don't get to play it.”
Donald Mustard
Creator of Shadow Complex(Making Shadow Complex, at Gamasutra.com)
Fans form communities
World of Warcraft
12 million subscribers150 SEK/month per subscription
The users are playing a game
User = Player = Gamer
Youtube “mirrors edge glitches” gives 1720 results
Why not play the same game?
Over 3.800.000 views on YouTube
Games and gamers
Feelings
Organisation
Testing tasks
Lessons learned
Recap
Questions
”When I design a game I imagine the
players’ face”
Shigeru Miyamoto
Creator of Mario Bros. and The Legend of Zelda(said during keynote speech, GDC 2007)
Let’s not make it this face...
Perly Nyström
@ FatShark AB(thanks for being a good sport!)
Bug types
Glitches Exploits Crashes
HangsTechnical
hangsMinor
Lag
Games and gamers
Feelings
Organisation
Testing tasks
Lessons learned
Recap
Questions
QA structure
Organisation
Embedded QA
Blackbox QA
Compliance
Community
Players Focus group Play group
QA Communication
Embedded QA
Blackbox QA
Feature cards
Programmers Artists Audio
Animation Designers Embedded QA
Classic team structure
AAA games, 50-200 people
Level Design Online AI
Core tech Movement Embedded QA
Scrum(ish) team structure
AAA games, 50-200 people
”A scrum team without a tester isn’t a team.”
Jeff Sutherland
Co-founder of Scrum(said at Scrummaster certification course, stockholm 2008)
Programmers Artists Audio
Classic team structure
Independent games, 5 people
Programmers Artists Audio
Wait, where’s QA?
Independent games, 5 people
Programmers Artists Audio
Everyone’s QA!
Independent games, 5 people
So we think it’s like this...
Create
Compile
Test
Feedback
...but it’s more like this.
Create
Compile
Test
Fix bugs
Test
Done / Screw it
Time’s up!
No time for feedback!
Sitting together
Games and gamers
Feelings
Organisation
Testing tasks
Lessons learned
Recap
Questions
Manually
Games are complex interactions
Game
Audio
Graphics
Input
Logic
Player in the center
The Truman Show
Spot the difference
”Something looks odd”
”This floor doesn’t sound right”
”I went here and got stuck”
”Tech hang”, yes the term confuses the coders
”Textures seems missing on
trees”
(actually, we just like white
trees)
”I just tried it on the console”
”I pulled the network cable and
pressed all the buttons”
’B’ is for bug
Everybody can test!
Games and gamers
Feelings
Organisation
Testing tasks
Lessons learned
Recap
Questions
Automated
Machines never sleep
Continous Integration
Boot up tests
Automatic build submissions
Framerate tests
Trigger tests
Stress test
AI test
Games and gamers
Feelings
Organisation
Testing tasks
Lessons learned
Recap
Questions
Heatmaps for game balancing
Halo 3Cold Storage multiplayer map kills
Progress Report
Not so good.
Memory Test
Worked really well.
Memory Fragmentation Test
Worked well, near end.
Full playthroughs
Worked well.
Blackbox QA
• Find bug
Blackbox QA
• Verify fix
Internal verification
Developer
• Fix bug
Embedd QA
• Verify fix
Worked really well.
Focustests
Worked a bit.
Playtest
Worked well.
Games and gamers
Feelings
Organisation
Testing tasks
Lessons learned
Recap
Questions
Recap!
Focus on user feelings
Testers in your teams
Test like your users
Observe the users
Automate, when you can
Games and gamers
Feelings
Organisation
Testing tasks
Lessons learned
Recap
Questions
