W6  Internet  of  Things  5/4/16  11:30  

The  Internet  of  Things  in  Action:  Testing  Anki's  OVERDRIVE  Racing  Game  

Presented  by:  

Jane  Fraser  

Anki,  Inc.  

Brought  to  you  by:  

350  Corporate  Way,  Suite  400,  Orange  Park,  FL  32073    888-­‐-­‐-­‐268-­‐-­‐-­‐8770  ·∙·∙  904-­‐-­‐-­‐278-­‐-­‐-­‐0524  -­‐  info@techwell.com  -­‐  http://www.stareast.techwell.com/  

Jane  Fraser  Anki,  Inc.  

Jane  Fraser  has  managed  tests  groups  from  just  herself  to  a  team  of  120.  Her  focus  is  in  developing  testers  into  key  members  of  the  company  with  true  influence  on  the  product  and  process.  Jane  has  spent  the  last  20  years  on  a  span  of  projects  including  publishing  programs,  web/e-­‐commerce,  games,  big  data,  mobile,  e-­‐ticketing  and  robotics.  As  Test  Director  at  Anki,  a  robotics  and  artificial  intelligence  company,  she  is  managing  the  test  team  on  the  holiday  hit  robotic  battle  racing  game,  Anki  OVERDRIVE.  Jane  has  spoken  at  several  testing  conferences  and  loves  being  able  to  "play"  with  hi-­‐tech  toys  for  a  living.  Â  Connect  with  Jane  on  LinkedIn.  

THE INTERNET OF THINGS IN ACTION: ANKI’S OVERDRIVE RACING GAME

Jane Fraser, Test Director, Anki, Inc.

20+ YEARS TESTING, MANY IN MANAGEMENT, TESTING WEB, MOBILE APPS AND INFRASTRUCTURE, GAMES, SOCIAL, E-COMMERCE, PUBLISHING & CURRENTLY ROBOTICS & AI

COMPANIES: ANKI, EA, VODAFONE, JUSTARRIVE, BIGWORDS, COREL

CERTIFIED JOHN MAXWELL COACH, TRAINER & SPEAKER

ABOUT ME

INTERNET OF THINGS ADDS A “THING”

SEE OUR TOOLS USED TO TEST THE THINGS TEST METHODOLOGIES REMAIN THE SAME YOUR TOOLKIT NEEDS ADDITIONS YOU’LL NEED TEAMWORK & COOPERATION

EXAMPLES & PROCESSES TO EXPAND YOUR TESTING

SESSION GOALS

Wikipedia: The Internet of Things (IoT) is the network of physical objects, devices, vehicles, buildings and other items which are embedded with electronics, software, sensors, and network connectivity, which enables these objects to collect and exchange data.

IOT DEFINITION

FBI Warns Barbie Doll Could Be Used for Child Porn

Internet of Things security is so bad, there’s a search engine for sleeping kids

FTC Report on Internet of Things Urges Companies to Adopt Best Practices to Address Consumer Privacy and Security Risks

How the Internet of Things Got Hacked

Not my area of expertise, so sufficient to say, it’s important.

SECURITY

It’s not just software anymore, but it is still testing

• Addition of a network layer, to communicate

• Need to test the connection between hardware

• Early Prototype Hardware is not always reliable

• Partnering with both software and hardware engineering and manufacturing

• Employing test fixtures in addition to test cases

• Adding different test tools to your arsenal

TESTING A “THING”

• With hardware, firmware, and software difficulty increases

• SD card on the vehicle for more robust logging

• Video cameras

• Device Logs (logcat)

• Replay Logs - virtually replace the game

• Reference Devices

• Android devices typically use custom roms

WORKS ON MY MACHINE

• Damage from dropping

• Temperature & Humidity

• Sunlight

• Dust and Dirt

• Water

• Static Electricity

• Network interference

“THING” IS IN THE REAL WORLD

• This is a factory tool that both tests and assigns the correct firmware to the “thing”.

• IN TEST…

• This helps confirm we have good hardware

• Can change one car to another (very beneficial to test a new hardware type car) before we have hardware.

• Partnering with manufacturing and hardware teams

TEST FIXTURE

Box Test Error Codes

PASS,000,

EMPTY_COMMAND,001,

ACK1,002,

ACK2,003,

RECEIVE,004,

UNKNOWN_MODE,005,

OUT_OF_RANGE,006,

ALIGNMENT,007,

FIXTURE_ENTER_DTM,008,

FIXTURE_ENTER_TX,009,

FIXTURE_END_TEST,010,

POWER_CONTACTS,100,

ENABLE_CAMERA_2D,210,

ENABLE_CAMERA_1D,211,

CAMERA_2D,220,

CAMERA_1D_RAW,221,

LENS_HALORIFFIC,230,

ENABLE_MOTORS,300,

DRIVE_MOTORS,310,

READ_ENCODERS,320,

GEAR_GAP,321,

ENCODERS_NON_ZERO,330,

ENCODERS_ZERO,331,

LEFT_MOTOR_ZERO,332,

RIGHT_MOTOR_ZERO,333,

LEFT_MOTOR_NON_ZERO,334,

RIGHT_MOTOR_NON_ZERO,335,

LEFT_WHEEL_NEGATIVE,336,

LEFT_WHEEL_POSITIVE,337,

RIGHT_WHEEL_NEGATIVE,338,

RIGHT_WHEEL_POSITIVE,339,

NOT_STATIONARY,340,

WRITE_FACTORY_BLOCK,400,

SERIAL_EXISTS,401,

LOT_CODE,402,

INVALID_MODEL,403,

READ_FACTORY_BLOCK,410,

ALREADY_FLASHED,411,

GET_VERSION,412,

DIAGNOSING

Results from one of our test fixtures.

Bluetooth Low Energy is a power-conserving variant of Bluetooth personal area network (PAN) technology, designed for use by Internet-connected machines and appliances.

Uses frequency hopping wireless technology in the 2.4 GHz unlicensed radio band to interconnect nearby devices

Different devices have different chips.

BTLE

• 802.11, 802.11b, 802.11g, 802.11n, 802.11ac

• 5 GHz, 2.4 GHz

• WPA2-AES, WPA2-TKIP

• Security, encrypted or not

• Hot spots

There are lots of different types and settings with networks.

WI-FI

DEVICE FRAGMENTATION

Ability to change game characteristics on the fly

CONSOLE VARIABLES

TRACK DETECTION

CAMERA VIEW

This was really difficult to detect before printing,

as it was a mask layer that was too wide.

TRACK TESTING

MANUFACTURING TROUBLESHOOTINGWhen things go wrong its all hands on deck!

I’ll review an actual manufacturing “emergency”.

It all started with a supplier shipment that many parts didn’t meet our standard. This dropped our yield to 60%. Hardware, software and test needed to test and understand what we needed to do. Figure out how to adjust the firmware to account for the problem or stop production.

Nuke_Mod_150ohm_gain5_blackToCode

Led Brightness - current car vs dim issue

Nuke_Mod_150ohm_gain5_laps5_hist

Nuke_Mod_150ohm_gain5_laps5_scan

GRAPHING LOGS

“The important thing is not to stop questioning. Curiosity has its own

reason for existing.”

- Albert Einstein

QUESTIONS

THANK YOU

PLEASE FILL OUT AN EVALUATION FORM

If you have any questions please feel free to contact me. jane@anki.com www.linkedin.com/in/janefraser/