©2013 DISNEY

SysAdmins Unleashed!Jon Geibel | Director of Systems

Ron Johnson | Technology Manager

Walt Disney Animation Studios

1

©2013 DISNEY

Founded, October 1923

First animated feature, Snow White and the Seven Dwarfs, 1937

Frozen - 53rd Animated Feature

800 artists & engineers in one vertically integrated studio

2

©2013 DISNEY

30,000 Core Renderfarm60M Render Hours on Frozen1.5MW Data Center6PB Raw Storage1000 Linux Workstations800 Macs

Our scale...

3

©2013 DISNEY

5,300

10,00030,000

2013

20122011

Scale is increasing...

4

©2013 DISNEY

50-person Systems Team5

©2013 DISNEY

21 Years Experience

50% Engineering, 50% Management

Engineers building an Engineering Organization

Use Scientific Method

Survived many top-down re-orgs

Who we are...

6

©2013 DISNEY

DIRECTOR OF SYSTEMS

LINUX

MANAGER

CORE SERVICES

NETWORK

DATA CENTER

RENDERFARM

MANAGER

STORAGE

DATABASE

TECH SUPPORT

MANAGER

PROCUREMENT

MAC OS

WINDOWS

BACKUPS

MANAGER

ARCHIVING

SYSTEMS OPERATIONS

AUDIO VISUAL

MANAGER

POST PRODUCTION

Our old traditional model...

7

©2013 DISNEY

DIRECTOR OF SYSTEMS

LINUX

MANAGER

CORE SERVICES

NETWORK

DATA CENTER

RENDERFARM

MANAGER

STORAGE

DATABASE

TECH SUPPORT

MANAGER

PROCUREMENT

MAC OS

WINDOWS

BACKUPS

MANAGER

ARCHIVING

SYSTEMS OPERATIONS

AUDIO VISUAL

MANAGER

POST PRODUCTION

Resource ContentionPigeonholing

Difficult to evolveIndirect communicationCareer path limitations

No empowerment

Problems...

8

©2013 DISNEY

LINUX

CORE SERVICES

NETWORK

DATA CENTER

RENDERFARM

STORAGE

DATABASE

TECH SUPPORT

PROCUREMENT

MAC OS

WINDOWS

BACKUPS

ARCHIVING

SYSTEMS OPERATIONS

AUDIO VISUAL

POST PRODUCTION

Let’s move the managers out of the way, and remove silos...

9

©2013 DISNEY

Small Autonomous Teams2-6 PeopleResponsible for R&D, Engineering & SupportAct as small Start-upsResults Oriented Focus

10

©2013 DISNEY

Co-locatedEmbedded warroomAvoid formal meetings

11

©2013 DISNEY

Team Roles

Team Lead

Primary Members Secondary Members

12

©2013 DISNEY

Technical expert in area

Can have any titleProvides Tactical & Strategic leadership

Works directly with stakeholders

Lead Role

13

©2013 DISNEY

Lead RoleLead works with managers to determine equipment, budget, & headcount

14

©2013 DISNEY

Primary Role

Data Center Team

Dedicated to a team

Everyone has a Primary Role on one of the teams

15

Secondary RoleSecondary Role Primary Role

©2013 DISNEY

Linux Team Storage Team

Data Center Team

Secondary Roles add technical and soft skills to a teamNo talent left behind

Playing Multiple Roles...

16

©2013 DISNEY

So... Where did all of the Managers go??

17

©2013 DISNEY

Same structure as other teamsManagers work as a team

18

©2013 DISNEY

LINUX

MANAGER

CORE SERVICES

NETWORK

DATA CENTERXManagers no longer “own” functional areas; that’s the job of the team leads

19

©2013 DISNEY

Career and Team Coaching Team Investment Strategies

Managers work as a team and provide two major functions

20

©2013 DISNEY

Career & Team Coaching

For those that like sports analogies: Managers act more like coaches on sidelinePeople can pick their own Manager based on skill set and personalityTeams can have members that report to different Managers

21

©2013 DISNEY

Fixes A Couple of Traditional Problems

When you change technical roles you no longer have to change Managers

You can change managers without changing what you do

22

©2013 DISNEY

Investment Strategies

‣ Decides on where to invest‣ Build new teams as technology evolves‣ Select new Team Leads based on who has

best ideas and vision‣ Provide resources (people + capital) for

Team Lead‣ Helps define team goals that align with

Studio direction

23

©2013 DISNEY

This is how we look today

LINUXCORE SERVICESNETWORKDATA CENTERSTORAGEDATABASESSYSTEMS

MANAGEMENTPROCUREMENTBACKUPSARCHIVINGTECH SUPPORTRENDERFARM

SYSTEMS OPERATIONS

MAC OS MEDIA MOBILE WINDOWS

No Hierarchy

Tech Leads are top of Food Chain

24

©2013 DISNEY

NEXT EXPERIMENT:

Breaking down the Physical Walls

25

©2013 DISNEY

Thank you.

26