Copyright © Institut Lean France 2013

3 & 4 October, 2013 Paris, France

European Lean IT Summit 2013

Lean Software Factory Yves Caseau – Bouygues Telecom

Yves CASEAU – October 2013 – Lean Software Factory 2/24

Lean Software Factory –

Applying The Toyota Way to the continuous

crafting of embedded evolving software

October 4th, 2013 – v0.1

European Lean IT Summit

Yves Caseau, EVP New Products & Innovation

Bouygues Telecom

Yves CASEAU – October 2013 – Lean Software Factory 3/24

Outline

First Part:

Motivations for our “Bbox” Software Factory

Agile, Extreme Programming & Lean Software

Second Part:

Lean Software Factory (LSF) with Four Practices

Third Part:

2012 – 2013 : Lessons learned

Yves CASEAU – October 2013 – Lean Software Factory 4/24

Overall Corporate Goals - 2011 1

1. Efficiency

• Reduce development costs

• Less « rework », faster (functional) convergence

• Better Quality of Experience through better code

2. Agility

• Reduce TTM, remove lost time, less waiting

• Early stakeholders (Marketing) integration into development cycle

• Continuous innovation flow

3. Capitalize vs. Turnover

• Give to everyone an opportunity

to contribute

• Satisfaction derived from

individual excellence

• Pride (collective & corporate)

Part

1:

Moti

vati

ons

Yves CASEAU – October 2013 – Lean Software Factory 5/24

What defines a good software product ? Generic goals:

Modularity “degree to which a system’s components may be separated and recombined” (Wikipedia) … or the

capacity of the architecture to maximize in its decomposition the independence of subcomponents

Evolvability “the property of having many abilities”, that is software that can serve many purposes, together with “the

capacity of adaptive evolution”

Openness expose API, open source (scrutiny), platform

Specific to GW/STB products:

HW/SW interface : HW changes frequently + instable (protocols)

Embedded Linux

Legacy Assets (older “boxes”)

Our « modular middleware » ambition:

TTM / Lifecycle control

Modularity across HW

Open to external innovation

1 Part

1:

Moti

vati

ons

Yves CASEAU – October 2013 – Lean Software Factory 6/24

Five Years of Software Strategy (2011)

Part

1:

Moti

vati

ons 2010 2011 2012 2013 2014

Plan Build Deploy Improve Delight

11 Mai

• Partner

selection

• unified MW

Proof-of-

concept

• SW Factory

outline

• Continuous

improvement

• Factory starts

• Unified &

Modular

Midelware

• Bbox Sensation

development

• Continuous

improvement

• Bbox sensation

launch on June

18th (on time)

• Three separate

products

• New UI

• Cloud Gaming

• Beginning of SW

deployment on

legacy box

• QoE tuning

• Open API – First

Hackathons

• Open Innovation

• Jinni

• Ijenko

• Next Gen

Hardware

• Additional

Features

•Open Innovation

(followed)

LSF I LSF II

1 Part

1:

Moti

vati

ons

Yves CASEAU – October 2013 – Lean Software Factory 7/24

Software Factory

Automation Build

Test

Industrial Tools & Practices Configuration & Source version Management

TQM & Continuous integration

Value the development process as much as produced software Agility / Quality / Openness

code data

process processors

practices models

process stakeholders

users ops

dev

Information Technology Information Systems Software Factory

Continuous Deployment

Co

ntin

uo

us

Fe

ed

ba

ck

1 Part

1:

Moti

vati

ons

Yves CASEAU – October 2013 – Lean Software Factory 8/24

Agile, Scrum & Lean

1. Small teams

2. Small batches

3. Time boxing

4. Coevolution of

code/design/architecture

5. Role of face-to-face

communication

6. User stories

1. Test-driven

development

2. Sustainable pace

3. Code is valuable

1. Visual Management

2. Practices and Rites

3. Reflection

1. Kanban

2. Kaizen

3. 5S and waste

removal

Extr

em

e

Pro

gra

mm

ing

SC

RU

M Agile

Toyota

1 Part

1:

Moti

vati

ons

Yves CASEAU – October 2013 – Lean Software Factory 9/24

Motivations: Agile & Extreme Programming

1. Small teams

2. Small batches

3. Time boxing

4. Coevolution of

code/design/architecture

5. Role of face-to-face

communication :

6. User stories

1. Test-driven

development

2. Sustainable pace

3. Code is valuable

Extr

em

e

Pro

gra

mm

ing

Agile

Axioms:

- smaller pieces = less risk

- smaller pieces for faster adjustment

- Common rhythms to stay synchronized

Oldest, most durable problem

(silos) + 100+ people-projects

Axioms:

(1)Innovation → agility & iteration

(2)Agility → test velocity

End-to-end testing is difficult !

Still too much overload … and

exhaustion

To hurry => to err

Most dramatic change in our Taylor-

inspired large-scale organizations

Stakeholders

alignment

1 Part

1:

Moti

vati

ons

Yves CASEAU – October 2013 – Lean Software Factory 10/24

1. Visual Management

2. Practices and Rites

3. Reflection

1. Kanban

2. Kaizen

3. 5S and waste

removal

SC

RU

M

Toyota

Motivation: Scrum & Lean

Cf. Aristotle:

We are what we repeatedly do – Excellence, then, is not

an act but a habit

Efficient communication channel

- leverage body language

- avoid redundancy

- foster collaboration

The only way to do faster and better is to do

less

The main symptom of dysfunctioning was (2011) and

still is (2013) …waiting for one another

Team problem solving is the best training tool … to

understand the complexity of our own systems !

Axiom:

Hyper-competition & complexity ⇒

(excellence ⇒ continuous improvement)

1 Part

1:

Moti

vati

ons

Yves CASEAU – October 2013 – Lean Software Factory 11/24

Part II

First Part:

Motivations for our Bbox Software Factory

Agile, Extreme Programming & Lean Software

Second Part:

LSF with Four Practices

Third Part:

2012 – 2013 : Lessons learned

Yves CASEAU – October 2013 – Lean Software Factory 12/24

The Art of Lean Software Development

Practice 0: Source Code Management and

Scripted Build

Practice 1: Automated Testing

Practice 2: Continuous Integration

Practice 3 : Less Code

Prioritized requirements – YAGNI (You ain’t gonna need it )

BDUF (Big Design Up Front) – Avoid complexity

Reuse, coding standards, design patterns, …

Practice 4: Short Iterations

Practice 5: Customer participation

2 Part

II:

LSF P

racti

ces

Already deployed in

our IT software

development center

(Nantes)

Cf. SCRUM

« Customer Participation is a two-way street »

Cf. Architecture’s role …

Yves CASEAU – October 2013 – Lean Software Factory 13/24

LSF Practices (1) – Team Problem Solving

1. Visual description of the problem a drawing is worth a thousands words

2. Search for root causes, using

the « Five Whys » approach team work with all stakeholders

3. Build a collective action plan detailed: what will be done, by who, how and why …

4. Regular check of the plan application and its

consequences need for discipline & perseverance

5. Trace those four steps: A3-like document

Why ?

Since we work on systems, with long causal chaines and feedback loops.

Since we need to fix causes and not symptoms … To avoid repeating the same mistakes

All viewpoints are required to find the best solutions Buy-in and empowerment from all

Most often the problem vanishes or evolves by itself since environment conditions have changed

Cf. ITIL : problem ≠ incident

LSF start:

May 2012 2

Part

II:

LSF P

racti

ces

Yves CASEAU – October 2013 – Lean Software Factory 14/24

2 Part

II:

LSF P

racti

ces

LSF Practices (2) – Project Room

1. Visualize planning and milestones Rite: Sprint Planning Meeting

2. Visualize « workflow » (WBS) see the process and the succession of steps

• Multi-scale if needed

3. Visualize « issues »

• Display the problem-related A3

• List of most important incidents « GdM »

4. A place for collective memory, hence reflection

• SCRUM rite after each sprint

• Experience Return after each project

Why ?

To stay synchronized, to avoid waiting

Understand « the big picture » and facilitate transitions

All viewpoints are required to find the best solutions Buy-in and empowerment from all

Learning and continuous improvement

LSF start:

May 2012

Yves CASEAU – October 2013 – Lean Software Factory 15/24

LSF Practices (3) – Reduce WIP

1. Visualize all tasks assigned to teams i.e.: to place post-its into cells

2. Reduce WIP (Work in Process) Add constraints to the work load

• No more than X post-its per cell

3. JIT management (Pull flows)

• In an ideal project setting, with an optimal scheduling,

there is no difference between push & pull

• « push » happens when the end of step N activity signals

the beginning of step N+1 activity (most often with some

delay compared to the optimal schedule) The more optimized the schedule, the more delays are amplified

• « pull » means that step N – 1 production is governed

by the capacity of step N, which is more robust

(for instance, design vs. development)

• Each post-it is a signal (Kanban)

•

Avoir overload, multi-tasking and stockpiling « work waiting to be handled »

Avoid « useless work » (muda) • waiting code • unused features • minimize reponsability handovers • avoid setup times necessary to switch from one type of activity to another

Only works with small batches

2 Part

II:

LSF P

racti

ces

LSF start:

May 2012

Yves CASEAU – October 2013 – Lean Software Factory 16/24

LSF Practices (4) – Love Your Code

1. Sort, organize and structure code (cf. Practice 0) Lean : 5S (Sort, Systematize, Shine, Standardize, Sustain)

2. Discipline (coding styles & rules) Define guidelines and automate checking

3. Code reviews & « elegant programming »

Display you code with pride to your colleagues

4. Less code (cf. Hibb)

• Remove what is no longer in use

• Avoid what will not be used much

5. « Gardening » (code refactoring)

• code is alive (life recycle )

• Iterative process leads to accumulation

Work better, more efficiently Collaboration & maintenance

Cf. « 5S with Java » - p. 192 from Poppendieck

Avoir accumulation, since it generates costs & complexity

Code quality and Experience quality are linked to one another Collaboration & Capitalization

Future is uncertain, hence favor agility over expressiveness

Cf. Google : 50% of code changes every month

LSF start:

May 2012 Part

II:

LSF P

racti

ces

2

Yves CASEAU – October 2013 – Lean Software Factory 17/24

LSF Deployment

Factory Code management tools (IBM RTC) – 100% useful, even if it generates lots of debates

Automated SW integration test (wall of boxes)

What worked better/faster than expected: Standups meetings (reduce email)

User stories (key to facilitate product owner’s role)

Visual Management (related to SCRUM)

What worked more slowly than expected: “All hands on deck” , Pull versus push (still too much waiting)

Unit testing (test-driven development)

What is hard: Reflection & Slowing down

Large-scale orchestration

Part

II:

LSF P

racti

ces

2

Yves CASEAU – October 2013 – Lean Software Factory 18/24

Part III

First Part:

Motivations for our Bbox Software Factory

Agile, Extreme Programming & Lean Software

Second Part:

LSF with Four Practices

Third Part:

2012 – 2013 : Lessons learned

Yves CASEAU – October 2013 – Lean Software Factory 19/24

Agile versus « V » Development Cycles

Not everything is « agile » … Clear & stable requirements, homogenerous code (e.g., technical patterns), Service Platform

integration ⇒ V cycle (strength of engineering, design methods, abstraction and anticipation)

Unclear / Unstable requirements ⇒ agile

Hardware projects (costly integration) ⇒ engineering + agile prototypes

Part

III:

Less

ons

learn

ed

3

System

Integration

System

Engineering

Building new “heavy & stable” components

Coordination

• … but almost all activities benefit from a « lean » approach.

Cf. key ideas from Toyota Product Design:

• Set-based design – parallel exploration, delay design choices with the most consequences

• Tight flow (no place for problems to hide )

Lean Factory:

continous integration

& deployment

Evolving existing components /

Build new (light) ones

Iterative design

Time-boxing

Small teams

Project Room

(time & location)

Etc.

Ménadier’s “W cycle”

Yves CASEAU – October 2013 – Lean Software Factory 20/24

Lean & Architecture (1)

Agile or lean software development does not prevent from building complex

products

Complexity requires sense & common vision

Architecture is about communication & story telling

Architects are one of the backlog stakeholders

In a continuous & incremental development process, one needs

an architectural framework to prevent from diverging

Refactoring is not enough (too much rework)

Architect must work in “pull mode” – they are here to assist

developers (not the other way around !)

A key Toyota principle is to share a systemic vision between all

process/product actors

Visual management → well-crafted artifacts

Design reviews are critical

Part

III:

Less

ons

learn

ed

3

Yves CASEAU – October 2013 – Lean Software Factory 21/24

Lean Architecture (2)

Lean ≠ Agile, truly complement each other

Agile: well suited to change, focus on present,

delay decisions to match environment

Lean : well suited to complex/ large-scale,

focus on future & long-term, anticipation is welcome

Architects are needed to reconcile

« small scale » & « large scale » visions

Building a « platform approach »

Architecture is the « grammar for cooperation »

Conway’s law: (Architecture ⇔ organisation)

Coplien & Bjørnvig:

You cannot always refactor your way

to a better architecture. The essence of Lean Architecture is to take careful,

well-considered analysis and distill it into APIs

Remember that architecture is mainly about people

Part

III:

Less

ons

learn

ed

3

Yves CASEAU – October 2013 – Lean Software Factory 22/24

LSF Gouvernance

LSF Steering Committee

Every two months, to re-align vision & goals

A moment to dissent & disagree (makes change leader stronger)

Educate managers about « change management »

LSF Support System

Tools

Engineering, Architecture

Methods et Project Management

Scrums Masters community

2.0 (share & exchange) practice to capitalize

Learn from other SCRUM-practicing divisions (IT, digital)

Get inspiration from agile communities in other companies

Values : 10 self-inspired principles

Steering Committee

Support System

Community

Part

III:

Less

ons

learn

ed

3

1. Customer Focused

2. Self-empowered teams »

3. Time Boxing

4. Less is more

5. Software Pride

6. Dare to innovate

7. Humble listeners

8. Continuous story tellers

9. Knowledge is worth sharing »

10. « Respect pleasure »

Yves CASEAU – October 2013 – Lean Software Factory 23/24

Gemba Walks Twice a week, one hour per visit

On-going, learning process

Build a rite = follow the same « script » each time

1. Show me what you do

show me your code / your product / your demo / your design

individual or group

2. Tell me why your are doing it this way.

This is the opportunity to share the vision

3. What are your current problems ? Who are your stakeholders ?

Seven tips for a healthy ‘Gemba Walk’ / MBWA

Visit everyone

Go alone – Daily standup meetings aren’t enough

Don’t bypass middle management e.g. don’t change priorities, requirements or design

Observe, ask and LISTEN

Be genuine, have fun and strive to catch your engineers doing something right and not something wrong (you

are not the “fun-police” ;)

Share your dreams and vision

Don’t “disturb” the Gemba – Timing is everything…

Karmona Pragmatic Blog Part

III:

Less

ons

learn

ed

3 I should have listened to

Michael Ballé and started

sooner

Yves CASEAU – October 2013 – Lean Software Factory 24/24

Reflection : A Moment of Truth

Lauched Bbox sensation on time

Approximately as many bugs as similar products,

longer than expected to stabilize

Sucess is mostly a matter of skills !

The good news is that we learned a lot, the bad news is that we did not know enough

A key methodological difficulty is that it is still hard to forecast how long it will take to

solve a problem

→ Stakeholder solidarity is still crucial

Part

III:

Less

ons

learn

ed

3

Too long to reach

desired stability

SW/HW coupling Skills to detect earlier & better

understand

Lack of experience + pressure

from stakeholders

Pressure → breaks the “small

batches” principle

Poor delivery time

forecasting

Slow delivery of

improvements SW mastery

Yves CASEAU – October 2013 – Lean Software Factory 25/24

Conclusion

• Consumer Electronics Products require extremely short

development cycles

• Focus on skills, what matters in the end

• SW production/delivery automation is a must for embedded

SW products

LSF works

• Lean (Toyota-style) maximizes motivation

Learning is a satisfaction growth engine

• (Toyota Kata) Practices ! Learn by doing …

Values are everything

• The way you build is as important as what you build

• … SW is a “live object”, constantly evolving

→ lean is a must, Taylorism does not work

• “best-of-breed integration” of agile SW practices

LSF : a « new » vision

about software

products

Copyright © Institut Lean France 2013

3 & 4 October, 2013 Paris, France

More Lean IT presentations and videos on www.lean-it-summit.com