the fab experience - pnsqc · the fab experience how i stopped ... /fix control measure identify...

The Fab ExperienceHow I stopped whining and started to appreciate Process

Presented by: Michael Stahl, SW Validation Architect, PerCDeveloped with: Ron Moussafi, SW Engineering manager, CSG

Oct, 2014

© Michael Stahl & Ron Moussafi, 2014, All Rights Reserved

Agenda

• A little bit about Chip Manufacturing

• Fab Quality Management Processes

• Chip Manufacturing VS SW Development

• Opportunities


Disclaimers

Names and brands referenced herein may be claimed as the property of third parties

The views expressed in this presentation are solely my own, and do not in any manner represent the views of my employer

Information in this presentation is provided “AS IS” without any warranties or representations of any kind


Chip Manufacturing

Sand

500 chips

1.5 Billion

Transistors

22nm

dimension

http://www.google.co.il/url?sa=i&rct=j&q=&esrc=s&frm=1&source=images&cd=&cad=rja&docid=qAB3ouLHEhsufM&tbnid=PJhSEZJVZqSbeM:&ved=0CAUQjRw&url=http://jennaolivia94.blogspot.com/2012/02/arrow.html&ei=rmQ7UpihHYrE0QXMj4DABA&bvm=bv.52434380,d.bGE&psig=AFQjCNFghFDyVgzUe3JNjTtsaPx9_ogMuw&ust=1379710428490361









• $5B Capital

• 20000 sqm Clean room

• 500 Machines

• 400 Manufacturing steps

• 50,000 Wafers per month

Chip Manufacturing


What can go wrong?

At 22nm?

EVERYTHING!

Human Hair diameter

= ~ 0.1 mm = 100 micron

= ~4500 Transistors !!!

../common files/African Bull Frog ant crusher.wmv

../common files/African Bull Frog ant crusher.wmv


What can go wrong?


Excursion =

$1,000,000s


Quality is #1 Priority in High Volume

Semiconductor Manufacturing

Quality must be

controlled, standardized, managed!

http://www.google.com/url?sa=i&source=images&cd=&cad=rja&docid=LMNIHLCh9wcieM&tbnid=0pcfR8v5wS7GBM:&ved=0CAgQjRwwAA&url=http://4youtechnology.com/clean-room-technology-then-and-now/&ei=I9o-UrWUHoGriALhpIHoCQ&psig=AFQjCNF6siBGCFXN8GfWoT5qCktjMqIVeQ&ust=1379937187544912

http://www.google.com/url?sa=i&source=images&cd=&cad=rja&docid=LMNIHLCh9wcieM&tbnid=0pcfR8v5wS7GBM:&ved=0CAgQjRwwAA&url=http://4youtechnology.com/clean-room-technology-then-and-now/&ei=I9o-UrWUHoGriALhpIHoCQ&psig=AFQjCNF6siBGCFXN8GfWoT5qCktjMqIVeQ&ust=1379937187544912

http://www.google.com/url?sa=i&source=images&cd=&cad=rja&docid=fx3xMkBoe820iM&tbnid=gamOBjzVr8qdsM:&ved=0CAgQjRwwAA&url=http://english.semi.cas.cn/st/rd/200804/t20080404_26098.html&ei=Kdo-UrrBJIiEiwLi7IHAAg&psig=AFQjCNG6dknu5ddvSx4se5iveuYGgYtHeA&ust=1379937193661933

http://www.google.com/url?sa=i&source=images&cd=&cad=rja&docid=fx3xMkBoe820iM&tbnid=gamOBjzVr8qdsM:&ved=0CAgQjRwwAA&url=http://english.semi.cas.cn/st/rd/200804/t20080404_26098.html&ei=Kdo-UrrBJIiEiwLi7IHAAg&psig=AFQjCNG6dknu5ddvSx4se5iveuYGgYtHeA&ust=1379937193661933

http://www.google.com/url?sa=i&source=images&cd=&cad=rja&docid=whRQ0Hq_vP9VhM&tbnid=m-lMrewpATY7WM:&ved=0CAgQjRwwAA&url=http://newscenter.lbl.gov/news-releases/2009/11/18/fabs21-benchmarking-tool/&ei=L9o-Uo7rNuLtiwKNr4DYDA&psig=AFQjCNGFJ-t45QbEfvWpyjxxD812LcAi3A&ust=1379937199962198

http://www.google.com/url?sa=i&source=images&cd=&cad=rja&docid=whRQ0Hq_vP9VhM&tbnid=m-lMrewpATY7WM:&ved=0CAgQjRwwAA&url=http://newscenter.lbl.gov/news-releases/2009/11/18/fabs21-benchmarking-tool/&ei=L9o-Uo7rNuLtiwKNr4DYDA&psig=AFQjCNGFJ-t45QbEfvWpyjxxD812LcAi3A&ust=1379937199962198

http://www.google.com/url?sa=i&source=images&cd=&cad=rja&docid=kpd15Qxe-TcTGM&tbnid=1GZds0-ixW0CGM:&ved=0CAgQjRwwAA&url=http://www.theguardian.com/technology/2009/sep/16/apple-ram-dram-64-bit&ei=KNo-UpTpGJDOigL3y4DAAw&psig=AFQjCNGOdI3Nnz-rOW14HrAgf_j3WZr5ig&ust=1379937192457999

http://www.google.com/url?sa=i&source=images&cd=&cad=rja&docid=kpd15Qxe-TcTGM&tbnid=1GZds0-ixW0CGM:&ved=0CAgQjRwwAA&url=http://www.theguardian.com/technology/2009/sep/16/apple-ram-dram-64-bit&ei=KNo-UpTpGJDOigL3y4DAAw&psig=AFQjCNGOdI3Nnz-rOW14HrAgf_j3WZr5ig&ust=1379937192457999


The Science and Art of

Manufacturing Quality Management

Design /Fix

Control

Measure

Identify Excursion

Contain

Disposition

Root Cause

Analysis Factory Level Focus

Lean Methodology

Change Control

Ownership

Automation

Empowerment


Design /Fix

Control

Measure

Identify Excursion

Contain

Disposition

Root Cause

Analysis Factory Level Focus

Lean Methodology

Change Control

Ownership

Automation

Empowerment

The Science and Art of


Process characterizationSpecificationsStatistical Process ControlAdvanced Process Control

TrainingChange Management ProcessAutomationInspectionsControl limits

8D/7-StepsData Driven paretoInspectionFailure AnalysisFailure Pattern defined“If it failed, there is a cause, someone owns it and it can be fixed” mentality

Risk Based AnalysisSign offMulti disciplinary team

Incoming materialWork in Process at each stepCorrelation to Process, EquipmentTrends and SPCTrack Quality results indicatorsEach individual/team has Quality goals and measurements

Risk level for each outlierIdentification of Systematic problems vs. random variationsManagement attention

Clear procedures and processGoal is to minimize effect, fastest recovery and minimal risk Clear management R&RMulti-Disciplinary Task Force


This is fine for the Fab…

… but in R&D we must

Innovate!


http://www.google.co.il/url?sa=i&rct=j&q=&esrc=s&frm=1&source=images&cd=&cad=rja&docid=Wo-VR99tC2wayM&tbnid=gxvfk2DKftVDlM:&ved=0CAUQjRw&url=http://www.tandexlabs.com/trillium.html&ei=JKlLUoX4LpD30gWV1YDgDQ&bvm=bv.53371865,d.d2k&psig=AFQjCNG3TwehlwaGGfJ82ayBORX7e3rZGg&ust=1380776603137221

http://www.google.co.il/url?sa=i&rct=j&q=&esrc=s&frm=1&source=images&cd=&cad=rja&docid=Wo-VR99tC2wayM&tbnid=gxvfk2DKftVDlM:&ved=0CAUQjRw&url=http://www.tandexlabs.com/trillium.html&ei=JKlLUoX4LpD30gWV1YDgDQ&bvm=bv.53371865,d.d2k&psig=AFQjCNG3TwehlwaGGfJ82ayBORX7e3rZGg&ust=1380776603137221


Quality Processes

DO NOT

block innovation!


And now ForSomething completely

different…

http://www.google.com/url?sa=i&rct=j&q=&esrc=s&frm=1&source=images&cd=&cad=rja&docid=ahHMCEBl_uQZmM&tbnid=m3119BEbTM2yXM:&ved=0CAUQjRw&url=http://en.wikipedia.org/wiki/Monty_Python&ei=dIZCUrexKKuK1AW_wYHQBQ&bvm=bv.53077864,d.d2k&psig=AFQjCNEqJopalnb7eQaV46Hn8G16cNFAzA&ust=1380177795141039

http://www.google.com/url?sa=i&rct=j&q=&esrc=s&frm=1&source=images&cd=&cad=rja&docid=ahHMCEBl_uQZmM&tbnid=m3119BEbTM2yXM:&ved=0CAUQjRw&url=http://en.wikipedia.org/wiki/Monty_Python&ei=dIZCUrexKKuK1AW_wYHQBQ&bvm=bv.53077864,d.d2k&psig=AFQjCNEqJopalnb7eQaV46Hn8G16cNFAzA&ust=1380177795141039


Software Quality


CMM – Capability Maturity Matrix


Manufacturing quality processes

CLEARLY MEET

CMM level 5.


Manufacturing Quality vs. CMM5

Managed (L2)

Defined (L3)

Q. Managed (L4)

Optimized (L5)


CMM5 L2: Managed

• Process follows Policy • Practices defined and Managed • Product meet Standards

(project level)• Process monitored, controlled &

reviewed• Risk management is mandatory • SW practices: CM, QA,

contractors management, project tracking and management, project planning, requirements management.

• Existing practices are retained in time of stress

Manufacturing Quality vs. CMM5: L2


Specs, Training, Quality Policy enforced Managed reviews, meetings, procedures Defined Spec, Controls. Measurements taken, Automated,

monitored. Well defined Risk Management

procedures. Equivalent: Product data, Quality in-line

and end of line, records for every step maintained, Specifications defined and measured


Manufacturing quality processes

CLEARLY MEET

CMM level 5.

Trust me on this one.


SW Development has Quality Processes equivalent to Manufacturing

Process targets

Process flow

Machine recipe

Change Control Board

In-line tests & monitors

Recipe control

Factory automation

End of Line Tests

Test sample plan

Root-cause analysis

PRD, Requirements - KPI

Architecture specs

Design documents

Reviews, RCR

Unit tests, Static analysis

Configuration management

Build automation & CI

Test and Test documents

Risk Based Analysis

Root-cause analysis

Software DevelopmentChip Manufacturing

Are SW quality processes inadequate?


With all these quality

processes available…

How come

almost no SW shop

is at CMM5?!!!


Why do Software developers

find it so hard…

…to do what Process

Engineers think of as

“way of life”?


Fab

Cost of Error

Software


Fab:• $ millions lost in the line

• $ millions of lost revenue or recall costs

• Months to recover

Software:• Hard to quantify cost of bad requirements, design

• Bug-fix timeline is short – small pain

• Hidden costs of bugs

• Death by 1000 paper-cuts

Cost of Error


SoftwareFab

Accountability; Ownership


Fab: • Root-cause analysis culture: Every Problem has an Owner

• Quality is owned by … Everybody! (QA owns the process)

• Risk of personal consequences

Software: • Design documents: “fire and forget”

• Development “expected” to create bugs

• Test expected to catch them…

• Quality ownership is QA. Escapes are Test fault



Fab

Culture

Software


Culture

Fab:• Quality and Compliance are Values

• “Zero tolerance” to non-compliance

• Enforcement support systems

• Everyone does it (לא רק הפריירים)

Software:• Flexibility is a Value

• Process seen as “stifling innovation”

• Management tolerates non-compliance

• Strict process is NOT a must for having a product

• The “Yes-but” phenomena


Fab

World view

Software


Holistic view of the Process

Fab:• The mission is Production

• Deep understanding that steps influence each other

• The end product is present on a daily basis

• Short feedback cycle

Software: • The mission is Research & Development

• Decoupling is a value

• (Some) Developers don’t “see” the end product

• Long feedback cycle


Fab

Engineering self image

Software


Recap

Fab Process Software

Cost of Error: Very clear Hidden; Distributed

Ownership: Very clear Unclear; Delegated

Compliance: A Value Not a value

Process view: Holistic Decoupled

Self Image: Scientist The Lone Ranger


East is East, and West is West,

And never the twain shall meet…

The Ballad of East and West, Rudyard Kipling


Or maybe…

West

East










What can we (software)

learn from the Fab world?


Cost of Error

• Devise methods to calculate the cost of bugs

• Track, publish the cost

• Associate cost to bug-injection point

• Make it visible & immediate

Who wants to “own”

a $100,000 bug?


Gerard J. Holzmann

• American computer scientist, best

known as the developer of the SPIN

model checker.

• Leads the NASA JPL Laboratory for

Reliable Software; a JPL fellow.

• Awarded the NASA Exceptional

Engineering Achievement Medal.

• Writer of several books


Cost of Error

Gerard Holzmann - https://www.usenix.org/conference/hotdep12/tbd

An example of creating a direct link between mistakes and costs (min. 17:45 in the video)

https://www.usenix.org/conference/hotdep12/tbd

00133 - Gerard Holzmann - SW quality pracices for the MSL - NASA - JPL 00_17_46-00_18_54 - Cost.wmv

00133 - Gerard Holzmann - SW quality pracices for the MSL - NASA - JPL 00_17_46-00_18_54 - Cost.wmv



• Everybody owns Quality, not just QA.

• Track documents’ delivery, accuracy (bugs!), current-ness

• Per module quality metrics

• Escapes analysis focus: – Why it happened

– Not why Test missed it

Bugs are Excursions !




An example how per-module, public quality metrics help achieve better quality (min. 34:25 in the video)


00133 - Gerard Holzmann - SW quality pracices for the MSL - NASA - JPL 00_34_26-00_35_16 - Accountability.wmv

00133 - Gerard Holzmann - SW quality pracices for the MSL - NASA - JPL 00_34_26-00_35_16 - Accountability.wmv


Culture

• “Zero tolerance” to non-compliance

• Everyone does it (not just the nerds)

• Enforcement support systems

• Reward high quality– Fire Fighting is good

– No fire – Better

Once started…

… it’s easier to maintain


Culture


An example of imposing “everyone has to do it” attitude to agreed-upon processes (min. 23:40 in the video).


00133 - Gerard Holzmann - SW quality pracices for the MSL - NASA - JPL 00_23_42-00_24_40 - Culture.wmv

00133 - Gerard Holzmann - SW quality pracices for the MSL - NASA - JPL 00_23_42-00_24_40 - Culture.wmv


Enforcement


Enforcement

http://www.google.com/url?sa=i&rct=j&q=&esrc=s&frm=1&source=images&cd=&cad=rja&docid=W7EvOrtvpNxqOM&tbnid=LAFScmil4zz06M:&ved=0CAUQjRw&url=http://en.wikipedia.org/wiki/Texting_while_driving&ei=4MBLUqC7IOWK0AWQ3YH4BA&bvm=bv.53371865,d.ZGU&psig=AFQjCNGCTLT-AxCWZ0KazNGbL-_RmyL6LQ&ust=1380782139996562

http://www.google.com/url?sa=i&rct=j&q=&esrc=s&frm=1&source=images&cd=&cad=rja&docid=W7EvOrtvpNxqOM&tbnid=LAFScmil4zz06M:&ved=0CAUQjRw&url=http://en.wikipedia.org/wiki/Texting_while_driving&ei=4MBLUqC7IOWK0AWQ3YH4BA&bvm=bv.53371865,d.ZGU&psig=AFQjCNGCTLT-AxCWZ0KazNGbL-_RmyL6LQ&ust=1380782139996562


Holistic view of the Process

• Think “Software Production”• Keep the product flexible; Keep the process repeatable

• Fix Root Causes

• While (1) {

Control();

Measure();

Improve();}


Summary

Some aspects of Fab quality management are

relevant to Software.

Adopting any or all will improve software quality:

• Know the cost of non-compliance

• Set and enforce ownership expectations

• Build a Quality Culture

• Think “Software Production”


Closing Thoughts

Process adherence removes daily annoyances


Process adherence protects from silly mistakes

Closing Thoughts


Closing Thoughts

Process adherence allows fast changes

http://www.google.com/url?sa=i&rct=j&q=&esrc=s&frm=1&source=images&cd=&cad=rja&docid=qZT6L0q8i-GT5M&tbnid=gONoD85TlnGxzM:&ved=0CAUQjRw&url=http://www.geograph.org.uk/photo/2274688&ei=_z9lUuzrEsyV0QWI6IHACA&bvm=bv.54934254,d.bGE&psig=AFQjCNH3y2O7Vk8OCC4B3_plpXRa7_AyyA&ust=1382453572561826

http://www.google.com/url?sa=i&rct=j&q=&esrc=s&frm=1&source=images&cd=&cad=rja&docid=qZT6L0q8i-GT5M&tbnid=gONoD85TlnGxzM:&ved=0CAUQjRw&url=http://www.geograph.org.uk/photo/2274688&ei=_z9lUuzrEsyV0QWI6IHACA&bvm=bv.54934254,d.bGE&psig=AFQjCNH3y2O7Vk8OCC4B3_plpXRa7_AyyA&ust=1382453572561826


Adherence to process gg

leaves more time

for Innovation

http://www.google.com/url?sa=i&rct=j&q=&esrc=s&frm=1&source=images&cd=&cad=rja&docid=ImIo2BDVIAI7bM&tbnid=WrKdEuIYcGRU3M:&ved=0CAUQjRw&url=http://commons.wikimedia.org/wiki/File:LightbulbGlow.jpg&ei=ILJCUoT1MYPV0QWTm4CYCQ&psig=AFQjCNGOupxDvqHnzXXHNxkUb3EvyCSE1Q&ust=1380187079458688

http://www.google.com/url?sa=i&rct=j&q=&esrc=s&frm=1&source=images&cd=&cad=rja&docid=ImIo2BDVIAI7bM&tbnid=WrKdEuIYcGRU3M:&ved=0CAUQjRw&url=http://commons.wikimedia.org/wiki/File:LightbulbGlow.jpg&ei=ILJCUoT1MYPV0QWTm4CYCQ&psig=AFQjCNGOupxDvqHnzXXHNxkUb3EvyCSE1Q&ust=1380187079458688

//upload.wikimedia.org/wikipedia/commons/d/de/Fluorescent_lightbulb.jpg

//upload.wikimedia.org/wikipedia/commons/d/de/Fluorescent_lightbulb.jpg

http://www.google.com/url?sa=i&rct=j&q=&esrc=s&frm=1&source=images&cd=&cad=rja&docid=7zYii1XmrdE0iM&tbnid=DZvSUTq-3mk6HM:&ved=0CAUQjRw&url=http://www.flickr.com/photos/kalleboo/4057044421/&ei=WbNCUoOaFaqg0wWxuYG4BA&bvm=bv.53077864,d.bGE&psig=AFQjCNFwutWwC7lCAAYHCuE5EN2pc5C-PQ&ust=1380189069666003

http://www.google.com/url?sa=i&rct=j&q=&esrc=s&frm=1&source=images&cd=&cad=rja&docid=7zYii1XmrdE0iM&tbnid=DZvSUTq-3mk6HM:&ved=0CAUQjRw&url=http://www.flickr.com/photos/kalleboo/4057044421/&ei=WbNCUoOaFaqg0wWxuYG4BA&bvm=bv.53077864,d.bGE&psig=AFQjCNFwutWwC7lCAAYHCuE5EN2pc5C-PQ&ust=1380189069666003

http://www.google.com/url?sa=i&rct=j&q=&esrc=s&frm=1&source=images&cd=&cad=rja&docid=_BE3cgiCdli0uM&tbnid=GD86i2f-NtpykM:&ved=0CAUQjRw&url=http://www.flickr.com/photos/caveman_92223/3346906435/&ei=s7NCUrr2HvOb1AX1sIGoBA&bvm=bv.53077864,d.bGE&psig=AFQjCNFwutWwC7lCAAYHCuE5EN2pc5C-PQ&ust=1380189069666003

http://www.google.com/url?sa=i&rct=j&q=&esrc=s&frm=1&source=images&cd=&cad=rja&docid=_BE3cgiCdli0uM&tbnid=GD86i2f-NtpykM:&ved=0CAUQjRw&url=http://www.flickr.com/photos/caveman_92223/3346906435/&ei=s7NCUrr2HvOb1AX1sIGoBA&bvm=bv.53077864,d.bGE&psig=AFQjCNFwutWwC7lCAAYHCuE5EN2pc5C-PQ&ust=1380189069666003

Thank You!


Images creditsSlide 7: http://en.wikipedia.org/wiki/File:Chemicals_in_flasks.jpg - Joe Sullivan

Slide 8: http://commons.wikimedia.org/wiki/File:Sneeze.JPG - James Gathany

Slide 9: http://www.flickr.com/photos/parksdh/6493273759/sizes/l/in/photostream - D.H. Parks

Slide 10: http://www.flickr.com/photos/xlibber/3870106899/lightbox/ - photographer: http://www.flickr.com/photos/xlibber/

Slide 11: http://commons.wikimedia.org/wiki/File:Abundances_of_Atoms_in_Earth's_Crust.png - Stephen Brooks

Slide 12: http://www.flickr.com/photos/tracy_olson/61056391/ - Tracy O

Slide 37: Hippies: http://www.flickr.com/photos/imagineitall/9426241971/sizes/h/in/photostream/ - By Elazar

(http://www.flickr.com/photos/imagineitall/)

Air Aerobic team: http://www.flickr.com/photos/24874528@N04/9381172687/sizes/l/in/photostream/ - Airwolfhound

Slide 39: Blass blowing: http://upload.wikimedia.org/wikipedia/commons/thumb/4/44/Bamboo_Framing.jpg/682px-Bamboo_Framing.jpg -

Original uploader was Dbritta2 at en.wikipedia - Glass Artist J.P. Canlis sculpting a glass bamboo installation

Manufacturing plant: http://anthonyandginnie.blogspot.co.il/2008_05_01_archive.html - Ginnie and Anthony

Slide 53: http://en.wikipedia.org/wiki/Texting_while_driving - Ed Brown as Edbrown05

Slide 58: http://www.geograph.org.uk/photo/2274688 - Brian Robert Marshall

Video clips are from a lecture by Gerard Holzmann, published here: https://www.usenix.org/conference/hotdep12/tbd

Other images are from PowerPoint “Clip Art”, Intel owned or belong to the author.

the fab experience - pnsqc · the fab experience how i stopped ... /fix control measure identify...

Documents