on project probabilistic cost analysis from lhc tender data ph. lebrun cern, geneva, switzerland...
TRANSCRIPT
![Page 1: On project probabilistic cost analysis from LHC tender data Ph. Lebrun CERN, Geneva, Switzerland TILC’09, Tsukuba, Japan 17-21 April 2009](https://reader030.vdocuments.us/reader030/viewer/2022032605/56649e855503460f94b87373/html5/thumbnails/1.jpg)
On project probabilistic cost analysisOn project probabilistic cost analysisfrom LHC tender datafrom LHC tender data
Ph. LebrunCERN, Geneva, Switzerland
TILC’09, Tsukuba, Japan17-21 April 2009
![Page 2: On project probabilistic cost analysis from LHC tender data Ph. Lebrun CERN, Geneva, Switzerland TILC’09, Tsukuba, Japan 17-21 April 2009](https://reader030.vdocuments.us/reader030/viewer/2022032605/56649e855503460f94b87373/html5/thumbnails/2.jpg)
Basis of probabilistic cost analysisBasis of probabilistic cost analysis
• Following the PBS, the project is split in i lots, the cost of which are random variables Xi with– mean value mi
– standard deviation i
• The total cost of the project is a random variable X = Xi
– with mean value m = mi
• In the case when the Xi are statistically independant, X = Xi is characterized by– standard deviation = (i
2)½
– probability density function (PDF) asymptotically tending to Gaussian (central-limit theorem)
• Statistical independance or correlations between Xi is more important to probabilistic analysis of total cost, than detailed knowledge of the specific PDFs of Xi
![Page 3: On project probabilistic cost analysis from LHC tender data Ph. Lebrun CERN, Geneva, Switzerland TILC’09, Tsukuba, Japan 17-21 April 2009](https://reader030.vdocuments.us/reader030/viewer/2022032605/56649e855503460f94b87373/html5/thumbnails/3.jpg)
Statistical modeling of component Statistical modeling of component costscosts
• Heuristic considerations– things tend to cost more rather than less ⇒ statistical
distributions of Xi are strongly skew– PDFs fi(xi) are equal to zero for xi below threshold values bi equal
to the lowest market prices available– commercial competition tends to crowd prices close to lowest ⇒
PDFs fi(xi) are likely to be monotonously decreasing above threshold values bi
• The exponential PDF is a simple mathematical law satisfying these conditions
f(x) = 0 for x < bf(x) = a exp[-a(x-b)] for x ≥ b
• Characteristics of the exponential law– only two parameters a and b– threshold b– mean value m = 1/a + b– standard deviation = 1/a = m – b– « mean value = threshold + one standard deviation »
![Page 4: On project probabilistic cost analysis from LHC tender data Ph. Lebrun CERN, Geneva, Switzerland TILC’09, Tsukuba, Japan 17-21 April 2009](https://reader030.vdocuments.us/reader030/viewer/2022032605/56649e855503460f94b87373/html5/thumbnails/4.jpg)
Densités de probabilité exponentielle et normale(m = 0, sigma = 1)
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
-5 -4 -3 -2 -1 0 1 2 3 4 5
x
f(x)
Exponentielle
Normale
![Page 5: On project probabilistic cost analysis from LHC tender data Ph. Lebrun CERN, Geneva, Switzerland TILC’09, Tsukuba, Japan 17-21 April 2009](https://reader030.vdocuments.us/reader030/viewer/2022032605/56649e855503460f94b87373/html5/thumbnails/5.jpg)
Fonctions de distribution exponentielle et normale(m = 0, sigma = 1)
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
-5 -4 -3 -2 -1 0 1 2 3 4 5
x
F(x
)
Exponentielle
Normale
![Page 6: On project probabilistic cost analysis from LHC tender data Ph. Lebrun CERN, Geneva, Switzerland TILC’09, Tsukuba, Japan 17-21 April 2009](https://reader030.vdocuments.us/reader030/viewer/2022032605/56649e855503460f94b87373/html5/thumbnails/6.jpg)
Comparing Gaussian and exponential Comparing Gaussian and exponential PDFsPDFs
• Gaussian– X ≤ m - at confidence level 15,9%– X ≤ m at confidence level 50%– X ≤ m + 1,28 at confidence level 90%– X ≤ m + 1,65 at confidence level 95%– X ≤ m + 2,06 at confidence level 98%
• Exponential– X ≤ m - at confidence level 0– X ≤ m at confidence level 63,2%– X ≤ m + 1,30 at confidence level 90%– X ≤ m + 2,00 at confidence level 95%– X ≤ m + 2,91 at confidence level 98%
![Page 7: On project probabilistic cost analysis from LHC tender data Ph. Lebrun CERN, Geneva, Switzerland TILC’09, Tsukuba, Japan 17-21 April 2009](https://reader030.vdocuments.us/reader030/viewer/2022032605/56649e855503460f94b87373/html5/thumbnails/7.jpg)
LHC cost structureLHC cost structure
54%
12%
1%
1%
3%
2%
2%
15%
3%
2% 3% 2%
Magnets
Cryogenics
Beam dump
Radio-frequency
Vacuum
Power converters
Beam instrumentation
Civil Engineering
Cooling & ventilation
Power distribution
Infrastructure & services
Installation & coordination
Total 2.2 BEuro
![Page 8: On project probabilistic cost analysis from LHC tender data Ph. Lebrun CERN, Geneva, Switzerland TILC’09, Tsukuba, Japan 17-21 April 2009](https://reader030.vdocuments.us/reader030/viewer/2022032605/56649e855503460f94b87373/html5/thumbnails/8.jpg)
90 main contracts in advanced 90 main contracts in advanced technologytechnology
![Page 9: On project probabilistic cost analysis from LHC tender data Ph. Lebrun CERN, Geneva, Switzerland TILC’09, Tsukuba, Japan 17-21 April 2009](https://reader030.vdocuments.us/reader030/viewer/2022032605/56649e855503460f94b87373/html5/thumbnails/9.jpg)
Cost variance analysisCost variance analysis
Cost variance
factor
Evolution of
configuration
Technical risk in
execution
Evolution of market
Commercial strategy of
vendor
Industrial price index
Exchange rates, taxes,
custom duties
Lot 1
Lot 2
Lot 3
…
Lot N
Total
Not addressed here
Deterministic & compensated, not addressed
here
Coped for in tender price variance
![Page 10: On project probabilistic cost analysis from LHC tender data Ph. Lebrun CERN, Geneva, Switzerland TILC’09, Tsukuba, Japan 17-21 April 2009](https://reader030.vdocuments.us/reader030/viewer/2022032605/56649e855503460f94b87373/html5/thumbnails/10.jpg)
Scatter of LHC offersScatter of LHC offersas a measure of cost varianceas a measure of cost variance
• Available data: CERN purchasing rules impose to procure on the basis of lowest valid offer ⇒ offers ranked by price with reference to lowest for adjudication by FC
• Postulate: scatter of (valid) offers received for procurement of LHC components is a measure of their cost variance due to technical, manufacturing and commercial aspects
• Survey of 218 offers for LHC machine components, grouped in classes of similar equipment
• Prices normalized to that of lowest valid offer, i.e. value of contract
• Exponential PDFs fitted to observed frequency distributions with same mean and standard deviation
![Page 11: On project probabilistic cost analysis from LHC tender data Ph. Lebrun CERN, Geneva, Switzerland TILC’09, Tsukuba, Japan 17-21 April 2009](https://reader030.vdocuments.us/reader030/viewer/2022032605/56649e855503460f94b87373/html5/thumbnails/11.jpg)
All data (218 offers)
0
20
40
60
80
100
120
140
160
1.25
1.75
2.25
2.75
3.25
3.75
4.25
4.75
5.25
5.75
6.25
6.75
7.25
7.75
8.25
More
Tender price relative to lowest bid
Nu
mb
er
Frequency
Exponential fit
![Page 12: On project probabilistic cost analysis from LHC tender data Ph. Lebrun CERN, Geneva, Switzerland TILC’09, Tsukuba, Japan 17-21 April 2009](https://reader030.vdocuments.us/reader030/viewer/2022032605/56649e855503460f94b87373/html5/thumbnails/12.jpg)
Piping & mechanical installation (26 offers)
0
2
4
6
8
10
12
14
16
18
20
1.25
1.75
2.25
2.75
3.25
3.75
4.25
4.75
5.25
5.75
6.25
6.75
7.25
7.75
8.25
More
Tender price relative to lowest bid
Nu
mb
er
Observed frequency
Exponential fit
![Page 13: On project probabilistic cost analysis from LHC tender data Ph. Lebrun CERN, Geneva, Switzerland TILC’09, Tsukuba, Japan 17-21 April 2009](https://reader030.vdocuments.us/reader030/viewer/2022032605/56649e855503460f94b87373/html5/thumbnails/13.jpg)
Electronics (24 offers)
0
2
4
6
8
10
12
14
16
18
1.25
1.75
2.25
2.75
3.25
3.75
4.25
4.75
5.25
5.75
6.25
6.75
7.25
7.75
8.25
Mor
e
Tender price relative to lowest bid
Nu
mb
er
Observed frequency
Exponential fit
![Page 14: On project probabilistic cost analysis from LHC tender data Ph. Lebrun CERN, Geneva, Switzerland TILC’09, Tsukuba, Japan 17-21 April 2009](https://reader030.vdocuments.us/reader030/viewer/2022032605/56649e855503460f94b87373/html5/thumbnails/14.jpg)
Cryogenics & vacuum (35 offers)
0
5
10
15
20
25
1.25
1.75
2.25
2.75
3.25
3.75
4.25
4.75
5.25
5.75
6.25
6.75
7.25
7.75
8.25
Mor
e
Tender price relative to lowest bid
Nu
mb
er
Observed frequency
Exponential fit
![Page 15: On project probabilistic cost analysis from LHC tender data Ph. Lebrun CERN, Geneva, Switzerland TILC’09, Tsukuba, Japan 17-21 April 2009](https://reader030.vdocuments.us/reader030/viewer/2022032605/56649e855503460f94b87373/html5/thumbnails/15.jpg)
Mechanical components for SC magnets (70 offers)
0
5
10
15
20
25
30
35
40
45
50
1.25
1.75
2.25
2.75
3.25
3.75
4.25
4.75
5.25
5.75
6.25
6.75
7.25
7.75
8.25
Mor
e
Tender price relative to lowest bid
Nu
mb
er
Observed frequency
Exponential fit
![Page 16: On project probabilistic cost analysis from LHC tender data Ph. Lebrun CERN, Geneva, Switzerland TILC’09, Tsukuba, Japan 17-21 April 2009](https://reader030.vdocuments.us/reader030/viewer/2022032605/56649e855503460f94b87373/html5/thumbnails/16.jpg)
A simple worked-out exampleA simple worked-out example
• Consider a project made of 5 lots according to the table below
Lot Seuil Ecart-type Moyenne Variance1 250 40 290 16002 150 20 170 4003 100 10 110 1004 300 20 320 4005 200 10 210 100
Somme 1000 100 1100 2600Sigma 50.9901951
![Page 17: On project probabilistic cost analysis from LHC tender data Ph. Lebrun CERN, Geneva, Switzerland TILC’09, Tsukuba, Japan 17-21 April 2009](https://reader030.vdocuments.us/reader030/viewer/2022032605/56649e855503460f94b87373/html5/thumbnails/17.jpg)
Densités de probabilité du coût des lots(lois exponentielles)
0
0.02
0.04
0.06
0.08
0.1
0.12
0 50 100 150 200 250 300 350 400 450 500
Coût
f(x)
Lot 1 (m=290, σ=40)
Lot 2 (m=170, σ=20)
Lot 3 (m=110, σ=10)
Lot 4 (m=320, σ=20)
Lot 5 (m=210, σ=10)
![Page 18: On project probabilistic cost analysis from LHC tender data Ph. Lebrun CERN, Geneva, Switzerland TILC’09, Tsukuba, Japan 17-21 April 2009](https://reader030.vdocuments.us/reader030/viewer/2022032605/56649e855503460f94b87373/html5/thumbnails/18.jpg)
Application of central-limit theoremApplication of central-limit theorem
• In case the elementary costs are statistically independent, the total cost is a random variable with– mean value m = mi = 1100– standard deviation = (i
2)½ ≈ 51• Its PDF tends towards a Gaussian law [1100, 51]
– X ≤ 1165 at confidence level 90%
– X ≤ 1184 at confidence level 95%
– X ≤ 1205 at confidence level 98%
• This law can be compared to the result of a Monte-carlo simulation based on exponential PDFs for elementary costs, treated as independent
![Page 19: On project probabilistic cost analysis from LHC tender data Ph. Lebrun CERN, Geneva, Switzerland TILC’09, Tsukuba, Japan 17-21 April 2009](https://reader030.vdocuments.us/reader030/viewer/2022032605/56649e855503460f94b87373/html5/thumbnails/19.jpg)
Fonction de distribution du coût total du projet(simulation Monte Carlo n = 100, comparée à une loi normale [1100, 51])
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1000 1050 1100 1150 1200 1250
Coût [MCHF]
Pro
babi
lité
cum
ulée
![Page 20: On project probabilistic cost analysis from LHC tender data Ph. Lebrun CERN, Geneva, Switzerland TILC’09, Tsukuba, Japan 17-21 April 2009](https://reader030.vdocuments.us/reader030/viewer/2022032605/56649e855503460f94b87373/html5/thumbnails/20.jpg)
Conclusion: proposed procedure for Conclusion: proposed procedure for probabilistic cost analysis probabilistic cost analysis
• Identify sources of cost variance and separate deterministic effects
• Identify correlated random effects and estimate their standard deviations (not to be added quadratically!)
• Estimate mean value and standard deviation of independant elementary costs and modelize by simple skew law, e.g. exponential
• Apply central-limit theorem and/or Monte-Carlo on sum of independant elementary costs
• Apply uncertainty due to correlated random effects on previous result
• Apply compensation of deterministic effects by established factors (e.g. currency exchange rates & industrial price indices)