forecasting in project monitoring system ......schedule variance: sv(t)=es-at schedule performance...
TRANSCRIPT
863
FORECASTING IN PROJECT MONITORING SYSTEM BASED ON
THE CONCEPT OF EARNED SCEDULE
Dr. Zoltan Sebestyen, Budapest University of Technology and Economics, Hungary
Dr. Gergely Babos, Budapest University of Technology and Economics, Hungary
ABSTRACT
The scope of this article is to come round the topic how to keep a project on track properly
with the fundamental parameters, and to provide an improved time-dependent solution for
monitoring. Since the milestone reporting is poor to monitor the projects, a much more
sophisticated system, the earned value management is proposed to be operated concurrently.
Recently, this method is criticized severely on its financial-oriented nature, which leaded to
its unreliability. Several solutions are suggested to provide an extension to earned value
management that surpasses these drawbacks. One of the suitable answers is to introduce and
use the concept of earned schedule. A method based on adaptive projective forecasting model
to earned schedule, which takes historical data into consideration, is proposed. The paper
also presents an illustration on the calculation of earned schedule using quantitative
forecasting models applied for a real-life data warehouse project at a financial institution.
Keywords: project monitoring, forecasting, earned value management, earned schedule, data
warehouse
864
INTRODUCTION
The project success is a primary interest in every organization. Since the achievements of a
project within the fundamental triple constraints correlate with project success, it is important
in every project to measure and monitor the time taken, the cost incurred, and the work
complete. Project managers were always motivated to use monitoring techniques not only for
identifying the variances, but also for forecasting the progress of the project and
implementing proper strategies. In data warehouse (DW) implementation and business
intelligence (BI) application development projects it is especially important to monitor the
project progress continuously, since the majority of these projects is business process
analysis. In case of business process analysis, project managers have some experience to
forecast the resource requirement, but significant variances may occur. The paper introduces
the application of earned value management (EVM) in a business intelligence development
project in order to indicate the variances.
Improving Earned Value Management System
An early milestone reporting system was introduced in the 19th century for North American
railroad construction projects, and later, in its entirety, for the projects of Department of
Defense in the US. The technique is particularly appropriate for linear construction works,
where the intensity of consequent activities must be aligned. Since the insignificant
performance of using milestones for monitoring more complicated projects, EVM method is
suggested recently. The terminology is based on the recommendations of Project Management
Institute, one of the most respected international project management organizations (PMI
2005; 2008). An EVM control system identifies and evaluates the variances from original
budget and control, also taking the progress into consideration. The method provides widely
used performance indices. Schedule performance index (SPI) measures the earned value (EV)
or budgeted cost of work performed against the planned value (PV). Schedule variance (SV) is
the difference between EV and PV. Cost performance index (CPI) can be calculated with the
ratio between the earned value and the actual value (AV). Cost variance (CV) expresses the
difference between EV and AC. The evaluation of the project is based on the relative positions
of earned value parameters. For earning these important data from the system, the project
must be monitored throughout the whole life cycle.
Lately, the system of earned value is improved, extended and applied in projects from
different industries. Kauffmann, Keating and Considine (2002) suggested to use earned value
methods to trace change-of-scope claims. Naeni, Shadrokh and Salehipour (2010) presented a
865
fuzzy approach for the earned value management. An effective control mechanism with EVM
is established in production environment (Bagherpour, Zareei, Noori and Heydari 2010).
Vandevoorde and Vanhoucke (2006) described different forecasting methods with the earned
value approach. Recent research (Lipke, Zwikael, Henderson and Anbari 2009) concluded
statistical tests on the reliability of forecasting methods. Based on their findings the use of
improved ES parameters produces sufficiently reliable predictions. It also provides an
effective mechanism for controlling cost and schedule in government contract works, such as
capital improvement programs (Storms, 2008). Howes (2007) proposed a detailed method on
the implication of earned value analysis on construction projects. An application for team
development and compensation tool is also pointed out (Vargas 2005). Owen (2008) showed
how to apply EVM in an R&D environment. A conceptual framework for better
implementation of earned value management methodology in different types of organizations
and projects on the basis of 40 key independent variables is added by Kim, Wells and Duffey
(2003). Pajares and López-Paredes (2011) proposed the use of new metrics for project
monitoring. The cost control index and the schedule control index are built on the concepts of
both earned value management and risk management.
Based on the drawbacks of EVM theory reviewed in different aspects below, some new
directions for improvement of original earned value concept were suggested (Warburton,
2010). Anbari (2003) introduced a quantity called time variance that is a time-based
counterpart of schedule variance and based on the spend rate and the planned value rate.
Numerous aspects of a new concept, the earned schedule were introduced and presented
(Fleming and Koppelman 2005; Lipke 2003) to measure the time delay between EV and PV
cost curves.
Since the EVM method can become unreliable after the first 15% of the duration of a project
(Christensen and Heise 1992), and the classic, well-known schedule performance indices, e.g.
SV and SPI, may provide optimistic results (Vandevoorde and Vanhoucke 2006; Cioffi 2006;
Lipke 2003), these earned value management-related, unreliable indicators must be integrated
or replaced. For EV is getting closer to PV towards the end of the project, the value of SPI
approaches 1 irrespectively of the delay. It means that at the end of the project, SPI becomes
equal to 1 in both on-time and late projects. As stated before, additional time-based
parameters were also suggested by Lipke.
The basis of the concept developed is analogous to earned value except that the cost based
performance measures are integrated with time based measures. The value of Earned
Schedule is the cumulative time to the point where EV in period t is equal to PV. For
866
determining ES, the following cumulative time based measures are introduced using actual
time (AT) as a point of reference.
Schedule variance: SV(t)=ES-AT
Schedule performance index: SPI(t)=ES/AT
The original schedule variance and the schedule performance index are fundamental elements
of any EVM system, but these are cost based parameters. This is the reason why the money-
oriented attribute of measures is indicated frequently in notations: SV($) and SPI ($).
In the coordinate system the horizontal axis maps the time, the vertical axis shows the costs
related to EV and PV (Figure 1). The equations (1) and (2) provided by Lipke are built on the
interpretation of a distance on the horizontal axis by drawing a horizontal line from the EV
curve to the PV curve. Earned schedule is used for determining the schedule delay or
acceleration reliably referring to actual time at period t.
Figure 1: Interpretation of earned schedule
yttES )1( (1)
)()1(
)()1(
tPVtPV
tPVtEVy
(2)
Forecasting of Earned Value Management Parameters
It is a very important issue to predict the different EVM related parameters in an on-going
project. The possession of these reliable and accurate estimated values improves the capability
of project managers for making informed decisions. Researchers of project management
realized the need to develop forecasting methods based on the latest monitoring techniques.
Lipke, Zwikael, Henderson and Anbari (2009) concluded a research to predict the project
outcome with CPI from EVM and SPI(t) from ES and found a reliable range of project cost
and schedule duration at different confidence levels. They also provided the related statistical
867
prediction technique and testing methods. Vandevoorde and Vanhoucke (2006) presented a
generic schedule forecasting formula for different project situations. The method produces the
estimated duration at completion (EAC) calculating with planned duration of work remaining
(PDWR). An important statement of the paper is that the schedule forecasting methods must
be used at least at the cost account level or higher of the work breakdown structure to be
reliable for early warning signals. Other sources reveal the reasons of the on-going or
sustained false optimism bias persists beyond the planning and execution phase, and
inaccuracy of forecasting (Kutsch, Maylor, Weyer and Lupson 2011).
Information Technology Environment for the Application
In order to prove our model, we monitored a live project with the method. The project goal
was to upgrade on a new application of the campaign management processes of the market
leading Hungarian bank, re-engineer and implement the necessary input data marts and output
monitoring reports.
The data warehouse is a software architecture that collects data from its source systems
(mainly analytical systems), historically stores and consolidates them and offers data through
data marts to decision support applications. A data warehouse needs to provide easy access to
organization’s information, consistent information delivery, easy adapting and resilience to
change, protection to the information asset and supporting improved decision making
(Kimball and Ross, 2002). Building a data warehouse means understanding the information
asset of an organization, restructure and utilize it in both operation and high level reporting.
The DW project initiation was in January 2011 where the project tasks were separated into
two phases. The first phase focused on installing the application and building the basic inputs
and outputs. The goal of the first phase is to release a simple campaign and learn the shortages
of the implemented solution. A milestone of the phase gate was planned to the beginning of
September 2011. The second phase focused on improving the solution architecture and
implementing both the complete input data mart and output report set. A milestone was
placed to February 2012.
Although the project sounds to be a technology project, we can rather say it is one of the most
complex business projects of the bank. The main success factor is not the release of the new
application, but the implementation of data mart and monitoring reports. As the data are
reliable, the application is also trustworthy to release proper campaigns. Concepts, methods
and applications that improve business decision making by using fact-based support systems
(usually data warehouses) are called business intelligence applications (Inmon, Strauss and
868
Neuschloss 2008). Business Intelligence applications are particularly sensitive to poor data
(Adelman and Trelepuk 2000). Implementing a data mart meeting the business requirements
means a lot of data profiling activity that can be hardly planned and scheduled (Inmon 2002).
Therefore such a project requires a project monitoring method to check if a project task is late,
or will be finished earlier. The first phase of the project contains the following tasks
respectively: Analysis, Planning, Implementation, Release to test environment, Training,
Testing, Pilot and Release to production.
The most sensitive task for data profiling is Analysis. In this task not only the business
requirements, but the available data quality and the exact business meaning of each data needs
to be identified. We will monitor the “Analysis of data sources” activity within this task with
our model.
The analysis of data sources activity started in the beginning of April 2011 and was planned
to be finished by the end of June 2011. Since during collecting all the necessary information
about source data some difficulties occurred, additional resources had to be involved and
some extra money was spent. One source system stored the data in an easily accessible
database, but the meta data describing the meaning of the data was only available for the third
party development partner, that had to be involved. This fact was unknown in the project
planning period. The involvement required continuous negotiations that resulted more costs
and more time spent (for instance long correspondence that practically means non-working
time). This difficulty occurred in week 6, therefore significant slowdown is realized around
these weeks.
Table 1: Data for a late project
Week 1 2 3 4 5 6 7 8 9 10 11 12
PV 200 400 650 860 1045 1210 1490 1900 2120 2320 2500 2710
AC 180 380 610 800 1010 1240 1420 1780 1990 2230 2480 2700
Progress 4% 8% 12% 18% 22% 31% 33% 38% 41% 44% 46% 48%
EV 112 220 323 503 628 863 924 1076 1140 1229 1288 1344
The involvement of a third party development partner started with official negotiations that
required resources, but did not result significant progress in this source system analysis. In
order to minimize the lost time the project manager restructured the task and started some
steps of Week 11, such as analyzing other source systems. This progress can be seen in
869
Weeks 8, 9, 10 and 11. In Week 12 the negotiations with the third party development partner
turned into the final phase that used lot of resources, but did not mean any progress in the
task. In Week 13, as the negotiations finished, the analysis activity continued rapidly. The
table shows both the planned and the actual progress of the activity. The percentages and
costs have been modified for security reasons (Table 1 and Figure 2).
We have already understood the reasons of non-linearity of Actual costs (involvement of extra
resources vs. saving planned resources), but we have not investigated the reasons of non-
linearity of earned value. We have to realize that the relation between Actual costs and
Progress percentages is not linear. However we involve extra resources, it is not sure that it
means better Progress, sometimes the Actual costs are higher and the Progress is less than it
was planned. We have experienced the same by saving some resources, reaching lower Actual
cost level and showing more Progress than it was planned.
Figure 2: Trend of input parameters for the calculations
ES Forecasting Method
As we stated above, the fundamental earned value parameters clearly show that traditional
SPI($) optimistically do not indicate any problem for a late project, because as the project
show progress, the value of EV approaches the all-time current value of PV. The time based,
earned schedule related index, SPI(t) falls abruptly, revealing a real situation for the managers
behind the numbers. Since the mastery of an accurate forecast for monitoring parameters is an
elemental need, the managers use models based on data describing real processes best.
0
500
1000
1500
2000
2500
3000
1 2 3 4 5 6 7 8 9 10 11 12
Planned value
Actual cost
Earned value
870
In project management practice, the fundamentally simple forecasting methods must not be
neglected. According to Fildes (1979) these forecasting models were more widely used and
accepted than more sophisticated procedures striving to labor desirable statistical results.
Hence, the scope of this research stays in the field of fundamental, quantitative and projective
forecasting models of operations management.
Most quantitative forecasting models are based on series of observations taken at regular
intervals of time. The models produce forecasts, if an underlying pattern can be identified in
the time series. In the sample project, a trend is obviously supposed, so for analyzing the
pattern of historical data of ES, the application of a smoothing technique, namely the Holt
method is obvious, because the set of data follows an additive trend without seasonality. As in
the case of almost any forecast, a random noise is superimposed on the underlying pattern.
Figure 3: Earned value and earned schedule indexes for a late project
One step Holt method provides the future value of ES to time period t+1 (Ft) in time period t.
The forecast is calculated with a recurrent formula containing the level (St) and the slope (Gt)
in t. Since a one-step forecast is used, the forecast is retrieved from (3). In the end of any t
period the level and the slope can be continuously recalculated (4 and 5) with the latest real
ES observation (At) as becomes available.
ttt GSF 1 (3)
ttt FAS )1( (4)
11 )1()( tttt GSSG (5)
0
0,2
0,4
0,6
0,8
1
1,2
1 2 3 4 5 6 7 8 9 10 11 12
SPI($)
CPI($)
SPI(t)
871
1,0 (6)
The Holt method is built on the concept of exponential smoothing, but the underlying pattern
is not constant and cannot be described with one parameter, two variable smoothing
coefficients (α,β) are built in. Using initial values of α=0,4, β=0,4 and F0=1,1 the model
produces forecasting for every week. Since we are at the end of the week 12, the forecast for
the week 13 is important to know. At the end of the week 12 the forecast is retrieved from the
sum of the level and the slope: F(13)=3,84.
The one-step forecast is continuously calculated from the beginning of the project, and all the
input of the model is available. The application of the model is started with initial values,
including the first forecast and the smoothing coefficients. After a series of calculation, the
errors, the way of their calculation and the ES values are at hand, with some analysis the
benefits of learning can be earned, the parameters can be refined. A more accurate smoothing
coefficient pair can be determined by minimizing the root mean squared error (RMSE) for the
whole interval containing n periods (7). The location of the minimum value gives the optimal
value for the smoothing parameters (α=0,645864278, β=0,482862563). Calculating a new,
adaptive forecast based on the modified coefficients gives a different, less optimistic value for
ES: Fadaptive(13)=3,12.
t
ten
RMSE 21min)min( (7)
It is obvious, that improved, time based models describe the progress much better than the
simple earned value method. The real life problem is even more complicated, because
sometimes foreseeable irregular events happen. The impact of these events can be estimated
with reliable historical data and an event pool can be introduced and built.
If we are in possession of historical project monitoring data of ongoing multiple projects, or
finished in the organization earlier, event pools of collected similar events can be built. Let us
investigate the event that was described earlier as a real life example. A third party supplier
had to be involved that resulted in unexpected negotiations and slowdown in project progress.
If this is not the first case as such event occurs, then we can investigate previous projects’
progress (t means the period when the necessity of the event was realized and t+1 means the
period when the event occurred).
In our case we focused on collecting ES(t) and ES(t+1) from previous projects in order to
build an event pool (
Table 3):
872
Table 2: Impact of an unexpected event
Project 1 2 3 4 5
ES(t) 4,2 3,968 8,623188 4,105556 8,2
ES(t+1) 3,092308 3,466667 8,423729 3,693333 6,388235
From these data we intend to calculate a constant (ci), that can be utilized as a correction
constant by forecasting for similar events in the future. One and maybe the easiest way to
calculate this constant is to form the average difference between ES(t+1) and ES(t) (8).
ci = i
tES
tES
i i
i
)(
)1(
(8)
We can examine other events and include them into the event pool that are usually
unexpected in the planning phase, but foreseeable in the current time period in BI projects to
forecast the project progress in late projects easier. We experienced the following typical
unexpected but foreseeable events in BI projects:
Infrastructure problems:
o Hardware: the necessary hardware components show defects, or need to be
replaced
o Software: the necessary software components need to be upgraded in order to
collaborate with new components
Unexpected involvement of new participants into the project, for instance third party
suppliers
The corrected way of calculation takes the impact of unexpected but foreseeable events into
consideration. The extent of the decrease is derived from the historical experiences, expressed
with parameter ci. The difference between the adaptive forecast and the adaptive corrected
forecast purely stems from the subversive effect of unexpected but foreseeable events (9).
These events are elementals of any projects, so their consideration expresses the aspiration for
having more realistic data and accordingly managing better projects. For the specific case of
unexpected involvement of a third party supplier calls forth ci=0,5.
adaptivei
adaptivecorrectedadaptive Fc
FF 100
, (9)
873
Summarizing the level and the slope in week 12, putting the impact of the new participant’s
involvement to it, the adaptive corrected forecast is produced for week 13:
Fadaptive,corrected(13)=1,56. The results of different forecasts are summarized in
Table 3 and Figure 4.
Table 3: Forecasted earned schedules for week 13
Forecast
(F)
Adaptive forecast
(F adaptive)
Corrected adaptive forecast
(F adaptive corrected)
3,84 3,12 1,56
Figure 4: Earned schedule values for different forecasts
CONCLUSION
Recently the importance of predicting the future progress of a project became more important.
Behind the prediction there are forecasted parameters based on measurable and accurate
monitored project data. In the last decades the use of EVM-based methods was spread around
the world, however lately for the drawbacks of SPI($) and SV($) several solutions were
offered. We chose Lipke’s ES approach to achieve a more accurate project management
monitoring solution. Consequently this article provided an answer to the questions what to
forecast and how to forecast in relation to the progress of projects. Employing the new
concept, ES(t) were forecasted in three ways. First, reliable quantitative data were available;
our forecasting was based on Holt method. Then, for a refined result the adaptive Holt method
0
1
2
3
4
5
6
7
9 10 11 12 13
F
F adaptive
F adaptive corr
874
was used, recalculating smoothing coefficients with investigating previous time periods.
Eventually, an event pool was set on typical unexpected but sometimes foreseeable irregular
events, and the database of the pool was employed to produce a forecast, which follows the
real life processes of the project best. A new, possible direction of our future research is to
understand the unexpected events in the event pool properly and to prepare a more
sophisticated model producing better ci parameters.
ACKNOWLEDGEMENTS
This work is connected to the scientific program of the "Development of quality-oriented and
harmonized R+D+I strategy and functional model at BME" project. This project is supported
by the New Széchenyi Plan (Project ID: TÁMOP-4.2.1/B-09/1/KMR-2010-0002).
REFERENCES
Adelman, S. and Trelepuk, M. 2000. Data Warehouse Project Management, Addison-
Wesley: 256
Anbari, F. 2003. Earned Value Project Management Method and Extensions. Project
Management Journal 34:12–23.
Bagherpour, M., Zareei, A., Noori, S. and Heydari, M. 2010. Designing a control mechanism
using earned value analysis: an application to production environment. The
International Journal of Advanced Manufacturing Technology 49 (5-8): 419-429.
Christensen, D. S. and Heise, S. 1993. Cost Performance Index Stability. National Contract
Management Journal 25, Spring: 7-15.
Cioffi, D. F. 2006. Completing Projects According to Plans: An Earned Value Improvement
Index. Journal of the Operational Research Society 57: 290-295
Evensmo, J., and Karlsen, J., T. 2005. Earned value – the hammer without nails? Proceedings
of the 49th Annual Meeting of AACE International, New Orleans, 26-29 June.
Fildes, R. 1979. Quantitative Forecasting -- The State of the Art: Extrapolative Models. The
Journal of the Operational Research Society 30 (8), Aug: 691-710
Fleming, Q. W. and Koppelman J. M. 2005. Earned Value Project Management. PA: Project
Management Institute, Newton Square, 3rd
ed.
Howes, R. 2000. Improving the Performance of Earned Value Analysis as a Construction
Project Management Tool. Engineering Construction & Architectural Management 7
(4), December: 399-411.
875
Inmon, B. 2002. Enterprise Intelligence – Enabling High Quality in the Data Warehouse/DSS
Environment. Ascential Software Corporation, Westboro, MA, WP-3011-0402: 5-6
Inmon, B., Strauss, D. and Neuschloss, G. 2008. DW 2.0 The Architecture for the Next
Generation of Data Warehousing, Morgan Kaufmann Publishers: 97-98
Kauffmann, P., Keating, C. and Considine, C. 2002. Using Earned Value Methods to
Substantiate Change of Scope Claims. Engineering Management Journal 14 (1), March:
13-20.
Kim, E., Wells W. G. Jr. and Duffey, M. R. 2003. A Model for Effective Implementation of
Earned Value Management Methodology. International Journal of Project Management
21 (5), July: 375-382.
Kimball, R. and Ross, M. 2002. The Data Warehouse Toolkit, John Wiley and Sons Inc., 2nd
ed.
Kutsch, E., Maylor, H., Weyer, B. and Lupson, J. 2011. Performers, trackers, lemmings and
the lost: Sustained false optimism in forecasting project outcomes - Evidence from a
quasi-experiment. International Journal of Project Management 29 (8): 1-12.
Lipke, W. 2003. Schedule is Different. The Measurable News, Summer: 31-34
Lipke, W., Zwikael, O., Henderson, K. and Anbari. F. 2009. Prediction of Project Outcome:
The Application of Statistical Methods to Earned Value Management and Earned
Schedule Performance Indexes. International Journal of Project Management 27(4):
400-407.
Naeni, L. M., Shadrokh, S. and Salehipour, A. 2010. A fuzzy approach for the earned value
management. International Journal of Project Management 15 (7): 28-32.
Pajares, J. and López-Paredes, A. 2011. An extension of the EVM analysis for project
monitoring: The Cost Control Index and the Schedule Control Index. International
Journal of Project Management 29 (5): 615-621.
Project Management Institute. 2008. A Guide to the Project Management Body of Knowledge
(PMBOK® Guide). 4th
ed.
Project Management Institute. 2005. Practice Standard for Earned Value Management
Owen, J. K. 2008. Implementing EVM in an R&D Environment: From Infancy to
Adolescence. Cost engineering 50 (10): 12-17.
Storm, K. 2008. Earned Value Management Implementation in an Agency Capital
Improvement Program. Cost Engineering 50 (12): 17-40.
Warburton, R. D. H. 2011. A Time-Dependent Earned Value Model for Software Projects -
Paper accepted for publication in the International Journal of Project Management
876
Vandevoorde, S. and Vanhoucke, M. 2006. A comparison of different project duration
forecasting methods using earned value metrics. International Journal of Project
Management 24: 289–302.
Vargas, R. V. 2005. Using Earned Value Management Indexes as Team Development Factor
and a Compensation Tool. Cost Engineering 47 (5), May:20-25.