purpose of this document - dtf.vic.gov.au · web viewin the case of an investment that only...

Investment Management Standard 2017Technical Guide for FacilitatorsProblem Definition workshop

The SecretaryDepartment of Treasury and Finance1 Treasury PlaceMelbourne Victoria 3002AustraliaTelephone: +61 3 9651 5111Facsimile: +61 3 9651 2062dtf.vic.gov.au

Authorised by the Victorian Government1 Treasury Place, Melbourne, 3002

© State of Victoria 2017

You are free to re-use this work under a Creative Commons Attribution 4.0 licence, provided you credit the State of Victoria (Department of Treasury and Finance) as author, indicate if changes were made and comply with the other licence terms. The licence does not apply to any branding, including Government logos.

Copyright queries may be directed to [email protected]

ISBN 978-1-925551-40-2Published May 2017

If you would like to receive this publication in an accessible format please email [email protected]

This document is also available in Word and PDF format at dtf.vic.gov.au

http://www.dtf.vic.gov.au/

mailto:[email protected]

http://creativecommons.org/licenses/by/4.0/

http://creativecommons.org/licenses/by/3.0/au/

Contents

Purpose of this document..........................................................................................1What are the differences between IMS edition 5.0 and IMS 2017?.............................................1

1. Context for the Problem Definition workshop......................................................31.1 The theory behind the IMS................................................................................................41.2 Applications for IMS workshops........................................................................................41.3 How many workshops?.....................................................................................................6

1.3.1 Not all investments will require all four workshops.............................................................7

1.4 Timing of workshops.........................................................................................................91.5 Keeping track of project complexity and fuzziness...........................................................91.6 Addressing risk and uncertainty......................................................................................121.7 Addressing relevant policy levers....................................................................................14

2. Problem Definition workshop.............................................................................142.1 Purpose of a Problem Definition workshop.....................................................................142.2 Responsibility of the facilitator........................................................................................152.3 Who should be there?.....................................................................................................152.4 What preparation is required?.........................................................................................162.5 Before the workshop.......................................................................................................172.6 At the workshop...............................................................................................................17

2.6.1 Structuring the workshop...................................................................................................172.6.2 Timing................................................................................................................................182.6.3 Set expectations................................................................................................................182.6.4 Unpacking the cause and effect using the problem trajectory.........................................222.6.5 Reiterate the feedback process........................................................................................30

2.7 After the workshop..........................................................................................................302.8 Templates, examples and other resources.....................................................................32

Appendix 1: 16 Questions – Investment decision-maker’s checklist.......................33Appendix 2: Shaping new investments using the IMS.............................................35Appendix 3: Fictional - Investment Logic Map.........................................................37Appendix 4: Design guidelines – Investment Logic Map.........................................38Appendix 5: Sample email – before the Problem Definition workshop....................39Appendix 6: Checklist – Problem Definition workshop............................................40Appendix 7: Sample agenda – Problem Definition workshop..................................42Appendix 8: Benefit framework................................................................................43Appendix 9: Sample email – after the Problem Definition workshop.......................45Appendix 10: Quality assessment form – Investment Logic Map............................46Appendix 11: Facilitator feedback form – Problem Definition workshop.................47

Page i

Purpose of this documentThis is the first of four guidance documents within the ‘shape a new investment’ series of IMS Technical guides for facilitators. It primarily targets facilitators1 and provides practical guidance on how to lead a successful initiative-level Problem Definition workshop and prepare supporting documentation which is consistent with the Investment Management Standard (IMS) Version 6.0.

This guidance assumes users have read and understood the IMS – its principles, practices and the theory on which it is built. (The IMS is available at the investment management website, www.dtf.vic.gov.au/investmentmanagement.).

Section 1 outlines the context and objectives for the Problem Definition workshop.

Section 2 describes how facilitators should approach, conduct, and conclude the workshop.

Appendices contain additional materials which may be useful for facilitators.

What are the differences between IMS edition 5.0 and IMS 2017?

IMS 2017 is the first update to the IMS since 2013. Over this period there has been an increased focus on the planning and delivery of infrastructure investments, and on using real options analysis 2 to manage related uncertainty. In response, the Department of Treasury and Finance has updated its Investment Lifecycle and High Value/High Risk (HVHR) framework to provide advice on incorporating real options analysis when developing business cases and procurement strategies. The related guidelines are available at www.dtf.vic.gov.au/Investment-Planning-and-Evaluation.

The 2017 update reflects these changes and includes several enhancements to refine the workshop process and the development of an Investment Logic Map (ILM), and the other documents in the IMS suite. These incorporate the feedback and experiences of both those involved in the workshops and the end-users of ILMs. The major changes are:

more detailed advice on preparing for a workshop; clearer definition of the preferred participant types for each workshop; greater and more explicit consideration of uncertainty during the workshops including identifying

investments which may need real options analysis; increased focus on determining the quality and availability of evidence throughout the workshop

process; a reshaped and more robust Benefit Definition workshop which tests alignment with

Government policy, or other relevant strategic drivers, and focuses on the integrity of KPIs and measures;

restructure of the Response Definition3 and Solution Definition workshops to clarify the objectives of each and to ensure both are more intuitive, robust and make a valuable contribution to decision-making;

consequential changes to the supporting documentation for all the workshops; and amendments to the 16 questions – decision-maker’s checklist (Appendix 1) to include more

consideration of uncertainty and reflect the changes described above.

1 Further detail on the facilitator training course, in which this guidance is used, can be found at www.dtf.vic.gov.au/investmentmanagement.2 Real options analysis is an investment evaluation and decision-making framework which introduces more flexibility to the management of infrastructure projects that are significantly affected by uncertainty. It assists Government make investments that are more adaptable over time and better able to meet the community’s evolving needs.3 Previously the Strategic Options workshop

Investment Management Standard 2017 – Technical Guide for Facilitators Page 1

http://www.dtf.vic.gov.au/Investment-Planning-and-Evaluation

http://www.dtf.vic.gov.au/investmentmanagement

1. Context for the Problem Definition workshopThe ability to select the investments that provide the most benefit to society is a key component of good government. This is often a complex exercise requiring the contributions of many people, each of whom bring their specialist skills and perspectives.

The Investment Management Standard (IMS) is a process for applying simple, common-sense ideas and practices that help organisations direct their resources to deliver the best outcomes from their investments. The IMS addresses many of the issues that arise during investment decision-making and aligns with the HVHR business case guidelines, and templates. In the context of the IMS and these guidance documents, DTF defines investment as ‘the commitment of the resources of an organisation with the expectation of receiving a benefit’.

The IMS helps decision-makers determine whether:

there is a real, evidence-based problem that needs to be addressed now; the benefits which will be delivered through successfully addressing the problem are of

high value to the organisation and the community; the benefits’ KPIs are meaningful, measurable and attributable to the investment and

are worth tracking and reporting; the way the problem will be addressed is strategic, feasible, and innovative; the solution is likely to be delivered within time and budget constraints; and the solution can be applied flexibly to manage and respond to uncertainty and adapt to

changing conditions and demand.

The IMS includes a set of 16 questions (the Investment Decision-maker’s Checklist) which address the four IMS elements - problem, benefits, response and solution. Each element asks key questions that enable decision-makers to make sensible and informed investment decisions. The depth of enquiry for each question will depend on the scale and complexity of the investment. These questions correlate with key elements of the Victorian government full business case template and aid business case writers and assessors.

The relevant questions that the Problem Definition workshop should explore and help to answer are:

Table 1: Investment decision-maker’s checklist – problem

Problem – Investment decision-maker’s checklist

5. Is it clear what the problem is that needs to be addressed, both the cause and effect?

Yes Maybe No Not sure

6. Is there sufficient evidence to confirm both the cause and effect of the problem?


7. Does the problem need to be addressed now and by this government?


8. Does the defined problem capture its full extent/scope including sources of future uncertainty?


Page 2 Investment Management Standard 2017 – Technical Guide for Facilitators

1.1 The theory behind the IMS

In the place of complex processes, the IMS is centred on three key concepts:

1. the best way to aggregate knowledge is through an informed discussion that brings together those people with most knowledge of a subject at the time their insight provides most value;

2. the logic underpinning any investment (the ‘investment story’) should be able to be depicted on a single page using language and concepts that can be understood by a lay person; and

3. every investment should be able to describe how it is contributing to the benefits the organisation is seeking.

The IMS practices are focused on the early stages of shaping investments and on the evidence required to understand and validate the investment need or problem, articulate the benefits that will be delivered, and shape a robust indicative solution (Appendix 2-Shaping a new investment using the IMS). The practices also enable benefits tracking and support evaluation of an investment’s effectiveness.

1.2 Applications for IMS workshops

The IMS workshops can support an organisation’s primary investment decision-making in several ways. Specifically, they help organisations to:

shape a new investment; prioritise investment proposals; develop new policies; monitor and measure the delivery of benefits; evaluate a program of investment; refocus itself to improve its effectiveness; and monitor its outcomes.

Organisations usually make investment decisions at three levels:

organisation level – these decisions affect the organisation’s overall direction and outcomes;

program level – these decisions affect programs or portfolios of individual investments; and

initiative level – these decisions affect individual investments, both asset and non-asset based

While these levels should naturally connect, organisations often manage them using entirely different processes, in different forums, and using different language. By applying IMS practices, including its workshop suite, across all levels of decision making, as illustrated in Figure 1, an organisation can achieve a much closer alignment within its decision-making. The logic of a single investment can also easily be tested within the broader organisational context.

Figure 1 also illustrates how a set of foundation questions or areas of enquiry, are common to all three levels of investment. This ensures that an organisation’s entire decision-making framework can achieve consistent understanding, language, priority, and direction.


However the advice in the four IMS workshop guidance documents focuses on an initiative-level investment. The IMS can be used for an investment of any type or complexity, including program and organisation level investment and prioritisation discussions.

Figure 1: Integrated view of Investment Management Standard


1.3 How many workshops?

There are four workshops in the IMS suite:

1. Problem Definition Workshop

Successful investments are made as a considered reaction to an identified or emerging problem. This workshop focuses on:

defining the problem that needs to be addressed; validating that the problem is real; and specifying the benefits that will result from addressing the problem.

For investments requiring resources over the medium-long term (especiallylarge infrastructure and asset investments), an assessment of how uncertainty of different future scenarios may impact the way the problem is defined is an important element of this workshop. Where participants identify significant uncertainty about how the future may turn out, you should consider and record its potential impacts on investment success.

The output of this workshop is the first version of an Investment Logic Map (ILM) with the problems and benefits defined.

2. Benefit Definition Workshop

Investments are often shaped with little understanding of the benefits expected to be produced. This workshop will:

identify the KPIs, measures, targets and timelines that the investment will need to deliver; and

specify how the delivery of the benefits will be measured and reported.

For investments requiring resources over the medium-long term (especially assets), an assessment of how uncertainty of different future scenarios may impact the way the benefits are defined or realised is an important element of this workshop. Where participants identify significant uncertainty about how the future may turn out, you should consider and record its potential impacts on investment success.

The output of this workshop is a Benefit Management Plan (BMP) made up of a Benefit Map and Benefit Profile.

3. Response Definition Workshop

Business cases for new investments often fail to consider the full range of things that could be done to address the identified problem. This workshop will:

• explore the interventions that could deliver the expected benefits and KPIs; • formulate and evaluate a mix of response options; and • assess response options and potentially select the preferred response.

For projects requiring resources over the medium-long term (especially assets), a deliberate assessment of how effectively each response option deals with uncertainty is an important element of this workshop. You should consider what circumstances would lead to the preferred response being ineffective or inappropriate, and where a different response would be preferable. Where significant uncertainty is identified, you should consider and record its potential impacts on investment success.


The output of this workshop is a Response Options Analysis Report (ROAR).

4. Solution Definition Workshop

This workshop ensures that the project team develops a solution which is consistent with the foundations established in previous workshops. This workshop will:

confirm the preferred response and the interventions it contains; identify and evaluate the changes and assets that are required to implement the

preferred response and deliver the benefits; define a recommended solution for the expected future state; confirm the circumstances (change in condition or an event) where the preferred

response may be inadequate or inappropriate, and the triggers requiring a change in response;

identify cost range, timeframe for project and benefit delivery, key risks and uncertainties, dis-benefits and critical interdependencies associated with the recommended solution; and

consider any policy levers that may impact this response and identify any action or areas to investigate further outside the workshops. This may include departmental policies and well as whole of government policies regarding value creation and capture and climate change.

At the end of this stage, you should review the problem definition to confirm you have identified the right investment need, and that the preferred solution is likely to support this need given a range of alternative future scenarios. You should consider the whether there are any conditions in which the preferred solution may be sub-optimal, you would prefer a different approach, or would regret the selected solution.

The output of this workshop is an Investment Concept Brief (ICB).

After each workshop, facilitator updates the other IMS documents in the suite to ensure that they reflect the organisation’s current ‘investment story’.

1.3.2 Not all investments will require all four workshops

While size and complexity are key factors in determining how many workshops will be needed, all discussions of initiative-level investments must follow the same ‘line of enquiry’ (Figure 2) as they develop their investment stories.

Figure 2: Line of enquiry

Some investments may only require one or two workshops, whereas other, more complex investments may need up to four workshops. In the case of an investment that only requires one or two workshops, each area of the line of enquiry is covered but in a faster and less rigorous manner, usually producing only an ILM and BMP.

Opportunities to reduce the number of workshops and combine some workshop elements should be identified following a conversation with the investor (in advance of the Problem Definition workshop), considering the complexity of an investment.


Some of the typical factors affecting investment complexity are illustrated in Table 2.

Table 1: Factors affecting investment complexity

Cost Cost is usually a good proxy for general complexity, as is a wide range of potential costs.

Risk and uncertainty The higher the risk or the more uncertain the external environment, the more complex the investment. Section 2.5 provides some more guidance on how the IMS workshop suite considers risk and uncertainty.

Solution certainty In some cases, (usually very low cost investments), there is a known, or ‘best likely’ solution that is unlikely to be challenged. The more possible solutions, the more complex the investment decision.

Stakeholder profile The larger the number of individual stakeholder groups needing to be engaged, the harder it will be to both develop the investment, and ensure its success. This will increase the investment’s risks and the complexity around benefit delivery and measurement.

Public accountability The greater the public interest and/or accountability of the investment, the more important it is that the investment logic is robust and evidenced.

Benefit maturity Defining the benefits that a successful investment will deliver is pivotal to the IMS practices. Not all organisations have well-developed benefit measurement practices and, if they do not, there will need to be a much greater focus on developing KPIs and measures of value during the workshop process

These factors can help determine whether an investment is likely to be low, medium or high complexity and how many workshops will be required. Investments that fall under the Victorian Government’s HVHR framework will almost inevitably require all four workshops.

Table 2 outlines how the complexity of an investment determines the number of IMS workshops, and the suite of documents produced.


Victorian Government’s budget process

Investments falling within the Victorian Government’s budget processes are recommended to complete workshops that support the development of an ILM, Benefit Management Plan, Response Options Analysis Report and an Investment Concept Brief.

Table 2: Workshops and their products

1.4 Timing of workshops

The IMS workshops are often held at around two-weekly intervals. Experience has shown that, in many cases, two weeks provides sufficient time for the thinking of the previous workshop to be absorbed and preparation for the next workshop undertaken, and is not so distant that the momentum is lost. However, timing is dependent on the scale, complexity and stage in the budget cycle with the time between workshops being up to eight weeks in some instances.

Whatever the time interval, it is important that the essential preparation is completed and the relevant evidence is validated in order to move on to the next workshop. Failure to do this will undermine the quality of the next workshop’s output, and the robustness of any future business case. More information on the essential preparation required between workshops is provided in the guidance for each workshop.

1.5 Keeping track of project complexity and fuzziness

Investment decisions are typically complex. There are a range of internal and external factors that can influence why a problem exists, and when and how to best address it. The IMS’s explicit function is to distil the core elements of a problem and communicate a clear and simple investment story. Its value is its ability to present a complex issue in an uncomplicated way that is easily-digested by the investor, Government or other stakeholders. It is a simple investment overview that can help the Investor or Government


decide if the problem or service need is sufficient to warrant further consideration, such as business case development.

The investor, and those shaping the argument for investment, will naturally make a range of assumptions when distilling a project to its core elements. During IMS workshops, participants are likely to discuss a wide range of factors that could impact an investment. By design, the final IMS products will largely distil these elements to produce a clear, simple narrative that will not always convey the uncertainty that underpins it.

Whilst it may be acceptable practice to develop an investment for an assumed future state at the investment conceptualise (preliminary business case) stage, the final business case must robustly test these assumptions and provide options for managing them.

One of the most significant changes introduced in this 2017 IMS update relates to how it treats uncertainty. This reflects a greater emphasis on dealing with uncertainty throughout the Victorian Government’s broader investment management practices.

Under previous editions of the IMS, participants were not explicitly required to consider how uncertainty impacts their investment (although it was commonly discussed within the workshops). Where the discussion did raise issues relating to how external forces may impact a problem or response, this discussion trail was sometimes lost from the investment logic or narrative as the IMS template provided nowhere to record it. Where participants identified significant uncertainty in an investment strategy, they typically defined this uncertainty from a risk perspective. This did not necessarily ensure the subsequent business case addressed it effectively.

Under the 2017 edition, DTF requires participants to consider uncertainty during each IMS workshop. Participants should consider how their investment may be impacted by uncertainty, and how the preferred response might change if assumptions don’t hold or future conditions do not unfold as expected.

Requirement: If uncertainty is identified during any workshop, this should be captured within the workshop outputs, and must be considered, tested and addressed in the business case (if the proposal proceeds to that stage).

This requirement relates to any type of investment proposal. Early consideration of how changing circumstances may impact the optimal response to achieve benefits can benefit any project type. By designing an investment to anticipate change and/or respond advantageously to changing conditions, an organisation can minimise its, or government’s, obligations under negative conditions, and take advantage of opportunities.

DTF has made this framework change to recognise the critical influence uncertainty can have on investment success. DTF also recognises that the IMS, as a stand-alone product, explicitly and by design does not address significant uncertainty. The preferred option and alternatives identified in the solution definition workshop assume an expected future state, making it difficult to explore alternative investment trajectories and real options within the set of IMS process. If significant uncertainty is identified a different set of tools may be needed to inform investment options.

Requirement: If the uncertainty identified is significant, and the proposal is asset-related, DTF expects departments to undertake real options analysis to inform business case development4.

4 Information about real options analysis and associated tools is contained in DTF’s “ Investing under uncertainty” technical supplement to the Investment Lifecycle Guidelines.


Important note: Although agencies are required to consider the impacts of uncertainty on non-asset based investments, they are not required to undertake real options analysis as part of the business case for these proposals. Non-asset based investments generally allow government greater flexibility to adapt an investment strategy in response to changing conditions. Uncertainty impacting on these types of proposals can generally be adequately managed through strategic planning or taking managerial decisions during implementation.

At what point should departments undertake real options analysis to help deal with uncertainty?

Where appropriate, real options analysis should be undertaken as early in the investment lifecycle as possible. Ideally, once government has committed to developing a business case for the investment, real options analysis should be undertaken as an integral input to business case development. However, there is no set point at which it should be considered within the IMS framework.

In some instances, the investor may identify that the investment need is subject to uncertainty to such an extent that real options analysis could help define the problem and benefits. High cost, infrastructure investments with a long investment lifecycle typically fall into this category. In other instances, there could be value in going through the first three or four workshops, and using the outputs to inform real options analysis.

Whilst the IMS does not provide the tools to address uncertainty itself, in clearly articulating the investment problem it provides the starting point for any real options analysis. The IMS therefore remains a valid and useful tool for investment decision-making, including for those investments impacted by significant uncertainty.

Real options analysis of an investment should document:

The investment need, including how demand might change in response to an event or altered conditions;

The expected benefits, and how these might change given different future states; Response options, and including responses to alternative investment trajectories and

future states; and The preferred solution(s), including assessing under what conditions this solution

would be sub-optimal, an alternative approach would be preferred, and/or the investment would be regretted.

Where real options analysis is undertaken, this should be used to inform the business case and would require the IMS outputs to be updated.

If participants do not identify significant uncertainty during the four IMS workshops, and real options analysis has not been identified as being warranted, the group must undertake a feedback loop at the end of the fourth workshop as a final check that real options need not be considered. These are:

Requirement: At the end of the Solution Definition workshop, review the Decision-Maker’s Checklist and test:

– Has the investment need been correctly defined – are we considering the right problem?

– Under what conditions would the preferred solution be a sub-optimal response?– Under what conditions would an alternative investment strategy be preferred?– Under what conditions would we regret this investment?


The next section outlines how to consider uncertainty in each of the four IMS workshops.

1.6 Addressing risk and uncertainty

Within the suite of IMS workshops the key concepts of ‘risk’ and ‘uncertainty’ are explored in different ways. The terms both refer to situations where the knowledge of outcomes is incomplete or imperfect. However, whilst practitioners sometimes use these terms interchangeably, they refer to slightly different concepts.

Risk is a variance (either positive or negative) from an expected outcome.

Risks usually apply to the delivery of a project. They are inside the project team’s control to minimise and mitigate to achieve the

defined scope and expected benefits.

The Response Definition and Solution Definition workshops identify the key risks related to achieving an investment’s benefits, for further analysis in the business case.

In comparison, uncertainty is an event or change in conditions.

Uncertainties usually relate to the investment need or problem. They are usually external factors outside the project team’s control. They can result in a different future state to that anticipated or assumed in the

business, and can impact the need for an investment or the benefits that are likely to be realised and can require a change in response.

These events include technological developments, major shifts in markets and economic conditions, the behaviour of other organisations, demographic and societal structures, or the natural environment. If such events occur they can have both positive and negative impacts on benefit delivery.

Investors should consider the potential impacts of uncertainty when developing any proposal. The problem definition developed in the first IMS workshop sets the direction of the investment and is the basis for how the problem is considered and investigated in the business case and beyond. It is therefore important that any uncertainty identified within the IMS process is captured and, if the investment proceeds, further pursued in the business case development.

Requirements for considering uncertainty at each stage of the IMS process are:

Problem Definition – consider the organisation’s operating environment and the expected future state where this problem exists. Is it characterised by pronounced uncertainty? Are there any external factors outside the control of the organisation that could significantly impact the cause, effect likelihood and severity of the problem or the need or demand for change? Consider the conditions or future state in which the investment need or problem materially changes. If there is any likelihood that the extent of the problem, or the nature of the demand for an investment, could change over time, this should be captured in the cause and effect diagrams and raised at subsequent workshops for further consideration.

Benefits Definition – consider uncertainty may materially affect realising any of the proposed benefits. Would the need for a benefit, or your ability to achieve that benefit, be materially different in different future scenarios? Is achievement of any of the benefits, KPIs or measures, contingent on significant interdependencies? Again, any material uncertainty should be captured within IMS documents and further considered in subsequent workshops.


Response Definition – consider the operating environment and supply markets. Would any of the response options become unfeasible if circumstances changed? Under what different scenarios or conditions would response options be inadequate or inappropriate? What are the triggers for changing the response? Are any of the interventions contingent on material interdependencies? Will this affect the need for, or approach to, the investment? At this stage, the level of uncertainty impacting a project, and the extent to which response strategies need to contemplate it, needs to be formally assessed and noted within the ROAR.

Solution Definition – consider the operating environment and supply markets. Are there any external conditions or uncertainties that would lead to the preferred solution being inadequate or inappropriate, and where we might regret the investment approach? Are there any external conditions or uncertainties that could impact on the deliverability of the recommended solution? What are the triggers for a change in response? Does the recommended solution need the flexibility to respond to uncertainty?

The level of uncertainty impacting a project, and extent to which response strategies need to contemplate it, needs to be formally assessed and noted within the ICB.

If any of the above questions is ‘yes’, and the investment relates to infrastructure, it is likely that real options analysis5 will be needed during the business case preparation. This will focus on building as much flexibility and adaptability as possible into the investment response and solution.

If participants do not identify significant uncertainty during the four IMS workshops, and real options analysis is not considered warranted, the group must undertake a feedback loop at the end of the Solution Definition workshop. You should review the Decision-Maker’s Checklist and test:

Has the investment need been correctly defined – are we considering the right problem?

Under what conditions would the preferred solution be a sub-optimal response? Under what conditions would an alternative investment strategy be preferred? Under what conditions would we regret this investment?

The role of the IMS is to identify uncertainty and its impacts on an investment and, where those impacts are potentially significant, recommend further investigation, Real options analysis is outside the scope of the IMS. DTF highly recommends agencies engage suitably qualified and experienced consultants to support this work.

1.7 Addressing relevant policy levers

The IMS workshops focus on prosecuting a core problem, and identifying a clear set of defined benefits and interventions that respond to that problem. Whilst it is not the intent of the IMS to consider in detail how a proposed investment aligns with broader policy levers, workshop discussions can help identify policy requirements and synergies. For example, IMS workshops may identify potential value creation and value capture opportunities6 within a proposal, or projects that can contribute to Victoria’s Climate Change Adaptation

5 Real options analysis is an investment evaluation and decision-making framework which introduces more flexibility to the management of infrastructure projects that are significantly affected by uncertainty. It assists Government make investments that are more adaptable over time and better able to meet the community’s evolving needs6 The Victorian Government has developed Victoria’s Value Creation and Capture Framework aimed at getting better value for the community from all future infrastructure projects. More detail on this framework is available at http://www.dpc.vic.gov.au/index.php/news-publications/value-creation-and-capture-framework .


http://www.dpc.vic.gov.au/index.php/news-publications/value-creation-and-capture-framework

Plan.7 Consideration of these opportunities will be important in defining benefits as well as option analysis and solution design.

Where this occurs, the IMS documents (in particular the ICB) should capture these additional proposal details and the business case should explore and test them further.

2. Problem Definition workshop

2.1 Purpose of a Problem Definition workshop

This is the first workshop in the IMS series and aims to shape an investment that will deliver the maximum benefit to the organisation, and the community. It focuses on determining the problem and the benefits, with greater or lesser emphasis on the KPIs, preferred response and the recommended solution, depending on the scale of the investment and the number of planned IMS workshops (see 2.3.1).

If the investment is simple, the Problem Definition workshop will generate an ILM in which all columns are completed. However, the rigor associated with all but the Problem column will be less robust. If the investment is more complex, the Problem Definition workshop will only generate content for the problem and some of the benefit sections of the ILM. The rest will be added in future workshops.

The primary intent of this workshop is to get an agreed understanding of the:

problem affecting the organisation; strength of available evidence to confirm both the cause and effect or consequence of

the problem; and benefits the organisation can expect if it successfully responds to the problem (these

will be developed further in the Benefits Workshop).

Figure 3 depicts the place of this discussion within the ‘line of enquiry’ that underpins the IMS, as well as the three individual steps within the discussion.

7 Victoria’s Climate Change Adaptation Plan lays out the government’s plan to help the State meet the challenges and act on the opportunities for climate change. http://www.dpc.vic.gov.au/index.php/news-publications/value-creation-and-capture-framework .




Figure 3: Line of enquiry – Problem Definition workshop

The key output of this workshop is the story of the potential investment in the form of an ILM (Appendix 3). In developing an ILM, the group is effectively producing the executive summary of the business case for the potential investment.

There is no ‘right answer’ to the investment story; it is just the story which the workshop participants have agreed.

2.2 Responsibility of the facilitator

It is your responsibility to help the participants develop the most compelling, evidence-based case for investment they can. To do this, you will need to be challenging, probing, and independent.

To be successful, you must:

Try to ensure that the right people will be attending and that two hours has been allocated for the discussion;

provide appropriate advice regarding the preparation and evidence required for an effective discussion and what is expected of the participants;

conduct the discussion following the guidance provided in this document; and finalise the discussion and the ILM within 48 hours.

You should refer to Design guidelines – Investment Logic Map (Appendix 4) for some tips on preparing a high-quality ILM.

2.3 Who should be there?

The key person who must attend the workshop is the investor – the person who owns the business problem and will be responsible for delivering the benefits. The investor may also be known as the senior responsible officer or sponsor.

It is the investor’s responsibility to identify the other workshop participants. They should bring together people who understand the problem(s), can provide evidence that confirms that the problem(s) are real, and are prepared to express their views and contribute actively to a discussion. The facilitator should be prepared to advise the investor on how to craft an effective workshop group.


http://www.dtf.vic.gov.au/CA25713E0002EF43/pages/investment-management-tools-and-templates-glossary-people

For example, it is usually valuable to involve people from broader strategic policy areas of the organisation to appropriately inform and challenge the problem(s) identified. This workshop also provides an opportunity to include key external stakeholders who will be important to ensuring that the investment is a success.

The complexity and significance of the investment will determine the seniority of the participants, but including participants solely on the grounds of seniority should be discouraged. For very complex problems and investments, however, the workshop will usually need senior executives with strategic responsibilities for key operational areas. For less complex investments, the appropriate mix may involve stronger representation from operational staff. If you are planning only one or two workshops and hope to complete the ILM relatively quickly, at least one person with implementation experience should be included in the workshop.

Typical participants for a Problem Definition workshop include:

the investor; senior executives with strategic responsibility for relevant operational areas; line managers with good understanding of the key issues; Chief Financial Officer/Chief Information Officer/Chief Strategy Officer or other

senior staff who have an overarching view of organisation and understand any relevant independencies;

representatives from other material agencies, with likely interdependencies, or other critical external stakeholders;

Department of Treasury and Finance and/or lead Department representative8; technical specialist with detailed knowledge of services, systems and/or

implementation; and business case developer, (if already identified), who can attend, as an observer, to

learn more about the potential investment.

It is a good idea for the investor to invite the desired participants to all the planned workshops at the same time. This will ensure that the core participant group is preserved for the whole series and augmented at the right time by other skills and experience 9. This does, however, require certainty about workshop timing and this will depend on whether, or not, the intra-workshop preparation has been satisfactorily completed (see Section 2.4)

The most effective group size is usually between six and eight of active participants, but could be up to ten depending on the nature of the investment.

2.4 What preparation is required?

To ensure that the problem definition workshop is useful and efficient it is important that all participants, including the investor, understand the intent of the workshop and the expectations of participants. You can support this by:

being clear about the roles and responsibilities of participants and the depth of understanding they should have of the issues. The workshop is very time-constrained and cannot become a forum in which issues are explained in detail to participants;

ensuring the investor directs all participants to make themselves very familiar with any existing relevant materials such as: briefing papers, background reports, external reviews, internal management reports etc.; and

8 Early engagement with DTF us valuable as it builds greater understanding of the need for investment and allows them to assist with shaping the case for investment.9 The other Tips and Traps guidance documents in this series describe the participants for the other workshops.


assisting the Investor to identify those participants who are the owners of key evidence and ensure that they can refer to it authoritatively during the workshop, when required. A key part of this workshop is to confirm that evidence exists to substantiate the problem(s).

It is not necessary for any of the participants to bring materials to the workshop.

2.5 Before the workshop

Ensure the investor is attending the workshop and assist them to identify the other participants.

Ask the investor to send an email outlining the purpose of the discussion to the participants. You may find it helpful to tailor the Sample email – before the Problem Definition workshop (Appendix 5) and provide this to the investor. The attachment referred to in this email is Appendix 1 Shaping new investments using the IMS. This will provide the participants with a useful overview of the thinking behind the IMS, and the decision-making pathway.

Make sure the venue has been reserved 10 minutes prior to the start of the workshop, can accommodate the number of participants comfortably and has at least one suitably sized whiteboard. it is often useful to have a flip chart available as well.

2.6 At the workshop

2.6.1 Structuring the workshop

The structure and time allocation within the Problem Definition workshop will depend on the size and complexity of the proposed investment, and the number of planned workshops. For a very low complexity investment, the facilitator should aim to move from defining the problem all the way through to the business changes that make up the recommended solution in one workshop. Completing the ILM in one workshop is the exception rather than the rule. The Checklist - Problem Definition workshop (Appendix 6) overviews the eight steps required if this approach is taken. If more than one workshop is planned, you will usually only complete steps one to four, and possibly part of step five, within the Problem Definition workshop.

For an investment of medium complexity, the aim would be to develop the problems, benefits and KPIs in the first workshop. For a very complex project you would probably only reach the stage of defining the problem and outlining the benefits, including an initial suite of KPIs, in the first workshop. For more guidance on how many workshops are needed, refer to section 2.3

DTF recommends you roughly map out how you aim to structure the workshop in terms of the time allocated to each section. However, this structure may well change depending on how the discussion progresses or if it is discovered that the investment is more, or less, complex than previously expected.

2.6.2 Timing

Maintain a time-pressure awareness throughout the discussion.

Allow for ‘venting’ of issues during the first period (10–20 minutes).


Focus on establishing very strong problem statements (no more than four and typically two or three). Extracting these statements is likely to take up a significant proportion of the workshop. It is time well spent though, as their identification is the most critical element of a successful business case, and investment.

Benefits need to be expressed in the context of why the organisation exists and must be supported by good KPIs (meaningful, attributable and measurable). If these are weak you are likely to find the ILM falls apart at the Benefit Definition workshop.

If you are trying to complete the ILM in one workshop, then you will require the last 10–15 minutes of the workshop to gather a list of potential interventions, changes and assets.

For major investments, don’t be concerned about determining measures, selecting interventions or defining the recommended solution. Participants will seriously consider these at subsequent workshops.

Appendix 7 provides a Sample agenda for a Problem Definition Workshop.

2.6.3 Set expectations

It is a good idea to set some general expectations at the start of the workshop:

you expect people to be at the workshop for the full two hours; phones should be turned off, including messaging and email; and polite behaviour within the group is expected but this should not compromise the

robust and probing nature of the discussion. This is a workshop where you expect high levels of participation and contribution from those at the table.

Step 1: Set the scene

Set the context and objective of the workshop, the roles of each participant, the approach, the time criteria and your role. You should aim to create a climate of robust and open discussion within a two hour ‘time-box’.


Investment Management Standard 2017 – Technical Guide for Facilitators

Why are we here?We are here to have an intelligent and open discussion that will extract the story of this potential investment. This investment story will be in the form of an Investment Logic Map. What is an Investment Logic Map?

Hand out a copy of the ‘Example – Investment Logic Map (initiative)’.This is to demonstrate the outcome sought in this discussion. An Investment Logic Map is a single-page depiction of the logic that underpins an investment. It represents an ‘agreed investment story’ that is created in an informed discussion. It is written in plain English in a way that will allow a layperson to understand the language and the concepts. An Investment Logic Map must speak to the funder in a language they understand. It should tell a compelling story for investment that is logical, able to be supported by evidence and easily understood. When completed it underpins the executive summary and the headings of the business case.The Investment Logic Map itself becomes the foundation document that will be modified to reflect the logic for an investment throughout its lifecycle. This session will go for two hours. In that time, we will shape and record the story of this investment. There are three possible outcomes… each of which is valid and valuable: we will like what we see (validation that the idea has merit). we won’t like what we see (seemed like a good idea at the time). we will have nothing coherent (we don’t know enough about the problem to consider investing

now).

What is my role?My role as the facilitator is to be content free. I am a storyteller and my aim is to help you tell the best investment story that is possible, achievable and can be supported by verifiable evidence. Part of my job as a facilitator is to challenge the assumptions and claims you make and to test their validity. I am paid to detect fiction and ensure what is depicted in the final Investment Logic Map is an accurate reflection of our discussion and the evidence that underpins it.

The investor’s roleAs the investor, you are the most important person in the room. This is your forum and you are responsible for delivering value from this investment. You are the person with the problem who will be making or advocating the investment decision, and who will ultimately be responsible for delivering the benefits. Final decisions in respect of this investment will be directed to you.

ParticipantsAs the other participants, you are here to assist the investor to define the problem, the expected benefits, preferred response, and recommended solution for this investment (if this is a single workshop). You have been selected to participate because you know most about this problem and its impact – you are the subject matter experts. It is your insights that will help to shape and bring rigour to this investment story. Our observers (one or two) are welcome but are expected to listen to the conversation rather than actively participate. Their time to participate is likely to come during subsequent workshops or in the development of the business case.

How far will we get?The workshop is the initial part of the consultation. The feedback process that takes place after this workshop is equally important and you are all expected to participate.It is unlikely there will be sufficient time during the workshop to get all the wording exactly right. Our focus is on getting clear agreement on the intent of the wording during the workshop and we will ‘wordsmith’ during the feedback stage and in the final draft of the ILM.

In this workshop, your challenge is to move the discussion away from a focus on the solution, where people typically want to start, to the need or negative impacts that have created the impetus to make a change.

It is useful to start by asking the investor to identify the overarching business outcome they want from this investment. This becomes the initial title for the investment and provides some scope and context for the discussion. This exercise should not take more than one minute. It is often more efficient to ask the investor to think about this prior to the workshop. Once you conclude the workshop , the participants will have a chance to review the title to ensure it reflects the investment story they have told. At this point a subheading can also be included for simpler projects; this will be an indicative solution whether it be a project, asset or service. Alternatively, you can leave this field vacant until the discussion around the solution has been completed.

For example:

Heading: Improving access to ambulance services in rural areasSubheading: Upgrade to the ambulance fleet

It is common practice during the process of developing an ILM to test the initial desired solution thoroughly and a new, better solution will often become apparent. This will be reflected in the subheading as the ILM develops.

Step 2: Gather the issues and identify the themes

Ask the investor to outline the issues that has caused them to consider an investment. Once the investor has finished (three or four minutes), open this up to the rest of the group. Let the group ‘vent’ for as long as it needs, typically five to ten minutes, and capture a list of issues on the whiteboard or flipchart.

This exercise helps to establish whether all participants have a shared understanding of the context and ensures that they are engaged. It is important at this stage not to challenge the validity of the issues presented (or allow others in the room to do so). This will happen quite rigorously when we start to unpack the problem.

Once the issues list reaches a natural conclusion, try to identify themes that you can use to categorise the issues, for example: inefficiency, service quality, injury and safety, cost, risk, productivity or waste. The most important theme will become the starting point for the first problem.

A maximum of four problems are permitted but, in most cases, two to three problems are sufficient to describe the need for the investment.

Step 3: Unpack the problem(s)

A problem is the reason action needs to be considered now, even if it may not have fully crystallised. Generally, it is couched in negative terms and is made up of two parts – cause and effect. The problem statements are critical to understanding the need for the investment and should enable the reader to get a sense of the significance and urgency of the problem(s), and form an early estimate of the scale of the investment.


For example:

Cause: High levels of toxicity Effect: Threaten the presence of rare flora and fauna in the park

A problem statement should:

be expressed in plain English and have a clearly defined cause and effect; be supported by evidence to verify both the cause and effect that these two elements

are correlated; have an end consequence that is measurable; and be compelling and something that the government, the organisation or community

cares about (i.e. if the effect or consequence is of little importance or concern, the problem is not compelling).

Once a problem has been unpacked and validated, the facilitator records it in the first column of the ILM on the whiteboard.

At the heart of a good problem statement is the articulation of the cause, or what is broken, and the evidentially linked effect, or consequence.

Participants often come to the workshop with some firm views about the cause, or need for investment. These may or may not be true. Often, they are symptoms of the problem rather than the heart of the problem.

Sometimes they initially see a problem statement within an external context, for example, it is an election promise, a Minister’s request, or a compliance requirement. You need to work hard to work back to identify the community need that stimulated the election


Key questions to interrogate and test problem statements

What is the cause and what is the effect or consequence and do we have evidence? How do we know?

What contributing factors and failures have contributed to this cause or breakage or failure i.e. why has this happened?

Has anything in the external environment changed or is likely to change? Are there organisational causes, human causes, or physical causes? Can we do something about remedying the cause or what is broken? What is the effect or impact that we care about? Who and what is affected? How bad is it and how do we know? What evidence is there to support the relationship between the cause and the effect that we

care about? What are the imperatives for this investment? What trigger has made us think that we need to respond now? What will happen if we do nothing? What external factors could impact on the cause or effect of the problem causing

uncertainty? Is the organisation operating in an environment of significant uncertainty and do the problem

statements reflect this?

What will happen if we do act and circumstances change?

promise, Ministerial request or compliance requirement. Typically, these situations have a real and evidence-based need that can be described in the problem statement.

On other occasions participants arrive with a view of the problem that starts at a very low level such as a broken or outdated asset. If you start here, then you are forced too early and often unproductively down an asset pathway for your solution. You will need to work with the group to lift the discussion to a more strategic level.

Getting a better understanding of cause and effect will help sharpen the problem statement and provide a better understanding of the problem story. A good way to do this is to try to unpack an issue by exploring the relationship between cause and effect. This can be recorded on the whiteboard using a range of techniques such as root cause analysis, decision-tree analysis, the 5 whys, or developing a problem trajectory. The evidence should provide insight into the nature, scale and scope of both the cause and effect. By unpacking an issue in some detail, and testing the evidence that supports both the cause and effect, the group starts to understand not only the real problem but also the benefit, potential KPIs and some of the strategic interventions that might be considered.

2.6.4 Unpacking the cause and effect using the problem trajectoryFigure 4: Developing a problem trajectory



Example – unpacking a problem (see Figure 4)

We have an issue around ‘High levels of toxicity in a national park’.

Write the cause, “High levels of toxicity in the waterways”, on the whiteboard. Now validate that there is evidence available to demonstrate the nature, scale and scope of the toxicity. Once this is confirmed move to identifying what the causal factors of this toxicity

Question: What caused the high levels of toxicity?

Answer: The water quality was declining.

Next question: What is causing the decline in water quality?

Answer: Pollution from a nearby factory and increasing pesticide run-off from agricultural land.

Focus first on the factory.

Question: What is causing the factory pollution?

Answer: Lack of standards-based filtration equipment.

Question: Why don’t they have appropriate filtration systems?

Answer: Cost and lack of monitoring to ensure compliance.

Now let us look at the pesticide run-off.

Question: What’s causing the increase in run-from pesticides?

Answer: Excessive pesticides being used by local farmers.

Question: Why are farmers using more pesticides that are of a greater strength than they need?

Answer: Outdated farming practice.

So, we get a better understanding of the factors that contribute to the problem as well as some areas that may need to be addressed within our response:

Cost of more effective filtration equipment, monitoring capacity and education for farmers.

Question: Is there anything else that is contributing to this high toxicity?

Answer: Low levels of water flowing through the waterways during summer

Question: Has this changed?

Answer: Yes, firstly we have had much lower rainfall over the last two years; we think that this is an effect of climate change.

Question: Is there a significant level of uncertainty around the impacts of climate change on the rivers in this area? Is this likely to make matters worse?

Answer: Unsure of the extent now but the range and level of impact might require us to consider different responses

Question: Is this the only thing that has changed?

Answer: No, there has been an increase in water taken from the river to support irrigators.

Questions: Why has this occurred?

Answer: New agricultural developments that have been encouraged by government policy and incentives and a change to the existing agreements because of low rainfall levels

Now let us look at the effect of the high level of toxicity in a national park.

Question: What is the effect/impact/consequence that is most closely related to this issue that is supported by evidence?

Answer: Flora and fauna is threatened.

Question: What evidence do you have for this effect?

Answer: Increased populations under stress – rare bird population and some indigenous plants; increased levels of pesticides and chemicals in the water.

Question: What’s the negative effect that this will contribute to if this threat to flora and fauna is not addressed?

Answer: Reduction in local biodiversity.

The further down this chain the harder it is to isolate the effect to our problem. It is at this point that we may be able to better understand the benefit if we rectify the negative effect. This often draws out the expected benefit. For example, the benefit that will arise from reducing or eliminating the threat to flora and fauna might be… protection of local biodiversity as well as increasing the appeal of the park to visitors

By unpacking the problem using the problem trajectory technique, the group and facilitator have a:

clear view of the problem (both the cause and the effect); the contributors to the problem; the scale and scope of evidence that is available to prove the cause and effect; and the consequence of doing nothing.

This approach tests at an early stage whether there is evidence to substantiate the problem and whether the logic for the argument can be sustained. It also provides some efficiencies as each part of the trajectory can be used in the development of the ILM.

The cause and effect becomes the problem statement. The first order contributors, or sub-causes to the problem, may inform the potential interventions that could be considered. The next order of contributors may be changes and assets that could form part of the recommended solution. By addressing the negative impact or consequence outlined in the effect part of the problem statement positive outcomes are delivered and these are often the reverse of the impacts of doing nothing. These can become the benefits. Finally, the compelling evidence that supports the effect (e.g. % of flora and fauna exhibiting stress) can then become relevant and powerful KPIs for demonstrating benefit delivery. This is illustrated in Figure 5.

Figure 5: How the problem trajectory builds the investment story

During this step, the facilitator should encourage the group to explore the potential for uncertainty to impact on investment success. This includes considering whether any external factors, such as changes in demographics, economic conditions, technological


advancements, organisational behaviour or politics, could lead to a change in demand for action. The group should think about whether there is likely to be a change in the cause(s) or effect(s) of a problem.

The problem definition developed in this workshop sets the direction of how the business case is developed.. If there is any uncertainty in a problem definition – i.e. any possibility that the nature and/or extent of a problem may change – this needs to be pursued further in the business case.

The facilitator should note any uncertainties impacting an investment in the ILM.

If participants identify significant uncertainty, and the proposal is asset-related, they will likely require real options analysis to inform business case development. The options considered in the business case may need to provide a level of flexibility to change the course of an investment under certain circumstances, and this flexibility may need to be costed in the cost-benefit analysis. It will probably not be possible to adequately consider real options within the ILM, however the IMS should be used to identify those projects that will need real options analysis and highlight further work required outside the IMS process.

Participants should consider whether they undertake IMS workshops 2, 3 or 4 as inputs to real options analysis, or simply move to real options analysis at this point.

Step 4: Rank and weight the problem(s)

It is often useful to rank and then allocate weightings to the problems at this stage. It ensures the workshop participants continue to discuss only the most compelling problems.

In an ILM a total of 100 per cent is distributed within each of the problem, benefit and response columns. This distribution indicates the relative importance of the various elements within each column. To ensure clarity and ease of prioritisation, no items should have the same weighting within one column.

If a problem is rated less than 15%, the facilitator should challenge their inclusion in the ILM. The ILM is there to capture those things that really matter and eliminate the rest.

Step 5: Determine benefits and KPIs

This step focuses on determining and articulating the benefits to the organisation, enterprise or community that will be delivered because of this investment.

If you are having more than one workshop, the initial identification of potential benefits and KPIs, from the information that has been captured when unpacking the problem(s), will be as far as you will probably get in the Problem Definition workshop.

If you are only having one workshop and are aiming to complete the entire ILM, then you will be completing the benefits column and checking logical alignment with the problem(s) before moving on to the rest of the map.

Benefit statements should provide an obvious connection to the government’s or the organisation’s outcomes but be set in the context of the organisational or local impact. This important alignment between the high-level enterprise benefits and specific investment benefits is illustrated in the Benefit framework (Appendix 8).

A benefit is defined as the value the investment will provide to the organisation, its clients, customers or other stakeholders. Benefits are normally positive consequences of


successfully responding to the identified problem. Each benefit is supported by one or more KPIs. The practical application of this is described in the Benefit framework.

A benefit needs to pass three tests, that it:

has removed or mitigated the defined problem, specifically a cause or effect, and is aligned to the outcomes valued and articulated by the organisation;

is supported by up to two KPIs that are meaningful, measurable and attributable to the investment; and

is cost-effective, i.e. the effort required to track the benefit and the KPIs are commensurate with the value and insight that they provide.

A maximum of four benefits is allowed but typically two or three benefits are identified in an ILM. There should be no more than eight KPIs but four to six are typically sufficient.

It is critical that benefit statements include high-quality KPIs. A KPI is an indicator which, together with its associated measures and targets, provides the evidence that expected benefits have been delivered. It helps determine whether the target group/community is ‘better off’ because of the investment.

Ideally, KPIs should be outcome rather than output or activity focused. Where there is no practical way to measure an outcome, a proxy or indirect indicator may be used. These are often more output focused and should be used judiciously.

A good KPI is:

meaningful – it is a reasonable indicator that the benefit has been delivered. It is clear how the investment contributes to the benefit;

attributable – the investment is the most likely reason for a change in this indicator; and

measurable – there is an existing baseline and it is cost-effective to measure progress against this baseline. The indicator can be expressed as an increase, or decrease, that shows improvement over time against the baseline.

The single workshop approach will not generally allow sufficient time to identify measures, baselines and targets for the KPIs and complete a Benefit Management Plan, or equivalent. This will need to be done after the workshop and should be a relatively straightforward exercise for very low complexity investments, if the problem statements are strong and evidence-based. If not, a Benefit Definition workshop may be required (see Section 2.3).

Weightings

The benefits are weighted in the same way as the problems. If time is short, it is often useful to at least rank the benefits in the workshop. Make sure that you check, either in the


Key questions to test benefit statements What benefits will the organisation expect in successfully responding to the problem? What outcomes will we get from remedying this problem? What part of the government /organisational agenda will this investment support? What benefit will government, other funders, and citizens get from this investment? What KPIs will demonstrate delivery of the benefits? Is deriving these benefits a key priority for Government?

workshop or later, that the allocation of % from the problems to the benefits aligns logically with the weighting of the benefits. If this distribution of percentages does not work it usually means that either the problems or the benefits have not been weighted properly.

Logic flow

Overall, there should be a clear and evidence based logic between the problem statement (the effect or consequence part of the statement) and the benefit statement to which it is linked. Figure 6 demonstrates this relationship.

Figure 6: Problem to benefit relationship

Step 6: Determine the response and its interventions

The response column is the third column of the ILM. It is the preferred response to an identified problem(s) and is made up of between one and four interventions. You will complete this column during the Problem Definition workshop only if you have one workshop and are aiming to complete the ILM in one session.

An intervention is a high-level action that responds to a problem. It must:

deliver some of the KPIs in the benefit statement(s); allow for more than one possible solution; and be a valid and achievable response to the problem(s).

Start by saying something like: ‘the purpose of an intervention is to deliver some of the KPIs that have been identified. However, an intervention needs to remain sufficiently high level so as not to lock us into a specific solution.’ So, can we intervene by:

changing demand (e.g. measures to stop, prevent or reduce demand for services); improving productivity (e.g. get better at what we are doing and change service

delivery mechanisms); and/or changing supply (e.g. treat the problem and add capacity to meet increased demand)?

This workshop attempts to identify the range of interventions that can address the problem and deliver the desired benefits.

To help participants think more broadly than any preconceived ideas they may have of what a solution may look like, a structured series of questions can help draw out a fuller range of potential interventions. In an environment of constrained resources, it is critical that the range of interventions considered includes those that can manage demand for


services, as well as those that can improve the productivity of the existing services being delivered.

Workshop participants should be invited to first consider:

What options does the Government have to manage demand for a service? This could include user charging, rationing or queueing.

What options does the Government have to increase the productivity of the existing service? This could include options such as removing seats in trains, or traffic treatments that improve the flow of vehicles.

Once the viability of options to manage demand and improve the productivity of existing investments have been considered, if there is a residual requirement for further services or investment, then supply side options can be considered:

What options does the Government have to increase the level of service provision through new investment in assets or programs.

Capture all the first order contributors from the problem trajectory exercises you completed earlier – these are some possible interventions. Ask whether there are other interventions that could be considered. Direct the group to select the strongest combination of interventions from this list and connect them to the relevant benefits they relate to. A minimum of four and a maximum of eight possible interventions are likely as you try to determine the best response. Ultimately you will select a combination of one to four interventions from this list to form a response.

Weightings

The interventions are weighted in the same way as the problems and benefits. If time is short, it is often useful to at least rank the interventions in the workshop. Make sure that you check, either in the workshop or later, that the allocation of % from the benefits to the interventions aligns logically with the weighting of the interventions. If this does not work it usually means that either the benefits or the interventions have not been weighted properly.

If you have more than one workshop, then weightings will be discussed in a subsequent workshop.

Step 7: Outline the recommended solution - changes and assets

The items in the changes and assets columns in the ILM form the recommended solution. This is often the area that the group is most comfortable with. You will complete the changes and assets columns only if you have one workshop and are aiming to complete the ILM in one session.


Key questions to test the preferred response Does the response align with the purpose of our organisation(s)? Have we canvassed ways to change demand, improve productivity and change supply? Have we thought ‘outside the box’? Can we address any of the factors that contribute to the cause part of the problem

statement(s)? Will our intervention/s deliver on the KPIs?

Changes are the things that must be done by the organisation if the expected benefits are to be delivered. Changes provide the detail of how the response will be implemented. Assets are any physical item that must be acquired to enable the identified changes to occur. Examples of these are hospitals, pipelines, plants or technology (software and hardware). A change must align with at least one intervention. An asset must align with a least one change.

An initial list of changes and assets for the group to consider can be derived from the second order contributors from the problem trajectory exercises you completed earlier.

Most investments have between four and six changes and between zero and four assets. It is recommended to not go beyond this number as adding further changes and assets makes the map complex to understand and evaluate.

Weightings

Weighting are not required for changes and assets.

Step 8: Finalise the workshop

At the end of the workshop, try to tell the investment story by reading across the map and testing its logic flow, as illustrated in Figure 7.

Figure 7:Logic Flow across an ILM


Key questions to test the recommended solution What changes are needed to the way we operate? What do we need to do differently? What do we need to modify, build, buy or lease to implement our changes and therefore

contribute to our intervention? Do these changes align with our interventions? Are there any orphaned interventions? Will these changes and assets deliver the benefits and KPIs? Are there orphaned problem statements or benefits? Is this approach feasible?

To sum up, ask the participants:

Does our title and subtitle reflect the story that we have articulated on this map? Was this the story that we expected/wanted to tell? Does the map accurately reflect the

discussion we had? What outcome did we achieve at the end of this workshop? Validate the idea has merit;

go back and do more work before proceeding; or stop the investment. Is this the most compelling story for investment and is it validated by evidence? Do I have licence to do some word smithing to sharpen the statements?

2.6.5 Reiterate the feedback process

At the close of the workshop, advise participants that within 24 hours (or on the following day) they will be provided with ILM version 0.1. They will be asked to provide suggested changes within 24 hours and ensure they copy in the other participants. Advise participants that there is no point in circulating this draft to people who were not present in the workshop as their comments would lack context; they will be free to make comment when this iteration of the ILM has been completed. Within a further 24 hours you will update the ILM based on participant feedback and distribute version 1.0.

2.7 After the workshop

As with all workshops of the IMS, it is important to wrap-up the Problem Definition workshop without delay.

Within 24 hours:

download the current version of the ILM format from the Investment Management website and complete it with the data from your workshop (version 0.1);

within 24 hours of the workshop, send version 0.1 of the ILM to all participants. Often taking a photo of the whiteboard output (as you go) and circulating this ensures that participants have a copy of the original output prior to word smithing and tidying up by the facilitator;

use your storytelling and word smithing skills to create a Plain-English story that encapsulates the discussion;

make any observations you have of the draft ILM and suggest where its strengths and weaknesses are and how it might be improved. Include the list of issues and any interventions that have been gathered during the workshop;

send an email with version 0.1 of the ILM, the whiteboard output, and your observations directly to each of the participants at the session avoiding, if possible, channelling it through a single person. It is important that all participants are involved in finalising the discussion; and

in the email ask participants to provide feedback (and copy in all other participants) of any suggested changes within 24 hours and advise them you will make changes and provide them with version 1.0 within 24 hours of that time. All of this reinforces the ‘what do we think today’ principle that underpins this approach – see Sample email – after the Problem Definition workshop (Appendix 9).

The effectiveness of the ILM that is produced can be assessed using the Quality assessment form – Investment Logic Map (Appendix 10). The effectiveness of the facilitator can be assessed using the criteria in the Facilitator feedback form – Problem Definition workshop (Appendix 11). This latter form need not be completed each time a


Problem Definition workshop is held but are required for facilitator accreditation and re-accreditation.

Within 48 hours:

send all participants version 1.0 of the ILM; respond to any feedback that was provided in response to version 0.1. If no feedback

was received finalise this as you see fit; and suggest or confirm any agreed next step (e.g. Benefit Definition workshop).

Prior to the Benefit Definition workshop:

You should advise the investor about next steps including preparation for the Benefit Definition workshop, if this is planned. This should cover any additional work on evidence substantiation and KPI development needed to get the most value out of this next workshop. Further advice for facilitators on the preparation needed for the Benefit Definition workshop is provided in the next technical guide document in this series.

During business case development and beyond:

The Problem Definition workshop is largely designed to support the development of a preliminary business case. A full business case, particularly one prepared for a HVHR investment is likely to require completion of the rest of the workshops in the series and considerable work to validate the ILM, in line with Stage2: Prove guideline and the Sustainability Investment Guidelines.

The project team or business case writer should update all IMS documents to reflect the new analysis and understanding which emerges during business case development. This may change the description, weighting or measurement of benefits. If major changes occur, the ILM will need to be reviewed more comprehensively. The failure to update IMS documents has been regularly reported in Gateway Reviews and has often undermined confidence in the case for investment.

2.8 Templates, examples and other resources

The templates and examples are available for download at www.dtf.vic.gov.au/investmentmanagement.



Appendix 1: 16 Questions – Investment decision-maker’s checklistThe 16 questions (the Investment decision-maker’s checklist) are a set of prime questions that any decision-maker should have answered before funding an investment. The depth of enquiry for each question will depend on the scale and complexity of the investment. These questions can be asked, in part or in their entirety, at various stages in the investment management process to test the robustness of the IMS workshop document suite and the business case, if developed.

The 16 questions correlate with key elements of the Victorian government full business case template and aid business case writers and assessors.

The focus for the Problem Definition workshop is the first four questions.


Figure 8: 16 Questions - Investment decision maker's checklist


Appendix 2: Shaping new investments using the IMSThe Victorian Government’s Investment Management Standard (IMS) establishes a set of simple practices that enable organisations to select the investments that matter most and shape and implement them so they deliver the maximum benefit and best value for money. The practices can also be used to help prioritise investments, develop policy, evaluate programs and improve the effectiveness of an organisation.

The IMS supports a way of thinking characterised by evidence-based discussion, robust logic and simple storytelling. It brings together the best thinkers on a subject to discuss and shape new investments in two-hour facilitated discussions (workshops) to address the four questions that are fundamental to investment decision making:

The number of workshops required is determined by the nature of an investment. Large and complex investments might require four separate workshops that would produce four documents critical to establishing a sound business case. Small and simple investments might require just one or two workshops and would produce an Investment Logic Map (ILM) and a Benefit Management Plan. The four workshops are described below.

PROBLEM

Successful investments are made as a considered reaction to an identified or emerging problem. This workshop focuses on:

defining the problem that need to be addressed; validating that the problem is real; and specifying the benefits that will result from addressing the problem.

The output of this workshop is the first version of an Investment Logic Map (ILM) with the problems and benefits defined.

BENEFIT

Investments are often shaped with little understanding of the benefits expected to be produced. This workshop will:

identify the KPIs, measures, targets and timelines that the investment will need to deliver; and specify how the delivery of the benefits will be measured and reported.

The output of this workshop is a Benefit Management Plan (BMP) including a Benefit Map and Benefit Profile.

RESPONSE

Business cases for new investments often fail to consider the full range of things that could be done


PROBLEM

to address the identified problem. This workshop will:

explore the interventions that could deliver the expected benefits; formulate and evaluate a mix of response options; and assess response options and potentially select the preferred response.

The output of this workshop is a Response Options Analysis Report (ROAR).

SOLUTION

This workshop ensures that a solution is developed which is consistent with the foundations established in previous workshops. This workshop will:

confirm the preferred response and the interventions it contains; identify and evaluate the changes and assets that are required to implement the preferred

response and deliver the benefits; define a recommended solution; and identify cost range, timeframe for project and benefit delivery, key risks, uncertainties, dis-

benefits and critical assumptions associated with the recommended solution.

The output of this workshop is an Investment Concept Brief (ICB).


Appendix 3: Fictional - Investment Logic Map


Appendix 4: Design guidelines – Investment Logic Map The function of Investment Logic Maps (ILM) is to portray the logic underpinning a potential investment on a single page and in a form that can be quickly understood by decision-makers. Already several thousand ILMs have been developed. Based on this experience those things that work well have been identified and these are provided in this design guideline. Use this guidance to develop your ILM. They will help to ensure that your map is an effective communication tool that articulates the argument clearly, exposes the logic of your argument and identifies the value proposition inherent in this investment.

ILM Checkpoint Yes/No

The investment story is clear and easy to read and understand and the language is plain but precise

The title describes the most significant overarching business outcome or benefit that will be achieved, and the sub-heading gives an understanding of the solution

Problem statements contain a clear cause and effect relationship that is supported by evidence and provides insight into who and what is involved and why this is important

There are no more than four problems and no problem is weighted less than 15 per cent

Benefits align with the priorities of the organisation

There are no more than four benefits with a total weighting adding up to 100% and the distribution of % in respect of the problem weightings makes sense

KPIs are outcome focused and are meaningful, measurable and attributable to this investment

The preferred response contains a combination of interventions that deliver the KPIs and solves the problem and includes more than just supply-based interventions

There are no more than four interventions with total weighting adding up to 100 per cent and the distribution of % from one element to another makes sense

Changes are business changes and are linked to the interventions

There are no more than six changes

Assets are hard physical infrastructure and are linked to at least one change

The ILM template is current and at the correct level – Initiative, Program or Organisation


ILM Checkpoint Yes/No

Box and font size has not been altered

Control fields are completed and updated

Appendix 5: Sample email – before the Problem Definition workshop

[Greetings]

[something about the proposed investment]

As a way to consolidate our thinking about this proposed investment, I would like to invite you to attend a Problem Definition workshop at xxx on xxx.

This two-hour workshop has been designed to extract the compelling logic that underpins our proposed investment. The key output will be an [partial/completed] Investment Logic Map (ILM) which will ‘tell the investment story’ on a single page. A fictitious example is attached. The complexity of our investment will determine how many workshops are required. Each workshop will enhance the rigour that underpins our investment story. The ILM will be updated and other workshop products prepared from each subsequent workshop. An outline of this process is attached (See Shaping New Investments using the IMS)

The ILM that we create will continue to evolve and will form the basis for subsequent decision-making, support the development of future funding submissions, and, ultimately, be used to measure the success of the investment.

You are invited because of your experience and expertise in the area. All participants in this workshop will be subject matter experts who know most about the problems and their consequences, often from a broad range of perspectives. While no formal documents will be presented the participants must be very familiar with any existing relevant materials such as: briefing papers, background reports, external reviews, internal management reports etc., associated with the problem and potential need for investment. As a key part of the workshop is to confirm that evidence exists to substantiate the problems, the facilitator will assume that you bring your subject matter expertise and understanding of the issues.

The workshop will be facilitated by [facilitator’s name], who is an accredited investment management facilitator.

If you would like to read more about the benefits and practices of this approach you should refer to the DTF website at www.dtf.vic.gov.au/investmentmanagement.

[Sign off]



Appendix 6: Checklist – Problem Definition workshopThis is a high-level summary of the steps involved in a Problem Definition workshop. If more than one workshop is planned, then Steps 1-4 and ideally Step 5 must be completed during the Problem Definition workshop. If only one workshop is planned, then Steps 1-8 should be completed.


Appendix 7: Sample agenda – Problem Definition workshopProblem Definition workshop for a low-complexity investment (single workshop)

5 minutes Introduction and outline of purpose and role

15 minutes Investor overview and issues gathering

50-60 minutes Problem statements development

10 minutes Benefits and KPIs

10 minutes Preferred response - interventions

5 minutes Recommended solution – changes and assets

5 minutes Conclusion and next steps

Problem Definition workshop for a medium- or high-complexity investment (two to four workshops)

5 minutes Introduction and outline of purpose and role

20 minutes Investor overview and issues gathering

60–70 minutes Problem statements development

10–15minutes Benefits and KPIs (initial)

5 minutes Conclusion and next steps


Appendix 8: Benefit frameworkEach time an organisation makes an investment there is an expectation that some form of benefit will be returned. It therefore follows that the ability to design investment solutions that provide the maximum benefits, and to be able to confirm that these planned benefits are delivered, is critical to every organisation. To do this, agencies need to be able to articulate the benefits they will provide, define how they will be measured, or measure the benefits that are ultimately delivered?

Historically there has been an inability within large organisations to define how each individual investment contributes to the primary benefits that are the point of the organisation’s existence.

It is typical and appropriate that everyone within, say a police force, believes their individual investment will provide the benefit of ‘reduced crime’ and everyone in an education department will claim theirs will result in ‘better learning outcomes’. But there has generally been limited ability to describe the specific contribution an individual investment makes to, ‘reducing crime’ or achieving ‘better learning outcomes’.

The benefits framework that is illustrated has been found to be effective at addressing this longstanding problem.

The framework is a three-level structure that links the contribution of an individual investment to the outcomes the enterprise is seeking.

In the example depicted here, at the enterprise level, the government is seeking to create ‘friendly, confident and safe communities’. To this end, it sets benefits and targets that must be met at the organisation level – in this example, the police force is required to ‘reduce crime’.

At the investment level, it is necessary to demonstrate how a particular investment will contribute to the benefits sought by the organisation. In the example, the forensic science division is seeking an investment to improve forensic management practices, acquire state-of-art forensic software and to renew its aged computer system. In return for this investment it claims that it will ‘reduce crime’. The evidence to support this claim is a reduction in the time taken to provide forensic matches by 30 per cent, and to obtain 20 per cent more forensic matches.


The head of the police force is then left to determine:

how significant will meeting the targets associated with these KPIs be to a reduction in crime? would the claimed KPIs and their targets be directly attributable to the proposed investment? would the likely impact on crime reduction be worth the cost of the proposed investment?

This benefit framework is used as the basis for considering the validity of potential benefits during the development of ILMs.


Appendix 9: Sample email – after the Problem Definition workshop[Greeting]

Thanks for your participation at yesterday’s Problem Definition workshop for [investment name]. I have attached the ILM that we produced and ask that you provide me with any suggested amendments by close of business today.

What happens over the next 48 hours is just the conclusion of our discussion? As such, there is no need to circulate this draft to people who were not present in our discussion as their comments would lack context – they will be free to make comment when we have completed this iteration of the ILM

You should copy your responses to all other participants so we continue to understand each other’s perspectives. Based on your responses I will make any necessary changes and have version 1.0 to you by close of business tomorrow

… my observations

To assist your input, I make the following observations as to the relative strengths and weaknesses of the ILM we produced:

[for example]

Both the problem and the benefit one are generally strong – they speak well and there is strong evidence to support the drivers.

While benefit two would be powerful I am not sure that it can be supported by a KPI that is attributable to this investment. It probably needs to change to ‘[rewording]’ and could then use ‘[name of KPIs]’.

… about Investment Logic Maps

An ILM is never finalised. It is an evolving document that tells the story of an investment at any point in the investment lifecycle. Its strength is measured by its ability to be easily read by anyone who can then understand why an investment is being considered (or is underway). The reader should be able to suggest how it might be reshaped to deliver a better outcome.

If you would like to read more about the benefits and practices of this approach you should refer to the DTF website at www.dtf.vic.gov.au/investmentmanagement.

[Sign off]



Appendix 10: Quality assessment form – Investment Logic MapThe purpose of an ILM is to clearly and honestly communicate the case for an investment. Whether the case for the investment is weak or strong then becomes a matter of judgement for the reader. This form should be completed by a facilitator as part of the IMS Facilitator Accreditation Process.

There are five tests an ILM must pass to be considered of an acceptable standard:

Test 1: Could a layperson read and easily comprehend the story of this investment to the point where they could have some opinion of it?

Assessment:YES / NO / MAYBE

Please explain why you have assessed it this way.

Test 2: Is each problem a ‘call to action’ that conveys what is broken (both the cause and effect)?



Test 3: Is there a logical connection between the effect of the problem and the benefits and their KPIs?



Test 4: Is the preferred response one that:

is likely to deliver the expected benefits and KPIs; allows for more than one project option; and seems to be a valid response to the problem(s).



Test 5: Does the recommended solution read as a set of logical and sensible business changes and assets that need to be undertaken to adequately deliver the preferred response?



How would you rate this ILM?Assessment:

SATISFACTORY/ UNSATISFACTORY



Appendix 11: Facilitator feedback form – Problem Definition workshopCritical to the success of the Investment Management Standard is the ability to have an intelligent facilitated discussion focused on the logic that underpins a potential investment. The capabilities of the facilitator are key to this discussion.

You have just completed a Problem Definition workshop and we would like your feedback on how well the workshop was facilitated. This form is required as part of the IMS Facilitator Accreditation Process.

How would you rate the truth contained within the following statements on a scale of 1 to 5?

SCALEFALSE (1) – TRUE (5)

At the commencement all participants were given a clear understanding of the role of the facilitator and the outcomes sought by the discussion.

The opinions of the key participants were obtained and properly considered.

The difficult questions that were pivotal to the success of this investment were identified and properly addressed.

Hard evidence was sought to validate each statement of the investment story.

The logic flow of the ILM was considered and tested.

The discussion concluded with the agreement of participants that the investment story that was documented was consistent with the discussion.

The workshop was completed within two hours. Yes / NoIf no, for how long did it run?

(__ hrs, __mins)

Date of the workshop:

Department / organisation:

Size of investment: Less than $500k $500k – $10m Above $10m

Your name:

Your position:

Would you be happy to be contacted by other ‘investors’ seeking further information on the capability of this facilitator?

Yes / No


purpose of this document - dtf.vic.gov.au · web viewin the case of an investment that only...

Documents