it advisory miami dade county public...

IT ADVISORY IT ADVISORY

KPMG LLPKPMG LLP

Miami Dade County Public SchoolsMarch 2009 ERP Project Planning and Resource Assessment May 15, 2009

Miami Dade County Public SchoolsMarch 2009 ERP Project Planning and Resource AssessmentMay 15, 2009

2© 2009 KPMG LLP, a U.S. limited liability partnership and a member firm of the KPMG network of independent member firms affiliated with KPMG International, a Swiss cooperative. All rights reserved. Printed in the U.S.A. 1461$MIA

2

MIAMI-DADE COUNTY SCHOOL BOARD

Table of Contents

Scope of KPMG’s AssessmentApproachExecutive SummaryAssessment Results

Project PlanningProject Staffing

Recommended Next StepsAppendix –

KPMG’s Review of M-DCPS Management Response


3


Scope of KPMG’s Assessment

On March 12, 2009, Miami-Dade County Public Schools (M-DCPS or District) agreed to an addendum to the Agreement with KPMG LLP (KPMG) dated July 15, 2008 that would continue KPMG’s point-in-time assessment of M-DCPS’s ERP project during the month of March, 2009. KPMG’s assessment provides observations and recommendations based on the review of the project related documents as well as interviews with key ERP project team members and stakeholders. Areas of focus included:

Revised M-DCPS ERP project plan as of March 26, 2009 ERP project resource plan.

KPMG assessed the structure and responsibilities of the ERP project team (both internal resources and third party consultants) as compared to teams deployed for SAP Finance implementations at other entities of comparable size and complexity and assessed the experience and skills of key team members.

KPMG focused primarily on the Finance portion of the ERP implementation planned for go-live in early 2010. This report constitutes KPMG’s observations and recommendations as a result of the field work conducted from March 24, 2009 through March 31, 2009.

UPDATE May 06, 2009: M-DCPS agreed to have KPMG review M-DCPS management’s response to this assessment. See Appendix –



4


Approach

KPMG’s assessment was based upon the following project team member interviews:

# Meeting / Interview Date M-DCPS Participant(s)

1 M-DCPS ERP Management Team (Kick-Off) 3/24/2009

Vera Hirsh – Assistant Superintendent HR/Project Co-sponsorDebbie Karcher – CIO/Project ManagerCraig Rinehart – Administrative DirectorNick Di Liello – Development / Functional TeamsRudy Rodriguez – Controller/Finance Procurement Release LeadDebra Graper – Change and Organization ReadinessVictor Diorio – Executive Director/DevelopmentJames O’Donnell – Security & ControlsLinda Jones – HR Release LeadWilliam Howell – PMOSharon Dixon – PMOJorge Fernandez – PMO

2 Functional Teams Meeting 3/24/2009

Craig Rinehart – Administrative DirectorRudy Rodriguez – Controller/Finance Procurement Release LeadLinda Jones – HR Release LeadVictor Diorio – Executive Director/Development

3 Basis Meeting 3/24/2009 Craig Rinehart – Administrative Director/Technology Infrastructure

4 Co-Sponsor Meeting 3/25/2009 Judy Marte – Chief Budget Officer/Project Co-sponsor

5 Finance Meeting 3/25/2009 Joanne Koski – Procurement SMEJany Marchana – GL/CO/Reporting SME


5


Approach (Continued…)

KPMG’s assessment was based upon the following project team member interviews:

# Meeting / Interview Date M-DCPS Participant(s)

6 PMO Meeting 3/26/2009

Jorge Fernandez – PMOWilliam Howell – PMOSharon Dixon – PMOJennifer Steven – PMO

7 Development Team Meeting 3/26/2009Victor Diorio – Executive Director/DevelopmentNick Di Liello – Development / Functional TeamsEddie Hernandez – Portal Developer

8 Deployment & Testing Team Meeting 3/26/2009

Debra Graper – Change and Organization ReadinessWilliam Howell – PMOJorge Fernandez – PMOJennifer Steven – PMO

9 Security Team Meeting 3/26/2009 James O’Donnell – Security & ControlsLarry Rubio – Security & Controls GRC

10 Sponsor Meeting 3/27/2009 Dr. Richard Hinds – CFO / Project Sponsor

11 HR Meeting 3/27/2009Maria Montoya – e-Recruiting SMESamuel Pongnon – Organizational Management SMEEverardo Verguizas - Personal Administration SME

12 Object Manager Walkthrough 3/27/2009 William Howell – PMO

13 Thread Manager Walkthrough 3/27/2009 William Howell – PMO

14 MDCPS ERP Portal Walkthrough 3/27/2009 William Howell – PMO

15 Co-Sponsor Meeting 3/31/2009 Vera Hirsh – Assistant Superintendent HR/Project Co-sponsor


6


Approach (Continued…)

KPMG’s assessment was also based upon reviewing the following ERP project documentation:

# Description Document Name

1 Industry Print Finance rationalized with MDCPS requirements IP Flows.docx

2 List of Custom Development Objects - Release 1.0 ERPObjects.xls

3 Risks, Issues, Activities and Opportunities by Process Team Area StatRep20mar09_wdh.xlsx

4 Deliverables by Process Team by Priority MDCPS Transition Checklist Final.xls

5 ERP FI/CO & HR Release 1.0 - Team Structure ERP_Team_Structurev3.pdf

6 Application Development Deliverables List ERP Checklist Transition Appl_Devel_v4.xls

7 Consultants Hiring Process - Checklist consultant_process.docx

8 ERP Status Report - Feb 2009 AMC_M773_-_ERP_Status_Report_for_Feb_2009.pdf

9 HR Release 1.0 - e-Recruiting Project Scope Document - Final eRecruiting Project Scope Final.doc

10 ERP Master Project Plan M-DCPS SAP ERP Masterv1 3_WDH_Edited.mpp

11 SAP Development Landscape Hard Copy

12 Object Manager - Walkthrough N/A

13 Thread Manager - Walkthrough N/A

14 MDCPS ERP Portal - Walkthrough N/A


7


Executive Summary

KPMG recognizes improvements in the ERP project scope, timeline,

and staffing as revised in February 2009, in that:

Finance was re-introduced as the first priority, becoming the foundation for the ERP implementation;

The chosen phased implementation approach will allow the District to adapt to the new capabilities of SAP and adopt the necessary business process changes in more manageable intervals;

The project team is staffed with District personnel with the appropriate institutional knowledge that was lacking in the prior team structure;

The project re-plan considers the District’s financial organization’s responsibilities and commitments so project activities should not conflict with scheduled calendar events such as financial closing and the

annual budget cycle.

The current market conditions have sped up the contractor and consultant on-boarding process and allow the District to recruit from a talent-rich resource pool.


8


Executive Summary (Continued…)

However, key areas of concern remain per the March Assessment: Insufficient time is planned for integration testing and break fix before the Finance go-live. Testing typically reveals complexities not previously recognized.

The full complexity of the implementation remains unclear because:the Finance configuration and development object specifications have not been completely finalized as of 3/31/2009;as of 3/26/2009, the details of each project task are not centrally managed in a single project plan with timely updates and necessary detail (e.g., some objects explicitly captured in master project plan while complete list is maintained in Object Manager);


9



The principal project plan’s lack of detailed activities more than three months out may contribute to resource constraints because:

The District has budgetary constraints that may challenge business process owners in staying fully engaged and available during traditionally demanding phases of the project like Integration Testing and Cutover. The District, however, has taken steps to improve the scheduling of subject matter experts;

this effort will need to continue through go-live.

Heavy reliance on shared resources may lead to bottlenecks and potentially shift more of the project work to the project consultants and contractors, which in turn could impact the District SAP learning curve, project schedule and total cost of ownership.


10



Projects are managed with “Four Levers”

of control:QualityScopeScheduleBudget (People, Dollars)

Our understanding of current status:Scope, Schedule, and Budget (by moving to lower cost consulting resources) levers have been adjusted to-date;Revised budget and schedule must

be maintained.

IMPLICATION: Quality and Scope are the only levers left to “pull”

Intermittent oversight will help promote transparency to the Board, but on-

going

diligence will be required to achieve Scope, Schedule, and Budget goals without sacrificing Quality.

Quality should be the only immovable lever


11


Executive Summary –

Appendix Summary

KPMG discussed these concerns with M-DCPS management in order to share our findings and allow for a formal response that documents any corrective actions or alternative perspectives.

KPMG’s review of management’s response has been included in this report. See Appendix –


KPMG recommends that this assessment be evaluated in conjunction

with management’s response.

12

Assessment Results

-

Project Planning

-

Project Staffing


13


Assessment Results -

Project Planning

KPMG team met with the Project Management Office (PMO), Finance, Human Resources (HR), Security & Controls, Development, Change & Organizational Readiness and Technology teams to discuss the scope of the project and timelines. The Finance scope includes SAP Purchasing, General Ledger, Accounts Receivable, Accounts Payable, Cash and Treasury, Inventory, Funds Management and Grants Management. The go- live is planned for January 2010. Additional scope including SAP e-Recruiting, SAP Organization Management and SAP Mini Personal Administration is planned for a go-live date of November 2009. Additional HR functionality is scheduled for release in July 2010 with Time, Payroll, and Benefits scheduled for January 2011.

In early 2009, M-DCPS re-assessed the ERP project approach and modified the Deloitte project plan by incorporating both SAP’s ASAP methodology and the District’s existing large project implementation methodology. At its core, the new project plan is encouraging in that it establishes the foundation of M-DCPS’ new ERP system by re-introducing Finance prior to major HR functionality and Payroll. This phased approach also helps to manage risk by allowing the District to adapt to new technology and capabilities. Overall, the new project plan takes the high level business needs and cycles of the District into account, incorporating key subject matter professionals into the plan at times when they can be most available to the ERP team.

This project plan is being managed to the task level without complete resource allocation; therefore, no direct correlation to the project financials exists. KPMG recognizes the challenges of maintaining a master project plan for an implementation of this size and complexity. Furthermore, informal communications and real-time team collaboration often contain details that are more current than what is formally captured in a project plan. However, day-to-day focus on detailed activities does not substitute for detailed project planning that extends out through post-production support and takes into account the impact of project changes on subsequent activities, timing and effort.

Background


14



Project Planning (Continued…)

• The project plan does not evidence timely update nor sufficient detail to evaluate the true health of the project:

• 70 out of 570 tasks due on or before 3/26/2009 showed as past due, of which 68 were showing over 10 days past due. The project team was actively updating the status of these tasks during our assessment period.

• The project plan is not loaded with resources nor their assignment to the individual teams (% of time);

• Process team timelines are identified at a very high level and it is not clear how teams are managing the work effort at a detailed level within each team.

• The project plan is missing key elements (e.g., detailed critical milestones; OM, and mini-PA activities/milestones; cutover activities; performance testing timelines).

• Activities identified in the project plan lack detail as it progresses further out in time:

• The project plan is consistent with our observations during interviews that the project team is focusing on immediate milestones and tasks, with limited regard for future planning.

• Functional teams are overly reliant on the PMO to look out for the big picture which may result in presently unknown resource constraints and unidentified project tasks.

• Project complexity may be underestimated, thereby increasing the risk for project delays and compromises in scope or quality.

• The scope of activities for functionality scheduled prior to January 2010 (Finance and e-Recruiting) configuration— baseline/ final—is not completely captured in the plan nor is the list of development objects finalized (planned for 4/17/2009)

• Establish recurring meetings between individual teams and the PMO with the expressed objective to update the project plan with new and updated activities. These meetings should be held weekly so dependencies and conflicts can be identified and resolved timely;

• Additionally:

• Any activity that has a 8 hour (1 man day) effort or greater should be explicitly captured and monitored;

• Plan and capture future activities with the same level of detail and planning as near-term activities;

• Establish additional team level project plans to help capture work effort and team level progress in greater detail. If teams are tracking detailed progress on a ‘plan vs. actual’ basis, ensure that each team is reporting uniformly so they roll up to the master project plan consistently;

• Project plan updates should be viewed as a critical project activity with the same priority as configuration, development, and testing.

• While e-Recruiting is being considered as a different release from Finance, key milestones for e-Recruiting should be included in the Finance project plan so risks of interdependencies and potential resource conflicts can be well understood.

RecommendationsObservations

KPMG recognizes that the project plan alone doesn’t constitute the sole form of planning and communication on a project of this size and complexity. However, the master project plan should be kept up-to-date and incorporate other project tools consistently at all times to allow for independent oversight, but more importantly, sufficient internal team communication.


15



Project Planning (Continued…)

• Integration Test timeline for Finance (35 days) appears to be underestimated given the project scope.

• The release of e-Recruiting coincides with the Integration Test schedule for the Finance release.

• Plan to actual cost allocations are not tracked in the project plan.

• A variety of project management tools are in use by the project team (Solution Manager, Thread Manager, Object Manager, ERP Portal).

• Assume additional time will be required to complete a thorough Integration Test so that all key functionality and integration points are adequately tested and retested once fixes are applied. Despite early testing efforts (unit testing), a significant portion of the data conversion and interface errors go undetected until a thorough Integration test is performed. Overlapping Finance Integration Testing with the e- Recruitment launch (regardless of complexity) adds additional risk to maintaining the schedule and ensuring the quality of testing for the Finance modules.

• Establish a formal reporting mechanism that links project progress (typically monitored in the project plan) with the project budget in a manner that can be independently evaluated.

• KPMG recognizes that the project team plans to consolidate project management tools for the HR and Payroll releases. However, an expanded use of Solution Manager’s capabilities should be explored as soon as possible for the e-Recruiting and Finance releases. Solution Manager is SAP’s strategic toolset for on-going SAP maintenance and support.



16



Project Staffing

With the departure of Deloitte Consulting, M-DCPS has staffed its ERP project with a combination of District, contractor and consulting resources. M-DCPS has integrated into the ERP project Information Technology Services (ITS), subject matter experts (SME) and business process owners (BPO) from across the District. The project team relocated from an off-site location to the District’s Administrative offices, placing them closer to end-users and each other. This integration of key District resources, in conjunction with the physical move, have had a positive impact on team members’ stated commitment and optimism regarding a successful ERP implementation.

Additionally, external resources have been contracted: the Finance Support Team (FST) and the Business Analyst (BA) Team. The BA’s are primarily dedicated to functional teams according to staffing needs. FST resources are provided by Top Resource Inc. to provide SAP consulting expertise to the PMO and process teams on an as-needed basis. The FST’s role is designed to be strategic rather than tactical so that District team members execute the actual implementation tasks, thus promoting knowledge transfer.

The on-boarding process has also greatly improved largely due to the current economic conditions. Contractors have been brought into the District as quickly as within one week from posting and supplement the District’s team where specific SAP skills are required. The current economy provides for a large talent-rich pool that allows the District to identify and staff very specific technical skills in a timely manner.

Background


• BAs and FSTs are writing functional specifications and actually performing and documenting the SAP configuration, despite the intention that FSTs function as strategic resources.

• The project team may be under-estimating potential staffing needs, specifically due to under-estimated activities in the Security and Control areas which may expose the District to an over simplification of the security and controls model increasing control and audit risk.

• The project team, executive leadership and the Board should acknowledge that using consulting and contractor resources in tactical tasks to avoid project delays increases the reliance on external support and accordingly, total cost of ownership. Knowledge transfer is reduced when District resources do not perform these key functions, thereby increasing the dependency on consultants and contractors long-term. While this is not an inherently bad practice, the risk should be understood by executive leadership and the Board.

• Perform an external and internal staffing needs assessment for each Security and Control task (e.g., the design, development, test of controls; role definitions; role assignments; segregation of duty analysis; remediation; etc.).


17



Project Staffing (Continued…)

• Shared resources and other factors may cause resource constraints during peak periods of the project.

• The District has budgetary constraints that may challenge BPOs in staying fully engaged and available during traditionally demanding phases of the project like Integration Testing and Cutover due to other pressing responsibilities, such as the year-end close. The District, however, has taken steps to improve the scheduling of SMEs; this effort will need to continue through go-live.

• Current project planning practices may exacerbate the risk:

• lack of detailed project planning for activities more than three months out;

• the project plan is not loaded with resources nor their assignment to the individual teams (% of time);

• process team timelines are at a high level and not clear on managing the work effort within each team.

• The deployment of e-Recruiting coincides with the existing Integration Test schedule for the Finance release.

• The Development team is investigating off-shore resources as an accelerant plan, but common project risks such as poor communication and lack of requirement clarity associated with off-shore resources could delay development of custom objects and potentially impact the quality of the final product.

• The project schedule calls for high resource demands during traditional holiday and vacation schedule periods (November through January). Finance and Payroll Integration Test and Cutover activities span this period.

• The project team should conduct a thorough staffing assessment after the project plan has been updated with sufficient detail and resource loading. Key areas of assessment include the:

• confirmation of BPO availability for the critical phases of the project;

• allocation of adequate contingency funds for potential staff and contractor overtime as well as the addition of currently unplanned resources;

• minimization of shared resources. At the very least, shared resources shouldn’t be planned for more than 80% total time allocation to account for the overhead of multi-tasking.

• Adjust the project plan where resource constraints leave no alternative.

• An off-shore resource model is a common accelerant and/or contingency plan used to address delays and budget overruns. As a part of the investigation of off-shore resourcing, the District should test pilot a few low complexity development objects with off-shore providers. This will allow the project team to “test” the plan so realistic estimates can be made regarding the quality, cost, and speed of service.

• The project team has acknowledged the project calendar and is staffed with motivated team members who have committed to the work at hand. Nonetheless, leadership and the Board should recognize that all projects of this size and complexity will warrant overtime and burnout is an unavoidable reality that may impact quality and timelines. Timely recognition of extraordinary efforts should be a priority to mitigate the effects of burnout.


18

Recommended Next Steps


19


Recommended Next Steps

While future point-in-time assessments may promote a degree of transparency, the aggressive project timeline will significantly

limit the potential for an oversight role performed on a point-in-time basis to help mitigate or avoid project risks.

KPMG recommends engaging on-going oversight to help identify mitigation strategies on a timely basis throughout implementation and post-production support.

20

Appendix



21


Scope and Approach of KPMG’s Review Over M-DCPS Management Response

On May 06, 2009, M-DCPS and KPMG entered into an agreement to perform and document a review of M-DCPS’s management response to the assessment results documented in the body of this report. KPMG reviewed management’s response and related attachments (ERP project plan as of May 08, 2009).

KPMG’s review focused on management’s responses and the mitigation plans/corrective actions taken to address the assessment results.

KPMG did not re-perform the review of the project plan in its entirety but only as it related to M-DCPS’

formal management response.


22



KPMG reviewed M-DCPS management’s response and found that it covered all key points and provided supplemental detail and clarity for additional risk management activities employed by the project team and District management.

Management’s response can be categorized into two groups:Supplemental information that describes management’s position regarding risks and issues that they believe are appropriately mitigated through existing project practices.

Acknowledgement of KPMG’s observations and corrective measures that the District has taken or will take in order to appropriately mitigate the risk.


23


KPMG’s Review of M-DCPS Management Response (Continued…)

In regard to supplemental information, KPMG reviewed management’s response and found that it is consistent with KPMG’s inquiry of project management; and if appropriately implemented, could reduce risk of project delay and overruns.

For the remaining responses, KPMG noted the following additional

corrective actions:

The PMO group will work with the process team leads to extract activities at a granular level into a sub-plan for management by each process team lead.M-DCPS is initiating mitigation plans to address parallel activities regarding the e-Recruiting implementation and Finance integration testing. Key considerations include eRecruiting

development/testing in a separate environment and utilizing a separate resource pool for eRecruiting

testing and rollout.


24


KPMG’s Review of M-DCPS Management Response (Continued…)

Additional corrective actions continued…M-DCPS will develop an action plan to accelerate the conversion to

a sole source tool set (Solution Manager) for maintenance, configuration and management of the SAP application. The District will be conducting a staffing needs assessment to assist in assessing the workload over security and control.Should the District go forward with utilizing off-shore resources as an accelerant, the District will fashion a pilot program to validate the quality, timelines and scope of the objects.

KPMG inspected the ERP Project Plan updated as of May 8, 2009 and inquired of project management to confirm that the corrective measures were either reflected in the project plan or acknowledged by project management and project sponsorship as additional project procedures going forward. If appropriately implemented, these corrective measures could reduce implementation risk.

it advisory miami dade county public...

Documents