Session 908 - Mapping Out A Plan: Accurately Estimating ... · Plan: Accurately Estimating Your Project Time Requirements ... The first thing we needed to do was collect some time
17
Session 908 - Mapping Out A Plan: Accurately Estimating Your Project Time Requirements Susan Prichard, Edward Jones Orlando, FL • March 22 – 24, 2017
Session 908 - Mapping Out A Plan: Accurately Estimating Your
Project Time RequirementsSusan Prichard, Edward Jones
Orlando, FL • March 22 – 24, 2017
Mapping Out a PlanSusan Prichard and Nathan WindeknechtOperations & Service Training
The Solve Uncertain projects deadlines Additional resource requests Partial data and no accountability
What are we solving for?
Presenter
Presentation Notes
Thank you for the opportunity to present today. Edward Jones is a fortune 500 financial service firm. We are privately held and tend to develop a lot of our own systems and tools. This is great benefit in some regards, however, one challenge it has presented for us is not having the benefit of Project Management Software and Tools. Early in 2016, we had several problems converging at once. Our business was getting ready to go through significant regulatory change, we were hiring at an increasing rate, and system enhancements were continuing to challenge the relevance of our training. We had more projects coming our way than ever before! We had just started to utilize an intake and analysis process, although it was still relatively ad hoc and our decisions were not rooted in data. Projects were taking longer than they should. Well, at least we thought they were. We really didn't know for sure how long they should take. This led to our stakeholders being frustrated, they didn't have a clear picture of how long a project should take to be done. When pressed, we would provide a timeframe, but it was just a ball park guess at best. Generally, it was overly optimistic. While it was understandable that our stakeholders were frustrated with us, our designers were also frustrated with them. They would miss deadlines for reviews or important meetings. This would contribute to projects being delayed. We thought we needed additional resources. It is difficult to gain headcount in our firm and it can be a rigorous process. We didn't have factual statistics to support our resource request. Our project leaders weren't able to be held to accountable as much as we would like because we didn't have a way to measure their ability to keep projects on time and under budget. We didn't have a visible metric that the department could rally around as a shared goal.
Data Collection
Type of Training per 1 hour
Low Hours
Per hour of Instruction
(2009)
High Hours
Per Hour of Instruction
(2009)
Low Hours
Per hour of Instruction
(2003)
High Hours
Per Hour of Instruction
(2003)
Stand-up training (classroom) 43 185 20 70
Self-instructional print 40 93 80 125
Instructor-led, Web-based training delivery (using software such as Centra, Adobe Connect, or WebEx-two-way live audio with PowerPoint)
49 89 30 80
E-learning Developed without a Template
Text-only; limited interactivity; no animations 93 152 100 150
High interactivity; multiple animations 154 243 400 600
Presenter
Presentation Notes
The first thing we needed to do was collect some time tracking data. We had asked our designers to track their time for ages, but it never seemed to produce the valuable data that we needed to be able to accurately estimate project timelines. We tried to follow the industry standards of tracking time for ILTs or OLTs, but 80% of our courses are blended, so it was difficult to pin down just how much of our time was dedicated to working on the ILT portions of a course versus the OLT portions.
Intake and analysisProject managementDesign Online module
PowerPointRecorded CallRecorded demoTest/quizJob aid
Task Solution Development
Metrics Breakdown
Presenter
Presentation Notes
By trial and error, we finally landed on a solution: Track by the [CLICK] name of the designer and project, the type of [CLICK] task (intake and analysis, project management, design, development, and implementation), the type of [CLICK] solution (Blended learning plan, bundle release, OLM, on-the-job solution, OLM maintenance, and written maintenance), and finally when tracking in the development task we would tag [CLICK] with what type of asset was being developed (coaching guide, facilitator guide, job aid, learning journal, online module, participant guide, PowerPoint, recorded call, recorded demo, test/quiz).
Data Collection
COMMUNICATING THE WHY
SETTING EXPECTATIONS
MAINTAINING MOMENTUM
TRACKING AND ANALYSIS
The Process
Presenter
Presentation Notes
I wanted our data to be accurate. The worse thing than having no data is having bad data! So, how would I convince my team to take the time to be very specific about their time tracking? I was asking them to track to the minute! This was a lot to ask a large team of very independent associates. I can still remember the team meeting – we were all gathered together and I had to somehow get my team to see the value in tracking their time. I had to convince them that I wasn't micromanaging them even though it felt like I was micromanaging them. I needed accurate data, so I had to convince the team that they wouldn't be in trouble if their projects lasted longer than we expected. I started with WHY. [CLICK WHY] Our big why was that my team expressed in surveys that they were expected to do too many projects in too short of time. They provided feedback that we needed to hire more designers to support the business areas appropriately. They had provided more feedback that said they felt some team members were doing their fair share, while others didn't. I promised my team that by tracking their time, we could set a level playing field for all associates. We could provide accurate estimates to our business areas and not overpromise anymore. We could use this data to determine if we truly needed to hire more designers or take on less projects. I promised the team POWER! [CLICK EXPECTATIONS] What I didn't ask for was an accounting of their whole day. I only wanted time related to project work. I didn't care about team meetings, vacation, sick, bathroom breaks, people breaks, etc. I said I only expected about 85% of their 40 hour week tracked. I was not looking for a 40 hour week in project work from anyone. In fact, if that did occur, we needed to discuss it and make adjustments. That was one of my heaviest meetings and yet it was a turning point for my team. In addition to the time tracking, we started daily huddles [CLICK MAINTAINING]. These are 15 minute meetings at the beginning of each day. Each person gets a minute to share what they are working on and if they need any help or if they can help anyone else. This provided transparency across my team so that everyone knew what everyone else was working on and I could tell if any projects were going off track. Each week, [CLICK TRACKING] I would look at the time tracking data to make sure the associates were keeping up with their tracking. I would also look for abnormal numbers, like someone working longer hours than I would expect. We tracked for 9 months before we started to analyze the data.
Comparative Time Estimate
An estimate based on other project estimates
SolutionCapacity Estimate
(hours)Stakeholder Impact
Estimate (days)Blended Learning Plan – New or Rewrite 345 Hours/ 3-day LP 16 Weeks
Bundle Release 65 Hours 12 Weeks
On the Job Solution 45 Hours 6 Weeks
Online Content Maintenance 22 Hours 8 Weeks
Written Content Maintenance 40 Hours 8 Weeks
Presenter
Presentation Notes
A comparative estimate is based on other project estimates. It is reasonable to assume that the current project will take about the same about of time as similar projects. Few projects are exactly the same size and complexity, so the estimate must be adjusted as the design is completed. We looked at different types of project that we have done in the past, broke them down into solutions, then used our data to gain capacity estimates vs. stakeholder impact estimates. In our department, we are concerned with how many hours will this project likely take us. That allows us to understand how many hours we could dedicate our project leads and designers to project. From a resource planning standpoint, this is beneficial. Our stakeholders aren't concerned about our capacity and staffing planning, they want to know when their project will be done. Period! We also utilized our historical data to show us days to complete a project. This helped us paint an accurate picture for our stakeholders. The analogy that we often use is if you are a speculating on a piece of real estate that you are looking to rehab, you may drive by or do a walk through to get an idea of what exactly you are getting into. If you have done enough work, you have a general idea of how long something will take to get done. If you are utilizing your own labor, you will want to understand the labor cost based on hours. This helps you plan the cost. You also may want to consider when the house may be able to be listed on the market. That could be more where your timeframe for stakeholder impact comes into play. This is type of estimate is most often used to help in our intake and analysis, set stakeholder expectations, and initial target for our project lead. It was also instrumental in making our case for additional resources. It allowed us to demonstrate what the business was asking of us, how long it would take to complete with our current department, and set expectations for how much more additional resources would allow us to accomplish.
Bottom-Up Time EstimateIdentifying time to complete each of the detailed solutions and tasks, then summing them to determine the total time.
Format Unit DescriptionCapacity
Hours/Unit
Job Aid Five page written text and screenshots outlining major transactions/steps
20
Facilitator/Participant Guide
One "playbook" with facilitation/speaking points to accompany training presentation per one-day learning plan
60
Coaching Guide 20 pages of written direction for an on-the-job coach to deliver 30
Classroom PowerPoint One training presentation to accompany a one-day learning plan 20
Recorded Demo/Call One recorded on-screen demo (~five minutes) of specific task / transaction, made with Captivate, Replay, or similar tool; assumes job aid documentation exists; or a recorded call for a call library
10
Online Module One application-focused module with practice and learner interactivity
90
Test/Assessment 20 questions covering material found in a learning plan 8
Presenter
Presentation Notes
The most accurate method of estimating the time to complete a project. Upon completion of design, each solution and task is assigned a time to complete. Finding the time to complete each of the detailed solutions and tasks, then summing them to determine the total time is "bottom up" estimating. Sticking with the real estate analogy, rehabbing a house rarely goes exactly as another project of the same type. You can get a general idea by utilizing a comparative estimate, but the real idea comes in better when you are designing the floor plan, knocking out walls and seeing more detail into the project. We take a similar approach during our design phase. As the various types solutions, formats, and delivery options are identified they are assigned a more specific detailed number of hours. This is utilized to set our actual number that our project leads are held accountable to, rolls us to our aggregate metric, and continues to be analyzed for accuracy.
Results
Intake and Analysis
10%
Project Management
10%
Design10%
Implementation10%
Development60%
Time Estimate
Alignment > 85%
Presenter
Presentation Notes
The results! On April 13, 2016, we put the stake in the ground and presented our finding to my team. We used the historical data to show that we typically spend 10% of our time in each of the following tasks: Intake and Analysis, Project Management, Design, and Implementation. We spend 60% of our project time in development. Using our comparative and bottom up estimates, we will now hold each other accountable to a time estimate alignment of 85%. We said that in any given project you have 6 times to hit the target time estimate. This means they could completely blow it on one task and still be on target for the measure. While 85% was a bit of a stretch goal, since our data pointed to 80%, the team felt that this was completely doable because the historical data consisted of a large scale project that went way off track. (It had to be rewritten a number of times and there were all sorts of issues.)
Delivery Dependencies
What we can't control Outside reviews Stakeholder approvals Compliance and legal approvals
JANUARY
6
13
20
27
7
14
21
28
1
8
15
22
29
2
9
16
23
30
3
10
17
24
31
4
11
18
25
5
12
19
26
SUN MON TUE WED THUR FRI SAT
Presenter
Presentation Notes
We would only hold ourselves accountable to what we can control. This means we do not hold ourselves accountable to the deadline estimate. There are too many outside reviews and approvals that need to occur in our projects that we do not have control over. This usually causes our projects to go over their deadline. What I did want to hold ourselves accountable to was what we could control. This is how much time we as designers spend on a project. This is important, because with this information, I could now accurately plan resources and projects. It no longer had to be a best guess. It now had the science to the art.
Additional resource requests.
Partial data and no accountability.
Uncertain project deadlines.
RESULT:Clearer definition of what can be delivered and when.
RESULT:Data supported the business case to bring on more resources.
RESULT:• 6 of 9 months we've hit our goal!• Team members are accountable
to their goals.• We can more easily identify and
communicate trade-offs to stakeholders.
Going back to the "Why"
Presenter
Presentation Notes
The results we have seen because of this: It goes back to our WHY! My team expressed in surveys that they were expected to do too many projects in too short of time. We now knew how long projects should take. We realized that while a job aid may only take 20 designer hours, it typically still takes 8 weeks to get through all of the reviews and approvals. We now have a clearer definition of what we can deliver and in what time frame to stakeholders. (Side note for another discussion – because of this data, we have been able to show our stakeholders how long their reviews take, and have been able to shore up that time in some cases to 1 week!) They provided feedback that we needed to hire more designers to support the business areas appropriately. We were able to make a business case and receive through data to get a full time resource and 2 additional contractor resources to support the number of business areas required by my team. They had provided more feedback that said they felt some team members were doing their fair share, while others didn't. 6 out of 9 months have hit the goal! We now have a goal to hold ourselves to and associates are held accountable to this goal in their trimester and annual reviews. We can more easily identify tradeoffs once we are in the project and confidently communicate those trade offs to stakeholders.
Do What We Did
Recommendations Track the time Gather enough data to make estimates Hold the team accountable
Presenter
Presentation Notes
If you find yourself having a difficult time estimating how long a project should take, going over your project estimates, or unable to hold your team accountable to realistic time estimates, I would like to recommend you do what we did. Track the time, get enough data to make estimates, and then hold your team accountable to those estimates.
