review 12 17 october 2015 - unfccc
TRANSCRIPT
A general perspective
Review 12 – 17 October 2015
CR-1 2015
• Introduction: key components of success
•The challenges: the challenges during the review and what opportunities they presented
•Good practices and tools: what was helpful in addressing the challenges
•Beyond the review week: how we kept the momentum going after the review week
Turning a good review into the greatA holistic approach:
• take a systemic view of the review
• apply project management techniques to LR work
• design and use a communication plan
The good is in details:
• early preparation and planning
• think through and plan for every step of the review (“what”, “so what”, “now what”)
Building a great team:
• communicate clearly
• share knowledge and responsibility
• have fun
Taking ownership in innovative solutions:
• identify the issues
• be proactive and solution-focussed
• take collaborative approach to solution design and testing
Review challenges: turning challenges into opportunities
The challenges of the review formed three major groups:
•New rules – due to changes in UNFCCC and IPCC guidelines as reporting and review rules
•Logistics – tools, templates, organisation & planning
•General – more generic challenges not related to review guidelines or technical issues
Turning challenges into opportunities: new rules
New guidelines for inventory reporting and review + IPCC 2006 methodological Guidelines:
the review process under the new rules had no precedents to follow
•An opportunity to develop a better understanding of the review process•A natural environment for in-depth group discussions•A great learning opportunity for new and experienced reviewers
Turning challenges into opportunities: tools and templates
New ARR template:
a streamlined table-based document with extensive internal cross-referencing
• An opportunity to apply innovative solutions to working with the ARR
• Saving time and efforts by applying the consistent approach to questions to the Party and potential recommendations in the ARR
• Design and test simple software tools to link wrap-up discussions with the new ARR
Tools: the usual software tools (like Locator, NK finder etc.) were not available
•Explore and understand both CRF data entry file and the output files
•Use software skills to make the use CRF input file more efficient and user-friendly
• Lesson learned: we can be less dependent on the standard review tools for data analysis and verification
Delay in the assessment report resulted in delays with the Party responses
•Apply a more flexible approach to the review planning and prioritise our work better
Turning challenges into opportunities: tools and templates
Communication: working across 22 different time zones was a challenge
•An opportunity to explore on-line meeting planning tools
•Use the time-difference to our advantage to schedule almost 24/7 non-stop work process
Language: one of the Inventories has been written in Russian with no English summary
•An opportunity to utilise our linguistic
skills and experiences
•Work more cohesively as a team
Turning challenges into opportunities: general
We have welcomed three new experts to the ERT (Energy, IPPU and LULUCF sectors)
•An exciting opportunity to meet the new people
•Strengthen our collaboration skills through group work, coaching and inter-disciplinary discussions
•Great learning opportunity for all experts through sharing knowledge and new ideas, responding to challenging questions and giving feedback
• Lesson learned: experienced reviewers working together with new experts produced excellent questions and insightful analyses all the way through the review
10
Turning challenges into opportunities: general
Good practices and tools
We found it a good practice to have a detailed plan for every part of the review, including:
•Timeline: Milestones schedule/ Hourly planning for the review week /Detailed schedule for completion of the ARR
•Roles and responsibilities
•Communications
•Q&A process
•Discussion process and tools
•ARR compilation process
all these will be introduced in the follow-up slides
Good practices and toolsWe have discussed, designed and tested agendas and tools for each aspect of the review week, making sure that every angle is covered:
•Role allocation (detailed)
•Team diary
•Meeting/conference planner (time matrix for more efficient communications – free on-line tool)
•Dashboard
•Agendas for: teleconferences; introduction meeting; daily work; wrap-ups
•Discussion tools (Wrap-up files 1 and 2)
Week: -5 -3 -1 0 0 0 0 0 0 2 3 4 6
Responsible ERT member (include
specific people once we know
responsibilities)
Access to VTR confirmed
(Due 14 Sept)
Prelim. Qs Sent (Due 28
Sept)
Sector-based ZOD provided
(Due 9 Oct) ZOD of ARR
LR comments on ZOD (by
17 October if possible)
UNFCCC comments on ZOD (by
17 Oct if possible)
Response to ZOD
comments (Due 17 Oct if possible)
Review transcript
(Due 17 Oct)
Preliminary main
findings (Due 17 Oct)
Inputs for consolidated ZOD (or FOD if respond to comments)
(Due 2 Nov.)
LR approved FOD (9
November)
Review officer
comments on FOD (Due
13 Nov)
Response to follow up
questions or comments
Submit to editing and QA (Due 27
Nov)
Respond to QA
comments
Send to Party (Due
25 Dec)
Response from Party
received (Due 22 Jan)
ERT response to comment (1
Feb)
Final ARR from ERT
(due 22 Feb)
Publication of final ARR (7 March)
Canada
General NA NA NA NA NA NA NA
Energy NA NA NA NA NA NA NA
IP NA NA NA NA NA NA NA
Agriculture NA NA NA NA NA NA NA
LULUCF NA NA NA NA NA NA NA
Waste NA NA NA NA NA NA NA
LRs NA NA NA NA NA NA NA NA NA NA NA NA
Secretariat NA NA NA NA NA NA NA NA NA NA NA NA
Ukraine
General NA NA NA NA NA NA NA
Energy NA NA NA NA NA NA NA
IP NA NA NA NA NA NA NA
Agriculture NA NA NA NA NA NA NA
LULUCF NA NA NA NA NA NA NA
Waste NA NA NA NA NA NA NA
LRs NA NA NA NA NA NA NA NA NA NA NA NA
Secretariat NA NA NA NA NA NA NA NA NA NA NA NA
Codes
Date finished
Not yet submitted
Not applicable
Good practices and tools: time-line (ex 1)The time line has been established in discussion between LRs, ERT & ROs
• First communication LR/RO – 7 weeks prior the review week• Long-term time-line created – 6 weeks prior the review week• Roles & responsibilities – 4 weeks prior the review week• Teleconferences – 3 weeks prior the review week
Good practices and tools: approach to communication
We approach this as a set of options
Elements of the plan included:
• Timing, frequency, triggers and methods (e.g., e-mail, teleconference, person-to-person call; workgroup/full ERT discussion);
• personal preferences in approach to communication; dealing with delays (plans A, B, C)
• On-line meeting planning tool was used to coordinate teleconferences across multiple time zones.
• Teleconferences were held in two consecutive days to ensure convenience for experts located across 22 time zones.
The plan was also used after the review week
Communication: teleconferences
•Timing could be challenging – we used an on-line meeting organiser (free) – matrix planner
http://www.timeanddate.com/worldclock/meeting.html
•Split group discussion to cater for different time zones (we had 2 groups)
•Sign-off process for each proposed teleconference option
•Detailed agenda
•Beware - teleconferences is an equipment challenge (BIG sound problems); detailed agenda and can help
Teleconference: sample agenda
• Introductions (LR)
• Overview of status of review tools (RO)
• Introduction of new processes and procedures (e.g. based on new guidelines or LR recommendations)
• Round of questions (all) – points of concern:• ZOD• Recommendation vs encouragement• Time-series consistency• Sector-specific• CRF-related technical questions
• Work plan for the review week
Good practices for role allocations (with example)
Expert fieldFamiliar with
the Party’s systems/NIR?
Knows Russian?
Party to review
Role options
Generalist (LR) (can help Energy & IPPU)
Yes (national system)
fluent Can
1. Primary (Canada)2. Help the other generalist with language, if needed3. Working with Energy expert on translation/ teleconference with Ukraine
• Timing: 4 weeks prior review
• The following attributes were taken into account:
– Field (extended for cross-sectoral collaboration)
– Experience
– Familiarity with the Party’s inventory/inventory systems
– Language skills
– Primary and secondary responsibilities
Good practice: Q & A process
•The process: all questions to the Party were peer-reviewed by
• fellow experts
•RO
• LRs
•The approach: “context-trigger-question” (CTQ) (this was initiated by LRs)
•CTQ was applied consistently to all
questions and later – to ARR entries
Good practice: CTQ (what? -> so what? -> now what?)
Each statement contained three elements:•The context where the issue was nested •A specific issue that triggered the question (a
contradiction, an identified gap, etc) •A question to the Party / a potential
recommendation that should help to resolve the matter
Why this was important:•This saved time on question clarifications with the
Party – practically, none were required• It helped to keep track the questions and potential
issues for the ARR
Discussion process and tools for the review week
We used process-based approach to ensure that the following aspects were given time and consideration:
• Individual work and quiet time for research & analysis.
•Work in small groups – when experts in each sector were given an opportunity to discuss their findings with other experts, peer-review each other’s work and discuss critical points with or get clarifications from the LRs.
•Whole group discussions (daily wrap-up sessions); the focus of each review day was factored in the discussion themes
Example: planning daily sessions during the review week
• Morning and early afternoon (9 am – 2 pm):
Individual/group work on questions and reports or/and teleconferences with the Parties
• Afternoon (2 - 4pm):
Expert group discussions of all issues for all Parties for each sector
• Late afternoon (4 – 6 pm):
Entire ERT discussion of day’s progress. This would include the ERT dashboard of identified issues and relevant questions, proposals for teleconferences with Parties and identifying potential recommendations.
• Evening (6 pm+):
LR finalised questions for the Party
AND we have finished each session in time
Q&A Review Transcript
ARR SummaryTable 2
Provisionalmain findings
Full ARR
Monday
Tuesday
Wednesday Last questions,evening if possible
Thursday
Friday Transfer RT issues to ARR documentSaturday Fully
elaborated
General Timeline of Review Week
Summary
table 2 may
help guide
wrap up
meetings
Example: flexible approach to review week schedule• We combined the timeline with review week agenda• The daily plan was not rigidly fixed – there was space for adjustments when
needed.
The review week: new discussion tools To further improve efficiency of discussions, two discussion tools were designed (file 1 and file 2)
File 1 (used on Tuesday-Thursday): •Based on the review transcripts • Issue identifier by: •Sector •Type (consistency, accuracy etc.)• Issue? (following the Issue flow chart) • Is it repetitive?
To streamline this process even further:
• Link RT to the discussion file
• Even better: direct link RT -> ARR
File 1: fields• Fields:• Issue ID • Category• Gas• Finding description• Is included in 2014 ARR? (incl. paragraph #)• Number of successive reviews when the issue not addressed
(Years X-X)• ERT assessment: Is the finding an issue?• Issue type (please select from the drop-list)• Recommendation/Encouragement?• Status (resolved, not resolved, not relevant, addressing)
• Process:
• Each expert populated a relevant sector/Party tab with the potential issues discovered during the day and sent them to the LR*
• LR compiled the sectoral entries in one file*
• ERT discussed the sector entries and discussed the status of each identified issue
• Product: a good ZOD for tbl 5 of the ARR (2015 template)
Review week: new discussion tools
File 2: •Consolidates and categorises the issues for tbl 2 of
ARR• It looks like tbl 2 of the 2015 ARR template•Compiled during the discussion•Best to be used for Friday’s discussion when the
majority of issues have been identified and discussed using File 1
The Process: •Experts name the issue identifiers as per File 1 to
populate each field of File 2•The ERT revisits some issues (if needed) and
finalises File 2•The product: ready-to-go ZOD for tbl 2 of the ARR
Review week: provisional main findings
•The final versions of files 1 and 2 provided a good summary of issues provisional main findings document
•We also applied the CTQ approach to improve consistency of the document and reduce its size
Files
1 & 2CTQ
Provisional
recommendations
Key lessons learned
• Each review comes with its challenges – turn them into opportunities to learn and grow
• Early planning saves time and helps the review to go smoothly
• Be creative – new tools and ideas could be helpful to more than one team
•New experts bring new ideas and ask good questions – we all can learn from them
• Collaborative work before, during and after review week helps to share the load, produce excellent solutions and makes the review a great fun