ron sokol, m.d. principal investigator, cctsi bonnie walters executive director, the evaluation...
TRANSCRIPT
Colorado Clinical Translational Science Institute Progress Tracking System
Ron Sokol, M.D.Principal Investigator, CCTSI
Bonnie WaltersExecutive Director, The Evaluation Center
Kathryn Nearing, Ph.D.Associate Director, The Evaluation Center
CCTSI is supported in part by Colorado CTSA grant 5UL1RR025780 from NCRR/NIH
The Evaluation Center 2010
System Components
Two report forms Meeting report form (specific to each core: features list of
meetings, associated participants, aims and “active” indicators)
Director report form (generic; completed by directors quarterly)
Performance dashboards Feature aims, indicators specific to each core, current phase of
grant
Each indicator has a status designation that supports progress monitoring
Master Calendar
The Evaluation Center 2010
Master Calendar
meeting event
master
calendar
email with
link to form
The Evaluation Center 2010
Progress Monitoring
The Evaluation Center 2010
Progress Monitoring
The Evaluation Center 2010
Automated Reports
Three summaries (generated monthly)
Meeting summary (how many of which meetings occurred)
Summary of indicators addressed and key decisions
Summary of participants within and across cores, as well as affiliate institutions (to support SNA to examine integration, collaboration)
Automated email sends a record of what was submitted via electronic forms
Archival summaries to respond to NIH reporting requirements (in development)
The Evaluation Center 2010
Manual Reports
Performance Dashboard
Real-time snapshot of how work is progressing during the current phase of the grant (updated monthly)
Stakeholder briefs
Highlight major accomplishments, emerging issues, future priorities, evaluation updates (often reviewed by steering committees)
Other reports may be generated “on the fly” by creating tailored views of data in SharePoint
Customized views can be created by any user of the system (with the proper permissions)
The Evaluation Center 2010
Document meetings (ongoing)
Review summaries (monthly),
director reports
(quarterly)
Update dashboards (monthly)
Generate stakeholder
briefs (quarterly)
Reporting Cycles: Daily, Monthly, Quarterly
The Evaluation Center 2010
Briefs created a responsive feedback mechanism Master calendar
Became a central repository for meeting, event information
Streamlined efforts to advertize CCTSI events
Indicators became more visible as an organizing framework to structure reporting Underscored the importance of well-written (i.e., “track-
able”, “actionable”) indicators
Agendas became more responsive to aims, indicators and the content of briefs
Advantages of System
The Evaluation Center 2010
Specific indicators -- critical benchmarks that indicate meaningful progress -- should be:
Written with the long-view in mind
In language understandable to diverse stakeholders
Timelines must be specified so that status designations are meaningful
Lessons Learned
The Evaluation Center 2010
Buy-in from leadership is essential
Orientation/ training is an ongoing process
Ongoing support should be provided in multiple ways
Ongoing access to tech support is essential
Administrators are integral to the success of the system
Lessons Learned
The Evaluation Center 2010
The PTS is more than an evaluator’s tool
The system is designed to: Support leaders and administrators at multiple levels
proactively manage (and reflect on) their work
Keep diverse stakeholders informed, thus enhancing their capacity to contribute meaningfully to organizational change
Key “Take Aways”
The Evaluation Center 2010
The PTS does not run on its own
The system requires ongoing commitment, time and attention Requires compliance with reporting requirements
A commitment to adequate, accurate and timely reporting (by both evaluators and program representatives)
A commitment to review reports and utilize information generated
Key “Take Aways”
The Evaluation Center 2010
System Requirements
Microsoft Office InfoPath© 2007 (optional) Microsoft Office SharePoint© server and client
access licenses (CALs) Internet Explorer browser (preferred)
The Evaluation Center 2010
Hosting Model
Dedicated PTS available for each CTSA
PTS site is self-contained and self-managed
Each CTSA would need an in-house administrator
We would provide:
Kick-off training and some level of ongoing support
Hosting and some application support
We are currently working with NCRR to explore whether funding might be available to support these efforts
The Evaluation Center 2010
For More Information
Kathryn [email protected]
Jeff [email protected]
On the Webhttp://cctsi.ucdenver.edu