going beyond counts: understanding impact
DESCRIPTION
Webinar: Librarians & #altmetrics: tools, tips and use cases Altmetrics are becoming an integral part of looking at the impact and reach of research. Tracking social and online outlets, altmetrics provide quick feedback from a wide range of sources. In this webinar, library experts will discuss how altmetrics work, tools available, and the application of altmetrics in a range of institutions and for various user groups. What steps should librarians be taking to incorporate altmetrics into their workflows and reports? - See more at: http://libraryconnect.elsevier.com/articles/2014-01/webinar-librarians-altmetrics-tools-tips-and-use-cases#sthash.Lfdao6rU.dpufTRANSCRIPT
Webinar
#LCwebinar
BioinformaticistWashington University in St. Louis
Kristi Holmes
Going beyond counts: understanding impact
Webinar
Going beyond counts: understanding impact Kristi Holmes
Translational Research
The bench to the bedside and beyond
http://icts.wustl.edu/
Webinar
Going beyond counts: understanding impact Kristi Holmes
Why do we need to think about research impact?• Quantify and document research
impact
• Justify future requests for funding
• Quantify return on research investment
• Discover how research findings are being used
• Identify similar research projects
• Identify possible collaborators
• Determine if research findings are duplicated, confirmed, corrected, improved or repudiated
• Determine if research findings were extended (different human populations, different animal models/species, etc.)
• Confirm that research findings were properly attributed/credited
• Demonstrate that research findings are resulting in meaningful health outcomes
• Discover community benefit as a result of research findings
• Progress reports• Tenure• Promotion dossiershttps://becker.wustl.edu/impact-assessment/model
Webinar
Going beyond counts: understanding impact Kristi Holmes
How do we think about productivity?
http://www.philnel.com/2010/10/14/procrastigrading/
Don’t knock publications• Citations
– Rates and rankings
– Networks
– Dissemination patterns
• Collaboration
• Dissemination
• Grant funding
• Research trends
Webinar
Going beyond counts: understanding impact Kristi Holmes
…but there’s so much more!
http://www.philnel.com/2010/10/14/procrastigrading/
Webinar
Going beyond counts: understanding impact Kristi Holmes
Research Impact
Wells R, Whitworth A. 2007. Assessing outcomes of health and medical research: do we measure what counts or count what we can measure? Australia and New Zealand Health Policy, 4:14
“It is no longer enough to measure what we can – we need to measure what matters.”
So how do we measure what matters?
Webinar
Going beyond counts: understanding impact Kristi Holmes
Going beyond the counts to find meaningful impact
In everyone’s CV, we’ll find• Reviews• New funding awarded• New types of funding
mechanisms• New research studies• Invited lectures, new focus areas
at conferences• Membership on committees• Awards
Consider specific pathways* to uncover meaningful impact…Advancement of KnowledgeClinical ImplementationLegislation and Policy Enactment Economic BenefitCommunity Benefit
*can be adapted to reflect any number of disciplines
Webinar
Going beyond counts: understanding impact Kristi Holmes
Going beyond the counts to find meaningful impact • New diagnostic criteria
• New standard of care• Curriculum guidelines• Measurement instruments• Continuing education materials• Clinical/practice guidelines• Quality measure guidelines• Private healthcare benefit plans• Cost-effective intervention• Consensus development conferences
American Medical Association Current Procedural Terminology (CPT) codes
• Change in delivery of healthcare services
http://icts.wustl.edu/
http://icts.wustl.edu/
Webinar
Going beyond counts: understanding impact Kristi Holmes
Slide title
https://becker.wustl.edu/impact-assessment
Webinar
Going beyond counts: understanding impact Kristi Holmes
The Becker Model• Provides a supplement to publication analysis to provide a more
robust and comprehensive perspective of biomedical research impact.– reporting templates, glossary of resources and terms, examples of relevant indicators
of impact across the research process, and readings
• Straightforward framework for tracking diffusion of research outputs and activities to locate indicators that demonstrate evidence of biomedical research impact– individual, core, and institutional-level; modify for different disciplines
• Guidance for quantifying and documenting research impact as well as resources for locating evidence of impact.
• Strategies for enhancing the impact of research
Webinar
Going beyond counts: understanding impact Kristi Holmes
Strategies for enhancing the impact of research
https://becker.wustl.edu/impact-assessment/strategies
Repetition, consistency, and an awareness of the intended
audience form the basis of most the strategies.
Suggestions for researchers and recommendations to reach out to their
library for assistance.
The strategies focus upon Preparing for Publication,
Dissemination, and Keeping Track of Your Research.
Optimizing discoverability and access of your research is the
surest way to enhance its visibility and impact.
Webinar
Going beyond counts: understanding impact Kristi Holmes
Understanding the impact of a single paper can be challenging.
How do we understand the impact of a person? A research center? A university?
How do we scale up and operationalize this process?
Webinar
Going beyond counts: understanding impact Kristi Holmes
Research networking and discovery systems
• Enable more efficient means of collecting & representing meaningful outputs en masse
• Showcase achievements and expertise
• Facilitate diffusion of research products
• Support team-based science and collaboration
• Allow better understanding of the research enterprise– Peer comparisons
– Strategic planning
– Emerging trends
Webinar
Going beyond counts: understanding impact Kristi Holmes
What is VIVO?
1. An open source semantic web application
2. An information model
3. An open community*
* A big, welcoming OS community! Let us know if you have questions or need information, connections, or
materials: http://vivoweb.org/contact
Webinar
Going beyond counts: understanding impact Kristi Holmes
Tying it all together
http://vivo.ufl.edu/display/n86607
Continue to represent typical profile information – absolutely!
Enhance profiles by incorporating meaningful outputs, as described by the Becker Model or other frameworks, to enhance their content and value
Can facilitate the dissemination of scholarship and more
Webinar
Going beyond counts: understanding impact Kristi Holmes
Tying it all together
http://blog.plumanalytics.com/post/57707501083/plumx-now-supports-vivo
PlumX supports VIVO
1. Aggregate information about our researchers
2. Harvest research outputs from the profile
3. Plum’s harvesting engine calculates metrics and provides both an analytics dashboard as well as visualizations.
4. Widgets can be embedded back into the VIVO profile!
Webinar
Going beyond counts: understanding impact Kristi Holmes
Tying it all together
http://blog.plumanalytics.com/post/57707501083/plumx-now-supports-vivo
PlumX supports VIVO
1. Aggregate information about our researchers
2. Harvest research outputs from the profile
3. Plum’s harvesting engine calculates metrics and provides both an analytics dashboard as well as visualizations.
4. Widgets can be embedded back into the VIVO profile!
http://vivo.ufl.edu/display/n86607
FigShare, lab notebooks, project outputs, code/data repositories SlideShare, etc.
Webinar
Going beyond counts: understanding impact Kristi Holmes
Acknowledgements
Funding:• Washington University Institute
of Clinical and Translational Sciences, NIH award UL1 RR024992
• VIVO - DuraSpace
Thanks:• Cathy Sarli, MLS, AHIP
• Karen Gutzman – NLM Fellow
• Jae Allen
• Washington University ICTS and WU ICTS Tracking & Evaluation Team
• Andrea Michalek at Plum @amichalek
• VIVO Community @VIVOcollab
• Becker Medical Library