applying ‘best fit’ frameworks to systematic review data extraction
Post on 07-Aug-2015
134 Views
Preview:
TRANSCRIPT
Applying ‘best fit’ frameworks to systematic review data extraction Andrea Mil ler-Nesbitt, Catherine Boden, Andrew Booth, et al. 7 t h I n te rna t i ona l Con fe rence on Qua l i t a t i ve and Quan t i t a t i ve Me thods i n L ib ra r i es , Pa r i s F rance
May 2015
Overview • Background • Systema2c review process • ‘Best fit’ framework methodology • Applica2on of methodology to our project
Background
Conduct a systema2c review to address one of the 15 ques2ons iden2fied in the MLA Research Agenda: Appraising the Best Available Evidence
Research question
What skills and knowledge must librarians possess in order to be able to design tools to help researchers visualize, mine, and otherwise manage large and complex data gathered during both quan2ta2ve
and qualita2ve research?
Research question
What skills and knowledge must librarians possess in order to be able to design tools to help researchers visualize, mine, and otherwise manage large and complex data gathered during both quan2ta2ve
and qualita2ve research?
*Catherine Boden
Brooke Billman
Lorely Ambriz
Andrea Miller-‐Nesbitt Martin Morris
Andrew Booth
Abby Adamczyk
Anne Woznica
Keith Engwall
Rienne Johnson
Betsy Clark
Search Databases: PubMed, Embase, ACM, LISA, LISTA, ERIC, Web of Science, WorldCat
Date limits: Ar2cles
• 2000 to May 2014 Books
• 2005 to May 2014
Preliminary results
Records a6er duplicates removed (n = 3910)
Records screened (n = 3910 )
Records excluded (n = 3745 )
Full-‐text arEcles assessed for eligibility (n = 165)
101 reviewed 64 in progress
Studies included in qualitaEve synthesis (by April 24 2015)
(n = 26 )
9 – data mining 7 – data visualizaEon 24 – data management
Note: arEcles could be coded in
more than one category
Full-‐text arEcles excluded to date (n = 70)
5 – not English 13 – not about research data
management, mining or visualizaEon 32 – not about designing tools 12 – did not address librarian
competencies 3 – librarian competencies were
described but not in relaEon to research data
3-‐ insufficiently complete for data extracEon
Records aTer duplicates removed (n=3910)
Records screened (n=3910)
Full-‐text ar2cles assessed for eligibility (n=165)
Ar2cles included in qualita2ve analysis
(n=26*)
Data visualiza2on (n=7) Data mining (n=9)
Data management (n=24)
Records excluded in 2tle abstract screen (n=3745)
Full-‐text ar2cles excluded (n=70*)
Data extraction
“Best fit” framework synthesis
Large result set
Time constraints
Large research team
MulEple facets within research quesEon
Framework synthesis • Deduc2ve process used for systema2c reviews • Highly structured approach to analyzing qualita2ve data • A priori framework is iden2fied or developed from a range of sources
• Clearly defined themes in order to ensure transparency, consistency and speed of data coding
‘Best fit’ framework synthesis
“The ‘best fit’ framework synthesis method offered a means to test, reinforce and build on an exis2ng published model,
conceived for a poten2ally different but relevant popula2on…this approach
produces a rela2vely rapid, transparent and pragma2c process.”
(Carroll et al., 2013, p1)
Research ques2on
Iden2fy ‘best fit’ frameworks, conceptual models or theories
Iden2fy relevant studies for analysis
Generate a priori framework using thema2c analysis
Extract data from included studies
Code evidence from included studies against a priori framework
Create new themes by doing thema2c analysis on evidence that cannot be coded against the framework
Incorporate new themes into a priori framework to produce new conceptual model
(Carroll et al., 2013, p.3)
‘Best fit’ framework(s) synthesis
• DigCCurr Data management
• TBD Data mining
Data visualiza2on • TBD
Data extraction form
Data extraction form “what are the competencies described in the ar2cle for designing tools that support archival storage?”
“what are the competencies in the ar2cle for designing tools for valida2on and quality control of digital objects/packages?”
…
“descrip2on of competencies relevant to the design of tools for data management that did not fit in the categories above”
“there is insufficient data to be extracted with regard to research data management”
“record any obvious issues about study quality”
Challenges
• Iden2fying appropriate frameworks • Lack of granularity • Missing various concepts (especially tools) • Did not adequately address ‘competencies’
• Maintaining objec2vity • Ensuring we do not force data into the a priori framework
Next steps
• New themes iden2fied • Relevance of data management, mining or visualiza2on frameworks to librarians’ roles • Applica2on of ‘best fit’ framework methodology to LIS research
Projected outcome
Generate an evidence-‐based model, that iden2fies the competencies required of librarians involved in the design of tools used for data management, mining or visualiza2on.
Selected references Barnek-‐Page, E., & Thomas, J. (2009). Methods for the synthesis of qualita2ve research: a cri2cal review. BMC Medical Research Methodology, 9(1), 1-‐11.
Carroll, C., Booth, A., & Cooper, K. (2011). A worked example of "best fit" framework synthesis: a systema2c review of views concerning the taking of some poten2al chemopreven2ve agents. BMC medical research methodology, 11(29).
Carroll, C., Booth, A., Leaviss, J., & Rick, J. (2013). "Best fit" framework synthesis: refining the method. BMC medical research methodology, 13(1), 37.
Dixon-‐Woods, M. (2011). Using framework-‐based synthesis for conduc2ng reviews of qualita2ve studies. BMC medicine, 9(1), 39.
Eldredge, J. D., Ascher, M. T., Holmes, H. N., & Harris, M. R. (2012). The new Medical Library Associa2on research agenda: final results from a three-‐phase Delphi study. J Med Libr Assoc, 100(3), 214-‐218. doi: 10.3163/1536-‐5050.100.3.012
Eldredge, J. D., Harris, M. R., & Ascher, M. T. (2009). Defining the Medical Library Associa2on research agenda: methodology and final results from a consensus process. J Med Libr Assoc, 97(3), 178-‐185.
Ritchie, J., & Spencer, L. (1994). Qualita2ve data analysis for applied policy research. In A. Bryman & R. G. Burgess (Eds.), Analyzing qualitaNve data (pp. 173-‐194). London; New York: Routledge.
Questions? Andrea Miller-‐Nesbi\ McGill University, Montreal, QC andrea.miller-‐nesbik@mcgill.ca
top related