using qdas in the production of policy evidence by non-researchers: strengths, pitfalls and...
TRANSCRIPT
![Page 1: Using QDAS in the production of policy evidence by non-researchers: strengths, pitfalls and implications for consumers of research Dr Chih Hoong Sin Head](https://reader036.vdocuments.us/reader036/viewer/2022083009/56649e4f5503460f94b46a0d/html5/thumbnails/1.jpg)
Using QDAS in the production of policy evidence by non-researchers:
strengths, pitfalls and implications for consumers of research
Dr Chih Hoong Sin
Head of Information and Research
Disability Rights Commission
![Page 2: Using QDAS in the production of policy evidence by non-researchers: strengths, pitfalls and implications for consumers of research Dr Chih Hoong Sin Head](https://reader036.vdocuments.us/reader036/viewer/2022083009/56649e4f5503460f94b46a0d/html5/thumbnails/2.jpg)
• Presentation from the perspective of research commissioner and of research provider
• Three key developments in UK:– evidence-based policy and practice– utilitarian view of research– effective dissemination
![Page 3: Using QDAS in the production of policy evidence by non-researchers: strengths, pitfalls and implications for consumers of research Dr Chih Hoong Sin Head](https://reader036.vdocuments.us/reader036/viewer/2022083009/56649e4f5503460f94b46a0d/html5/thumbnails/3.jpg)
• Implications:– ‘marketised’ research relationships– increasing heterogeneity of ‘providers’ and
‘clients’– different skills sets required– ‘quality guarantee’ in doubt or not primary
concern?– different ‘normative worlds’ in collision
![Page 4: Using QDAS in the production of policy evidence by non-researchers: strengths, pitfalls and implications for consumers of research Dr Chih Hoong Sin Head](https://reader036.vdocuments.us/reader036/viewer/2022083009/56649e4f5503460f94b46a0d/html5/thumbnails/4.jpg)
• Example of consultancies:– cross pollinators
• reduce ‘silos’, enhance transferability
– match makers• more effective partnership working
– translators and processors• information usable and relevant
– multiple dissemination routes, formative techniques
• wider audience, timely
![Page 5: Using QDAS in the production of policy evidence by non-researchers: strengths, pitfalls and implications for consumers of research Dr Chih Hoong Sin Head](https://reader036.vdocuments.us/reader036/viewer/2022083009/56649e4f5503460f94b46a0d/html5/thumbnails/5.jpg)
• Company X:– SME research and consultancy company– Works solely with public sector clients (i.e.
national, regional, local government, public bodies)
– Six employees use QDAS
![Page 6: Using QDAS in the production of policy evidence by non-researchers: strengths, pitfalls and implications for consumers of research Dr Chih Hoong Sin Head](https://reader036.vdocuments.us/reader036/viewer/2022083009/56649e4f5503460f94b46a0d/html5/thumbnails/6.jpg)
• Prior experience:– 4 had general undergrad social research
training– 1 did qualitative postgrad research– 1 no background in qualitative research at all– None had used any QDAS before
![Page 7: Using QDAS in the production of policy evidence by non-researchers: strengths, pitfalls and implications for consumers of research Dr Chih Hoong Sin Head](https://reader036.vdocuments.us/reader036/viewer/2022083009/56649e4f5503460f94b46a0d/html5/thumbnails/7.jpg)
• Training (not mutually exclusive):– 1 had formal external training by specialist– 4 had ‘on the job’ training– 3 had formal internal training by colleague -
implication?– 1 asked colleague– 1 read a manual
![Page 8: Using QDAS in the production of policy evidence by non-researchers: strengths, pitfalls and implications for consumers of research Dr Chih Hoong Sin Head](https://reader036.vdocuments.us/reader036/viewer/2022083009/56649e4f5503460f94b46a0d/html5/thumbnails/8.jpg)
• Type of research QDAS used on:– All were large-scale mixed-method national
policy evaluations– Mostly semi-structured interviews, one
structured focus group– Volume of data - from around 30 to more than
100 documents– All individuals used QDAS on actual projects
immediately after training
![Page 9: Using QDAS in the production of policy evidence by non-researchers: strengths, pitfalls and implications for consumers of research Dr Chih Hoong Sin Head](https://reader036.vdocuments.us/reader036/viewer/2022083009/56649e4f5503460f94b46a0d/html5/thumbnails/9.jpg)
• Perceived adequacy of training:– All felt training was adequate, irrespective of:
• background in qualitative research/data• experience in using QDAS• mode of training• timing of training
![Page 10: Using QDAS in the production of policy evidence by non-researchers: strengths, pitfalls and implications for consumers of research Dr Chih Hoong Sin Head](https://reader036.vdocuments.us/reader036/viewer/2022083009/56649e4f5503460f94b46a0d/html5/thumbnails/10.jpg)
• Functions used:– All used QDAS for preparing and uploading
documents; code; perform matrix node searches
– Fewer used it to design coding structure; define codes; generate reports; create memos
– 2 used Merge function
![Page 11: Using QDAS in the production of policy evidence by non-researchers: strengths, pitfalls and implications for consumers of research Dr Chih Hoong Sin Head](https://reader036.vdocuments.us/reader036/viewer/2022083009/56649e4f5503460f94b46a0d/html5/thumbnails/11.jpg)
• Confidence and weakness:– All confident in functions with regular use– Less confident in functions with sporadic use
or never used– Awareness of more ‘sophisticated functions’
that they had never used but no indications of knowledge of what these functions actually are
![Page 12: Using QDAS in the production of policy evidence by non-researchers: strengths, pitfalls and implications for consumers of research Dr Chih Hoong Sin Head](https://reader036.vdocuments.us/reader036/viewer/2022083009/56649e4f5503460f94b46a0d/html5/thumbnails/12.jpg)
• Project management:– All trained in specific project teams– Division of labour - data management, data
analysis– ‘Need to know’ and consistency
![Page 13: Using QDAS in the production of policy evidence by non-researchers: strengths, pitfalls and implications for consumers of research Dr Chih Hoong Sin Head](https://reader036.vdocuments.us/reader036/viewer/2022083009/56649e4f5503460f94b46a0d/html5/thumbnails/13.jpg)
• Data analysis:– ‘Core’ analysis team– Structured coding design– Descriptive or topic codes– Largely descriptive analysis, lack of theorising
![Page 14: Using QDAS in the production of policy evidence by non-researchers: strengths, pitfalls and implications for consumers of research Dr Chih Hoong Sin Head](https://reader036.vdocuments.us/reader036/viewer/2022083009/56649e4f5503460f94b46a0d/html5/thumbnails/14.jpg)
• Discussion:– Need to engage. Pragmatic rather than
idealistic response. Can’t ignore or shun as ‘wrong’ or ‘unorthodox’
– QDAS can offer some tools to help mitigate against the worst of ‘bad practise’, depending on:
• type of research• type of team management• type of outputs and hence analysis required
![Page 15: Using QDAS in the production of policy evidence by non-researchers: strengths, pitfalls and implications for consumers of research Dr Chih Hoong Sin Head](https://reader036.vdocuments.us/reader036/viewer/2022083009/56649e4f5503460f94b46a0d/html5/thumbnails/15.jpg)
• Discussion:– Allows things that can be systematised to be
systematised– Easy checking– Not overwhelm individuals, e.g. ‘need to know
everything’
![Page 16: Using QDAS in the production of policy evidence by non-researchers: strengths, pitfalls and implications for consumers of research Dr Chih Hoong Sin Head](https://reader036.vdocuments.us/reader036/viewer/2022083009/56649e4f5503460f94b46a0d/html5/thumbnails/16.jpg)
• What to look for:– Good guidance exist, but tend to target
people with some understanding of research– What to look for and what to ask for when it’s
not there. Inability to articulate causes frustrations on both sides, fuel continued misunderstanding
– QDAS not the only way, but can help. Some risks (e.g. ‘wow’ factor).
![Page 17: Using QDAS in the production of policy evidence by non-researchers: strengths, pitfalls and implications for consumers of research Dr Chih Hoong Sin Head](https://reader036.vdocuments.us/reader036/viewer/2022083009/56649e4f5503460f94b46a0d/html5/thumbnails/17.jpg)
• What to look for:– Samples of documents– Numbers of documents, all ‘analysed’– Codes– Use of codes– …and, dare we hope, a theoretical ‘model’?
![Page 18: Using QDAS in the production of policy evidence by non-researchers: strengths, pitfalls and implications for consumers of research Dr Chih Hoong Sin Head](https://reader036.vdocuments.us/reader036/viewer/2022083009/56649e4f5503460f94b46a0d/html5/thumbnails/18.jpg)
Thank you for your attention and enjoy the rest of the conference!
Dr Chih Hoong Sin
Email: [email protected], [email protected]