Using QDAS in the production of policy evidence by non-researchers:
strengths, pitfalls and implications for consumers of research
Dr Chih Hoong Sin
Head of Information and Research
Disability Rights Commission
• Presentation from the perspective of research commissioner and of research provider
• Three key developments in UK:– evidence-based policy and practice– utilitarian view of research– effective dissemination
• Implications:– ‘marketised’ research relationships– increasing heterogeneity of ‘providers’ and
‘clients’– different skills sets required– ‘quality guarantee’ in doubt or not primary
concern?– different ‘normative worlds’ in collision
• Example of consultancies:– cross pollinators
• reduce ‘silos’, enhance transferability
– match makers• more effective partnership working
– translators and processors• information usable and relevant
– multiple dissemination routes, formative techniques
• wider audience, timely
• Company X:– SME research and consultancy company– Works solely with public sector clients (i.e.
national, regional, local government, public bodies)
– Six employees use QDAS
• Prior experience:– 4 had general undergrad social research
training– 1 did qualitative postgrad research– 1 no background in qualitative research at all– None had used any QDAS before
• Training (not mutually exclusive):– 1 had formal external training by specialist– 4 had ‘on the job’ training– 3 had formal internal training by colleague -
implication?– 1 asked colleague– 1 read a manual
• Type of research QDAS used on:– All were large-scale mixed-method national
policy evaluations– Mostly semi-structured interviews, one
structured focus group– Volume of data - from around 30 to more than
100 documents– All individuals used QDAS on actual projects
immediately after training
• Perceived adequacy of training:– All felt training was adequate, irrespective of:
• background in qualitative research/data• experience in using QDAS• mode of training• timing of training
• Functions used:– All used QDAS for preparing and uploading
documents; code; perform matrix node searches
– Fewer used it to design coding structure; define codes; generate reports; create memos
– 2 used Merge function
• Confidence and weakness:– All confident in functions with regular use– Less confident in functions with sporadic use
or never used– Awareness of more ‘sophisticated functions’
that they had never used but no indications of knowledge of what these functions actually are
• Project management:– All trained in specific project teams– Division of labour - data management, data
analysis– ‘Need to know’ and consistency
• Data analysis:– ‘Core’ analysis team– Structured coding design– Descriptive or topic codes– Largely descriptive analysis, lack of theorising
• Discussion:– Need to engage. Pragmatic rather than
idealistic response. Can’t ignore or shun as ‘wrong’ or ‘unorthodox’
– QDAS can offer some tools to help mitigate against the worst of ‘bad practise’, depending on:
• type of research• type of team management• type of outputs and hence analysis required
• Discussion:– Allows things that can be systematised to be
systematised– Easy checking– Not overwhelm individuals, e.g. ‘need to know
everything’
• What to look for:– Good guidance exist, but tend to target
people with some understanding of research– What to look for and what to ask for when it’s
not there. Inability to articulate causes frustrations on both sides, fuel continued misunderstanding
– QDAS not the only way, but can help. Some risks (e.g. ‘wow’ factor).
• What to look for:– Samples of documents– Numbers of documents, all ‘analysed’– Codes– Use of codes– …and, dare we hope, a theoretical ‘model’?
Thank you for your attention and enjoy the rest of the conference!
Dr Chih Hoong Sin
Email: [email protected], [email protected]
