azair mixed methods 07
TRANSCRIPT
-
8/13/2019 AZAIR Mixed Methods 07
1/22
Using Mixed Method Approaches
in Institutional Research
Rick Kroc
University of Arizona
Richard Howard
University of Minnesota
Arizona Association for Institutional Research
Annual Meeting
Tucson, Arizona March 2007
-
8/13/2019 AZAIR Mixed Methods 07
2/22
Purpose of Decision Support
is
to reduce the risk to the decision
maker
-
8/13/2019 AZAIR Mixed Methods 07
3/22
Structure and Subjectivity
-
8/13/2019 AZAIR Mixed Methods 07
4/22
Research Process
-
8/13/2019 AZAIR Mixed Methods 07
5/22
Ways of Knowing
Two Classic Paradigms
Positivist Constructivist
there are multiple realities
and that truth is ever
changing, dependent oncontext and the individual
(subject and researcher)
there is a single reality or truth
across time and contexts, and
that this truth can be understood
through the objective study ofindependent variables
-
8/13/2019 AZAIR Mixed Methods 07
6/22
Conceptual Model of the
Constructivist Research ProcessAfter a literature review and pilot study,
define the Study:ProblemPurposeQuestion
Data Collection &
Management
Data Restructuring/Coding &
Analyzing/Read-Theme
Data Verification withRespondents:
FACTUAL ERRORAnalysis Verification with
Reviewers:
Interpretation Error
COMMUNICATE
New Theory/Model
TEST
Theory/Model
-
8/13/2019 AZAIR Mixed Methods 07
7/22
Conceptual Model of the Positivist
Research Process
After a literature review and pilot study
define the project
ProblemPurpose
QuestionOperationalize theory or model constructs
Data Collection and
Management
Data restructuring and
analysis
(Interpretation of Results)
Communication
of new
knowledge
For internal decision support-
test meaning of the findings
For external contexts-test/replicate forgeneralization
-
8/13/2019 AZAIR Mixed Methods 07
8/22
RESEARCH PARADIGMS
APPROACHES
Constructivist Positivist
QuantitativeQualitative
Consumer Generalizes Researcher Generalizes
-
8/13/2019 AZAIR Mixed Methods 07
9/22
-
8/13/2019 AZAIR Mixed Methods 07
10/22
Use of Mixed Method Approaches
in Institutional Research
Provides answers to two questions:
What- What happened and what was theoutcome? Answers tend to be
arrived at using quantitative methods
Why- Why did people behave the way theydid? Answers tend to be arrived at using
qualitative methods
-
8/13/2019 AZAIR Mixed Methods 07
11/22
What - goals are - how measuredSupports Accountability (External)
Whygoals were met or notpeople
issuesOften the focus ofAssessment/Evaluation (Internal)
So What?
-
8/13/2019 AZAIR Mixed Methods 07
12/22
Ways of Knowing
A Third Paradigm
Positivist Constructivist
there are multiple realities
and that truth is ever
changing, dependent on
context and the individual
(subject and researcher)
there is a single reality or truth
across time and contexts, and
that this truth can be understood
through the objective study of
independent variables
PragmatismUse what ever approach
and methodology that will
best answer the question.
-
8/13/2019 AZAIR Mixed Methods 07
13/22
Charge from President Likins
As Chair of the NCAA Division I Presidential Task Force
on the Future of Intercollegiate Athletics, I have both the
opportunity and the responsibility to examine the
culture of higher education athletics programs in
America. My responsibilities begin at home. My task
force experience as well as events at colleges anduniversities around the country led me to conclude that
it is appropriate to examine the environment defined by
the culture of high-level athletic competition here at The
University of Arizona. At the national level a seriousmovement to strengthen the academic experience of
student-athletes is well underway. We must examine the
presence of any cultural influences here that have the
potential of distorting the shared values of academics
and athletics in higher education.
-
8/13/2019 AZAIR Mixed Methods 07
14/22
Focus of the Study
The study was divided into two phases. Phase I,mostly quantitativein nature, was designed tolook somewhat broadly at the two fundamentalstudy areas: Student athletes academic experience; and
Student athletes culture and environment.
Using qualitativemethods, Phase IIwas designedto dig more deeply into the critical issuesidentified from Phase I. Detailed, targetedrecommendations would be developed from thesetwo phases.
-
8/13/2019 AZAIR Mixed Methods 07
15/22
Domain of the Study
Athletes academic
experience
Academic preparation
Progress to degree Student engagement
Academic support
Post-eligibility academic
experience Other academic issues
Athletes culture and
environment
Alcohol and drugs
Dishonesty and cheating Gambling
Racism
Sexism
Sexual behavior Violence
Weapons
-
8/13/2019 AZAIR Mixed Methods 07
16/22
Phase 1Quantitative
Board of Regents and NCAA reports on athletes gradesand graduation rates (ten year history).
Institutional student records data.
Health and Wellness Survey, administered annually toassess student alcohol, drug, and sexual behaviors (sevenyear history).
Dean of Students Code of Conduct and Code of AcademicIntegrity violations (five year history).
Coalition on Intercollegiate Athletics (COIA) report.
Annual University Survey of Graduating Seniors, assessing
student engagement, satisfaction, and other academicissues (five year history).
NCAA GOALS survey (student-athletes and student non-athletes).
Additional UA-specific survey questions appended toGOALS survey (student-athletes and student non-athletes).
-
8/13/2019 AZAIR Mixed Methods 07
17/22
Phase IIQualitative
Focus groups
Interviews
Follow-up surveys
Populations:
Students (athletes and non-athletes)
Coaches
Faculty Staff
Advisors
-
8/13/2019 AZAIR Mixed Methods 07
18/22
The Value of a Mixed Methods
Approach
Athletic culture spans a spectrum of
areas, some of which (alcohol use, for
example) have become fairly wellunderstood over the years from
quantitative methods, others of which
(physical violence, weapons use) needthe deeper exploration and understanding
provided by qualititative methods.
-
8/13/2019 AZAIR Mixed Methods 07
19/22
The Value of a Mixed Methods
Approach
The formative and summative aspects of
this study, which is more like program
evaluation in many ways than like atraditional research study, lend
themselves to a mixed methods
approach.
-
8/13/2019 AZAIR Mixed Methods 07
20/22
The Value of a Mixed Methods
Approach
Audiences and stakeholders consist of
some people who respond best to
systematic, inferentially rigorous,quantitative data, but also others who
want to hear the complex, richly detailed
stories derived from the individuals whoare immersed in the culture. Different
audiences resonate to different
approaches.
-
8/13/2019 AZAIR Mixed Methods 07
21/22
The Value of a Mixed Methods
Approach
Mixed methods also means that multiplemethods have been employed, which isessential to using triangulation as a
means to establish greater validity of thefindings. Any single approach used in themessy world of social science researchwill be flawed. If a wide array of methodsleads to consistent conclusions, thenanalysts and stakeholders feel moreconfident about conclusions and
recommendations.
-
8/13/2019 AZAIR Mixed Methods 07
22/22
Questions??