134621811 advance research methods
TRANSCRIPT
1 Muhammad Azeem Rafi NUML Lahore
Advance Research Methods
ADVANCED RESEARCH METHODOLOGY
Overview of Research and its Methodologies
• Concepts of research
• The need for research
• Types of research
• Steps in conducting research
Definition of Research
• Hunting for facts or truth about a subject.
• Organized scientific investigation to solve problems, test hypotheses, develop or invent new products.
What is Research? Research is systematic, because it follows certain steps that are logical in order. These
steps are:
• Understanding the nature of problem to be studied and identifying the related area of knowledge.
• Reviewing literature to understand how others have approached or dealt with the problem.
• Collecting data in an organized and controlled manner so as to arrive at valid decisions.
• Analyzing data appropriate to the problem.
• Drawing conclusions and making generalizations.
High Quality Research!
• It is based on the work of others.
• It can be replicated (duplicated).
• It is generalize able to other settings.
• It is based on some logical rationale and tied to theory.
• It is doable!
• It generates new questions or is cyclical in nature.
• It is incremental.
• It is apolitical activity that should be undertaken for the betterment of society.
What is bad research?
The opposites of what have been discussed.
• Looking for something when it simply is not to be found.
• Plagiarizing other people’s work.
• Falsifying data to prove a point.
• Misrepresenting information and misleading participants.
The general systematic characteristic of research is illustrated below.
Problem Identification
Reviewing Information
Data Collection
Analysis
Drawing Conclusion
2 Muhammad Azeem Rafi NUML Lahore
Advance Research Methods
• Research follows a scientific method. This means that it makes an integrated use of inductive and
deductive reasoning.
• This makes it very useful for explaining and/or predicting phenomena.
• The basic assumption of the scientific method is that every effect has a cause.
• It starts with the construction of hypotheses from casual observations and background knowledge
(inductive reasoning) to reasoning out consequences or implications of hypotheses (deductive reasoning)
followed by testing of the implications and confirmation or rejection of the hypotheses.
• Integrated use of inductive and deductive reasoning is, therefore, the essence of scientific method.
Why do we need research?
• To get PhDs, Masters and Bachelors?
• To provide solutions to complex problems
• To investigate laws of nature
• To make new discoveries
• To develop new products
• To save costs
• To improve our life
• Human desires
Classifying Research. Reviewing related past research studies is an important step in the process of carrying
out research as it helps in problem formulation, hypothesis construction and selection of appropriate research
designs. It is beneficial if you can classify a research study under a specific category because each category or
type of research uses a specific set of procedures.
There are two ways of classifying research.
• One way is to classify research on the basis of its purpose i.e. the degree to which the research findings are
applicable to an educational setting and the degree to which they are generalizable.
• The other is to classify research on the basis of the method employed in research.
Research can be classified into two types
Purpose Method
3 Muhammad Azeem Rafi NUML Lahore
Advance Research Methods
Classifying Research by Purpose
Classifying Research by Methods. The other basis for classifying research, is by the method it employs.
• Research method is characterized by the techniques employed in collecting and analyzing data.
• On the basis of method, research can be classified as historical, descriptive, correlational, ex-post facto and
experimental.
Historical Research. The purpose of historical research is to arrive at conclusions concerning trends, causes
or effects of past occurrences.
• This may help in explaining present events and anticipating future events.
• The data are not gathered by administering instruments to individuals ,but …
Rather, they are collected from original documents or by interviewing the eye-witnesses (primary source of
information).
• In case primary sources are not available, data are collected from those other than eye-witnesses
(secondary sources).
• The data thus collected are subjected to scientific analysis to assess its authenticity and accuracy.
Descriptive Research. Descriptive research studies deal with collecting data and testing hypotheses or
answering questions concerning the current status of the subject of study.
• It deals with the question “WHAT IS” of a situation.
• It concerns with determining the current practices, status or features of situations.
• Another aspect of descriptive research is that data collection is either done through asking questions from
individuals in the situation (through questionnaires or interviews) or by observation.
Correlational Research. Descriptive and historical research provides a picture of events that are
currently happening or have occurred in the past.
• Researchers often want to go beyond mere description and begin discussing the relationship that certain
events might have to one another.
• The most likely type of research to answer the relationship among variables or events is called
correlational research.
• A correlation study aims at determining the degree of relationship between two or more quantifiable
variables.
• Secondly, the relationship thus determined could be used for making predictions.
• A high value of relationship, however, does not signify a cause and effect relationship which must be
verified through an experimental study.
Type of Research
Basic Applied / Developmental
Type of Research
Historical Descriptive Co relational Ex Post Facto Experimental
4 Muhammad Azeem Rafi NUML Lahore
Advance Research Methods
• Correlational researches are studies that are often conducted to test the reliability and predictive validity
of instruments.
• Used for division making concerning selection of individuals for the likely success in a course of study or a
specific job.
• Some authors consider this research as a type of descriptive research, since it describes the current
conditions in a situation.
• However, the difference lies in the nature of conditions studies.
• A correlational study describes in quantitative terms the degree to which the variables are related.
Ex-Post Facto Studies. There is some research where both the effect and the alleged cause have already
occurred and are studied by the researcher in retrospect.
• Such research is referred to as Ex-Post Facto (after the fact).
• Systematic empirical inquiry in which the scientist does not have direct control of independent variables
because their manifestations have already occurred or because they are inherently not manipulable”.
• In ex-post facto or causal-comparative research the researcher has no control on the variables or he
cannot manipulate the variables (independent variables) which cause a certain effect (dependent
variables) being measured.
Experimental Research. Correlational research can help establish the presence of a relationship among
variables but not give us any reason to believe that variables are causally related to one another.
• How does one find out if the characteristics or behaviors or events are related in such a way that the
relationship is a causal one?
• Two types of research can answer this:-
• Experimental Research is where participants are assigned to groups based on some selected criterion
often called treatment variable.
• Quasi-Experimental Research is where participants are pre-assigned to groups based on some
characteristic or quality such as differences in sex, race, age, neighborhood, etc.
• These group assignments have already taken place before the experiment begins, and the researcher has
no control as to what the people will belong to each group
• The primary characteristic of experimental research is manipulation of at least one variables and control
over the other relevant variables so as to measure its effect on one or more dependent variables.
• The variable (s) which is manipulated is also called independent variables, a treatment, an experimental
variables or the cause.
• Some of the examples of independent variables could be: temperature, pressure, chemical concentration,
type of material and conductivity.
Applied vs. Basic Research
The most basic distinction between the two research is that basic research is research that has no immediate
application, whereas applied research is research that does.
• However, such distinctions are somewhat ambiguous as almost all basic research eventually results in some
worthwhile application in the long range.
Steps in Conducting Research. Irrespective of the type of a research study, the steps followed in
conducting it are the same. These are:-
5 Muhammad Azeem Rafi NUML Lahore
Advance Research Methods
1 Selecting and Defining a Problem. This marks the beginning of a research study and is
the most difficult and important step. This involves:
a. Identifying and stating the problem in specific terms;
b. Identifying the variables in the problem situation and defining them adequately;
c. Generating tentative guesses (hypotheses) about the relation of the variables or in
other words the solution of the problem, or writing explicitly the questions
(research questions) for which answers are sought; and
d. Evaluating the problem for its research ability.
All this is not done in a vacuum. To achieve this, you review the literature related to the
problem to know what other researchers have done and discovered and to identify the
possible methodology for conducting the research.
2 Describing Methodology of Research. You need to state the purpose of the study
and to define the problem clearly. This guides you in deciding the methodology of
research which involves :
a. Identifying the method of research;
b. Specifying the subjects of study (e.g. heat flow problem, etc.);
c. Selecting an adequate representative sample of subjects;
d. Selecting/constructing valid and reliable instruments for measuring the variables in the problem;
e. Selecting a research design and describing the procedure to be employed for conducting the research
study.
3 Data Collection
4 Analysis and Interpretation
CHAPTER 4 – THE MEANING AND METHODOLOGY
The three Approaches
Positivist Social Science
It is defined as the approach of the natural sciences.
Most people never even heard of alternative approaches.
Use of this approach has declined sharply in European Journals, but it is still very dominant in North
American Journals.
Positivism says that “there is only one logic of science”.
Social Sciences and natural Sciences should use the same method.
Positivism sees social science as an organized method for combining deductive logic with precise empirical
observations of individual behavior in order to discover and confirm a set of probabilistic causal laws that
can be used to predict general patterns of human activity.
Positivists believe hold that social and physical reality is real. It exists “out there” and is waiting to be
discovered.
The reality is outside and objective.
We can learn about people by observing their behaviour, what we see in external reality. It is more
important than what happens in internal, subjective reality.
Human events can be explained with reference to causal laws.
Positivists see a clear separation between science and non science.
6 Muhammad Azeem Rafi NUML Lahore
Advance Research Methods
Scientific knowledge is better and will eventually replace the inferior ways of gaining knowledge (e.g.,
magic, religion, astrology, personal experience, and tradition).
People can recognized truth and distinguish it from falsehood by applying reason.
In positivism, explanations must meet four conditions: they must:
Have no logical contradictions.
Be consistent with observed facts.
Replication is also needed.
The findings should be generalizable (generalizability is very important condition).
Positivism is dualist:
It assumes that the cold, observable facts are fundamentally distinct from ideas, values, or theories.
Empirical facts exist apart from personal ideas or thoughts.
Interpretive Social Science
It is a systematic analysis of socially meaningful action through the direct detailed observation of people in
natural settings in order to arrive at understanding and interpretations of how people create and maintain
their social worlds.
It is also called qualitative method of research.
Interpretive social science is related to hermeneutics, a theory of meaning that originated in the
nineteenth century.
Hurmeneutics emphasizes a detailed reading or examination of text, which could refer to conversation,
written words, or pictures.
Interpretive researchers often use participant observation and field research.
Other ISS researchers analyse transcripts of conversations or study videotapes of behaviors in
extraordinary details.
It has practical orientation.
Interpretive researchers study meaningful social action, not just the external or observable behavior of
people.
Social action is the action to which people attach subjective meaning: it is activity with a purpose of intent,
e.g. a physical reflex such as eye blinking.
The interpretive approach notes that human action has little inherent meaning. It acquires meaning among
people who share meaning system that permits them to interpret the action as a socially relevant sign or
action, e.g. raising one finger.
Interpretive approach holds that social life is based on social interactions and socially constructed meaning
systems.
For interpretive researchers, social reality is based on people’s definition of it.
The interpretive approach is ideographic and inductive. Ideographic means the approach provides a
symbolic representation or “thick” description of something else.
An interpretive research report may read like a novel or a biography than like a mathematical proof.
An interpretive theory gives the reader a feel for another’s social reality.
For ISS, a theory is true if it makes sense to those being studied and it allows others to understand deeply
or enter the reality of those being studies.
Critical Social Science
The purpose of social science is to reveal what is hidden to liberate and empower people.
7 Muhammad Azeem Rafi NUML Lahore
Advance Research Methods
Social reality has multiple layers.
People have unrealized potential and are mislead by reification; social science is relational.
The bounded autonomy stance is towards human agency.
Scientific knowledge is imperfect but can fight false consciousness.
Abduction is used to create explanatory critiques.
Explanations are verified through praxis.
All evidence is theory dependent, and some theories reveal deeper types of evidence.
A reflexive-dialectic orientation is adopted towards knowledge that is used from a transformative
perspective.
Social reality and the study of it necessarily contain a moral political dimension, and moral political
positions are unequal in advancing human freedom and empowerment.
CHAPTER 6 – STRATEGIES OF RESEARCH DESIGN
Introduction. Quantitative researchers are more concerned about issues of design, measurement, and
sampling because their deductive approach emphasizes detailed planning prior to data collection and analysis.
Qualitative researchers are more concerned about issues of the richness, texture, and feeling of raw data
because their inductive approach emphasizes developing insights and generalizations out of the data collected.
Triangulation. By observing something from different angles or viewpoints, the researchers get a fix on its
true location. This process, called triangulation, is used by quantitative and qualitative social researchers.
8 Muhammad Azeem Rafi NUML Lahore
Advance Research Methods
Applied to social research, it means it is better to look at something from several angles than to look at it in
only one way. There are four types of triangulation.
Triangulation of Measures. Researchers take multiple measures of the same phenomena.
Triangulation of Observers. Multiple observers or researchers add alternative perspectives,
backgrounds, and social characteristics and will reduce the limitations.
Triangulation of Theory. It occurs when a researcher uses multiple theoretical perspectives in the
planning stages of research, or when interpreting the data.
Triangulation of Method. It means mixing qualitative and quantitative styles of research and data.
Qualitative and Quantitative Orientations toward Research
Qualitative and Quantitative approaches differ in the nature of the data. Soft data, in the form of
impressions, words, sentences, photos, symbols, and so forth, dictate different research strategies and
data collection techniques than hard data, in the form of numbers.
Qualitative and Quantitative researchers often hold different assumptions about social life and have
different objectives.
Qualitative researchers often rely on interpretive or critical social science. They use a transcendent
perspective, apply “logic in practice”, and follow a nonlinear research path.
Qualitative researchers speak a language of “cases and contexts”.
Quantitative researcher uses a technocratic perspective, apply “reconstructed logic”, and follow a linear
research path. They speak a language of “variables and hypotheses”. Quantitative researchers s emphasize
precisely measuring variables and testing hypotheses that are linked to general causal explanations.
Reconstructed Logic and Logic in Practice
The way social researchers learn and discuss research usually follows one of two logics: reconstructed logic
or logic in practice.
Quantitative researchers apply more of the reconstructed logic, whereas qualitative researchers tend to
apply logic in practice.
Reconstructed logic means the logic of research based on reorganizing, standardizing, and codifying
research knowledge and practices into explicit rules and procedures, and techniques.
Logic in practice is the logic of research based on an apprenticeship model and the sharing of explicit
knowledge about practical concerns and specific experiences.
Linear and Non Linear Paths
The path is a metaphor for the sequence of things to do: what is finished first or where a researcher has
been, and what comes next or where he or she is going.
Quantitative researchers follow more linear path than do qualitative researchers. A linear research path
follows a fixed sequence of steps.
Qualitative research is more nonlinear and cyclical. A nonlinear research path makes successive passes
through steps, sometimes moving backward and sideways before moving on.
Objectivity and Integrity
Qualitative researchers emphasize the human factor and the intimate firsthand knowledge of the research
setting; they avoid distancing themselves from the people or events they study.
Quantitative researchers stress objectivity and more “mechanical” techniques. They use principle of
replication, adhere to standardized methodological procedures, measure with numbers, and then analyze
9 Muhammad Azeem Rafi NUML Lahore
Advance Research Methods
the data with statistics, an area of applied mathematics. Quantitative research eliminates the human
factor.
Qualitative researchers emphasize trustworthiness as a parallel idea to objective standards in quantitative
research design. This ensures that research activities are dependable and credible. Quantitative research
addresses the issue of integrity by relying on an objective technology such as precise statements, standard
techniques, numerical measures, statistics, and replication.
Qualitative researchers ensure that their research accurately reflects the evidence and have checks on
their evidence.
Preplanned and Emergent Research Questions
All research begins with a topic but a topic is only a starting point that researchers must narrow into a
focused research question.
Qualitative researchers often begin with vague or unclear research questions. The topic emerges slowly
during the study.
Quantitative researchers narrow a topic into a focused question as a discrete planning step before they
finalize study design. They use it as a step in the process of developing a testable hypothesis and to guide
the study design before they collect any data.
The qualitative researcher begin data gathering with a general topic and notions of what will be relevant.
Major limitations include time, costs, access to resources, and approval by authorities, ethical concerns,
and expertise.
Techniques for Narrowing a Topic into Research Question
Examine the Literature. Published articles are excellent sources for ideas for research questions. They
are usually at an appropriate level of specifying and suggesting research questions that focus on following:-
Replacing a previous research project exactly or with slight changes.
Exploring unexpected findings discovered in previous research.
Following suggestions an author gives for future research at the end of the article.
Extending an existing explanation or theory to a new topic or setting.
Challenging the findings or attempting to refute a relationship.
Specifying or intervening the process and considering any linking relations.
Talk Over the Idea with Others
Ask people who are knowledgeable about the topic or questions it that they have thought of.
Seek out those who hold opinions that differ from yours on the topic and discuss possible research
questions with them.
Apply to a Specific Context
Focus the topic onto a specific historical period, or time period.
Narrow the topic to a specific society or geographic unit.
Consider which subgroups or categories of people/ units are involved and whether there are
differences among them.
Define the Aim and Desired Outcome of the Study
Will the research question be for an explanatory, explanatory or descriptive study?
Will the study involve applied or basic research?
10 Muhammad Azeem Rafi NUML Lahore
Advance Research Methods
Qualitative Design Issues
The Language of Case and Contexts. Qualitative researcher use a language of cases and contexts,
employ bricolage, examine social processes and cases in their social context, and look at interpretations or
the creation of meaning in specific settings.
Grounded Theory. A qualitative researcher develops theory during the data collection process.
This more inductive method means that theory is built from data or grounded in the data.
The Context is Critical. Qualitative researchers emphasize the social context for understanding the
social world. They hold that the meaning of a social action or statement depends, in an important way, on
the context in which it appears.
Bricolage. Qualitative researchers are bricoleurs; they learn to be adept at doing many things, drawing on
a variety of sources, and making do with whatever is at hand. A bricolage technique means working with
one’s hands and being pragmatic at using an assortment of odds and ends in an inventive manner to
accomplish a specific task.
The Case and Process. In quantitative research, cases are usually the same as a unit of analysis, or the
unit on which variables are measured. Qualitative researcher tends to use a “case oriented approach
places cases, not variables, center stage”. The passage of time is integral to qualitative research.
Quantitative researchers typically measure variables of their hypotheses across many cases.
Interpretation. A qualitative researcher interprets data by giving them meaning, translating them, or
making them understandable. Quantitative research is expressed in numbers, and a researcher gives
meaning to the numbers and tells how they relate to hypotheses.
Quantitative Design Issues
The Language of Variables and Hypotheses
Variation and Variables
The variable is a central idea in quantitative research.
A variable is a concept that varies.
Quantitative research uses a language of variables and relationships among variables.
The values or categories of a variable are its attributes.
The attribute of one variable can itself become a separate variable with a slight change in
definition.
Quantitative researchers redefine concepts of interest into the language of variables.
Types of Variables
Researchers who focus on causal relations usually begin with an effect, and then search for its
causes.
Variables are classified into three basic types, depending on their location in causal relationship.
The cause variable is the independent variable.
The effect variable is the dependent variable.
Independent variables come before other type. Independent variables affect or have an impact on
other variables. Research topics are often phrased in terms of the dependent variables because
dependent variables are the phenomenon to be explained.
A third type of variable is the intervening variable, appears in more complex causal relations. It
comes between the independent and dependent variables and shows the link or mechanism
11 Muhammad Azeem Rafi NUML Lahore
Advance Research Methods
between them. In a sense, the intervening variable acts as a dependent variable with respect to
the independent variable and acts as an independent variable toward the dependent variable.
Simple theories have one dependent and one independent variable, whereas complex theories can
contain dozens of variables with multiple independent, intervening, and dependent variables.
Causal Theory and Hypotheses
The hypothesis and Causality. A hypothesis is a proposition to be tested or a tentative statement of a
relationship between two variables.
Five Characteristics of Causal hypotheses
Has at least two variables
Expresses a causal or cause effect relationship between the variables
Can be expressed as a prediction or an expected future outcome
Logically linked to a research question and a theory
Falsifiable and can be tested against empirical evidence and show to be true of false
Words that can be used to stated causal relations
Causes Leads Is associated with Produces
Is related Influences Results If then The higher, the lower Reduces
Testing and refining hypothesis. Knowledge develops over times as researchers throughout the
scientific community test many hypotheses. A hypothesis needs several tests with consistent and
repeated support to gain broad acceptance. The strongest contender or the hypothesis with the
greatest empirical support is accepted as the best explanation at the time.
Types of Hypotheses
It is not a must that a hypothesis is always proved, it can be disproved.
Because a hypothesis make predictions, negative and disconfirming evidence shows that the
predictions are wrong.
Positive or confirming evidence for a hypothesis is less critical because alternative hypotheses may
make the same prediction.
Researchers test hypotheses in two ways: a straightforward way and a null hypothesis way.
Null hypotheses. A hypothesis stating that there is no significant effect of an independent
variable on a dependant variable. Researchers use the null hypothesis with a corresponding alternative
hypothesis or experimental hypothesis.
Alternative Hypothesis. A hypothesis paired with the null hypothesis that says an independent
variable has a significant effect on the dependant variable. When null hypothesis testing is added to
confirming evidence, the argument for an alternative hypothesis can grow stronger over time.
Double Barreled Hypothesis. It is a confusing and poorly designed hypothesis with two independent
variables in which it is unclear whether one or the other variable or both in combination produce
effect. A double barreled hypothesis puts two relationships in one hypothesis.
Clarity about Units and Levels of Analysis
A level of analysis is the level of social reality to which theoretical explanations refer. A level of analysis can
be macro or micro.
The level of analysis delimits the kinds of assumptions, concepts, and theories that a researcher uses.
12 Muhammad Azeem Rafi NUML Lahore
Advance Research Methods
The unit of analysis refers to the type of unit a researcher uses when measuring.
Common units are individual, group, organization, social category, social institution, and the society.
The units of analysis determine how a researcher measures variables or themes.
Researchers use levels and units of analysis to design research projects, and being aware of them helps
researchers avoid logical errors.
Potential Errors in Causal Explanation
Tautology. A tautology is a form of circular reasoning in which someone appears to say something new
but is really talking in circles and making a statement that is true by definition. Tautologies cannot be
tested with empirical data, e.g. IT project failure causes low funding of IT project.
Teleology. A teleology is something directed by an ultimate purpose or goal, and it takes several forms.
Teleology cannot be empirically measured. They violate the temporal order requirement of causality and
they lack a true independent variable because the “causal factor” is so extremely vague, e.g. If we invest a
lot of funding in IT project, it will be successful
Ecological Fallacy. The ecological Fallacy arises from a mismatch of units of analysis. It refers to a poor fit
between the units for which researcher empirical evidence and the units for which he or she wants to
make statements. Ecological fallacy occurs when a researcher gathers data at a higher or an aggregated
unit of analysis but wants to make a statement about a lower or disaggregated unit, e.g. majority of IS
project are unsuccessful, therefore this case study fails.
Reductionism. Reductionism or fallacy of nonequivalence occurs when a researcher explains macro
level events but has evidence only about specific individuals. It occurs when a researcher observes a lower
or disaggregated unit of analysis but makes statements about the operations of higher or aggregated units.
Spuriousness. Spuriousness occurs when two variables are associated but are not causally related
because there is actually an unseen third factor that is the real cause.
From the Research Question to hypotheses
Hints about hypotheses are embedded within a good research question.
Hypotheses are tentative answers to research question.
Several hypotheses can be developed for one research question.
A researcher can formulate a tentative research question, and then develop possible hypotheses; the
hypotheses then help the researcher state the research question more precisely.
Researchers use general theoretical issues as a source of topics.
Theories provide concepts that researchers turn into variables as well as the reasoning or mechanism that
help researchers connect variables into research question.
CHAPTER 7 – QUALITATIVE AND QUANTITATIVE MEASUREMENT
The Measurement Process
• Conceptualization. The process of developing clear, rigorous, systematic conceptual definitions for
abstract ideas/ concepts. It is a process of thinking through the various possible meanings of a construct.
• Conceptual Definition. A careful, systematic definition of a construct that is explicitly written down.
• Operationalization. The process of moving from the construct’s conceptual definition to specific activities
that allows a researcher to observe it empirically.
• Operational Definition. A variable in terms of the specific actions to measure or indicate it in the
empirical world.
13 Muhammad Azeem Rafi NUML Lahore
Advance Research Methods
• Steps
• Identify concept of interest
• Develop conceptual definition. Decide on units of analysis
• Operationalize to create a variable. Links concept to measurement technique
Quantitative Conceptualization and Operationalization
• Rules for Correspondence. Standards that researchers use to connect abstract constructs with
measurement operations in empirical social reality.
• Conceptual Hypothesis. Type of hypothesis that expresses variables and the relationship among them
in abstract, conceptual terms.
• Empirical Hypothesis. A type of hypothesis in which the researcher expresses variables in specific
empirical terms and expresses the association among the measured indicators in observable empirical
terms.
Qualitative Conceptualization and Operationalization
• Conceptualization. It is a process of forming coherent theoretical definitions as we organize the
data and our preliminary ideas about it.
• Operationalization. It involves developing a description of how we use working ideas while making
observations.
Empirical Observations Working Ideas Concepts Generalizations/Theories
Reliability and Validity
Reliability. The consistency or dependency of a measure. Does the measure consistently give the same
results?
• Reliability and Validity in Quantitative Research. Measurement reliability is the dependability and
consistency of a measure of variables. It means that the numerical results an indicator produces do not
vary because of characteristics of measurement process or measurement instrument itself. There are
three types of reliability.
• Stability Reliability. Measurement reliability across time, the measure that yields consistent results
at different time points assuming what is being measured does not itself change. Does the measure
deliver the same answer when applied in different time periods?
14 Muhammad Azeem Rafi NUML Lahore
Advance Research Methods
• Representative Reliability. Measurement reliability across groups, a measure that yields
consistent results for various social groups. Does the indicator deliver same answers when applied to
different groups?
• Equivalence Reliability. Measurement reliability across indicators, a measure that yields
consistent results using different specific indicators, assuming that all measure the same construct.
Does the measure give consistent results across different indicators?
• How to Improve Reliability. It is rare to have perfect reliability. Four things can improve reliability:-
• Clearly conceptualize constructs.
• Use of most precise (or highest) level of measurement possible.
• Use multiple indicators of a variable.
• Use of pretests, pilot studies, replication.
Validity. The truthfulness of a measure. Is it measuring what the researcher thinks it is measuring?
Measurement validity is how well an empirical indicator and the conceptual definition of a construct that the
indicator is supposed to measure ‘fit’ together. There are four types of measurement validity:-
• Face Validity. A type of measurement validity in which an indicator makes sense as a measure of a
construct in the judgment of others, especially in the scientific community. ‘On the face of it’.
• Content Validity. A type of measurement validity that requires that a measure represents all aspects of
the conceptual definition of a construct. Is the full content of definition represented in the measure?
• Criterion Validity. Measurement validity that relies on some independent, outside verification.
• Concurrent Validity. Measurement validity that relies on a preexisting and already accepted
measure to verify the indicator of the construct.
• Predictive Validity. Measurement validity that relies on the concurrence of future event or
behavior that is logically consistent to verify the indicator of the construct.
• Construct Validity. The type of measurement validity that uses multiple indicators.
• Convergent Validity. A type of measurement validity for multiple indicators based on the idea that
indicators of one construct will act alike or converge.
• Discriminant Validity. A type of measurement validity for multiple indicators based on the idea that
indicators of different constructs diverge.
Reliability and Validity in Qualitative Research
• Reliability. A wide variety of techniques such as interviews, participation, photographs and document
studies is conducted. We use range of data sources and employ multiple measurement methods.
• Validity. It means offering a fair, honest and balanced account of social life from the view point of the
people who live it every day instead of realizing a single version of ‘Truth’.
Relationship between Reliability and Validity
15 Muhammad Azeem Rafi NUML Lahore
Advance Research Methods
Internal Validity. It means we have not made errors internal to the design of a research project that
might produce false conclusions.
External Validity. It refers to whether we can generalize a result that we found in a specific setting with
a particular small group beyond that situation or externally to a wider range of settings and many different
people.
Statistical Validity. It means that we have used the proper statistical procedure for a particular purpose
and have met the procedure’s mathematical requirements.
Levels of Measurement. A system of organizing information in the measurement of variables into four
levels, from nominal to ratio level.
Continuous and Discreet Variables
• Continuous Variables. Variables that are measured on a continuum in which an infinite number of
finer graduations between variable attributes are possible.
• Discreet Variables. Variables in which the attributes can be measured with only a limited number
of distinct, separate categories.
Four levels of measurement
• Nominal Level Measurement. The lowest, least precise level of measurement for which there is a
difference in type only among categories of a variable, e.g. religion: Islam, Christianity, Judaism etc
• Ordinal Level Measurement. The level of measurement that identifies a difference among
categories of a variable and allows the categories to be rank ordered as well. Letter grades A, B, C, D, E etc
• Interval Level Measurement. The level of measurement that identifies differences among variable
attributes, rank categories and measure distance between categories but has no true zero, e.g. Celsius
temperature: 50C, 100C, 150C etc
• Ratio Level Measurement. The highest, most precise level of measurement; variable attributes can be
rank ordered, the distance between them precisely measured, and there is no absolute zero, e.g. money
income: $ 100, $ 500, $ 1000 etc
Principles of Good Measurement
• Mutually Elusive Attributes. The principle that the variable attributes or categories in a measure
are organised so that the responses fit into only one category and there is no overlap.
• Exhaustive Attributes. The principle that attributes or categories in a measure should provide a
category for all possible responses.
• Unidimensionality. The principle that when using multiple indicators to measure a construct, all
indicators should consistently fit together and indicate a single construct.
Scales and Indexes
• Index. The summing or combining of many separate measures of a construct or variable to create a
single score.
• Standardization. Procedures to adjust measures statistically to permit making an honest comparison by
giving a common basis to measures of different units.
• Scales. A class of quantitative data measures often used in survey research that captures the intensity,
direction, level or potency of a variable constructs along a continuum; most are at the ordinal level of
measurement.
16 Muhammad Azeem Rafi NUML Lahore
Advance Research Methods
• Likert Scaling. A scale often used in survey research in which people express attitudes or other
responses in terms of ordinal level categories (e.g. agree, disagree) that are ranked along a continuum.
• Thurstone Scaling. Measuring in which the researcher gives a group of judges many items and
asks them to sort the items into categories along a continuum and then considers the sorting results to
select items on which the judges agree. It is also called method of equal appearing intervals.
• Bogardus Social Distance Scale. A scale measuring the social distance between two or more
social groups by having members of one group indicate the limit of their comfort with various types of
social interaction or closeness with members of the other group(s).
• Semantic Differential. A scale that indirectly measures feelings or thoughts by presenting people a
topic or object and a list of polar opposite adjectives or adverbs and then having them indicate feelings
by marking one of several spaces between the two adjectives and adverbs.
• Guttman Scaling. A scale that researchers use after data are collected to reveal whether a
hierarchical pattern exists among responses so that people who give responses at a ‘higher level’ also
tend to give ‘lower level’ ones.
CHAPTER 8 – QUALITATIVE AND QUANTITATIVE SAMPLING
Reasons for Sampling
• Sample. A small set of cases a researcher selects from a large pool and generalizes to the population.
• Population. The abstract idea of large group of many cases from which a researcher draws a
sample and to which the results from a sample are generalized.
• Sampling Element. The name for a case or single unit to be sampled.
• Sampling Frame. A list of cases in a population, or the best approximation of them.
17 Muhammad Azeem Rafi NUML Lahore
Advance Research Methods
Sampling Strategies
Probability Sampling Technique
• Random Sample. A sample using a mathematically random method, such as random number table or
computer program, so that each sampling element of the population has an equal probability of being
selected into the sample.
• Simple Random. A random sample in which a researcher creates a sampling frame and uses a
pure random process to select cases so that each sampling element in population will have an equal
probability of being selected.
• Systematic Sampling. A random sample in which a researcher selects every kth(e.g. 3rd or 12th) case
in the sampling frame using a sampling interval.
• Stratified Sampling. A random sample in which the researcher first identifies a set of mutually
elusive and exhaustive categories, divides the sampling frame by the categories, and then uses random
selection to select cases from each category.
• Multi Stage Cluster Sampling. A type of random sample that uses multiple stages and is often used
to cover wide geographic areas in which aggregated units are randomly selected and then samples are
drawn from the sampled aggregated units or clusters.
Non Probability Sampling Techniques
• Convenience Sampling. A non random sample in which the researcher selects anyone he or she
happens to come across.
• Snowball Sampling. It uses a small pool of initial informants to nominate, through their social
networks, other participants who meet the eligibility criteria and could potentially contribute to a specific
study. The term “snowball sampling” reflects an analogy to a snowball increasing in size as it rolls downhill.
• Quota Sampling. A non random sample in which researcher first identifies general categories into which
cases or people will be placed and then selects cases to reach the predetermined number in each category.
• Theoretical Sampling. A non random sample in which the researcher selects specific times, locations,
or events to observe in order to develop a social theory or evaluate theoretical ideas.