RESEARCH IN EDUCATION: THE ESSENTIALSFIRTH MCEACHERNINTERNATIONAL RESEARCH AND EDUCATION CONFERENCE, ADVANCED AND INNOVATIVE MINDS FOR EXCELLENCE INC. NOV 28–NOV 29, 2015SUMMER PLACE HOTEL, BAGUIO CITY, PHILIPPINES

THE NATURE OF RESEARCH

Following slides adapted from Fraenkel, J & Wallen N (2007). How to design and evaluate research in education

EXAMPLES OF EDUCATIONAL CONCERNS• A department head wants to improve the

morale of her faculty• A biology professor wonders if discussions

are more effective than lectures• A P.E. instructor is curious if performance in

one sport correlates with performance in another sport

• A principal wants to get parents more involved in school activities.

WAYS OF KNOWING

• Sensory experience• Agreement with others• Expert opinion• Logic• Scientific research

SCIENTIFIC METHOD

=Testing ideas in the public arena1. Identify a problem or question2. Clarify the problem/question3. Determine what info is needed and how to

obtain it4. Organize the info5. Interpret the results

TYPES OF RESEARCH

• Experimental research• Correlational research• Causal-comparative research• Survey research• Ethnographic research• Historical research• Action research

BASICS OF EDUCATIONAL RESEARCH

THE RESEARCH PROBLEM

• A problem that someone would like to research

• Usually posed as a question

THE RESEARCH PROBLEM

PROBLEM

1. Does acupuncture provide better pain relief than painkillers?

2. What goes on in a college classroom near exam time?

3. How do parents feel about school counseling?

4. Do teachers treat the genders differently?

METHODOLOGY

1. Traditional experimental research

2. Ethnographic research

3. Survey research

4. Causal-comparative research

A GOOD RESEARCH QUESTION

• Feasible• Clear

• Significant• Ethical

VARIABLES

Variable• A characteristic/quality that varies among

members of a particular group

Constant• A characteristic/quality that is the same for

all members of a group.

SPECIFIC TYPES OF VARIABLES

• Independent variable • Selected• Manipulated

• Dependent variable• Moderator variable• Extraneous variable

HYPOTHESIS

• Refers to prediction of results usually made before investigation

• Can be advantageous and not.• Should be significant.• Directional vs. non-directional

ETHICAL RESEARCH

• Protecting participants from harm• Ensuring confidentiality • Gaining consent• Avoidance of deception except in special

circumstances

LITERATURE REVIEW

• Locate similar work and evaluate it in terms of relevance to the research question

• Beneficial for:• Gleaning ideas from other researchers• Learning about the results of similar studies

• Different sources: • General references • Primary sources• Secondary sources

STEPS OF LITERATURE REVIEW

1. Define problem as precisely as possible2. Look at relevant secondary sources3. Select and peruse one or two general

references4. Formulate search terms / keywords

pertinent to problem5. Search general refs for relevant primary

sources6. Obtain and read primary sources

SAMPLING

• Sample: the group in a study on which information is obtained

• Population: the larger group to which the researcher would like to apply the results

• Target population vs. accessible population

SAMPLING METHODSRANDOM

• Simple• Stratified• Cluster• Two-stage

NON-RANDOM

• Systematic• Convenience• Purposive

HOW MUCH IS ENOUGH?

No clear-cut answer…the sample should be as large as the researcher can obtain in reasonable amount of time and energy.

Guidelines:• Descriptive studies: 100• Correlational studies: 50• Experimental and causal-comparative: 30

per group

GENERALIZING

• Population generalizing• Generalizing results to population

• Ecological generalizing• Extending results to other settings or conditions

(e.g. other textbooks, methods, technology, content areas, etc.)

INSTRUMENTATION

= the whole process of preparing to collect data.Includes:• Selection and design of instruments• Procedures and conditions under which

instruments will be administered.

AN INSTRUMENT MUST BE…

Valid• Measures what it is supposed to measure• Accurate inferences can be made

Reliable• Gives consistent results

Objective• Subjective judgments are eliminated or minimized

Useable• Can be used in a reasonable amount of time,

instructions are clear, appropriately sensitive to audience, etc

WHERE DID THE INSTRUMENT COME FROM?• Find and administer an existing instrument

(see searchERIC.org)Or…• Develop and administer your own

instrument

WHO PROVIDES THE DATA?

• Researcher instruments • rating scales, interview schedules, observation

forms, tally sheets, flowcharts, performance checklists, anecdotal records, time-and-motion logs

• Subject instruments• Questionnaires, self-checklists, attitude scales,

personality inventories, achievement or aptitude tests, performance tests, projective devices, sociometric devices, etc.

• Informant instruments

ACHIEVEMENT & APTITUDE TESTS

NORM-REFERENCED

• Scored at 50th percentile of his group

• Ran faster than all but one of his teammates

• One of fours students to receive A’s.

CRITERION-REFERENCED

• Spelled every word in the vocab list correctly

• Attempted to solve over 75% of the questions

• Did at least 25 pushups in 5 minutes

TYPES OF MEASUREMENT SCALES

SCALE

• Nominal: uses numbers to indicate category

• Ordinal: uses numbers to rank from high to low

• Interval: uses numbers to represent equal intervals along a continuum

• Ratio: uses numbers to represent equal distances from a known zero point.

EXAMPLE

• Gender

• Position in race• Temperature

• Money

INTERNAL VALIDITY

• The outcomes of the study are closely related to the factors being studied

• The relationships between variables is unambiguous, rather than due to something else

THREATS TO INTERNAL VALIDITY• Subject characteristics• Mortality• Location• Instrumentation (e.g. instrument decay, data

collector bias)• Testing• History • Maturation• Attitude of subjects (e.g. Hawthorne effect)• Regression

DATA ANALYSIS

STATISTICS

Statistic: an index calculated for a sample which provides general numerical information about the sample

Parameter: an index calculated for a entire population

TWO MAIN TYPES OF STATISTICS

Descriptive• Used to describe, analyze, or summarize data

Inferential• Used to make inferences or tentative conclusions

about a population based on the findings from a sample.

TECHNIQUES FOR SUMMARIZING QUANTITATIVE DATA• Frequency distribution• Frequency polygon• Histogram• Stem-leaf plot• Measures of central tendency• Measures of spread• Boxplot

TECHNIQUES FOR SUMMARIZING CATEGORICAL DATA• Frequency tables• Bar graphs• Pie charts• Crossbreak tables

CORRELATION

Correlation: a relationship, either positive or negative, between two variables

Correlation coefficient: an index measuring the degree to which two variables are related.

Don’t try to correlate something that’s not correlated!

INFERENTIAL STATISTICS: COMMON CONCEPTS• Distribution of sample means• Standard Error of the Mean (SEM)

• SEM=SD(ideally pop, but can use sample) / sqrt[n-1] , where n is sample size

• 95% Confidence Interval• Confidence boundaries = sample mean +- z-value

for 2 SD * SEM

HYPOTHESIS TESTING

• Research hypothesis: specifies the predicted outcome of a study

• Null hypothesis: most commonly specifies that there is no relationship in the population (e.g. there is no difference between the pop mean of Students A and Students B)

• Rejecting the null hypothesis provides support for the research hypothesis.

TECHNIQUES FOR ANALYZING QUANTITATIVE DATAPARAMETRIC – MAKE ASSUMPTIONS ABOUT POPULATION

• T-test for means• Analysis of variance

(ANOVA)• Analysis of co-variance

(ANCOVA)• Multivariate analysis of

variance (MANOVA) and co-variance (MANCOVA)

NON-PARAMETRIC (MAKE LIMITED OR NO ASSUMPTIONS)

• Mann-Whitney U test• Kruskal-Wallis one-way

analysis of variance• Sign test• Friedman two-way

analysis of variance• Chi-square test (for

categorical data)

INFERENCE STATS (SUMMARY)

1. The end product of all inference procedures is the same: a statement of probability relating the sample data to hypothesized population characteristics

2. All inference techniques assume random sampling

3. Intended to only answer one question: given the sample data, what are the probable population characteristics?

DATA ANALYSIS TIPS FOR RESEARCHERS 1. Use graphic techniques before calculating

numerical summary indices. 2. Use both graphs and summary indices to

interpret the result of a study3. Make use of external criteria (such as prior

experience or scores of known groups) to assess magnitude of a relationship

DATA ANALYSIS TIPS FOR RESEARCHERS 4. Consider using inferential statistics only if

you can make a convincing case for the importance of the size of the relationship found in a sample (must have practical significance, not just statistical)

5. Use tests of statistical significance only to evaluate generalizability, not to evaluate the magnitude of relationships.

RESEARCH PROPOSALS AND REPORTS

RESEARCH PROPOSAL VS. REPORT

PROPOSAL

• Communicates a researcher’s plan for a study

REPORT

• Communicates what was actually done in a study and what resulted.

BOTH CONTAIN…

• Problem to be investigated• Statement of the problem or question• Research hypotheses and variables• Definition of terms

• Review of literature• Procedures

• Description of the sample• Instruments• Research design• Procedures to be followed• Identification of threats to internal validity• Description and justification of any statistical procedures used

DIFFERENCES

Proposal• Written in future tense• also includes budget.

Report• Written in past tense• Includes results • Includes discussion of implications• Includes suggestions for further research.

THE END