Research Methodology

April 19, 2023 2

Research?

• what is it? (Definition)

• should you be doing it? (Motivation)

• how do you do it? (Steps and Methods)

April 19, 2023 3

Research: Definition (1)

• “Systematic investigation towards increasing the sum of knowledge”(Chambers 20th Century Dictionary)

• “an endeavor to discover new or collate old facts etc. by the scientific study of a subject or by a course of critical investigation.”

(The Concise Oxford Dictionary)

April 19, 2023 4

Research: Definition (2)

• Research: an activity that contributes to the understanding of a phenomenon

– phenomenon: a set of behaviors of some entity(ies) that is found interesting by a research community

– understanding: knowledge that allows prediction of the behavior of some aspect of the phenomenon

– activities considered appropriate to the production of understanding (knowledge) are the research methods and techniques of a research community

April 19, 2023 5

Motivation for Research Methodology

• (qualitatively) control research process• validate research results• compare research approaches• respect rules of good scientific practice

Types of Research

• Applied vs. Fundamental• Descriptive vs. Analytical• Quantitative vs. Qualitative• Empirical vs. Conceptual• Other type of research (e.g.):

– One-time vs. longitudinal (Repeated Observation)– Field-setting research vs. laboratory or simulation

research

Steps in Research• Define research problem• Review the literature• Formulate hypothesis• Research design• Collect data• Analyse data• Test Hypothesis• Interpret

All these process could be iterative. For example, research design may need to be modified depending on data.

Quantitative or Qualitative

Quantitative Qualitative

Inquiry from the outside Inquiry from the inside

Underpinned by a completely different set of epistemological foundations from those in qualitative research

An attempt to take account of differences between people

Are simply different ways to the same end?

Aimed at flexibility and lack of structure in order to allow theory and concepts to proceed in tandem

Involves the following of various states of the scientific research

The results are said to be “hard generalized data”

The results are said to be through theoretical generalization, “deep, rich and meaningful”

Deductive Inductive- where propositions may develop not only from practice, or literature review, but also from

04/19/23 10

Research Methods

• Research methods are generally categorised as being either quantitative or qualitative.

• What matters is that the methods used fit the

intended purposes of the research!

04/19/23 11

Qualitative and Quantitative Paradigms

• The qualitative paradigm concentrates on investigating subjective data, in particular, the perceptions of the people involved. The intention is to illuminate these perceptions and, thus, gain greater insight and knowledge.

• The quantitative paradigm concentrates on what can be measured. It involves collecting and analysing objective (often numerical) data that can be organised into statistics.

04/19/23 12

Qualitative and Quantitative Research

  Qualitative Research

Quantitative Research

Also known as

interpretative / responsive

positivist /hypothetico-deductive

Type of reasoning

(usually) deductive (usually) inductive

Link withconcepts

identifies concepts identified concepts and investigates relationships

04/19/23 13

  Qualitative Research

Quantitative Research

Action sometimes only describes a situation BUT in action-research openly intervenes

tests relationships between concepts

Outcome illuminates the situation

accepts or rejects proposed theory

Approach to validity

truth seen as context bound (socially constructed)

truth seen as objectiveand universal

Qualitative and Quantitative Research

04/19/23 14

Methodology Comparison

Quantitative

• Explanation, prediction

• Test theories• Known variables• Large sample• Standardized

instruments• Inductive

Qualitative

• Explanation, description

• Build theories• Unknown variables• Small sample• Observations,

interviews• Deductive

19/04/23

Quantitative research • Involves information or data in the form of

numbers• Allows us to measure or to quantify things• Respondents don’t necessarily give numbers as

answers - answers are analysed as numbers• Good example of quantitative research is the

survey

19/04/23Multimedia Training Kit

www.itrainonline.org

Qualitative research

• Helps us flesh out the story and develop a deeper understanding of a topic

• Often contrasted to quantitative research • Together they give us the ‘bigger picture’• Good examples of qualitative research are face-to-

face interviews, focus groups and site visits

04/19/23 17

Other classification of types of research

The purposes of research can be categorised as:

• Description (fact finding) • Exploration (looking for patterns) • Analysis (explaining why or how) • Prediction (forecasting the likelihood of particular events) • Problem Solving (improvement of current practice)

04/19/23 18

Descriptive Research

• Seeks to accurately describe current or past phenomena - to answer such questions as:

a) What is the absentee rate for particular lectures? b) What is the pass rate for particular courses? c) What is the dropout rate on particular degree

programmes? d) What effect does a particularly quality audit process

have on teacher morale?

04/19/23 19

Analytical Research• Seeking to explain the reasons behind a

particular occurrence by discovering causal relationships. Once causal relationships have been discovered, the search then shifts to factors that can be changed (variables) in order to influence the chain of causality. Typical questions are:

a) Why is there a preponderance of female students on 1st level teacher training programmes?

b) What factors might account for the high drop-our rate on a particular degree programme?

04/19/23 20

Predictive Research

• Seeks to forecast the likelihood of particular phenomena occurring in given circumstances. It seeks to answer such questions as: a) Will changing the start time achieve a higher attendance rate at

our lectures? b) Will introducing anonymous marking reduce the gender

imbalance in the achievement of 1st class degrees? c) Will increasing the weighting for course work encourage students

to adopt deep learning strategies?

04/19/23 21

Problem Solving Research / Action Research

• Action-research is a form of problem solving based on increasing knowledge through observation and reflection, then following this with a deliberate intervention intended to improve practice.

Educational action-research describes a family of activities in curriculum development, professional development, school improvement programmes, and systems planning and policy development.Participants in the action being considered are intricately involved with all of these activities.

04/19/23 22

Typical Methods Descriptive Research

Statistical SurveysSamplingInterviews

Analytical Research

Case Studies Attitude SurveysObservations Statistical SurveysHistorical Analysis

Predictive Research

identifying and / or defining measurable (quantifiable) variables and manipulating them to cause measurable.

Problem Solving/Active Research

action-research spiral: observe reflect plan act observe reflect plan act ………

04/19/23 23

Research Methods Categorised by Activity

Experimental Research

The causal effects of phenomena are tested on one group by comparison with a control group which is otherwise similar but upon which the phenomena is not allowed to act.

Quasi-Experimental Research

Causal effects of phenomena are investigated in a way similar to experimental research BUT full control is not possible

Non-Experimental Research

The investigation of conditions as they really are without an attempt to change any of them - at least while the research is in progress.

19/04/23

Participatory research

• Allows participation of community being researched in research process (e.g. developing research question; choosing methodology; analysing results)

• Good way to ensure research does not simply reinforce prejudices and presumptions of researcher

• Good for raising awareness in community and developing appropriate action plans

Quantitative Research

ContentTheoretical Research: Concepts and MethodsStatistical Research: E.g. Multivariate Analysis (Examples)

– Paradoxes (e.g. Simpson’s Paradox)– Cluster Analysis– SEM (Structural Equation Modeling)

Mathematical Programming– Optimization– Game Theory (and Equilibrium Analysis)

Integrative Analysis (Qualitative & Quantitative Research)– Scenarios Analysis

Theoretical/Fundamental Research: Concepts

• Concepts – Probability (Objective vs. Subjective)– Risk

• Axioms (Assumptions)• Proofs in Mathematics

– Proof by construction– Proof by contradiction

Statistical Research Methods

• Experimental• Non experimental

Multivariate Analysis

• Many statistical techniques focus on just one or two variables

• Multivariate analysis (MVA) techniques allow more than two variables to be analysed at once– Multiple regression is not typically included under

this heading, but can be thought of as a multivariate analysis

Outline of Lectures

•Why MVA is useful and important•Simpson’s Paradox•Some commonly used techniques

– Principal components– Cluster analysis– Correspondence analysis– Others if time permits

•Market segmentation methods•An overview of MVA methods and their niches

Simpson’s Paradox (1)

• Example: 44% of male applicants are admitted by a university, but only 33% of female applicants

• Does this mean there is unfair discrimination?

• University investigates and breaks down figures for Engineering and English programmes

Male Female

Accept 35 (44%) 20 (33%)

Refuse Entry

45(56%) 40 (67%)

Total 80 (100%) 60 (100%)

Simpson’s Paradox (2)

• No relationship between sex and acceptance for either program– So no evidence of

discrimination• Why?

– More females apply for the English program, but it is hard to get into

– More males applied to Engineering, which has a higher acceptance rate than English

• Must look deeper than single cross-tab to find this out

Engineering Male Female

Accept 30 (50%) 10 (50%)

Refuse entry

30 (50%) 10 (50%)

Total 60 (100%) 20 (100%)

English Male Female

Accept 5 (33%) 10 (33%)

Refuse entry

15 (67%) 30 (67%)

Total 20 (100%) 40 (100%)

Simpson’s Paradox (3)• In each of these examples, the bivariate

analysis (cross-tabulation or correlation) gave misleading results

• Introducing another variable gave a better understanding of the data– It even reversed the initial conclusions

• The practical significance of Simpson's paradox: – Which data should we consult in choosing an action,

the aggregated or the partitioned?

Treating Many Variables

Many Variables• Commonly have many relevant variables in market

research surveys– E.g. one not atypical survey had ~2000 variables– Typically researchers pore over many crosstabs– However it can be difficult to make sense of these, and the

crosstabs may be misleading• MVA can help summarize the data

– E.g. factor analysis and segmentation based on agreement ratings on 20 attitude statements

• MVA can also reduce the chance of obtaining spurious results

Cluster Analysis• Techniques for identifying separate groups of similar

cases– Similarity of cases is either specified directly in a distance

matrix, or defined in terms of some distance function• Also used to summarize data by defining segments

of similar cases in the data– This use of cluster analysis is known as “dissection”

Clustering Techniques

• Two main types of cluster analysis methods– Hierarchical cluster analysis

• Each cluster (starting with the whole dataset) is divided into two, then divided again, and so on

– Iterative methods• k-means clustering (PROC FASTCLUS)• Analogous non-parametric density estimation method

– Also other methods• Overlapping clusters• Fuzzy clusters

Applications• Market segmentation is usually conducted using

some form of cluster analysis to divide people into segments

• It is also possible to cluster other items such as products/SKUs, image attributes, brands

• Search result grouping: For intelligent grouping of the files and websites, clustering may be used to create a more relevant set of search results compared to normal search engines like Google. There are currently a number of web based clustering tools such as Clusty.

Cluster Analysis Issues• Distance definition

– Weighted Euclidean distance often works well, if weights are chosen intelligently

• Cluster shape– Shape of clusters found is determined by method, so choose method

appropriately• Hierarchical methods usually take more computation time

than k-means• However multiple runs are more important for k-means, since

it can be badly affected by local minima• Adjusting for response styles can also be worthwhile

– Some people give more positive responses overall than others– Clusters may simply reflect these response styles unless this is

adjusted for, e.g. by standardising responses across attributes for each respondent

Structural Equation Modeling (SEM)

• SEM is a statistical technique for testing and estimating causal relations using a combination of statistical data and qualitative causal assumptions.

• General method for fitting and testing path analysis models, based on covariances

• Fits specified causal structures (path models) that usually involve factors or latent variables– Confirmatory analysis

SEM Example:Relationship between

Academic and Job Success

ACT: American College TestingCGPA: Cumulative Grade Point Average

Mathematical Modeling

What use a model?• To reflect complex systems in a simplified form.• Helps to organize large amounts of data.• Examples:

– Economic Policy Models (I-O Analysis) – Environment Policy Models – Product Mix Models– Optimal Pricing Models

Modeling and Simulation

MARKet ALlocation Model:MARKAL

Multi-period linear programming formulation Decision variables in different time periods(e.g.): Energy consumption Electricity generation Capacity utilization Investment in technologies Emissions

MARKAL Model: Overall Architecture

Techno-economic Database

Economic Scenario

EmissionScenarioMARKAL

• Consumption and production of energy• Marginal ‘values’ of energy resources & technologies• Shadow price of external constraints (e.g. emissions)• Introduction and retirement of technologies

Bottom-up 3E (Energy-Economy-Environment) Model System

MINING IMPORT COLLECTION RENEWABLE EXPORT

COALN. GAS

OILBIOMASSNUCLEAR

RENEWABLE

45

ENVIRONMENT

ELECTRICITY PRODUCTION

COAL GAS HYDRO

NUCLEAR SOLAR

ENERGY

FUEL PROCESSING

PETROLEUM REFINERY GAS PROCESSING

75

ENDUSE DEVICES

PUMP TRACTOR

FURNACE MOTOR

BUS TRAIN

CFLTVOVENAC

AGRICULTURE

INDUSTRY

TRANSPORT

COMMERCIAL

RESIDENTIAL

67

235

TECHNOLOGY CAPITAL

EMISSIONS

ENERGY

ECONOMY SECTORS

Agr

Ind

Tran

Bldg

Typical Reference Energy System

MINING

IMPORT

IMPORT

IMPORT

IMPORT

COAL

GAS

REFINERY 2

REFINERY 1

ELECTRIC TRAIN

DIESEL VEHICLE

TRANSPORTEND USE

TECHNOLOGIES

GENERATION+T&D

PETROLEUM PROCESSING

EN

ER

GY

RE

SO

UR

CE

S

D E

M A

N D

ELECTRICITY TECHNOLOGY

EXTRACTION

GAS

COAL

OIL

EXTRACTION

EXTRACTION

OIL

Model Formulation

Objective FunctionTo minimize the discounted sum, over up to 100 years, of investment, operating and maintenance cost of all technologies plus the cost of energy imports and carbon tax

Subject to1. Demand Constraints (one for each end use demand)

Cig(t) >= demandk (t)

i DMD G GRD

V k DM, t T

WhereDMD…end-use demand technologyGRD…set of grades technologies/energy sourcesDM….class of all end use demandsT…..set of time periodsCig(t)…capacity of technology i of grade G in period t

Objective Function (cntd)

1. Capacity transfer constraints (to account for technology vintage carry over time periods)

2. Energy carrier balance constraints (supply >= demand of fuel)

3. Cumulative reserve constraints (fuel extraction <= total reserves)

4. Electricity balance constraints(day and night time modeling for electricity system)

Model Constraints

5. Process technology capacity utilization constraints (process activity <= available capacity)

6. Electricity production capacity constraints (electricity generation <= available capacity)

7. Electricity peaking constraints (extra capacity to meet peak demand)

8. Total emissions constraints (Carbon, SO2 etc)

Model Constraints (cntd.)

Game theory is a study of strategic decision making. More formally, it is "the study of mathematical models of conflict and cooperation between intelligent rational decision-makers.

Game Theory is descriptively also referred to as interactive decision theory.

Game Theory

Prisoners'-Dilemma• A paradox in decision analysis in which two individuals

acting in their own best interest pursue a course of action that does not result in the ideal outcome.

• Dominant strategy: Not to cooperate– If both remain silent, both win– If one betrays the other, than he benefits and the other loses– If both betrays, both loses

Prisoner B stays silent (cooperates)

Prisoner B betrays (defects)

Prisoner A stays silent (cooperates) Each serves 1 year Prisoner A: 3 years

Prisoner B: goes free

Prisoner A betrays (defects)

Prisoner A: goes freePrisoner B: 3 years Each serves 2 years

Prisoners‘ Dilemma

Computational General Equilibrium (CGE) Model

AGRICULTUREgrains and oil seeds

animal productsforestry

food processingother agricultural services

HOUSEHOLDSdemographics

labor supplyENERGY land supply

oil production household savingsPRIMARY FACTORS gas production final product demands

OF PRODUCTION petroleum refiningland surface gas distribution

subsurface resources coke and coal productslabor biomass production

capital uranium productionhydro and solar electric power GOVERNMENT

electricity production general governmentnational defense

education

EVERYTHING ELSEpaper and pulp manufacture

chemical manufacturecement manufacture

primary iron and steelprimary non-ferrous metals

other manufacturingpassenger transport

freight stransportother services

SGM: Second Generation Model is a Global CGE Model (PNL, USA)

Integration of Qualitative and Quantitative Analysis

• Scenarios Analysis• Projections of Future vs. Prediction• Qualitative Storyline• Used for long-term analysis (e.g. economic and

environmental futures)

• Case Study• Projections of Future vs. Prediction• Qualitative Storyline• Used for long-term analysis (e.g. economic and

environmental futures)

Case study

• Definition : an empirical inquiry that investigates a contemporary phenomenon within its real-life context; when the boundaries between phenomenon and context are not clearly evident(Yin, 1984)

• Case study research emphasize detailed contextual analysis of a limited number of conditions and their relationships

• Scenarios: Images of future or alternate futures.

• Scenarios are not Predictions or Forecasts.• Used as a methodology in energy &

environmental analysis: Account for future uncertainties in energy planning and to study likely implications of current policy pathways.

Scenario Analysis

Example

Reference : http://www.ft.com/cms/s/1/4d46cbae-2b4f-11da-995a-00000e2511c8.html

Thank You