1

Doing Research in Economics

Professor Charlie Karlsson

Jönköping International Business School and CESIS

2

Agenda

• What is research?• The research process in economics• Surveying the literature• Writing as a tool for economic research• Writing as a product of economic research• Critical reading• Theorising• Locating data• Manipulating data• Empirical testing• Communication of results

3

1. What is research?

• Research is the creation of new valid knowledge, i.e. an extension of the research frontier

• Research is based upon the existing up-to-date knowledge in the field

• Knowledge is more than data or information – it is structured information

• Knowledge is created by constructing a line of arguments– An argument is an assertion or a claim supported by

reasons or evidence

4

The construction of knowledge

• Scholars create knowledge by constructing competing arguments using the following tools:– Mental Processes – thinking about an argument– Oral Discourse – verbalising the argument– Diagram Techniques – illustrating the argument– Mathematic Techniques – manipulating equations to

test the logic of the argument– Writing – to present and spread the argument

5

How are arguments evaluated?

• What are the reasons behind the argument?• Does the argument make sense? Why or why not?• Is the logic flawed?• What are the underlying explicit and implicit

assumptions? Are they flawed?• How critical are the assumptions, i.e. would different

assumptions lead to different conclusions?• What is the empirical evidence? Does it support the

conclusion?• In light of the reasons and evidence provided, is the

argument persuasive? If so, the argument is valid until it is invalidated by new arguments.

6

The scientific method

• Select a scientific problem or question• Apply a theory to derive hypotheses about

the problem or question• Test the hypotheses by comparing its

predictions to evidence from the real world• If a hypothesis fails the test check the line

of arguments and if it is OK reject it• If you can’t reject a hypothesis you

complete your line of arguments

7

2. The research process in economics

• Develop a well defined research question• Survey the literature in the field• Define a clear purpose• Select a theory proper for the research question• Develop hypotheses• Test your hypotheses• Interpret your results and draw conclusions• Communicate your research

8

The objective of a research project

• is to analyse some aspect of a significant issue or problem

9

Step 1: Defining the scope of the research

• What is the research topic?

• What is the research question?

• What is the tentative research hypotheses?

• What is the current knowledge, i.e. where goes the research frontier?

• What is the purpose?

10

What characterises a good research question?

• Problem-oriented

• Analytical

• Interesting and significant

• Amendable to economic analysis

• Feasible, given the time and resources available

11

Strategies for selecting research questions (1)

• Pick a general topic area that interests you, ideally one in which you have some background

• Start reading the literature, not merely to see what has been done, but also to identify what research questions remain to be answered or what problems remain to be solved.

12

Strategies for selecting research questions (2)

• Select a promising from what you have found in the literature:– Could an interesting previous study be

applied to a new place or time?– Are there conflicting findings on some

question, which you might try to reconcile? – Studies often conclude suggesting questions

for future research.– The literature survey may reveal gaps in the

current knowledge that you can explore.

13

Step 2: Surveying the literature

• What is currently known?

• What has been discovered to date on a given topic?

• Objective: to identify and become familiar with the major studies that have been published on a topic

• Start with the most recent publications and work backwards to the roots

14

Step 3. Selecting one theory

• Based upon your literature survey you must choose one theory, which is relevant for explaining your research problem

• Motivate your choice

• Never spend time on long presentations of alternative theories – instead give references to literature, where competing theories are presented

15

Step 4: Analysing the problem

• The theoretical analysis of the research problem, is the process, where theory is applied to shed light on the problem:– What are the essential concepts comprising the problem being

analysed?– How are these essential concepts related?– What do these relationships imply?

• The result of this analytical process is the research hypothesis (hypotheses).

• A research hypothesis is the proposed answer to your research question.

• Hypotheses are derived from economic theory (and earlier empirical research)

16

Step 5: Testing your analysis (1)

• The scientific method in economics is strongly dependent upon empirical testing

• Empirical testing implies comparing the predictions of theory, i.e. the hypotheses, with appropriate real-world evidence

• You must decide how you will go about to test your hypotheses, i.e. decide your research design, which involves

1. Finding a good, large and relevant data set, and2. Selecting an econometric method

17

Step 5: Testing your analysis (2)

• Key questions: – How to choose an econometric method, which

produce the best possible and most reliable results?– How to adequately test the hypotheses?

• Remember that a hypothesis is either rejected or not rejected. It is never accepted!

• If you can not reject a hypothesis, then the theoretical proposition is provisionally accepted until it eventually is rejected

18

Step 5: Testing your analysis (3)

• If a hypothesis is rejected then one must ask the following questions:– Is there something wrong with the theory?– Is there something wrong with the data?– Is there something wrong with the econometric

method?

• If you find that something is wrong you have to retake the process, otherwise you continue to present your results in the thesis

19

Step 6: Interpreting the results and drawing conclusions (1)

• What are the results of the empirical testing of the research hypothesis?

• Are they consistent with the predictions of theory?

• Are they consistent with the results form earlier empirical research?

• Are there any problems (multicollinearity, heteroskedasticity, etc.) with the econometric testing that need to be corrected?

20

Step 6: Interpreting the results and drawing conclusions (2)

• Are there shortcomings with the econometric method that limit or weaken the results?

• Given the answers to the above questions: what can be concluded about the results?

• To what extent are they in line with the hypotheses?

• What can be concluded about your analysis and about your research question more broadly?

21

What is good research?

• Good research, is research that follows the scientific method, wherever the results lead, even if they reject one or several hypotheses

• A research projects that rejects a hypotheses is not failed because it still advances our knowledge – in this case by eliminating one hypothesis as an explanation to the research problem

22

Step 7: Communicating the findings of the research project

• A written report where the author makes a case for the validity of her/his results based on the logic, rigour and empirical evidence of the research.

• Oral presentation (and defence) at the final seminar

• Oral presentation at conferences• Publication as a working paper• Publication in a scientific journal (or in an edited

book)

23

Writing a research proposal

1. Statement of the nature of the problem

2. The research question

3. Survey of the literature

4. Research design

5. References

24

3. Surveying the literature

• Why is a literature survey necessary?

• Where to search: popular literature vs scholarly literature vs internet sources

• How to search: Developing an effective search strategy

• Obtaining the resources

25

Why is a literature survey necessary?

• To advance the state of knowledge, you need to know what the state of knowledge is

• You must create your own sense for what is known and what is not known

• The literature provide ideas for your own research

• The literature helps you to design your own study by showing how previous approaches either were or were not successful

26

The quality hierarchy

• Scientific journals with peer review

• Dissertations

• Edited books with peer review

• Monographs published by scientific publishers

• Working papers

• Internet sources

27

Where to search?

• EconLit• JStore• IDEAS/Repec• Palgrave’s• Handbooks• Collected volumes• Journal of Economic Literature (JEL)• Journal of Economic Perspectives• Course literature

28

How to search?

• To locate information efficiently, one needs to use a search strategy

• Two general approaches:– Browsing– Keyword searching

29

Browsing

• Manual examination of written material for useful information or references to useful information

• The American Economic Association and JEL use a hierarchical system to classify information in economics: http://www.aeaweb.org/journal/elclasjn.html

30

Keyword Searching

• Keyword searching use search engines – on the World Wide Web such as Goggle and

Google Scholar or – on specialised data bases such as EconLit or

Social Science Citation Index (SSCI)

• Remember to – start with the most recent years– Use Boolean searching (AND, OR, NOT) to

make your search more precise

31

A basic search strategy

• Begin by stating your research topic or question• Identify important concepts related to your topic• Brainstorm to create a list of keywords that describe

these concepts• Determine which search features may apply• Choose the appropriate database• Read the search instructions for the database• Create a search expression using the appropriate syntax• View the results• Modify the search if necessary• Try the same search with an other data base

32

Obtaining the material

• More and more journal articles and working papers are available in full-text

• Some of them you can reach via EconLit at KTH Library.

• You may also test: http://rfe.org

• Other resources you may test are www.nber.org http://papers.ssrn.com http://econwpa.wustl.edu

33

REMEMEBER!

• You need to collect a substantial material to cover the current knowledge and to get ideas for your own research

• Think like this: your thesis must contain a list of relevant references at least two pages long, i.e. at least 40 references

34

Scholarly references and citation styles (1)

• We strongly recommend that you use the parenthetical form for references in the text, for example Lööf (2005)

• You may consult – Turubian, K. (1996), Manual for Writers of Term

Papers, Theses and Dissertations, 6th ed., for general references and citation and

– Harnock, A. & E. Kleppinger, Online: A Reference Guide to Using Internet Sources available at http://www.bedfordstmartins.com/online for citation of online documents

35

Scholarly references and citation styles (2)

• There are three major schools for referencing and citations:– Modern Language Association (MLA)

available at http://www.wisc.edu/writing/Handbook/DocMLA.html

– American Psychological Association (APA) available at http://owl.English.purdue.edu/handouts/research/r_apa.html

– The Chicago style avilable at http://www.liu.edu/cwis/

cwp/library/workshop/citchi.htm

36

4. Using writing as a tool for economic research

• Writing to learn• Composition as a creative process• The structure of an argument• Examining an argument• Three types of reasoning: deductive,

inductive and warrant-based• What makes for a persuasive argument?• An important caveat

37

Writing to learn

• Economist use writing for two purposes:– Writing as a Product, a form of communication

to disseminate research results– Writing as a Process for deriving the research

results

• Writing forces you to think concretely, i.e. to figure out exactly what you mean.

• Writing is a tool of discovery

38

Composition as a creative process (1)

• The process of writing is called composition, which includes– analysis, i.e. taking something apart to

understand it, and – synthesis, i.e. putting pieces together to make

a whole

• It involves searching for relationships between facts, theories and ideas that make up the raw material for your research

39

Composition as a creative process (2)

• Composition needs time to develop and to mature

• Hence, it is important – to start early– to write a draft– to discuss the main ideas with supervisors

and colleagues– to take time off from your writing to allow your

sub-conscious to do its job

40

Composition as a creative process (3)

• Refining a thesis or a paper implies– re-viewing the information– re-thinking the way it is organised– questioning the theoretical framework– questioning the hypotheses– re-constructing the arguments– re-viewing the data– questioning the econometric methods– looking for new patterns of meaning

41

Composition as a creative process (4)

• Creative and critical writing almost always implies that you have to throw away parts of what you have written

• Always concentrate on the most relevant and most important aspects

• In Economics, we always disregard aspects of lesser relevance and importance

• What you shall provide is a structure, which highlights the most influencing factors in explaining a phenomena but never try to make a catalogue of all factors that may influence a phenomena

42

The structure of the argument (1)

• The purpose of scholarly writing is to make an argument that is persuasive to experts in the field

• When completed, scholarly writing follows a logical, hierarchical structure, in which the main thesis is supported by a series of nested arguments that lead logically to the thesis as a conclusion

43

The structure of the argument (2)

• The thesis is at the top

• It is supported by a number of major reasons

• Under each major reason there is a number of supporting arguments

• Points that do not lead to the thesis either directly or indirectly shall be omitted

44

What does it mean to say that a conclusion follows from the evidence?

• An inference is a conclusion reached after reasoning logically about facts and relationships

• If the evidence leads us logically to the inference as a conclusion, then we say that the conclusion ”follows”

• A logic fallacy is an argument that is flawed because the conclusion does not actually follow from the reasons stated

45

Logical fallacies (1)

• Straw man – mischaracterizing a position by omitting its strongest reasoning

• Special pleading – selectively using the available evidences

• Begging the question – making an assertion in which the reason given doesn’t really support the conclusion

• Affirming the consequent – drawing conclusions based on unexamined premises

46

Logical fallacies (2)

• Ad hominem – refuting the argument by attacking the person, rather than his/her arguments

• Appeal to authority – accepting an argument because an expert endorses it

• Appeal to the people – accepting a position because many others do, without examining the argument

• Post hoc, ergo propter hoc – what comes before was the cause

47

Logical fallacies (3)

• Fallacy of composition – what is true at the micro level must be true at the macro level and vice versa

• Appeal to pity – using sympathy for one issue as justification for another issue

• False analogy – Drawing parallels between two cases where there are enough substantive differences to question the comparison

48

Examining an argument

• Identify the major claim• Identify the major evidences• Identify the supporting points• Is there a logical sequence of reasoning?• Are there any logical fallacies?• Does the evidences and the supporting points

lead to the claim?• Don’t be afraid to criticise reasoning that isn’t

logical and supported by evidence

49

Three types of reasoning (1)

• Deductive reasoning starts from one or more general principles and derives specific predictions from them

• The predictions are deductions• A valid deduction is one which the

conclusion must follow from the premises• When scholars in economics theorize,

they are typically using deductive reasoning

50

Three types of reasoning (2)

• An inductive argument is one that reasons in the opposite direction from deduction

• Given some specific cases, what can be inferred about the underlying general rule?

• The reasoning process follows the same steps as in deduction

• The difference is the conclusions: an inductive argument is not a proof, but rather a probalistic inference

• When scholars use statistical evidence to test a hypothesis, they are using inductive logic

51

Three types of reasoning (3)

• Warrant-based reasoning• Warrants are un-stated or underlying

assumptions on which an argument stands• Often warrants are higher-order assumptions or

axioms that are not testable, e.g. the assumption that consumers maximise utility

• The purpose of a warrant is to establish the relevance of the evidence in supporting some claim

52

What makes for a persuasive argument? (1)

• For an argument to be persuasive, the reasons supporting it must be true, and the conclusions must follow from the reasons

• The evidence should be accurate, authorative, precise, clearly explained, complete and representative

53

What makes for a persuasive argument? (2)

• Factual evidence needs to be accurate – always control that the facts you build your arguments upon are true

• Evidence should also be authorative, e.g. data must come from a reliable source

• Evidence needs to be precise – try, for example to estimate the size of an effect instead of just claiming that many are affected

• Evidence must be clearly explained – it is not enough to just present tables with econometric estimations, they must be explained in writing

54

What makes for a persuasive argument? (3)

• Evidences need to be complete, i.e. they should have depth and breadth

• The evidences presented in your argument should be representative of thought on an assertion, i.e. you shall not only report those evidences, which supports your assertions – with other words you should be intellectually honest

55

An important caveat

• A conclusion can follow from the evidence, the evidence can be correct, and the argument may still be incorrect

• Internal consistency in an argument is a necessary but a sufficient condition, if another conclusion explains the evidence better and more correctly

56

5. Writing as a product of economic analysis

• What is economic writing?

• Writing steps

• Writing the first draft

• Revising the paper

• Writing style

• Writing mechanics

• Proof-reading

57

Writing as a product

• Writing as a product is the report from the research process

• The audience is someone other than the author – in your case economists at the master level

• Scholarly writing embodies an argument that attempts to persuade experts in the field

• The writing needs to be explicit and formal• All important elements needs to be spelled out clearly

and in sufficient detail to get your point across• Proper standards of punctuation and grammar must be

followed

58

What is economic writing?

• What fundamentally distinguishes economic from other disciplinary writing and what all types of economic writing share is the use of economic analysis

• All economic writing applies economic theory to derive insights about and explain answers to a question or a problem

59

Writing steps

1. Pre-writing or exploration – writing the research proposal (spend relative much time for this step)

2. Writing the first draft (spend relative less time for this step)

3. Revising (several times) (spend much time for this step)

4. Editing (if you apply the writing rules from the very beginning and are careful with details you need relative little time for this step)

60

Writing the first draft

• Features of good economic writing

• Getting the ideas down on paper

• Giving credit for intellectual property

61

The first draft

• One of the hardest steps in completing a paper

• Prepare the first draft by writing throughout the research process:– Taking notes as you research the topic– Writing critiques– Drafting your own ideas

• As a first step define the audience for your paper

62

Features of good economic writing

• Good writing should be – Focused, i.e. have a clearly defined purpose– Organised, i.e. follow a logical, hierarchical

structure– Solidly developed, i.e. major points must be

explained in detail and supported by evidence– Clear, concise and precise– Free of grammatical errors

63

Getting the ideas down on paper (1)

• Focus on getting your basic ideas down on paper

• Don’t worry about details, mechanics or grammar at this step

• Always take notes about what you have done and where you are going next before you end a work session

• Try to get long work sessions, preferably 3-4 hours

64

Getting the ideas down on paper (2)

• Create an outline of what you are trying to say, i.e. a disposition and a synopsis, OR

• do brainstorming, writing down your ideas as they appear and only then create an outline to organise them

• Try to find out which approach suits you best• Remember that a research paper should explain

what you found, not what you did, i.e. readers are not interested of the steps in the process that you went through writing the paper

65

Giving credit for intellectual property

• Totally avoid plagiarism, i.e. taking credit for someone else’s words or ideas, even when it’s unintentional

• Two types of plagiarism:– Using someone else’s words as if they were your own

– thus, if you quote use quotation marks and give proper reference but do not use quotations excessively

– Using someone’s unique idea without attribution – thus, be generous with references

66

Revising the paper

• Every first draft can be improved by revision!

• There must be several revisions between the first draft and the final draft

• To attain the full potential of the writing process you need start drafting far enough before your deadline to have time to do multiple revisions

67

Is the thesis clear?

• The purpose of revision is to craft your paper so it better embodies the features of good writing:– Does your paper has a clear focus, i.e. does it only

contain one idea or theme?– Is the thesis or principal assertion of the paper clear?– Can you underline the thesis sentence in the paper’s

introduction?– Can you underline the corresponding sentence in the

paper’s conclusion?

68

Is the paper well organised? (1)

• Is the organisation logical?

• Each chapter, section and paragraph should be the explication of a single thought, idea or theme – if not you should split them up

• They should all have a thesis sentence, typically the first sentence

69

Is the paper well organised? (2)

• It must be possible to identify a ”red thread” throughout the paper, i.e. it must be easy to follow your line of argumentation

• You must provide services to the reader by making the theme of each part explicit but also by summarising your arguments at the proper places

70

Are your points supported by evidence? (1)

• Examine the development of each major point in your paper

• Each point is itself an assertion that needs to be supported by evidence

• If the first sentence in a chapter, section or paragraph spells out the main point, the reminder should flesh out and support that main point

71

Are your points supported by evidence? (2)

• Does the main point need to be explained in more detail?

• Can you provide examples of what the main point says?

• What makes you think that the main point is valid?

• What evidence can you provide to bring the reader to that conclusion?

72

Writing style

• Strive for clarity

• Use the active voice

• Describe action with a verb

• Be precise and concise

• Let Microsoft Word help you

• Dictionary and thesaurus

73

Strive for clarity

• Compose each sentence so that the subject is the main actor of the story, and the verb is the main action

• Whenever possible use strong verbs over weak ones

• Don’t try to express complex ideas that you do not fully understand

• Take responsibility for what you write – don’t be vague

74

Use of active voice

• Passive voice is not incorrect per se, but it makes it harder for the reader to figure out what exactly you are saying

• Don’t think that the use of passive voice make your writing sound more objective

• The reader needs to know who stands behind the arguments – you should not be afraid of arguing actively

75

Describe action with a verb

• Academic writers tend to nominalize, i.e. to turn verbs into nouns

• However, it makes the writing sound more pretentious and harder to read

• Nominalization tends to add words to sentences without adding meaning

• To write clearly choose strong verbs to describe the actions in your sentences

76

Be precise and concise

• Discussing complex issues often requires nuance, i.e. word choice matters, since synonyms have slightly different meanings

• Always choose the word that means exactly what you wish to say to achieve precision

• Also try to make every point as concisely as you can, i.e. avoid empty words

• The quality of a paper never depends upon its length but – a shorter paper making the necessary points is always better than a longer paper

77

Let Microsoft Word help you

• Go to Tools/Options/Spelling and Grammar and select ”Grammar & style” under Writing style

• Under Spelling see to that you don’t ”Hide spelling errors in this document”

• Under Grammar see to the you don’t ”Hide grammatical errors in this document”

78

Dictionary and thesaurus

• If English isn’t your mother tongue you a large, good dictionary

• Everyone needs a large and good thesaurus

79

Writing mechanics

• Use complete sentences

• Don’t let sentences run on

80

Use complete sentences

• Serious writers employ complete sentences

• A complete sentence implies a complete thought, while a sentence fragment implies a fragmented or incomplete thought

• A sentence fragment is a sentence without a subject or without a verb or a sentence that is incomplete in another way

81

Don’t let sentences run on

• A run-on sentence is two or more independent clauses that are not separated with the proper punctuation

• In a run-on sentence several complete thoughts are jammed together, making it difficult for the reader to determine where one ends and the next begins

82

Proof-reading

• We naturally assume that every student uses the spell-checking to avoid spelling errors and simple grammatical errors

• However, spell-checking is not enough, which implies that you must always leave yourself time to proof-read your final draft very closely.

• To perform this task properly requires considerable time and concentration, because proof reading quickly turns into superficial skim-through unless it is done in ‘quality time’ and with academic breaks.

83

6. Critical reading or how to make sense of published research

• Making sense of published research

• Taking research notes and writing abstracts and critical reviews

84

Making sense of published research

• Understanding format

• Evaluating the argument: reading critically

• Questions to guide critical reading

• Evaluating published research

• More questions to guide critical reading

85

Understanding format

• Economics research papers tend to follow a common format that illustrates the scientific method

• In economics, there are three types of scholarly work:– A survey of the work of others– A purely theoretical study– An empirical study

86

The typical empirical research study

• An introduction

• A theoretical analysis of the problem

• An empirical test of the hypotheses derived in the theoretical analysis

• A conclusion

87

The introduction

• should – define the general topic and the specific

research question– explain the motivation for the research– review the work of previous researchers on

the topic, especially• what is lacking in the existing literature, and • how the current study proposes to address that

shortcoming

88

The theoretical analysis

• is the heart of any economics study

• is the application of theoretical economic analysis to shed light on the research question

• develops the theoretical model used by the study

• derives the testable implications or hypotheses of the model

89

The empirical analysis

• explains how the proposed analysis from the theoretical exercise is tested

• states explicitly what results would confirm the theoretical propositions

• presents the results obtained from the testing procedures and interprets them

• answers the question: To what extent is the theoretical propositions rejected or not

90

The concluding sector

• explains the insights learned from the research

• What answer did economic theory suggest for the research question?

• Was this answer rejected or not by the empirical evidence?

• How to interpret the results?• What research questions should be given

priority in future research?

91

Evaluating the argument: reading critically

• To understand a research paper you need to read deeply and critically and to understand it’s arguments

• This implies that you don’t just read the papers, you study them to discern and evaluate the author’s arguments

• To save on time you the first time skim the paper to see whether it is useful

• If you find it useful you read it again more carefully

92

Questions to guide critical reading (1)

• What question is the author asking?

• What answer does the author propose (i.e. what are the principal assertions of the study)?

• In what way does the study improve upon previous research?

• How does the proposed answer compare with those provided by previous research?

93

Questions to guide critical reading (2)

• What are the major logical or theoretical reasons for the author’s argument?

• What empirical evidence does the author provide?

• What type of econometric techniques do the author apply?

• Are there any problems with the econometric results?

• What assumptions are the author making in his her reasoning?

94

Evaluating published research

• Once you have identified the argument in a published work, the next step is to evaluate the argument, i.e. to assess its validity and reliability

• Does the author have an apparent conflict of interest?

• Is the study published in a refereed journal?• If not, in what way has the quality of the

argument been checked?

95

More questions to guide critical reading

• Does the theoretical analysis make sense?• Are the data used adequate to the task?• Does the empirical methodology adequately test

the hypotheses?• Are the assumptions reasonable?• Is the analysis (theoretical and empirical) clearly

explained?• Do the conclusions follow from the evidence

presented?• On balance, is the author’s argument convincing

to you?

96

REMEMBER!

• The questions for critical reading are also very useful when you shall prepare the discussion of a paper at a seminar or conference

97

Taking research notes and writing abstracts and critical reviews (1)

• When you take notes on a reading you read it more carefully

• The notes are addressed to yourself and preferably you save them in a computer file

• Always record the complete bibliographic information

• Make your own copies of complete articles and book chapters that are important to your research project

98

Taking research notes and writing abstracts and critical reviews (2)

• You can write more formal notes in form of– an abstract (a summary of the author’s

arguments), or – a critical review, i.e. an abstract augmented

with a critical evaluation of the work

• You may also construct an annoted biography, i.e. a list of references that includes a few sentences summarising and critiquing each item

99

7. Theorising or conceptualising the research

• What does it mean to apply theory to a research topic?

• What is theorising?

• Narrative reasoning

• Mathematical reasoning

• A commonly used shortcut: modifying an existing model

• What makes a good research hypothesis?

100

The development of a theoretical framework

• is the most abstract part of the research process,

• requires both analysis of the research problem and synthesis of an appropriate theoretical framework to explain it, and

• requires sufficient knowledge of the appropriate economic theory on which to build

101

What does it mean to apply economic theory to a research topic?

• Survey the potential economic theories of relevance for your research topic

• Assess which of the potential theories that is closest related to your research topic

102

What is theorising? (1)

• Theorising is the process of brainstorming about an issue so as to identify the logical connections that explain the issue

• The result of the process is a theory that analyses the research question, and in particular provides propositions in the form of research hypotheses

• Theorising involves constructing a conceptual or theoretical argument

103

What is theorising? (2)

• When you theorise, you ask three basic questions:– What are the essential concepts involved in the

problem being researched?– How are the essential concepts related?– What implications or predictions can be drawn from

these relationships?

• Notice that there is a distinction between existing economic theory and the resulting theory that is developed for a specific research project

104

Narrative reasoning

1. The creation of a ”primary narrative”, i.e. a document that gives a detailed description of the research topic

2. ”Concept creation”, i.e. a review of the primary narrative so as to identify the essential concepts

3. The creation of a ”higher order narrative”, i.e. a revised version of the primary narrative that focuses on the concepts developed in step 2

4. The examination of the rewritten narrative to identify possible relationships between concepts

5. The postulation of hypotheses from the theoretical relationships

105

Mathematical reasoning

1. Identify the relevant economic assumptions for the problem

2. Use mathematics to manipulate the assumptions so as to derive a conclusion or hypothesis

3. Mathematical models can be optimisation models or ad hoc models

106

A commonly used shortcut: modifying an existing model

• Rarely do economists create entirely original models

• Often researchers take an existing model, which has already been applied to the topic they are interested in, and modify it in some way that seems to be an improvement over the original

• This can be done both with optimisation and ad hoc models

107

What makes a good research hypothesis?

• It should be stated clearly and specifically in a way that can’t be misinterpreted

• It must be non-trivial• It must be able to discriminate clearly from

alternative hypotheses• It must be capable of being proved false• It should be empirically testable• It must be derived from the theoretical

analysis

108

8. Locating (and collecting) economic data

• Data creation

• The structure of economic data

• Organisations that collect and publish data

• Major primary data collections

• Major secondary data collections

109

Data and empirical research

• Data collection and manipulation is a key part of any empirical research project

• Start early on to look for potential data sources

• Check the data sources used, when you review the literature

• BUT, don’t let data availability govern, which variables you include in your theoretical discussion

110

Data creation

• Don’t view data as facts – the vast majority of data are constructed rather than collected

• Data construction

• Sample data

111

Data construction

• Steps involved in the construction of data series:– Definition of the concept– Decision on how the concept should be

measured– Determination of how to define the sample on

which the data is based

• Illustrating example: how to measure average family income?

112

Sample data

• Much social science statistics are based on sample data rather than population data

• Thus, the data that are published are extrapolated from samples

• Only if the sample is random, and thus truly representative for the population, will the sample statistics correctly measure the population

• Therefore, think of much data as estimates rather than facts

113

The structure of economic data

• It is important to differentiate between those organisations that collect or produce data and those who publish it, i.e. between primary and secondary sources of information

• Existing data are typical the result of a specific data collection effort or process

• The product of this process is a specific data set that includes a collection of certain variables

114

Characteristics of data sets

• Time-series data, which are available at different frequencies

• Cross-section data, which vary in terms of the unit of analysis

• Longitudinal data, i.e. a cross-section data set that is followed over time, is an example of a micro data set

115

Organisations that collect and publish data

• Central bureaus of statistics, e.g. Statistics Sweden (SCB)

• Institutes for economic analysis• Labour market agencies• Central banks• Patent agencies• International organisations, such as UN,

IMF, The World Bank, OECD, EU (Eurostat)

116

Major primary data collections

• National accounts• International financial flows and balances of payment• Imports and exports• Population data• Income data• Employment data• Education data• Consumer surveys• Industrial data• Financial company data

117

Major secondary data collections

• World economic outlook database http://www.imf.org/search97cgi/s97is_eng.dll/search97cgi/inetsrcheng.ini?action=FilterSearch&filter=spquery.hts&Query/Text=weodb

• Penn world tables http://datacentre.chass.utotronto.ca/pwt/index.html

• Joint BIS-IMF-OECD-World Bank statisitcs http://www.oecd.org/statistics/jointdebt

• Eurostat http://epp.eurostat.cec.eu.int/portal/page?_pageid=1090,1137397&_dad=portal%_schema=PORTAL

• OECD main economic indicators http://www.oecd.org/statsportal/0,2639,en_2825_293564_1_1_1_1_1,00.html

118

REMEMBER!

• Always consult your supervisor concerning available data

• Never start collecting your own primary data without the permission of your supervisor

119

9. Putting together your data set

• Developing a search strategy for finding your data

• Data manipulation

120

Developing a search strategy for finding your data

• Step 1: Before you search– What are the desired variables?– How should each variable be defined?– What data frequency and sample period or what levels of

analysis?– What are the potential sources for data on each variable?

• Step 2. As you search– What data are available?– Are there suitable proxy variables for variables that are

unavailable?– If not, how can the empirical model be modified to use the data

available but still test the hypotheses?

121

Data manipulation

• Level of variable• Change in variable• Real versus nominal magnitudes• Index numbers• Quantity indices versus real quantities• Price indices versus implicit price deflators• How inflation distorts nominal values• Rebasing data series• Data smoothing• Constructing a data appendix

122

Different forms of data

• Levels• Per capita• Changes• Rates of change/percentage change (or growth rates)• Annualised growth rates• Proportions• Nominal, i.e. running prices• Real, i.e. fixed prices• Index numbers for prices and quantities• Categorial data• Count data

123

10. A first look at empirical testing: creating a valid research design

• Key issues in research design

• How does one analyse data?

• Random variation in human behaviour

• Statistical methods

• Simple statistical hypothesis testing

• Confounding variables

• Casual validity

124

Key issues of research design

• Two general types of empirical methods:– Experimental methods– Survey or non-experimental methods (the

traditional method in economics)

• A critical factor in designing an empirical study is the degree to which the method is valid

125

Validity

• A study has internal validity if the impact observed can be attributed to the study variable:– Instrument validity – does the test instrument

adequately measure what it purports to?– Relationship validity – how conclusive is the empirical

testing?– Casual validity – can one be sure that the

hypothesised causal relationship is valid?

• A study has external validly if the results can be generalised to other situations, applications or circumstances

126

Empirical testing (1)

• Purpose: to search for evidence in the data to evaluate the hypotheses

• A good empirical test rejects alternative hypotheses

• However, the data in the real world may be consistent with alternative hypotheses

• Thus, you must select an empirical test that adequately discriminates between alternative hypotheses

127

Empirical testing (2)

• The power of a test is the probability of correctly rejection the null hypothesis when it is not true

• When selecting a test method always ask: ”If the test yields the strongest possible statistical results, how confident can I be that my hypothesis is not rejected?”

• If the test does not adequately discriminate between alternative hypotheses, you should consider a more powerful test

128

How does one analyse data?

• Thinking about empirical testing

• Casual empiricism

129

Thinking about empirical testing (1)

• Start by asking the following questions:– What are the implications (or predictions) of

my theoretical analysis?– If the hypothesis is not rejected, what

evidence should one expect to see?

• The answer to these questions is called the theoretical prediction of the analysis

130

Thinking about empirical testing (2)

• Once the predictions of the theory are identified, the researcher next ask:– Is the evidence of the real world consistent with these

predictions?– How exactly does one examine the evidence to

answer this question?

• We can here differentiate between three methods:– Casual empiricism– Simple hypothesis testing– Multiple regression analysis

131

Casual empiricism

• This is the type of analysis you do when you present your data and it includes– Data tables– Data graphs– Simple descriptive statistics– ”Visual” examination

132

Descriptive statistics

• Measures of central tendency are averages:– Mean– Median– Mode, i.e. the most common value in the sample

• Measures of dispersion:– Range, i.e. maximum and minimum values in the data– Standard deviation– Variance

• Measures of relationships between variables:– Covariance– Correlation

133

Major problems in data analysis

• The effects of random variation in human behaviour

• The fact that relationships between two variables can be concealed by the effects of other variables

• The fact that correlation is not the same as causation

134

Random variation in human behaviour

• This poses a problem for relationship validity because the effects of random variation can obscure any underlying relationship

• With large random samples the positive and negative errors tend to cancel each other

• However, truly random samples are rare, which introduce bias into the results as almost any data sample will include a non-random selection of errors due to sampling errors

135

Statistical methods

• The way to deal with random variation in human behaviour is to incorporate statistical methods to determine the likelihood of a sampling error

• When employing statistical methods it is critical to discriminate between two concepts:

• The null hypothesis or the statistical hypothesis• The maintained hypothesis, i.e. the theoretical

prediction of a model• Economists generally test the null hypothesis

136

Level of significance (1)

• How certain do you need to be to reject the null hypothesis?

• The level of significance is the risk that the researcher is willing to take that the null hypothesis will be rejected when it is true or alternatively the probability that the researcher will not reject the maintained hypothesis when it should be rejected

137

Level of significance (2)

• Two schools:– Select one level of significance, normally the

5 % level– Mark the different levels of significance, i.e.

10 %, 5 %, 1 %

138

Simple statistical hypothesis testing

• Simple statistical hypothesis testing is one case of what is known more generally as statistical hypothesis testing

• If you are uncertain consult your textbooks on statistics and econometrics on t-tests and the use of p-values

139

Confounding variables

• Confounding of explanatory variables, i.e. the fact that relationships between two variables can be concealed by the effects of other variables

• It affects both casual empiricism and simple t-tests

• The general solution to this problem is provided by multiple regression, which offers an opportunity to control for other variables that could influence that studied variable

140

Casual validity

• Correlation between two variables does not necessarily imply causation

• Regression analysis does not prove causation

• Theory can help you make a case for causation

• However, be observant for the endogenously problem, i.e. that causation may go both ways

141

11. Introduction to regression analysis

• The 6 steps of regression analysis

142

Steps in regression analysis (1)

• Step 1: State the hypothesis

• Step 2: Choose a proper mathematic model and motivate your choice

• Step 3: Test the hypothesis– Choose an estimation method and motivate

your choice– Estimate the model

143

Steps in regression analysis (2)

• Step 4: Interpret the test results– To what extent do the parameter estimates conform

to the maintained hypothesis identified in Step 1?– Are the parameter estimates statistically significant?– Are they economically significant?– Are the parameter estimates

• plausible for the real world?• consistent with economic theory?• within the range of previous estimates?

– How ”good a fit” is the overall regression model?

144

Steps in regression analysis (3)

• Step 5: Check for and correct common problems of regression analysis

• Step 6: Evaluate the test results– To what extent are the results in line with the

hypotheses?– Can you go on to analyse and present your

results or do you need to retake the process?

145

Step 1: State the hypotheses

• Identify your dependent variable and your explanatory or independent variable

• State clearly the sign (positive or negative) of the expected influence of each of the independent variables on the dependent variable

146

Step 2: Choose a proper mathematical model

• There are numerous mathematical models used in regression analysis

• You have to choose one model dependent upon your expectations about the relationship between the independent variables and the dependent variable and between the independent variables

• You must motivate your choice

147

Step 3. Test the hypothesis

• Choose an estimation technique – you can not always rely on OLS!

• Motivate your choice – the validity of each estimation method depends upon a number of technical assumptions hold true

• Estimate the relationship• Sometimes it is necessary to test alternative

model formulations or alternative models or to use alternative estimation methods

148

Step 4: Interpret the test results (1)

• To what extent do the parameter estimates conform to the maintained hypothesis?– Are the parameter estimates statistically

different from zero, i.e. are they statistically significant?

– Are the signs the expected signs?– Are the size of the parameter estimates in the

expected range?

149

Step 4: Interpret the test results (2)

• Has the results economic significance, i.e. are the size of effects of such magnitude that they matter?

150

Step 4: Interpret the test results (3)

• Are the parameter estimates plausible?– What are the units of the parameter estimates?– Given the units, are the parameter estimates

plausible?• Check the size of the elasticities!

– Are the parameter estimates reasonable stable between different model formulations?

– Are the results consistent with economic theory?– How do the estimates compare with previous

research?

151

Step 4: Interpret the test results (4)

• How ”good a fit” is the regression model?• There are several test statistics:

– R-square– Adjusted R-square– The F-statistic– Etc.

• Don’t overestimate the importance of a high R-square in hypothesis testing

• However, a low R-square may be an indication of missing explanatory variables

• R-square is normally lower for cross-section data than for time series data

152

Step 5: Check for and if necessary correct for common problems of regression analysis

• Problem 1: autocorrelation (and spatial autocorrelation)

• Problem 2: heteroskedasticity

• Problem 3: simultaneous equations bias

• Problem 4: specification error

• Problem 5: multicolliniarity

• Always consult your textbooks in econometrics at this stage!

153

Step 6: Evaluate the test results

• This is the main point

• What do the results mean with respect to my hypotheses?

• Interpreting regression results is more an art that a science

154

Some additions

• Transformation of data – natural logarithms

• Non-linear relationships – squared terms

• Inclusion of time trends

• Dummy variables

• Interaction terms

• Qualititative or limited dependent variables – logit models, tobit models, etc.

155

12 Communicating the results of a research project

• Writing the research report

• Presenting research orally

156

Writing the research report

• Introduction

• The written literature review

• Writing the literature review

• Theoretical analysis

• Empirical testing of the analysis

• Other components of the paper

157

The purpose of a written report

• The purpose of a written report– is to present the results of your research, but– more importantly to provide a persuasive argument to

readers of what you have found

• You must think of both format and argument• Always remember that the purpose of research

is to advance knowledge in a field by providing convincing arguments supported by logic and empirical evidence

158

Components of an empirical research paper in economics

• Title• Abstract• Table of contents• Acknowledgements (not necessary)• Introduction and literature survey• Theoretical analysis• Empirical testing• Conclusions• List of references• Data and other appendices

159

Introduction (1)

• The purpose of the introduction is to provide a rationale for the research

• What is the nature of the issue or problem the research investigates?

• Why is it worthy an investigation?• What have previous researchers discovered

about the issue or problem?• What is the purpose with your research, i.e. what

is the contribution that your research will make to the literature?

160

Introduction (2)

• The introduction is the most critical part of the research report

• You must design the first part of the introduction so that it catches the readers interest

• You may present some dramatic statistical figures or you may start by questioning some recent research

• Avoid to start the introduction in a ”traditional” manner

161

Introduction (3)

• Start the introduction by sketching out the research problem.

• What evidence can you offer to describe the issue?

• Why is it a problem?• Is it a public policy issue, a social

problem?• Is it purely an intellectual puzzle?• Who would be interested of the problem?

162

Introduction (4)

• Explain to the reader why the topic is interesting, significant and substantial.

• Is it, for example a hot topic among experts in the field, among politicians, in industry, etc.

• Show that there are some aspects of the problem that have yet not been understood

• Next step: summarise what has been done already to study the problem

163

The written literature review (1)

• The literature survey is usually included as a part of the introduction but can in rare cases be a separate chapter

164

The written literature review (2)

• The literature review shall accomplish three things:1. It should identify the major findings on a topic up to

the present with a concentration on the most recent contributions to the knowledge in the field

2. It should point out the principal deficiencies of these studies and/or provide a sense of what is lacking in the literature

3. It should conclude by leading into your research question, by explaining how your research proposes to contribute to the literature or address some shortcomings of earlier studies

165

Writing the literature review (1)

• Novice researchers tend to write excessively long literature surveys

• The literature survey should be brief but cover the major contributions

• The trick is to cast your net wide enough

• What number of sources to consult depends on the number of studies completed in the field

166

Writing the literature review (2)

• For each study you include you shall– Give complete citation information –

”Johansson (2002)”– Indicate the question that the author

examined– The author’s findings– Anything about the methodology relevant for

your study– Sometimes it is enough with one sentence for

one reference

167

Theoretical analysis (1)

• The purpose is to present the theoretical analysis of the issue or the problem that you are studying

• This section presents the logical evidence as a deductive argument

• You must clearly– describe the theory you are applying to your research

problem– explain in detail why it is relevant, and then– sketch out how it diagnoses the problem

168

Theoretical analysis (2)

• As you sketch out the logic of your theoretical analysis it is helpful to remember that you are trying to develop a deductive argument that culminates in the research hypothesis(es), which are contained in the form of an equation of the model

• Remember that you need to do a complete job of explaining your theoretical analysis – the theoretical analysis is more important than you think!

169

Empirical testing of the model

• This chapter must contain– A presentation of the data and where it comes from– Descriptive statistics– The empirical model, its motivation and the

hypothesized results– The econometric testing methodology and its

motivation– Actual results with the proper statistical indicators– A description of how you have dealt with common

econometric problems– Interpretation and analysis of the results

170

Conclusions (1)

• The purpose of this part is to summarise your findings, i.e. to restate your arguments and conclude whether your hypotheses were rejected or not

• In light of the statistical results, what can you infer about your hypothesis(es)?

• To what extent are the empirical results in line with the theoretical expectations

171

Conclusions (2)

• If your hypotheses are rejected you must suggest reasons why

• Something flawed with the theory?

• Something flawed with the data?

• Something flawed with the econometric techniques?

172

Conclusions (3)

• What can be concluded about the research question more broadly?

• Are there any policy conclusions that can be drawn?

• What research questions should be taken up in future research?

173

Other components of the paper

• Title should be carefully composed• You must include an abstract (together with 5-8

key-words and relevant JEL-codes)– What was the research problem?– How have you dealt with it?– What is your results?

• Table of contents • Acknowledgements (optional)• Reference list (complete and with all necessary

details)• Appendices (optional)

174

Presenting research orally

• An oral presentation is different from a written one

• Preparing the presentation

• Using visual aids: handouts, transparencies, PowerPoint

• Practicing the presentation

• Giving the presentation

• The role of the discussant

175

The seminar

• The purpose of the seminar is to– present the research work to a larger

audience, – provide time for criticism of the research work

to make it possible to improve it and for a scientific dialogue

• All participants are expected to be active in the discussion at the seminar

176

An oral presentation is different from a written one

• You will not be able to present all your material – you must a selection of the most strategic parts, i.e. the presentation shall summarise the major points of your research paper– The research problem in relation to earlier research– Your purpose– Your theoretical analysis and in particular your

hypotheses– Your data– Your econometric technique– Your major results– Your major conclusions

177

Preparing the presentation (1)

• Remember that your presentation shall highlight what is new with your research

• Prepare presentation ”notes”, i.e. put down on paper the points you want to make

• NEVER READ FROM YOUR NOTES WHEN YOU MAKE YOUR PRESENTATION – BUT DON’T FULLY MEMORISE THEM EITHER!

• You are supposed to ”talk about” your research, i.e. you need to know the material but you don’t need to know every world

178

Preparing the presentation (2)

• When you prepare your presentation, you should explicitly and carefully think about its introduction.

• How can you engage the interest of the audience?

• Do a careful time planning and allocate extra time to discuss the principal contributions of your research

• Also, think explicitly upon how you shall end the presentation

179

Use of visual aids

• Making presentations of research reports needs some visual aids, at least to show figures, diagrams, tables, equations, etc

• You can use handouts, transparencies or PowerPoint – each technique has its advantages and disadvantages

180

Practicing the presentation

• Always practice a few times before you make the presentation so you know what time it takes and if there is some flaws with your presentation structure

• Remember that the presentation takes much less time when you rehearse on your own

181

Giving the presentation

• Dress professionally for your presentation• Bring a few extra copies of your paper• Arrive in time so that you can make the necessary

technical preparations• Stand up to get better contact with the audience• Start by introducing yourself• Think about your body language• Avoid filler words like umm, aah, and so on• Don’t exceed the time limit - it is unprofessional• End your presentation with a thank you• Be prepared for questions

182

The role of the discussant (1)

• A two-fold role:– To offer a well-thought-out, educated reaction to the

paper– To give the author creative feed-bask on how to

improve the paper

• Always read the paper carefully and prepare your comments

• Never by rude but be frank about the less satisfactory aspects of the paper

• Never start with saying that this was a very good paper – in particular not if it isn’t true

183

The role of the discussant (2)

• Start by briefly summarising the paper and highlight the contribution it attempts to make to the literature

• What does the paper try to accomplish?

• How does it go about doing it?

• What results are reported?

• To what extent did the paper succeed in reaching its goal?

184

The role of the discussant (3)

• What can you suggest to improve the paper?• Could any parts benefit from more detailed explanation?• Are there parts that you are unable to understand?• Are there any problems with the data that weakened the

paper’s argument?• Are there any problems with the econometric techniques

and the statistical tests employed?• Are there any errors in interpreting the results?• Are there any previous studies that the author might

benefit from reading?

185

The role of the discussant (4)

• Specific suggestions are always more helpful than general comments

• The content of the paper is more important than the writing but technical details and language deficiencies matter to the extent that they detracts from the argument

• End with summarising what contribution the paper makes to understanding the issue or problem it seeks to explore

• Give a copy of your comments to the author and also the paper if you have annoted it