impact evaluation (ie) concept note how effective the...

22
Impact Evaluation (IE) Concept Note How Effective the Matching Grant Scheme in Improving Firm Performance and Export Outcomes Tunisia P158446 September 8, 2016 JEL Codes: D22, L20, O31 Keywords: 1 Export Subsidy, Trade Diversification, Firm Productivity, Innovation, Matching Grant, Randomized Control Trial 1 Please refer to JEL classification codes http://papers.ssrn.com/sol3/displayjel.cfm .

Upload: others

Post on 27-Sep-2020

1 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Impact Evaluation (IE) Concept Note How Effective the ...pubdocs.worldbank.org/en/479431510247692379/... · Present IE questions and main outcome(s) the intervention aims to affect

Impact Evaluation (IE) Concept Note

How Effective the Matching Grant Scheme in Improving Firm Performance and Export Outcomes

Tunisia

P158446 September 8, 2016

JEL Codes: D22, L20, O31

Keywords: 1 Export Subsidy, Trade Diversification, Firm Productivity, Innovation, Matching Grant, Randomized

Control Trial

1 Please refer to JEL classification codes http://papers.ssrn.com/sol3/displayjel.cfm .

Page 2: Impact Evaluation (IE) Concept Note How Effective the ...pubdocs.worldbank.org/en/479431510247692379/... · Present IE questions and main outcome(s) the intervention aims to affect

2

Table of Contents

IE PROFILE INDICATORS ......................................................................................................................................... 3

1. EXECUTIVE SUMMARY ................................................................................................................................... 4

2. BACKGROUND AND KEY INSTITUTIONAL FEATURES ...................................................................................... 5

3. LITERATURE REVIEW (E) .................................................................................................................................. 6

4. POLICY RELEVANCE ........................................................................................................................................ 8

5. THEORY OF CHANGE (E) .................................................................................................................................. 9

6. HYPOTHESES/EVALUATION QUESTIONS (E,R) ................................................................................................ 11

7. MAIN OUTCOMES OF INTEREST (E,R) ............................................................................................................. 12

8. EVALUATION DESIGN AND SAMPLING STRATEGY (E,R) .................................................................................. 13

8.1 TREATMENT AND CONTROL GROUPS ................................................................................................................... 14

8.2 SAMPLE SIZE CALCULATIONS ............................................................................................................................. 14

9. DATA COLLECTION (E,R) ................................................................................................................................. 15

9.1 QUANTITATIVE INSTRUMENTS ............................................................................................................................ 16

9.2 MANAGEMENT OF DATA QUALITY ...................................................................................................................... 16

9.3 ETHICAL ISSUES ............................................................................................................................................... 16

9.4 QUALITATIVE INSTRUMENTS .............................................................................................................................. 16

9.5 IE IMPLEMENTATION MONITORING SYSTEM (R) ..................................................................................................... 17

10. DATA PROCESSING AND ANALYSIS .......................................................................................................... 17

10.1 DATA CODING, ENTRY, AND EDITING (E) ............................................................................................................... 17

10.2 MODEL SPECIFICATION FOR QUANTITATIVE DATA ANALYSIS .................................................................................... 17

11. STUDY LIMITATIONS AND RISKS (E) .......................................................................................................... 18

12. IE MANAGEMENT (E,R) .............................................................................................................................. 19

12.1 EVALUATION TEAM AND MAIN COUNTERPARTS .................................................................................................... 19

12.2 WORK PLAN AND DELIVERABLES ........................................................................................................................ 19

12.3 BUDGET ........................................................................................................................................................ 20

13. PLAN FOR USING DATA AND EVIDENCE FROM THE STUDY ...................................................................... 21

REFERENCES ......................................................................................................................................................... 21

Page 3: Impact Evaluation (IE) Concept Note How Effective the ...pubdocs.worldbank.org/en/479431510247692379/... · Present IE questions and main outcome(s) the intervention aims to affect

3

IE PROFILE INDICATORS

No. Indicator Description

1 IE code IE-P158446-IMPE-TF0A3399

2 IE Title Tunisia Matching Grant Impact Evaluation

3 IE TTL Aminur Rahman

4 IE Contact Person Aminur Rahman, GTCME

5 Region MEN

6 Sector Board/Global Practice T&C

7 WBG PID (if IE is evaluating a WBG operation) P132381

8 WBG Project Name (if IE is evaluating a WBG operation)

Third Export Development Project (EDP III)

9 Project TTL (if IE is evaluating a WBG operation) Mariem Malouche

10 Intervention Impact of Matching Grant in Promoting Exports

11 Main Outcomes Firms’ export performance in terms of export volume, export of new products or variety, and/or exports to new markets.

12 IE Unit of Intervention/Randomization Firms in Tunisia

13 Number of IE Units of Intervention 1500 firms (approximately)

14 IE Unit of Analysis Firms

15 Number of IE Units of Analysis 1500 firms (approximately)

16 Number of Treatment Arms 3 (2 treatments and 1 control)

17 IE Question 1 (Treatment Arm 1) What is the impact of export subsidy rebate on firms’ export performance?

18 Method IE Question 1 Random assignment at the firm level

19 Mechanism tested in IE Question 1 Package

20 IE Question 2 (Treatment Arm 2) What is the impact of subsidizing the cost of standards and quality certification on firms’ ability to export high value added products and/or to new markets?

21 Method IE Question 2 Random assignment at the firm level

22 Mechanism tested in IE Question 2 Package

23 IE Question 3 (Treatment Arm 3) N/A

24 Method IE Question 3 N/A

25 Mechanism tested in IE Question 3 N/A

25 Gender-specific treatment (Yes, No) No

27 Gender analysis (Yes, No) To be determined as we don’t know what percent of matching grant applicants will be female owned in treatment and control groups.

28 IE Team & Affiliations Giacomo De Giorgi (New York Federal Reserve Bank, ICREA/MOVE, Barcelona Graduate School of Economics; Co-PI); Aminur Rahman (World Bank Group, T&C, Co-PI and IE TTL); Eric Verhoogen (Columbia University; Co-PI)

29 Estimated Budget (including research time) 499,645

30 CN Review Date 10-2016

31 Estimated Timeframe for IE 10-2016 to 12-2019

32 Main Local Counterpart Institution(s) Tunisia Export Promotion Agency (CEPEX)

Page 4: Impact Evaluation (IE) Concept Note How Effective the ...pubdocs.worldbank.org/en/479431510247692379/... · Present IE questions and main outcome(s) the intervention aims to affect

4

1. EXECUTIVE SUMMARY (1 page)

Describe the proposed IE in non-technical language in one paragraph or less. This could be an abstract of your IE. Include broad motivation/background and policy/research contribution. (E,R)

Present IE questions and main outcome(s) the intervention aims to affect.

Briefly explain how you are proposing to test your main evaluation question(s).

Tunisia seems to be trapped into a low productivity-low value added export-low economic growth situation. To

promote export diversification, Tunisia, with the support from the World Bank Group, is implementing a $50 million

Export Development Project III (EDP III), a significant component of which is a $22 million export matching grant

component, TASDIR+. TASDIR+’s objective is to increase exports in a sustained manner over time in addition to

promoting export diversification toward more high value added exports and/or to new markets. While both export

subsidy and matching grant are popular policy tools, to date there is very little rigorous evidence on their

effectiveness, a gap this proposed IE aims to address.

While the scope of TASDIR+, which aims to provide 50-70 percent co-financing to the firms to help firms improve

their export competitiveness, is fairly broad, we aim to test two specific interventions through RCT: (i) does export

subsidy rebates (treatment 1) encourage firms to increase export volume, export new product variety, and/or export

to new (advanced) markets, and (ii) does subsidizing the costs to obtain quality standards certificates and

accreditation help firms to export to new markets or new product variety. Through these interventions, we will also

evaluate if and how a firm’s export performance contributes to its productivity and growth by enhancing its

production possibility frontier.

Through communication campaign, we aim to encourage a wide number of applicants for each of the treatment

arms and we aim to roll out the applications 2-3 times a year if not more. Through a simple screening process, we

aim to select eligible pool of firms for each treatment arms, from which we will randomly assign a total of

approximately 250 firms into treatment group and 500 firms into control group for each intervention over a period

of 2-3 years to rigorously measure the effect of each of these interventions on firm’s export performance.

The underlying idea for the first intervention is that firms may not be efficient enough to compete on price to new

markets/new product space with more efficient competitors from other countries; hence the export subsidy rebate

would make Tunisian firms’ pricing more competitive in international markets. If exporting makes those firms more

efficient either by moving them along the production possibility frontier or expanding such frontier then a short-

term intervention can have long lasting consequences. Second intervention focuses on short term liquidity constraint

and/or investment under uncertainty situation. Exporting to advance economies often requires obtaining necessary

quality certification and accreditations, which the cash strapped firms might not be able to obtain, or the firms may

not be willing to undertake such costly investment as success to enter into the advanced market/product space is

uncertain. Thus by providing subsidy to the firms to obtain the necessary certificates and accreditation, we aim to

either overcome the potential liquidity constraints of the firms or to reduce their investment risks, and if these are

some the binding constraints for the firms to exports to advanced economies, then the second intervention would

help unleash the productivity gains of the firms from entering into the advanced market/product space.

Thus this IE tackles the questions that lie at the heart of Tunisian development strategy of how to promote economic

development through export diversification. As the new empirical literature on international trade argues, it is

Page 5: Impact Evaluation (IE) Concept Note How Effective the ...pubdocs.worldbank.org/en/479431510247692379/... · Present IE questions and main outcome(s) the intervention aims to affect

5

important to understand the factors that influence a country’s transition from the production of low-quality to high-

quality products since the production of high-quality goods is often viewed as a pre-condition for export success

and, ultimately, for economic development. However, there is not much rigorous evidence on what firm level

supports the government can provide to enable them to transit from low-value added to high-value added exports

(product diversification) or accessing new export markets (market diversification). This IE attempts to fill this gap.

The findings from our IE also expect to influence the government’s design of the matching grant scheme for

subsequent years. TASDIR+ plans to run until 2021. Therefore, the IE findings from the initial years of the matching

grant scheme will greatly benefit the government to make any necessary adjustment to the subsequent years to

make the scheme more effective.

2. BACKGROUND AND KEY INSTITUTIONAL FEATURES (1 page)

Present an overview of the local context.

Identify and define the problem: what is the policy/research problem this IE is proposing to study? Which

groups are affected by the problem?

Describe the intervention whether existing or new, implementing organization, institutional setting and any

important consideration.

Describe the intervention geographic/demographic scale and scope: Does it represent the “mode” of

delivery in the country? (R, E)

Tunisia appears to be trapped into a low productivity-low value added export-low economic growth situation. While

Tunisia has been one of the best performers in MENA region with an annual average GDP growth of 4.8 percent over

most of the 2000s, its growth rate is significantly lower than that of the high growth emerging economies. Tunisia’s

growth has been fueled by exports, but its export sector is characterized by lack of market and products

diversification, low value added, and little innovation. Its exports remain highly concentrated in the European market

(70 percent of Tunisia’s exports in 2010), and predominantly in three countries: France, Italy, and Germany (which

together accounts for 56 percent of total exports in 2010). This makes Tunisia vulnerable to the vagaries of EU growth

and the Euro.

In addition to lack of market diversification, the value addition of Tunisia’s exports, which is dominated by textiles,

garments, mechanical, and electrical engineering products (accounting for 60 percent of total exports), is low. While

the ratio of exports value to GDP is 47 percent, in terms of value addition, exporting manufacturing sectors generate

less than 15 percent of GDP. This is because Tunisian exporters still largely remain sub-contractors to large foreign

companies (at the lower end of the latter’s value chain) rather than being direct exporters, and continue to produce

relatively simple goods subject to stringent competition in the EU market. These products can be easily replaced

with similar products from other lower cost locations, such as China, India, and Bangladesh.

To transform the economy from a low-wage, low value-added, economy to a knowledge-based and skill-intensive

one (World Bank, 2012), Tunisia needs to pursue a two pronged strategy of repositioning in traditional sectors and

diversification into new sectors and products. First, it needs to move up the value chain in traditional export sectors

by deepening integration, products sophistication and supporting initiatives that enhance logistics and innovation

investment. Second, it needs to promote investment in newer, knowledge intensive products and services. Overall,

a gain in efficiency is demanded for Tunisia to grow at a higher pace. In this regard, the Government of Tunisia (GoT),

Page 6: Impact Evaluation (IE) Concept Note How Effective the ...pubdocs.worldbank.org/en/479431510247692379/... · Present IE questions and main outcome(s) the intervention aims to affect

6

with the support from the World Bank Group is implementing the Export Development Project III (EDP III), a $50

million investment project whose objective is to help increase and diversify exports by supported enterprises. This

project includes a $22 million component, named TASDIR+, for an export matching grant to beneficiary enterprises.

TASDIR+ in EDP III (i.e. the matching grant project that we aim to evaluate) has been designed based on lessons and

achievements of the FAMEX I and II (matching grants under EDP I and EDP II). Building on FAMEX I and II, the objective

of TASDIR+ is to increase exports in a sustained manner over time (well beyond the period of export subsidy), in

addition to promoting export diversification toward more value added exports and/or to new markets.

The key objective of this proposed impact evaluation (IE) of TASDIR+ is to rigorously evaluate if and to what extent

subsidizing a firm’s export cost through matching grant improves the firm’s export performance in terms of one or

more of the following outcomes: (i) increased export volume in the same market, (ii) accessing new market, and (iii)

introducing new product/product-variety in the existing and/or new market.

Why export subsidy through matching grant? Financing of export is a critical constraint especially for smaller SMEs

and new exporters in Tunisia. Available data from the 2012 World Bank’s Enterprise Survey show that Tunisian

companies typically consider access to and cost of financing as one of their greatest barriers to their growth. Around

40 percent of firms consider finance as their biggest binding constraint just after political instability, macroeconomic

uncertainty and employment readiness. Challenges to SME finance is also associated with heightened risk aversion

in a fragile environment, increased government financing by banks, additional loan concentration linked for instance

to large investment programs.

Entering into new market and innovation also involve substantial risk and investment, which in the current uncertain

environment, Tunisian firms are unlikely to take even if they had the financing power. To export new products, to

enter into new market, or to do both, firms have to (i) identify the right target market, product segment, and selling

channel; (ii) learn how to adapt their products for these markets; (iii) understand their competitors; (iv) launch

marketing and selling campaign; (v) train and hire new workers to be ready for the job; (v) obtain necessary quality

and standards certification, (vi) deliver the product on time and collect on sales. These activities require significant

investments with uncertain outcomes and thus often not financed from traditional financing sources (e.g.

commercial banks). All these in turn justify the government intervention in the form of providing export subsidy

through matching grant to encourage the “first movers” firms to take the risk for export diversification as their

success could lay the path for the followers to venture into those new markets or to export those new products.

Moreover, the success of the first movers could also create an overall “brand” effect of Tunisian products (a la Italian

shoes, Swiss watches, and so on) to the new markets, thereby making subsequent entry easier for the “followers”,

which adds to the rationale for the matching grant scheme.

The IE will be implemented in close collaboration with the World Bank Operation Team who is in charge of the EDP

III project and with Tunisia Export Promotion Agency (CEPEX); the latter is in charge of implementing TASDIR+, the

matching grant component of the EDP III project. A government team including the representative from CEPEX

participated and discussed the IE concept at the DIME and T&C IE Workshop in Istanbul in 2015. Subsequently the

IE research team conducted a detailed discussion on the IE methodology and implementation issues with CEPEX

team.

3. LITERATURE REVIEW (E) (1 page or less)

Page 7: Impact Evaluation (IE) Concept Note How Effective the ...pubdocs.worldbank.org/en/479431510247692379/... · Present IE questions and main outcome(s) the intervention aims to affect

7

Describe most relevant literature/scientific background specifically linked to your problem/evaluation question(s).

Both matching grants and export subsidies are popular policy ideas. As Campos et al (2014) write, "[o]ver the past twenty years matching grants have been a mainstay of World Bank projects to enhance private sector competitiveness." Export promotion schemes of various types have also been implemented in a large number of developing countries. But to our knowledge there has not been a randomized evaluation of a matching-grant program specifically targeted toward increasing exports. Moreover, there have been relatively few rigorous evaluations, either randomized or quasi-experimental, of similar or closely related programs. Overall, although the trend is positive, the evaluation literature has not kept pace with the policy initiatives in this area. One way to categorize the related literature is to distinguish between randomized and non-randomized (generally quasi-experimental) evaluations. Focusing first on randomized evaluations (of matching-grant programs generally, since we are not aware of one specifically targeted to export promotions), the most salient fact is that there have been few successful evaluations. Campos et al (2014) report that as of 2012 there had never been a successful randomized evaluation of a matching grant program, and seven World Bank-funded evaluations had recently been derailed because of political pressures and other implementation difficulties. Bruhn et al. (2013) evaluated a program offering subsidized consulting services for small and medium enterprises in Mexico, and found positive impacts on sales, profits and productivity but not employment. McKenzie et al. (2015) report on an experiment in Yemen that gave $10,000 as a 50 percent subsidy towards the cost of business services to small and medium enterprises, and find positive short-term impacts on product innovation, marketing and adoption of accounting systems, but they were not able to look at longer-term impacts because of political instability. We are aware of at least one other World Bank project currently underway that is aiming to do a randomized evaluation of a matching-grant program evaluating a high-impact entrepreneurship program of the Mexican government. But it is safe to say that the randomized evaluation of matching-grant programs is an under-researched area. There is a small literature on randomized evaluations of cash grants for firms (which have not required co-investment by the firms). In a sample of very small firms in Sri Lanka, De Mel et al. (2008) find that cash transfers generate high returns. McKenzie (2015) evaluates a massive business-plan competition in Nigeria, and finds positive impacts on survival rates, employment, sales and profits for firms that win the competition and receive cash transfers. Other studies have found mixed evidence on the effects of cash transfers. Little is known about the importance of the co-investment aspect of matching-grant programs relative to cash-transfer programs. Turning to non-randomized studies, the closest studies to the current proposal are of a previous generation of matching grants to promote exports in Tunisia, referred to as the FAMEX II program. Using a matching difference-in-differences design, Gourdon et. al. (2011) found that the program initially had positive impacts on export growth, but using a similar design Cadot et al (2016) found that the positive effects vanished within three years. Both papers also note the methodological challenges faced by “ex post” evaluations such as theirs and called for an evaluation design built into an export matching-grant program “ex ante”, which our current proposal aim to design. There have been several other non-randomized studies of export promotion policies in developing countries. For instance, using difference-in-differences (with and without matching), Volpe and Carballo (2008) analyze export-promotion efforts in Peru and find positive effects, especially on the extensive margin (increases in the number of products exported and the number of destinations served.) Alvarez and Crespi (2000) examine the effects of several export-promotion instruments in Chile (exporter committees, presence in international fairs, and utilization of business information systems) in a fixed-effects framework and find positive effects on the intensive margin (increased exports). Additional non-randomized studies of matching grants of various forms include Crespi et al. (2011), which studies an innovation program in Colombia that showed positive effects on product diversification and labor productivity;

Page 8: Impact Evaluation (IE) Concept Note How Effective the ...pubdocs.worldbank.org/en/479431510247692379/... · Present IE questions and main outcome(s) the intervention aims to affect

8

and Castillo et al. (2011), which studies a matching grant for technical assistance in Argentina and showed positive effects on employment and wages. A well-known concern with studies in this vein is that it is hard to rule out definitively that there is selection bias in assignment to the program, with better firms more likely to receive the matching grants. One additional study that bears mention is Atkin et al (2016), which randomly allocated a different form of export promotion: actual initial contacts with a foreign buyer to Egyptian rug producers. The authors find positive effects on the quality of the rugs produced (even identical types of rugs produced under controlled conditions) and measured productivity. While this project is ingenious, the form of export promotion studied is likely to be available only in very particular settings where the cooperation of foreign buyers can be secured. Matching grants, in contrast, are likely to be applicable in a much broader array of contexts and can potentially be scaled up in a way that the Atkin et al intervention cannot. Overall, there is a clear need for additional rigorous evidence on the impacts of matching grant programs and export promotion programs, and in particular on matching grants targeting increased exports; our current proposal aim to address this gap in the literature.

4. POLICY RELEVANCE (1/2 page or less)

Assess the extent to which the study may influence policy and institutional capacity at the national, regional,

and international level. Explain how you plan to track the policy influence of your study (see Appendix on

i2i sample indicators of IE influence on program/policy. These indicators, which are currently under revision,

will be collected through Grant Monitoring and Reporting on annual basis from all i2i supported IEs).

This IE tackles the questions that lie at the heart of Tunisian development strategy of how to promote economic

development through high value added export. As the new empirical literature on international trade argues, it is

important to understand the factors that influence a country’s transition from the production of low-quality to high-

quality products since the production of high-quality goods is often viewed as a pre-condition for export success

and, ultimately, for economic development. There is substantial evidence to suggest that firms in high income

countries produce and export higher quality goods than those in less-developed nations. Some recent studies also

indicate the positive benefits of import tariff liberalization on firm’s product quality upgradation and innovation (e.g.

Amiti and Khandelwal, 2011). However, there is not much rigorous evidence on what firm level supports the

government can provide to enable them to transit from low-value added to high-value added exports (product

diversification) or accessing new export markets (market diversification). This IE attempts to fill this gap.

The findings from our IE also expect to influence the government’s design of the matching grant scheme for

subsequent years. TASDIR+ plans to run until 2021. Therefore, the IE findings from the initial years of the matching

grant scheme will greatly benefit the government to make any necessary adjustment to the subsequent years to

make the scheme more effective.

Finally, as articulated in Campos et. al (2013), while matching grants are one of the most common policy instruments

in WBG operations to provide firm-level support, we have very limited rigorous evidence on the effectiveness of this

instrument. As discussed in Section 3, while there have been some attempts to evaluate the effectiveness of

matching grant using quasi-experimental methods, unfortunately non-experimental methods have severe limitation

Page 9: Impact Evaluation (IE) Concept Note How Effective the ...pubdocs.worldbank.org/en/479431510247692379/... · Present IE questions and main outcome(s) the intervention aims to affect

9

in dealing with selection issues for matching grant programs because of inherently forward looking behavior of both

the firms and the government – firms self-select into whether or not to apply for the program, and then government

selects which firms receive the grants. Firms that receive support are more likely to be perceived (by the firm and by

government) as having faster growth potential in the future than firms that do not apply or do not get selected, even

when they are observably similar in terms of current size and recent growth history. As such, matching on observable

characteristics and differencing out of firm fixed effects is still unlikely to control for the differences between the

firms which participate in the program and those which do not. Similarly, while in theory the regression-discontinuity

designs could be of use, in practice the programs often suffer from lack of proper use of the selection criteria

(mistargeting), very small sample of firms close enough to the scoring threshold, or the endogeneity issues related

to the selection threshold itself. Thus providing rigorous evidence on the effect of matching grant on firm’s export

performance, this IE aims to contribute to the design of the future matching grant schemes of the governments

supported by WBG and other donors.

5. THEORY OF CHANGE (E) (1 Figure and 2-3 paragraphs)

Describe the main elements of the intervention, and the hypothesized causal chain from inputs, through

activities and outputs, to outcomes.

Describe the main assumptions and other factors underlying the causal chain (internal and external).

TASDIR+ aims to provide 50-70 percent co-financing to the firms to help firms improve their export competitiveness.

Its scope is fairly broad. Some examples of activities of the firms that would be eligible for financial support under

TASDIR+ includes (i) the cost to obtain special accreditation for exporting goods and services that are required by

the importing countries, (ii) cost of development and marketing new product; (iii) cost of hiring and/or capacity

building of workers, (iv) cost of opening up the office in destination market, and so on. The scheme has also a focus

on supporting women entrepreneurs (at least 20% of the matching grant fund will be allocated to target women

entrepreneurs) and entrepreneurs from the lagging regions.

Given this scope, the key elements of our impact evaluation design include the following treatments:

Treatment 1: export subsidy rebates to encourage increased export volume, exporting new product variety,

and/or exporting to new (advanced) markets

Treatment 2: subsidy for obtaining required quality standards certificates and accreditation to export to

new market or new product variety

The underlying idea for first intervention is that firms may not be efficient enough to compete on price to new

markets/new product space with more efficient competitors from other countries; hence the export subsidy rebate

would make Tunisian firms’ pricing more competitive in international markets. If exporting makes those firms more

efficient either by moving them along the production possibility frontier or expanding such frontier then a short-

term intervention can have long lasting consequences, and that is precisely the overarching objective of TASDIR+,

i.e., to increase export in a sustainable fashion over time to new markets and/or to high value products.

Second intervention will focus on short term liquidity constraint and/or investment under uncertainty situation.

Exporting to advance economies often requires obtaining necessary quality certification and accreditations, which

the cash strapped firms might not be able to obtain, or the firms may not be willing to undertake such costly

Page 10: Impact Evaluation (IE) Concept Note How Effective the ...pubdocs.worldbank.org/en/479431510247692379/... · Present IE questions and main outcome(s) the intervention aims to affect

10

investment as success to enter into the advanced market/product space is uncertain. Thus by providing subsidy to

the firms to obtain the necessary certificates and accreditation, we aim to either overcome the potential liquidity

constraints of the firms or to reduce their investment risks, which could be the reasons why firms are not exploiting

the potential advanced market/product space. If these are the binding constraints for the firms, then the second

intervention would help unleash the productivity gains of the firms from entering into the advanced market/product

space.

Thus the simplified theory of change as depicted in figure 1 is as follows. The underlying assumption is that there

are a number of Tunisian firms with innovative ideas of either exporting their products to new markets or upgrading

their current products to higher value added variety for either existing or new markets. But the investment required

for this is risky as the firms do not know ex ante if they will be successful and hence the firms are either unwilling to

invest in this venture or do not have enough resources (cash constrained and credit market imperfection) to invest

in case they are willing. Matching grant interventions will address this suboptimal investment under uncertainty or

credit constraint and thus enable firms to export new high value product, or to new market, or both. This export

success in turn will make the firms more productive. Thus, another critical underlying assumption is that the

matching grant intervention is addressing a critical binding constraint that is preventing these firms from innovation

and/or exporting to the new markets.

The initial input in this theory of change is sufficient number of firms with proposals for developing high value

product variety or to access new markets. The immediate output is firms’ investment in areas that they view critical

to increase export volume, export to new markets, and/or export new/high value product variety, and for second

intervention, obtaining the needed certificates and accreditation. The immediate outcome would be increased in

export volume, export of new product variety, and/or exporting to new (advanced) markets. The medium term

outcomes would be increased firm productivity and growth through expansion of the firms’ production possibility

frontier.

Figure 1: Theory of Change

Inputs: Tunisian firms with potential for increased exports, developing new product variety for export, and/or

with the potential to export to new (advanced) markets.

Activities: Firms’ self-selection into most relevant matching grant window for their export success and developing

business plan to use the matching grant financing for either increasing export, exporting new product variety or

exporting to new market.

Outputs: Firms investment (co-financed by matching grant for 2nd intervention) to improve the relevant export

performance.

Intermediate outcomes: Firms’ increased export, firms’ export of new product variety to existing or new

(advanced) markets; firms’ export to new market.

Page 11: Impact Evaluation (IE) Concept Note How Effective the ...pubdocs.worldbank.org/en/479431510247692379/... · Present IE questions and main outcome(s) the intervention aims to affect

11

A theory of change describes how the intervention is expected to affect the outcomes of interest (based on theory)

but it does not demonstrate whether the intervention causes the observed outcomes. It usually includes the most

important outcomes (intermediate and final) that are critical to the causal chain, even if not all will be measured (see

Appendix for example).

A theory of change sets the structure for the hypotheses, evaluation questions, and outcomes of interest. It also lists

key indicators for developing the implementation protocol and IE monitoring system aimed at understanding what

is being evaluated, and whether the critical intervention activities/components were implemented/taken up as

planned.

6. HYPOTHESES/EVALUATION QUESTIONS (E,R) (1/2 page)

List the hypotheses derived from your theory of change.

List the main evaluation question(s) to be addressed by the proposed study. Evaluation questions connect

the specific intervention/treatment variation to the outcomes of interest, and end with a question mark.

They should be in the following format: What is the impact of <intervention/intervention variation> on

<outcomes>? E.g., What is the impact of a parental workshop on financial literacy on student knowledge,

attitudes and behavior?

You may have a broad evaluation question based on the knowledge gap and the strategy proposed.

However, the number of specific questions in this section should be perfectly aligned to the number of

your treatment arms (i.e., if you have 3 treatment arms you should have three specific evaluation

questions). Each question can be evaluated on a vector of outcomes (i.e., you may organize them as sub-

questions). Methods to answer sub-questions on heterogeneous treatment effects and spillovers should

be described in the methods section.

Describe how the evaluation questions were derived.

Evaluation Question 1: What is the impact of export subsidy rebate on firms’ export performance in terms of

export volume by product variety and market destinations?

Hypothesis 1a: Export subsidy rebate will encourage treated firms to increase their export volume.

Hypothesis 1b: Export subsidy rebate will encourage treated firms to export new product variety to traditional

Tunisian export markets.

Longer-term outcomes: Improved firm productivity through expansion of its production possibility frontier.

Page 12: Impact Evaluation (IE) Concept Note How Effective the ...pubdocs.worldbank.org/en/479431510247692379/... · Present IE questions and main outcome(s) the intervention aims to affect

12

Hypothesis 1c: Export subsidy rebate will encourage treated firms to export new product variety to new export

markets.

Hypothesis 1d: Export subsidy rebate will encourage treated firms to export to new (advanced) markets.

Evaluation question 2: What is the impact of subsidizing the cost of standards and quality certification on firms’

ability to export high value added products and/or to new markets?

Hypothesis 2a: Subsidy will enable the treated firms to successfully acquire the needed quality standard

certification and accreditation that is required for exporting a given product variety to a given market.

Hypothesis 2b: Obtaining necessary quality certificate and accreditation will enable the treated firms to export the

given product varity(ies) to given market(s).

Evaluation question 3: What is the impact of firms’ export performance on firm productivity and growth?

Hypothesis 3a: Exporting new product varity(ies) will positively affect treated firms’ productivity and growth.

Hypothesis 3b: Exporting to new (advanced) markets will positively affect treated firms’ productivity and growth.

7. MAIN OUTCOMES OF INTEREST (E,R) (1 table)

Briefly list and define main outcomes of interest (primary and secondary/intermediate) as in Table 1.

Further details on how the outcomes will be measured/collected will go in the data collection section.

Table 1. Main Outcomes of Interest

Outcome Type Outcome Name Definition Measurement Level

Primary/Secondary Export-product diversification (number of products)

Number of items exported by the firm to different destinations (Annual)

Firm-level survey Customs database

Export-market diversification (number of markets)

Number of destination markets a firm exported to (Annual)

Firm-level survey Customs database

Product diversification in terms of export volume

Export share of different product varieties

Firm-level survey Customs database

Market diversification in

Export share to different markets

Firm-level survey Customs database

Page 13: Impact Evaluation (IE) Concept Note How Effective the ...pubdocs.worldbank.org/en/479431510247692379/... · Present IE questions and main outcome(s) the intervention aims to affect

13

terms of export volume

Export volume Export volume Firm-level survey Customs database

Sales Annual sales of the firm

Firm-level survey Tax authority data

Employment Number of employees (permanent and temporary) of the firm (Annual)

Firm-level survey

Secondary Product diversification

Number of product variety produced by the firm (Annual)

Firm level survey

Inputs used Number and volume of inputs used by the firm

Firm level survey Custom database (in case of imported inputs)

8. EVALUATION DESIGN AND SAMPLING STRATEGY (E,R) (2 pages or less)

Present the main features of the proposed evaluation design to address the evaluation question(s).

Describe precisely the identification strategy (e.g., trial design including clustering, factorial, stratification

details) for each evaluation question.

Report all inclusion/exclusion criteria to define the target population/population studied, providers,

settings, and clusters (as relevant).

Report any ethical issues that may arise concerning the evaluation design and the sampling strategy (not

related to data collection).

The IE methodology is an RCT. We will have a control group and the two treatment groups corresponding to two

interventions as mentioned above.

Interventions: Considering the sample size (which is a typical problem with matching grant programs) and cost, we

focus our impact evaluation around two specific interventions: (i) export subsidy rebate with a focus on new product

variety or new market, and (ii) subsidy for obtaining required quality standards certificates and accreditation to

export to new market or new product variety. First intervention will take the form of a randomly assigned rebate to

the treatment firms based on the volume of firm’s export and will be potentially tilted towards new product variety

and/or new and/or advanced markets.

Communication campaign: To encourage wider participation of firms in the application process, we will embark on

a multi-pronged communication campaign to announce the different windows under which the firms can apply for

Page 14: Impact Evaluation (IE) Concept Note How Effective the ...pubdocs.worldbank.org/en/479431510247692379/... · Present IE questions and main outcome(s) the intervention aims to affect

14

the matching grant. The communication campaign will leverage social media, traditional newspaper and electronic

media announcement, roadshow and workshops across different regions in country. Leveraging tax and custom

database, we will also send mails to the firms’ addresses (obviously the start-ups who are not registered or do not

trade will not be covered under the mailing service but we aim to cater them through regional road shows,

workshops, traditional and social media).

Application and Selection: We will conduct a two-step application process. First, the firms will submit a short

expression of interest (EoI) indicating the intervention window from which they intend to seek support and why (i.e.,

the nature of their product or service for which the sought intervention is critical). This will provide us a sense of

number of potential applicants under the different intervention windows. At the same time, for the implementer of

the matching grant scheme this will be a “market test” for the services they are offering. Assuming there will be

sufficiently large number of applicants with EoI for our intervention windows, we will undertake a simple screening

of the EoIs through an independent panel of industry experts essentially to ensure that the firm’s business

proposition is viable. For first window, the screening will be minimal as the export subsidy rebate will be contingent

on actual export outcome of the firm, i.e., a firm will get x% of their export volume (verified through customs records

back as cash rebate (which is analogous to the x% cash back feature of many credit card companies in the US for

instance). For second intervention window, the panel will also evaluate if the firm is in a financially viable position

to undertake the project with matching grant (or in other words there is no risk of the firm running away with the

grant). For both windows, the screened applicants will then be invited to submit a full application with some detailed

firm specific information and information pertaining to their export plan. The applications will be scored on the same

criteria used for screening. All the applicants who receives a pass score for each window will be the eligible pool of

candidates; who will then be randomly assigned to the treatment and control group for each of the windows. As

mentioned above, for window 1 we then randomly assigned the rebate rate to each of the treatment firms with

some tilting towards exporting to new advanced markets and/or high-value product variety. We hope to conduct 2-

3 rounds of applications annually for next 2-3 years and applicants in the control for a given round could apply for

subsequent rounds.

8.1 TREATMENT AND CONTROL GROUPS

Provide specific description of features of each control and treatment arm (one paragraph per arm).

From the pool of eligible firms for each treatment arm, we will randomly assign the firms in the treatment and

control group. To increase the statistical power, we will undertake stratified randomization based on firm size, age,

current exporting status, and geographical location. At this stage, we don’t know if we will have enough number of

firms (also the nature of the firms aiming to cater to the export market is different from the micro scale informal

firms) to stratify on the gender of the main owner but this should become clear during the EoI stage and if

stratification on gender is possible, we will certainly do that as well. We hope to conduct 2-3 rounds of application

process under each of the two intervention windows annually for next 2-3 years, and applicants in the control for a

given round could apply for the subsequent rounds.

8.2 SAMPLE SIZE CALCULATIONS

Present the sample size estimates. Describe how the sample size was determined, including the sampling

frame, and main assumptions including Minimum Detectable Effect (MDE), variance estimates, intra-cluster

correlation, and units per cluster (if applicable).

Page 15: Impact Evaluation (IE) Concept Note How Effective the ...pubdocs.worldbank.org/en/479431510247692379/... · Present IE questions and main outcome(s) the intervention aims to affect

15

The matching grant programs often suffer from a small sample size problem (see for instance the discussion in

Campos et. al 2013). This is partly due to lack of communication and stringent selection criteria of the matching grant

scheme, both of which we try to alleviate in our IE design as mentioned above. However, for an export promotion

intervention, the nature of potential beneficiaries are quite different from the micro and small firms that dominates

the private sector in developing countries. This along with the fact that the treatment offered by matching grant is

quite expensive also have limiting effect on the sample size. Having said that, the previous round of matching grant

scheme in Tunisia on export promotion catered about 900 firms out of 1200 firms that applied for the program

between 2005 and 2010. The quasi-experimental study on the previous round of matching grant program included

about 391 beneficiary firms and contrast their performance against a control group of 2319 firms. The current

matching grant scheme aims to support approximately 225 firms between 2016-2018 and altogether about 575 firms

between 2016 and 2020 under the treatment arm 2, which is to obtain certification/accreditation of products and

services.

Previous quasi-experimental studies (e.g. Cadot et. al 2015) note that the export share of the recipient of the

beneficiaries of the previous phase of the matching grant scheme was about 20-40 percent higher than the control

group (standard deviations not reported). The average number of destinations exported to by the treatment firms

was 5.33 with a standard deviation (STD) of 9.37, while the corresponding mean and STD for the control group were

3.47 and 5.88. The average number of products produced by the treatment firms was 3.58 with a standard deviation

of 4.67, while the corresponding mean and STD for the control group were 1.76 and 2.88.

Given that treatment is expensive (substantial amount of grant money in the magnitude of tens of thousands of

dollars is involved in the treatment arms), for each treatment arm we aim to operate with a 1:2 ratio of treatment

and control group. For a given window, our power calculation suggests that we will be able to detect an effect size

of 0.2 (20%) on increase in export share of our treatment firms compared with the control firms with a STD of 0.8

for both groups with 80% power and 5% significance level with a treatment group of 190 firms and a control group

of 379 firms. For the above stated effects reported in Cado et. al (2015) in terms of number of products and

destination markets, we will be able to detect such differential effects with a sample of 63 treatment firms and 126

control firms for the difference in number of products and 240 treatment and 480 control firms for the number of

destination markets. Thus for each of our treatment arms, we aim to have a sample size of 250 treatment firms and

500 control firms, resulting in a total of 500 treatment firms and 1000 control firms for two treatment arms. These

total number of treatment and control firms will be accumulated through multiple rounds of call for applications

between 2016 and 2018 and these total number of sample size is well within the beneficiary target number of the

project (e.g. 2000 firms).

It should be noticed that the above power calculation is the upper bound of the sample size as we further aim to

increase the power by baseline survey and stratification in terms of key firm characteristics as mentioned above,

e.g., size, location, exporting status, age (and potentially gender).

9. DATA COLLECTION (E,R) (1 page if basic, 1-2 pages if include all sections for registration and ethical clearance)

Describe main instruments for data collection

Page 16: Impact Evaluation (IE) Concept Note How Effective the ...pubdocs.worldbank.org/en/479431510247692379/... · Present IE questions and main outcome(s) the intervention aims to affect

16

The data collection for the IE will be conducted through the application process for the matching grant, follow-up

annual survey of the treatment and control firms, and from customs database on actual trade (export and import)

data of the sample firms.

9.1 QUANTITATIVE INSTRUMENTS

Describe how primary and secondary outcomes (from section 7) will be measured, their timing and

frequency.

The primary and secondary outcome indicators (as reported in section 7) at the baseline will be measured from both

the application forms for the call for matching grant application. The data related to export performance indicators

would be cross-checked using the customs database. We will then collect on the annual basis the data from both

treatment and control firms through firm survey for a period of three years for each firm (as different firm will enter

in the sample either as a treatment or control firm in different period between 2016-2018) and trade data of the

firms will be collected from custom database for the entire duration of the WB project (i.e. until 2021) to assess the

potential medium term effects of our interventions on export performance.

9.2 MANAGEMENT OF DATA QUALITY

Describe methods used to enhance the quality of measurements (e.g., multiple observations, training of

surveyors), electronic data collection, protocols for quality assurance.

All the PIs have extensive experience in conducting large scale firm surveys in developing countries particularly under

RCT setting. We will employ a range of standard protocols for ensuring data quality. These include random callback

to a sample of firms to verify the authenticity of the interview, double-entry process for data entry, built-in protocol

in the data-entry program for quality checks on parameters exceeding certain ranges, selection of a highly competent

survey firm and provide necessary training to the surveyors.

9.3 ETHICAL ISSUES

Describe if this IE will require ethical approval, informed consent procedures, and important ethical

considerations related to data collection.

All the PIs have NIH certification. Since the objects of the analysis are firms, human subjects clearance will not be

required for comparing firm outcomes. The surveys will use standard informed consent procedures for collecting

information about firms.

9.4 QUALITATIVE INSTRUMENTS

Provide a description of all qualitative instruments (if applicable).

The IE will be primarily based on data collected in the application form, follow-up quantitative survey, and from

administrative data, such as customs database. Nonetheless, we will conduct detail qualitative interviews or focus

group discussions with about 10-20 firms to understand their current export behavior and bottlenecks to improve

export performance. The findings from qualitative interviews and FGDs will be used to design the application form

and follow-up survey questionnaire. Finally, if any unexpected findings arise from the quantitative surveys,

qualitative interviews and/or FGDs will be used to decipher those findings.

Page 17: Impact Evaluation (IE) Concept Note How Effective the ...pubdocs.worldbank.org/en/479431510247692379/... · Present IE questions and main outcome(s) the intervention aims to affect

17

9.5 IE IMPLEMENTATION MONITORING SYSTEM (R)

Describe the IE implementation monitoring system, particularly, what specific indicators and system will be

used to follow up the studied population, their treatment participation, treatment actually delivered and

received based on activities, and outputs (see the theory of change section).

EDP III project component has specific funding allocation for developing the M&E system for CEPEX (implementing

government counterpart for matching grant) to properly capture all the necessary information of matching grant

applicants, outcome of the application process, recording grant disbursement, and performance monitoring of the

beneficiaries once the grant is disbursed to ensure that grant is utilized for the given purpose. We will use this M&E

system to keep track of treatment delivery and participation in the treatment by the beneficiary and their activities

and outputs in terms of measures taken related to export performance including obtaining the required

accreditation and firms’ investment related to developing new product variety or accessing new markets.

10. DATA PROCESSING AND ANALYSIS (~ 1-2 pages)

10.1 DATA CODING, ENTRY, AND EDITING2 (E)

Describe planned methods for data entry, and for handling missing data, imputations.

Information from the firm application for matching grant will be entered in the database in Stata. Administrative

data on the treatment and control groups will be collected from customs database. Firm survey data will be entered

using double-entry procedure into the database with pre-specified consistency check protocol in place. No

imputation for missing data will be done, except as a robustness check. We will test for selective attrition and item-

non-response, and if this differs by treatment group status, employ bounding approaches to examine the robustness

of our findings to these issues.

10.2 MODEL SPECIFICATION FOR QUANTITATIVE DATA ANALYSIS

Describe the statistical method(s) that will be used to compare groups for primary and secondary outcomes

(the specific equation should be included), any transformations to quantitative data. Specify whether the

standard errors will be clustered or corrected.

Specify what IE parameter of interest will be estimated (e.g., ITT, TT, MTE, LATE).

Describe how you plan to address multiple hypothesis testing.

Describe methods for additional analyses, including spillovers and subgroup analyses.

Provide a list of any variables to be collected to check balance and correct for potential selection due to

attrition, non-response, take-up rate issues (all theoretically important variables to be measured at

baseline, including, those thought to be related to participation/dropout/non-response and the outcomes

of interest).

Lay out a strategy to follow up, test and correct for (if required) sources of bias (e.g., non-random attrition,

non-response, endogenous take-up).

State if you plan to register this IE (see selected links below)

2 This subsection is optional

Page 18: Impact Evaluation (IE) Concept Note How Effective the ...pubdocs.worldbank.org/en/479431510247692379/... · Present IE questions and main outcome(s) the intervention aims to affect

18

o AEA RCT Registry (https://www.socialscienceregistry.org/)

o 3ie Registry (http://www.3ieimpact.org/evaluation/ridie/)

The key objective of the empirical model is to test if a given treatment (e.g., treatment arm 1 and 2) has any

significant effect on the treated vis-à-vis the control group. Thus our focus will be on the intention-to-treat (ITT). As

we employ the RCT approach, the basic econometric specification of our empirical model is straightforward and is

of the following form:

Outcome = a + b*Treat + c *Randomization Strata +e

We will further attempt to improve the power of our analysis by controlling for export performance outcome at the

baseline in line with the ANCOVA specification of McKenzie (2010), i.e,:

Outcome = a + b*Treat + c*Randomization Strata +f*Baseline of export performance outcomes + e

We expect that all firms in the treatment group will undertake at least some part of the treatment albeit differing in

intensities. However, if a non-trivial fraction of the treatment firms do not take part in the treatment and drops out,

we will estimate the LATE by instrumenting taking part in the program with treatment assignment. Since none of the

control will receive the treatment, the LATE will also be the TOT.

Finally, if there is statistically significant differential attrition by treatment status, we will use Lee (2009) bounds to

see how sensitive our results are to this differential. As our target firms will be larger firms we expect attrition to be

fairly low.

We plan to register this IE in the AEA RCT Registry and 3ie Registry.

11. STUDY LIMITATIONS AND RISKS (E) (1/2 page)

Provide an assessment of risk and threat to internal validity (related to previous section)

Discuss issues related to external validity, particularly (i) representativeness of the sample; (ii)

representativeness of the institution(s) delivering the intervention, and (iii) feasibility that the intervention

can be scaled up.

The IE relies on RCT. The internal validity is the strength of such design. The study, however, has the potential risks

of (i) non-random attrition, and (ii) spillovers between treatment and control groups. As discussed above, we will

analyze from follow-up firm surveys and M&E system for the beneficiaries if there is any non-random attrition and

if so will use bounds to evaluate the sensitivity of our results due to attrition. We view that the risks of potential

spillover between the treatment and control group is limited as essentially the treatment and control group are

competing for the matching grant subsidy. Moreover, the intervention involves matching investment from the firm

side so firms would like to internalize the benefits of matching grant as much as possible. Nonetheless, we will

attempt to obtain the list of businesses a firm interact with in the application form as well as in the firm follow-up

survey and will attempt to measure any spillover effects on these interacted firms on the latter’s export performance

based on the customs database.

Page 19: Impact Evaluation (IE) Concept Note How Effective the ...pubdocs.worldbank.org/en/479431510247692379/... · Present IE questions and main outcome(s) the intervention aims to affect

19

Our implementing partner is the key agency in GoT to implement the matching grant scheme nationwide. Similarly,

the key objective of our communication and outreach initiative is to have a representative sample of Tunisian firms

with export potentials from which we will draw the sample of our treatment and control group. All these positively

contributes to the external validity of our IE findings. Also as mentioned above, findings from our IE will be used as

valuable inputs in designing the matching grant scheme for subsequent years (ex-post our evaluation). Thus there is

the potential of scaling up the interventions that would deem most effective in promoting firms’ export performance

based on our IE.

12. IE MANAGEMENT (E,R) (All tables)

12.1 EVALUATION TEAM AND MAIN COUNTERPARTS

Provide list of all IE team members with their position, affiliation, and responsibilities (including lead

researcher, other research team members, and all project staff involved in the IE work, and main

implementing agency counterparts).

Table 2. IE Team and Main Counterparts

Name Role Organization/Unit

Aminur Rahman IE TTL and Principal investigator (PI) (specify Lead Researcher)

WBG/GTCME

Giacomo de Giorgi

Co-PI New York Federal Reserve Bank and ICREA/MOVE, Barcelona Graduate School of Economics

Eric Verhoogen

Co-PI Columbia University

Mariem Malouche TTL, WBG Project WBG/GCTME

Jade Salhab Field Coordinator and WBG Project Team Member

WBG/GCTME

Cheffia Chalbia Main implementing and policy counterparts

Tunisia Export Promotion Agency (CEPEX)

12.2 WORK PLAN AND DELIVERABLES

Table 3. Milestones, Deliverables, and Estimated Timeline

Milestones Deliverables Completion Date

Peer-reviewed Concept Note

Methodology note October 14, 2016

Page 20: Impact Evaluation (IE) Concept Note How Effective the ...pubdocs.worldbank.org/en/479431510247692379/... · Present IE questions and main outcome(s) the intervention aims to affect

20

Data collection plan and pilot

TORs Questionnaires

November, 2016

Data collection (Baseline) Cleaned data Dictionaries

December-January, 2017

First data analysis Presentation Data file Do files Baseline report

March 2017

Implementation of intervention aligned to evaluation

Rollout plan Monitoring reports verifying treatment and control status

Interventions will be on a rolling window basis in every quarter or so from January 2017 onwards until December 2018 or so.

Follow-up data collection plan

TORs Questionnaire

Annual firm survey in 2017, 2018, and 2019; Customs data collection on a quarterly basis from March 2017 onwards till June 2021.

Data collection (Follow-up) Cleaned data Dictionaries

As above.

Final report and policy notes

Technical note Policy note Data file Do files

December 2019

Dissemination of findings Presentations June 2019 onwards

12.3 BUDGET

(1 paragraph)

Present total budget and disaggregated by staff time, data collection, and travel. Include all sources of

funding, both Bank-executed and client-executed (BB resources, trust fund and grants, FBS, EFO, project

financing for the IE, such as data collection, and other client financing). Estimate and include all

research/staff time (not only the time charged).

Table 4. Total Budget per Category

Category USD %

Staff 48,655 10

STC 145,990 29

Data Col lection 240,000 48

Travel 65,000 13

Total 499,645 100.00

Total Budget per Category

Page 21: Impact Evaluation (IE) Concept Note How Effective the ...pubdocs.worldbank.org/en/479431510247692379/... · Present IE questions and main outcome(s) the intervention aims to affect

21

The main sources of funding are the DIME i2i preparation grant ($25,000), three years of DIME i2i implementation grant funding ($50,000 x 3), and WBG Project Supervision Fund (approximately $200,000). The remaining costs are covered by work program time allocation of the PI and Co-PIs from their respective employers.

Attach detailed budget (see excel file template).

Excel file attached.

13. PLAN FOR USING DATA AND EVIDENCE FROM THE STUDY (1 paragraph)

Describe communication, participation, and dissemination strategy (potential users of findings, media

channels) at all stages of the IE (design, baseline analysis, mid-corrections, follow-up analysis, and final

results).

As mentioned above, we will deploy communication campaign in association with the government counterpart to

raise awareness and encourage wider participation from the businesses to two intervention arms. As the application

process will be continued 2-3 times (or even more) a year for a period over 2-3 years, the interim findings from

different rounds of application processes will be used for policy dialogues with the government by the WBG team

and for media and communication campaign by the government counterpart. The research paper based on the IE

will be published as World Bank Policy Research Working Paper and will be submitted for publication to the relevant

academic journal(s). We will also present the findings in relevant workshops organized by the WBG and academia.

REFERENCES Alvarez, R. and Crespi, G. 2000. “Exporter Performance and Promotion Instruments: Chilean Empirical Evidence.”

Estudios de Economía 27: Universidad de Chile.

Amiti, Mary and Amit Khandelwal. 2011. “Import Competition and Quality Upgrading.” Review of Economics and

Statistics.

Atkin, David, Khandelwal, Amit and Osman, Adam. 2016. “Exporting and Firm Performance: Evidence from a

Randomized Experiment” Unpub. paper, MIT.

Bruhn, M., Karlan, D.S. and Schoar, A., 2013. The impact of consulting services on small and medium enterprises:

Evidence from a randomized trial in Mexico. World Bank Policy Research Working Paper, (6508).

Cadot, Olivier, Ana M. Fernandes, Julien Gourdon, and Aaditya Mattoo (2015). “Are the benefits of export support

durable? Evidence from Tunisia.” Journal of International Economics, forthcoming.

Campos, Francisco, Aidan Coville, Ana Fernandes, Markus Goldstein and David McKenzie (2014). "Learning from the

Experiments that Never Happened: Lessons from Trying to Conduct Randomized Evaluations of Matching Grant

Programs in Africa," Journal of the Japanese and International Economies, vol. 33, pp. 4-24.

Castillo, Victoria, Alessandro Maffioli, Sofá Rojo and Rodolfo Stucchi (2011) “Innovation policy and employment:

Evidence from an impact evaluation in Argentina”, IDB Working Paper no. IDBTN-341

Page 22: Impact Evaluation (IE) Concept Note How Effective the ...pubdocs.worldbank.org/en/479431510247692379/... · Present IE questions and main outcome(s) the intervention aims to affect

22

Crespi, Gustavo, Alessandro Maffioli, and Marcela Melendez (2011) “Public support to innovation: The Colciencas

experience”, IDB Working Paper no. IDB-TN-264

De Mel, Suresh, David McKenzie and Christopher Woodruff (2008) “Returns to Capital in Microenterprises: Evidence

from a Field Experiment”, Quarterly Journal of Economics 123(4): 1329-72.

Gourdon, Julien, Jean Michel Marchat, Siddharth Sharma and Tara Vishwanath (2011) “Can matching grants promote

exports? Evidence from Tunisia’s FAMEX II program”, pp. 81-106 in Olivier Cadot, Ana Fernandes, Julien Gourdon

and Aaditya Mattoo (eds.) Where to spend the next million? Applying impact evaluation to trade assistance. World

Bank: Washington, D.C.

Karlan, Dean, Ryan Knight and Christopher Udry (2014) “Consulting and Capital Experiments with Microenterprise

Tailors in Ghana”, Journal of Economic Behavior and Organization, forthcoming.

Klinger, Bailey and Matthias Schundeln (2011) “Can Entrepreneurial Activity be Taught? Quasi- Experimental

Evidence from Central America”, World Development 39(9): 1592-1610

McKenzie, D. (2012). “Beyond Baseline and Follow-up: The Case for More T in Experiments.” Journal of Development

Economics 99(2): 210-21.

McKenzie, David J., Assaf, N. and Cusolito, A., 2015. The Additionality Impact of a Matching Grant Program for Small

Firms: Experimental Evidence from Yemen. World Bank Policy Research Working Paper, (7462).

McKenzie, David J. and Christopher Woodruff (2008) “Experimental Evidence on Returns to Capital and Access to

Finance in Mexico”, World Bank Economic Review 22(3): 457-82

McKenzie, David J., 2015. Identifying and spurring high-growth entrepreneurship: experimental evidence from a

business plan competition. World Bank Policy Research Working Paper, (7391).

Volpe, C. and Carballo, J. 2008. “Is Export Promotion Effective in Developing Countries? Firm-level Evidence on the

Intensive and Extensive Margins of Export Growth.” Journal of International Economics 76, 89-106.

World Bank. 2012. Development Policy Review: Creating jobs and accelerating inclusive growth. Report No. 71799-

TN. The World Bank, Washington D.C.