use of prospective studies on policy analysis · 2019. 2. 7. · use of prospective studies on...

European Journal of Scientific Research

ISSN 1450-216X / 1450-202X Vol. 148 No 1 December, 2017, pp. 161-178

http://www. europeanjournalofscientificresearch.com

Use of Prospective Studies on Policy Analysis:

A Bibliometric Study1

Sidarta Ruthes

Universidade Tecnológica Federal do Paraná – UTFPR, Brazil

Sistema Federação das Indústrias do Estado do Paraná

Observatórios Sistema Fiep

Christian Luiz da Silva

Universidade Tecnológica Federal do Paraná – UTFPR

Programa de Pós-graduação em Tecnologia – PPGTE

Programa de Pós-graduação em Planejamento e Governança Pública – PGP

Departamento de Gestão e Economia – DAGEE

Abstract

The aim of this article is to illustrate the status of the scientific production related to

the use of prospective studies on the policy analysis process. Two questions guided this

investigation: how are future studies used on policy analysis? Are prospective studies used

on all stages of the policy analysis? Considering that, an attempt was made to conduct a

bibliometric analysis in order to quantitatively measure the scientific production on this

theme, exploring databases and showing the parameters and variables from a set of articles

(bibliographic portfolio), such as authors, citations, references, relevance of journals, etc.

The bibliometric analysis was conducted according to the guidelines of the process known

as ProKnow-C. In addition, the EndNote X6 software was used to support the bibliometric

analysis. As a result, it is highlighted that prospective studies are used mainly as

information input to define the agenda of public policies, and they may not be more

effectively extrapolated into other stages of the policy analysis. Furthermore, the

bibliometric analysis has shown the emergence of the theme, with the identification of only

13 articles discussing this theme, considering that 7 of these documents were published

between 2012 and 2013.This is an emerging theme on the scientific community that works

with prospective studies, and it is discussed among a small group of researchers, with

citations among peers.

Keywords: Development Planning and Policy, Planning Policy, Industrial Policy,

Government Policy

1. Introduction Public policies are established to create solutions, organize the collective life and improve the daily life

of people; or, at least, this should be their main focus. Several problems have been minimized and even

solved through effective public policies, broadening the roles of the State to the detriment of the

demand of the society. It is within this context that public policies have a significant position on

1Award-winning paper at the Altec 2015 Congress.

162 Sidarta Ruthes and Christian Luiz da Silva

academic discussions, becoming the target of several researches in Brazil and worldwide (Heidemann,

2009).

With the advance of these researches, public policies were improved and several countries had

a significant development through successful implementations of these instruments. However, on the

one hand, there is a group of countries that reached satisfactory results with their public policies; on the

other hand, there is a significant group of countries that have faced major difficulties due to the fact

that they do not have efficient policies customized to their respective realities. Successful practices in

certain regions were not necessarily able to reach similar results when implemented in other places.

This fact may be explained by the cultural differences, or even by the different meanings of certain

actions, whose understanding and importance should be relativized. However, the fact is that

differences occur, and they may be related to adjacent political contexts that affect the policy

formulation process, which is the subject-matter of Policy Analysis.

The difficulty in analyzing effective policies and those under formulation is clear, since this is a

multidisciplinary area, involving significant amounts of methods, variables and agents within this

process (Gerston, 2010; Howlett, 2011). For Gerston (2010), the methods used for policy analysis are

different from those used on "hard" sciences. The author also highlights that, since this theme is related

to Social Sciences, public policies are filled with emotions, unexpected events and irrationality, which

makes them extremely difficult to quantify or replicate.

The traditional introspective, linear and deterministic tools to elaborate public policies are not

able to meet the challenges imposed by the complexity of society and the economy (Mureddu et al.,

2012). Policy analysis must be able of minimizing the limitations of the systemic understanding of

these policies, evaluating their non-linear impacts, mainly within a context of intensive growth of the

available data (Frey, 2000; Farah, 2011), but also in relation to the potential transformations that are

happening and that should be part of the scope of these analyses (Ughetto, 2007; Habegger, 2010;

Calof and Smith, 2010; Havas et al., 2010; Solem, 2011; Mureddu et al., 2012; Ahlqvist et al., 2012;

Johnston, 2012).

This complexity is even more difficult for those who formulate the policies, due to the limited

rationality of the decision-makers, which is in part the result of incomplete orimperfectanalysis

informational bases (Simon, 1979). In addition, the principle of functionality that is established in the

definition process of public policies creates a segmentation of the policy itself, and it may generate a

short-sighted and fragmented process (policy focus). Sousa (2006) highlights that it is necessary to

incorporate other variables to the formulation and analysis of public policies, broadening the scope of

the rationality to a more systemic view of the process. In that sense, prospective may contribute for

policy analysis, offering the necessary subsides for a rational decision-making process within a context

of complexity (Eriksson and Weber, 2008; Nelson, 2010; Könnöläet al., 2012; Horton, 2012;

Mendonça et al., 2012).

Therefore, the fact is that there are limitations to policy analysis, and one of the challenges may

be related to the restricted view of the complexity by the formulators and decision-makers. Thus, there

is the need for new approaches, able of minimizing some obstacles on the analysis, searching for new

solutions for old problems and, mainly, attempting to walk on paths that had been previously identified

as promising paths.

For such, policy formulators use several information inputs to try to develop their purposes,

such as those who use the results from prospective studies for policy design (Volkery and Ribeiro,

2009; Calof and Smith, 2012; Yoda, 2011; Ahlqvist et al., 2012; Green, 2012). However, the use of

these approaches for policy analysis is not yet extensively explored in the literature. In other words,

these approaches are frequently used to generate input for agenda setting, and they are not part of the

other stages2of policy analysis (Rijkens-Klomp, 2012; Van Der Steen and Van Twist, 2013; Hughes,

2013).

2 Basically, the stages are: (i) definition of the agenda; (ii) identification of alternatives; (iii) assessment of the options; (iv)

selection of options; (v) implementation; and (vi) assessment of the options.

Use of Prospective Studies on Policy Analysis: A Bibliometric Study 163

The challenge is related to the policy formulation that would be able to analyze the relationship

possibilities among the fundamental variables for the success of the proposal and assure that the

necessary prospected elements (such as trends and scenarios) are preserved by the policy, promoting

their long-term development (sustainability).

Some researches indicate that prospective studies are under used by formulators and political

decision-makers, and they are only used to generate ideas for agenda setting, and do not contribute to

more complex decisions and other stages of the political process within a maximized use of

prospective analysis. It is also noteworthy that a possible response to this underuse may be related to

the institutional barriers, due to the lack of understanding of the possible obstacles of the process

(Volkery and Ribeiro, 2009; Rijkens-Klomp, 2012).

Some prospective approaches defend the proactive behavior, that is, a behavior that may

transform the future due to the actions taken by the agent now (Godet and Durance, 2011). However,

several long-term studies developed for policy setting were only able to be established in the realm of

ideas. These studies reached the “proactivity” on public policies only from an idealistic, abstract and

emotional manner, that is, only by the perception of formulators and decision-makers (restricted use on

the political agenda setting). In order for the results of the prospective studies to go beyond the realm

of ideas and agenda setting, a more realistic, concrete and rational prospective approach is necessary,

that is, a proposal that is able to understand the obstacles of the political system, observing the

institutional and legal questions that may affect the development of policies aligned to the future

prospects in a more realistic manner.

Currently, prospective studies are used to offer subsides for the formulation of policies;

however, considering a context of institutional complexity (from an economic, social and/or political

perspective), these approaches are not able of reaching the effectiveness of their proposals, penetrating

on the other stages of policy analysis. Therefore, it is discussed whether this approach could go beyond

agenda setting, getting further into the other Policy Analysis stages, in addition to the factors that

hinder a more effective (maximized) use of prospective studies on public policies. In that sense, some

questions emerged from this theme, for example: which are the prospective models that have

effectively gone beyond agenda setting within the policy analysis cycle? What is the effectiveness

of prospective studies in policy formulation?

Considering that, an attempt was made to conduct a bibliometric analysis in order to elucidate

the status of the scientific production related to the cross between the areas of Prospective Studies and

Public Policies, mainly related to its analysis. An attempt was made to explore the merger between

both areas, since it is our intention to defend the argument that prospective studies do not lead to the

advancement of the other stages of policy analysis since there are institutional obstacles that were not

considered on the conception of the prospective proposals, mainly due to the differences in the

understanding of the interested groups (“production industry” versus “policy-makers”) on the studied

system and/or sector.

The aim of the bibliometric analysis is to measure the scientific production in a quantitative

manner, exploring databases and showing parameters and variables of a set of articles (bibliographic

portfolio), such as authors, citations, references, relevance of journals, etc. (Kleinubing, 2010; Afonso

et al., 2011; Lacerda et al., 2012; Vaz et al., 2012). Bibliometrics contributes for the identification of

scientific advances, illustrating the status of the knowledge produced and published by the academy,

also allowing the definition of the relevance and the originality of a research, since it shows the gaps

on certain areas of knowledge that may be explored. The bibliometric analysis shown on this study was

conducted through a process called ProKnow-C or Knowledge Development Process – Constructivist3

(Lacerda et al., 2012).

3 Process submitted to INPI for patenting by the researchers Leonardo Ensslin, Sandra Rolin Ensslin, Rogério Tadeu de

Oliveira Lacerda and Jorge Eduardo Tasca in 2010.


The methodological procedures that guided this research were carried out in December 2013,

exploring the bases indexed at portal Periódicos Capes4. The bibliometric analysis process used was

established in 9 stages, according to Figure 1.

Figure 1: Bibliometric Analysis Process

Source: made by the author.

The following sections describe in details each stage of the bibliometric process used for this

research, following the order graphically represented in Figure 1.

2. Definition of Keywords The definition of keywords was a process that implied a detailed understanding of Prospective and

Public Policy, as knowledge areas. We chose to use keywords in the English language since this is the

main language used for the indexation of articles on the databases. Even so, in the case of articles in

other languages, such as Spanish and Portuguese, the bibliometric research would contemplate these

articles since their title, abstract and keywords were in English, which is rather common on indexed

articles.

The exception is the term “prospective stratégique” in French, whose origin is French and it

requires a careful translation, since, according to Godet and Durance (2011, p. xiii – preface), “there

has never been a true equivalent” for the English language. This procedure was used since the French

approach is one of the main branches of the Prospective area.

The terms related to Public Policy that were broke down into their possible derivations and that

were subjected to the adherence test are: (i) policy making; (ii) policy analysis; (iii) policy formulation;

and (iv) policy design. For analysis and crossing purposes, this is group 1 of keywords.

The same process mentioned above was conducted for the terms related to the Prospective area,

which generated group 2 of keywords: (i) strategic foresight; (ii) technology foresight; (iii) strategic

prospective; and (iv) prospective stratégique.

The terms policy making, policy analysis, policy formulation and policy design are not

restricted to public policy. Several articles related to organizational policy, for example, could show up

in the scanning process of databases. In order to assure that the search on these bases would come up

only with articles on public policies, the terms “public policy” and “industrial policy” were used to

refine the search, creating group 3 of keywords. In short, the 3 groups of keywords were crossed,

creating the refinement process shown on Figure 2.

4 Portal Periódicos Capes is a virtual library that gather and makes international scientific publications available to

educational and research institutions in Brazil (Coordenação de Aperfeiçoamento de Pessoal de Nível Superior – CAPES,

2010, Institutional Text).

Definition of keywords

Keyword adherence test

Selection of article bases

Scanning on article bases

Refinement of articles

Classification of articles

Analysis of the potential portfolio

Definition of the focus portfolio

Final considerations

Use of Prospective Studies on Policy Analysis: A Bibliometric Study

Source: made by the author

3. Keyword Adherence TestThe adherence test is fundamental to verify the importance

investigated area. Through this

quantitative manner, measuring the return each

in a simple

adhere to the theme

at portal Periódicos

3 groups of keyw

Table 1:

Category

1

2

3

4




ted area. Through this


in a simple manner in

the theme or that are poor

Periódicos Capes

3 groups of keywords. The result for group 1 is shown on Table 1.

Adherence Test for Group 1 of Keywords

Keyword

"policy making"

"policy-making"

"policies making"

"policies-making"

"policy analysis"

"policy-analysis"

"policies analysis"

"policies-analysis"

"policy formulation"

"policy-formulation"

"policies formulation"

"policies-formulation"

"policy design"

"policy-design"

"policies design"

"policies-design"

Total


Figure 2:



ted area. Through this procedure, it


in Boolean fields

or that are poor

Capes and on Google Scholar, on December 2, 2013. Tests were conducted for the

ords. The result for group 1 is shown on Table 1.


Keyword

"policy making"

making"

"policies making"

making"

"policy analysis"

analysis"

"policies analysis"

analysis"

"policy formulation"

formulation"

"policies formulation"

formulation"

"policy design"

design"

"policies design"

design"


Figure 2: Refinement

3. Keyword Adherence Test The adherence test is fundamental to verify the importance

procedure, it


fields. The aim of this testis to

or that are poor representatives of

on Google Scholar, on December 2, 2013. Tests were conducted for the



Periódicos Capes

88.075

88.075

1.480

1.480

74.517

74.517

979

979

14.261

14.261

101

101

8.391

8.391

675

675

376.958

Portfolio of Articles

Group 2 of

keywords


Refinement Process Used

The adherence test is fundamental to verify the importance

procedure, it is possible to

quantitative manner, measuring the return each keyword offers

. The aim of this testis to

representatives of the studied area. The referred test was conducted




Scientific production

Periódicos Capes R%

23.36%

23

0.39%

0.39%

19.77%

19.77%

0.26%

0.26%

3.78%

3.78%

0.03%

0.03%

2.23%

2.23%

0.18%

0.18%

100.00%


Group 3 of

keywords

Group 2

keywords

Group 1 of

keywords


Process Used on Article Bases

The adherence test is fundamental to verify the importance of the

possible to verify the relevance of the term in a

keyword offers to the investigator, when searched for

. The aim of this testis to eliminate the

the studied area. The referred test was conducted



Scientific production

R% Google Scholar

23.36%

23.36%

0.39%

0.39%

19.77%

19.77%

0.26%

0.26%

3.78%

3.78%

0.03%

0.03%

2.23%

2.23%

0.18%

0.18%

100.00%


Group 1 of

keywords

on Article Bases

of the keywords for

verify the relevance of the term in a

to the investigator, when searched for

liminate the keywords that do not



Google Scholar

1.720.000

1.720.000

4.590

4.590

1.050.000

1.050.000

2.600

2.600

127.000

127.000

696

696

93.400

93.400

1.850

1.850

6.000.272

keywords for the studied or



keywords that do not



Google Scholar R%

28.67%

28.67%

0.08%

0.08%

17.50%

17.50%

0.04%

0.04%

2.12%

2.12%

0.01%

0.01%

1.56%

1.56%

0.03%

0.03%

100.00%

165

the studied or



keywords that do not



R%

28.67%

28.67%

0.08%

0.08%

17.50%

17.50%

0.04%

0.04%

2.12%

2.12%

0.01%

0.01%

1.56%

1.56%

0.03%

0.03%

100.00%



Note: R% = percentage representativeness.

The analysis of Table 1shows that the use of keywords has an homogeneous behavior when

testing for searches with or without hyphen, that is, there are no changes in the return for such terms

whether or not they are hyphenated, both in the plural and in the singular form. In relation to the use of

keywords in the singular and/or plural form, the results show a significant difference on both search

mechanisms. That is, the results show that within the academic environment, it is more usual to use the

terms in the singular form. As a result, the terms “policy formulation” and “policy design”, as well as

their respective derivations, showed little expressivity for the test. The stronger keywords are,

respectively, “policy making” and “policy analysis”.

In relation to the second group of keywords (see Table 2), the test reveals that the most used

term among the 4 terms analyzed is technology foresight, with approximately 63% of representativity

on both mechanisms (Periódicos Capes and Google Scholar). The documents with the French term

“prospective stratégique” represent approximately 1% of the articles on the area on Periódicos Capes

and 8% on Google Scholar.

Table 2: Adherence Test for Group 2 of Keywords

Category Scientific production

Keyword Periódicos Capes R% Google Scholar R%

1 "technology foresight" 958 62.66% 8.310 63.85%

2 "strategic foresight" 483 31.59% 3.120 23.97%

3 "strategic prospective" 74 4.84% 515 3.96%

4 "prospective stratégique" 14 0.92% 1.070 8.22%

Total 1.529 100.00% 13.015 100.00%



The last test shows that the term “public policy “is more usual in the singular form, with a

significant representativity of 88.43% on Periódicos Capes, and a little less significant on Google

Scholar, with 65.96% of representativity. The keyword “industrial policy” is more usual in the singular

form in both mechanisms, with results above 80%, as shown on Table 3.

Table 3: Adherence Test for Group 3 of Keywords

Category Scientific production

Keyword Periódicos Capes R% Google Scholar R%

1 "public policy" 268.047 88.43% 1.300.000 65.96%

"public policies" 35.071 11.57% 671.000 34.04%

Total 303.118 100.00% 1.971.000 100.00%

2 "industrial policy" 18.820 81.61% 234.000 84.32%

"industrial policies" 4.242 18.39% 43.500 15.68%

Total 23.062 100.00% 277.500 100.00%



The result of the tests conducted on groups 1, 2 and 3 of keywords resulted in 16 combinations

of terms for the bibliometric process, as shown synthetically on Table 4. Two terms related to how to

make and how to analyze policies (group 1), four related to the analysis approach proposal (group 2);

and two restricted to the type of policy to be analyzed (group 3), resulting in 16 combinations

(permutation: Pn=n! or 2x4x2=16). Table 4: Crossings Validated by the Adherence Test


Groups of keywords

Group 1 Group 2 Group 3

Combinations

"policy making"

X

"technology foresight"

X

"public policy" "strategic foresight"

"policy analysis" "strategic prospective"

"industrial policy" "prospective stratégique"

R% 86.27% 100.00% 87.95%


In addition, the test shows that all final keywords are significantly representative, which allows

the qualification of their use for the bibliometric research.

4. Selection of Article Bases The article bases were selected from tests conducted at portal Periódicos Capes. Boolean searches

were conducted with the terms from the three groups, and the most significant bases for the theme of

this research were selected. Scielo was selected since it is the most usual national article basis in the

country. Table 5shows the results of this selection stage for the article bases.

Table 5: Selected Article Bases

Adopted code Article bases

B1 ISI - Thomson Scientific

B2 Science Direct Elsevier

B3 SCOPUS

B4 EBSCO:host

B5 SCIELO

B6 Arts & Sciences

B7 Oxford Journals

B8 ProQuest

B9 GALE Academic

B10 Emerald Journals


A thorough study was conducted regarding all bases on how the Boolean search process for

articles works, as well as understanding the data export process to the EndNote X6software.

5. Scanning on Article Bases The scanning process occurred between December 3 and 6, 2013, on the selected article bases, and for

all 16 keyword combinations. The result of this process is shown on Table 6.


Table 6: Raw Portfolio of Articles

C Groups of keywords (G) Article bases (B)

Total G1 G2 G3 B1 B2 B3 B4 B5 B6 B7 B8 B9 B10

1

Policy

Making

Technology

Foresight

Public

Policy

0 76 81 63 0 23 1 0 0 21 265

2 Strategic

Foresight 2 23 14 15 0 10 1 0 1 13 79

3 Strategic

Prospective 0 4 2 0 0 1 0 0 0 0 7

4 Prospective

Stratégique 0 1 1 0 0 0 0 0 0 0 2

5

Policy

Analysis

Technology

Foresight

Public

Policy

0 41 32 19 0 9 0 0 0 5 106

6 Strategic

Foresight 0 13 10 0 0 7 1 0 0 4 35

7 Strategic

Prospective 0 1 0 0 0 0 0 0 0 2 3

8 Prospective

Stratégique 0 1 1 0 0 0 0 0 0 0 2

9

Policy

Making

Technology

Foresight

Industrial

Policy

0 12 7 20 0 7 1 1 0 2 50

10 Strategic

Foresight 0 2 1 1 0 3 0 0 0 0 7

11 Strategic

Prospective 0 1 0 0 0 1 0 0 0 0 2

12 Prospective

Stratégique 0 0 0 0 0 0 0 0 0 0 0

13

Policy

Analysis

Technology

Foresight

Industrial

Policy

0 6 3 4 0 5 0 0 0 1 19

14 Strategic

Foresight 0 0 0 0 0 2 0 0 0 0 2

15 Strategic

Prospective 0 2 0 0 0 0 0 0 0 1 3

16 Prospective

Stratégique 0 0 0 0 0 0 0 0 0 0 0

Total of articles 2 183 152 122 0 68 4 1 1 49 582


Notes: C = keyword combinations.

Boolean operators were used to optimize the search. In addition, an attempt was made to

delimitate the scientific production between 1976 and 2014. The choice for this time period is due to

the important milestone in the prospective literature, with the publication, in 1976,of the Doctorate

Thesis5by Michel Godet, considered by many as the father of Strategic Prospective, a branch that

influenced the broadening of the scope of the Anglo-Saxon approach of “Technology Foresight” to

“Strategic Foresight”, however, with significant differences in relation to the French prospective.

6. Refinement of Articles It is important to highlight that the results shown on Table 6resulted in the raw portfolio of articles,

which has duplicated articles. Three situations could be observed that generated the duplicity of

5 Doctorate in economic Sciences by the University of Paris-1 Panthéon-Sorbone (l’Université Paris-1 Panthéon-Sorbonne)

in 1976. Title of the doctorate thesis: “Crise de la prévision, essor de la prospective”, in English – “Foresight crisis,

prospective development” (DURANCE, 2004, our translation).


articles: (i) replication on the same article basis – when the article had more than one combination of

keywords on its content; (ii) shown on more than one basis – when the journal in which the article was

published is indexed on different bases; and (iii) replication due to spelling problems – when the article

has different spelling forms of its identification components (title, author, year, etc.) in different bases.

For example, the symbols “ - ” and “ – ” are treated as different symbols by the EndNote X6 software.

The last situation created exclusion problems on the software, since it was not able to identify the

articles with spelling problems using the find duplicates function. Their identification was not

automatic, leading to a manual work of identification and exclusion of the articles.

After identifying and excluding the duplicated articles, the refined portfolio of articles resulted

in 344 publications, as represented on Figure 3.

Figure 3: Duplication of Articles


7. Classification of Articles After excluding the repeated articles, the following components of the remaining articles were read in

full: (i) title and subtitle; (ii) abstract; and (iii) keywords. The purpose of this reading is to eliminate

articles that are not related to the objectives of this research. In total, 166 articles that did not

correspond to the subject-matter and to the objectives of this study were eliminated.The classification

of the articles observed the decision flow that is graphically represented on Figure 4.

582 articlesexportedto

EndNote X6

344 unduplicatedarticles

238 duplicatedarticles


Figure 4: Classification Process Flow of the Articles


The objective of the first phase (see Figure 4) of this flow is to identify articles that are related

to the proposal of this research, that is, documents related to prospective studies (methods, cases,

theories, etc.). This focus was used, since, during the scanning process of the bases, the keywords were

related to the public and industrial policy. Therefore, this first stage should identify articles that had a

strong relationship with prospective studies, precisely in order to verify the contributions of this

research line to the theme of public and industrial policy. The second phase of this classification flow

has the purpose of verifying the originality of the research thesis that is being suggested. In other

words, an attempt was made to verify the existence of any similar approach to the one that was being

suggested, or that could contribute for the discussion of this research.

The aim of the third phase (Figure 4) was to classify the articles according to the type of

prospective study. The result of this phase may be considered as a sub-product of this research, since it

shows how prospective studies are being used in the area of public and industrial policies. Table

7shows the result of the classification process of the articles according to the prospective studies. Table 7: Result of the Bibliometric Process Classification

Is the article related to prospective

studies?

Is the article aligned with the thesis of this

research?

Yes

No

Eliminatedarticle

Yes No

It could be classified as which type of

prospective study?

Analysis Portfolio

General Portfolio

Reading ofthearticlecom

ponents

Fase 1

Fase 2

Fase 3


Type of classification Number of articles

0 – Sem alinhamento (eliminado do portfólio) 166

1 – Delphi 10

2 – Roadmapping 11

3 – Scenarios 31

4 - Future-oriented Technology Analysis / Horizon Scanning 17

5 - Technology Forecasting 4

6 - Technology Foresight 22

7 - Strategic Foresight 59

8 – Prospective 7

9 - Multi-criteria Decision-making 7

10 - Technology Assessment (TA) 7

11 - Futures Studies 3

Total 344


In general, 178 articles were classified as prospective studies related to public and industrial

policies. However, only 34 of them, at first, were considered as potentially being in alignment with this

research (Figure 5).

Figure 5: Analysis Portfolio


Therefore, summing up the bibliometric process conducted up to this stage, important

international bases were scanned, which consolidate thousands of scientific articles from several parts

of the world. From this universe, 582 articles were identified through the Boolean search and,

consequently, had their identification components exported to the EndNote X6 software. On this

software, a refinement and analysis process of the article components was conducted, resulting in 178

articles related to prospective studies; from them, 34 were selected for a deeper analysis and definition

of the final portfolio.

8. Analysis of the Potential Portfolio After completing the classification of the potential portfolio, the articles were downloaded using the

EndNote X6 software (Find Full Text function). The articles that were not selected by the system were

directly collected on the article bases. Upon the completion of this stage, the portfolio was analyzed,

starting by reading the selected articles in full.

The reading of the selected articles was conducted in a systemic and continuous manner,

characterized as an informative reading, whose purpose, according to Cervo and Bervian (2002), is to

Prospective studies

(178 articles)

Potencial portfolio

(34 articles)


collect data or information in order to answer to specific questions, such as the questions from

scientific studies. In that sense, the selective reading technique was used, which, also according to

Cervo and Bervian (2002), tries to locate the critical research information, eliminating what may be

disregarded in order to focus on what is really the object-matter of the analysis.

Some questions were used to guide the selective reading, knowingly: (i) how is policy analysis

treated on the article? (ii) which is the role of prospective on the analyzed article? (iii) what is the

relationship between prospective and policy analysis? and (iv) is there a similarity with the proposal of

this research? In addition, some criteria were used to qualify the potential articles, knowingly: (i)

strong, when the argumentation is strongly pertinent to the proposal of this investigation; (ii) medium,

when there are elements that may help in the discussion of this research; (iii) weak, when punctual

elements appear, with little use for the discussion of this investigation; and (iv) irrelevant alignment,

when the article does not contribute to what is defended on this research.

The result of this analysis process led to the identification of 10 articles that were strongly

aligned to the theme of this research, 3 publications with medium adherence level, and 11 articles that

were weakly aligned to the central proposal here. Also in that process,10 documents that did not adhere

to the purposes herein were identified, and they could be disregarded with no damage to the

argumentation and discussion of this research. This fact occurred because the possible alignment

identified by the title or abstract was not in agreement with what was being discussed in the full text.

Therefore, the final portfolio of this bibliometric analysis resulted in 24 article, from which 13 are

considered as significant for the discussion of this research (strong and medium alignment), as seen on

Table 8.

Table 8: Distribution of Articles by Adherence Type

Adherence type Number of articles Representativeness

Strong adherence 10 29.4%

Median adherence 3 8.8%

Weak adherence 11 32.4%

No adherence 10 29.4%

Total 34 100.0%


Reading the articles in full allowed to classify the according to their potential in contributing

for this research, analyzing them according to the following perspectives: (i) objective(s); (ii) problem;

(iii) justification; and (iv) method. The same article may contribute for more than one perspective

(research phase or stage). The result of this classification may be seen on Table 9.

Table 9: Potential Focus of Contribution of the Article for the Research

Potential focus of contribution Number of articles(a) Representativeness

Problem 12 38.7%

Method 8 25.8%

Justification 7 22.6%

Objectives 4 12.9%

Total 31 100.0%


Note: (a) the sum is greater because one single publication may contribute for more than one research phase.

Analyzing the articles that were somehow aligned to this research (strong, medium and weak

alignment), the main journals for this bibliometric analysis could be identified (Figure 6), highlighting:

(i) Foresight – Emerald Group Publishing, England, with 6 articles; (ii) Futures– Elsevier Inc., United

States, with 5 articles; (iii) Technological Forecasting and Social Change – Elsevier Inc., United


States, with 4 articles; and (iv) Science & Public Policy – Oxford University Press, England, with 4

articles.

Figure 6: Distribution of Articles by Journal


By removing from the analysis the articles with a weak adherence to this research, that is,

maintaining only documents with strong and medium alignment, the framework of the journals is

changed, and the following order of importance is instituted: (i) Futures, with 4 articles; (ii)

Technological Forecasting and Social Change, with 3 publications; and (iii) Science & Public Policy,

with 3 selected documents. All other journals contribute with 1 article each, in a total of 3 more

articles. Therefore, under this configuration, there is a total of 13 publications (10 with high adherence

and 3 with medium alignment to this research).

Considering that the bibliometric analysis was conducted with the articles from the period

between 1976 and 2014, the emergence of the theme on the academic environment is noticed by the

year in which the articles that are in strong adherence with this research were publishes, as seen on

Figure 7. The first article identified with a strong adherence is from, and 2012 had the most number of

publications.

1

1

1

1

1

4

4

5

6

0 2 4 6 8

Research Evaluation

6th ICEGOV

Journal of Hazardous Materials

Resources Policy

Electronic Journal of Business Research Methods

Science & Public Policy

Technological Forecasting & Social Change

Futures

Foresight

Articles

Jo

urn

als


Figure 7: Year of Publication of the Articles with Strong Adherence


9. Definition of the Focus Portfolio Table 10shows the focus portfolio, which evidences the emergence of the approach developed on this

research, that is, the set of articles that are more strongly aligned to it. In addition, the systemic analysis

of this portfolio shows the interactions, discussions and crossings among these researchers, as shown

on Appendix A (at the end of this article).

Table 10: Focus Portfolio: Strong Adherence

Year Article Author(s) Journal Adherence

2008 Adaptive Foresight: Navigating the

complex landscape of policy strategies.

Eriksson, E. A.;

Weber, K. M.

Technological Forecasting

and Social Change Strong

2009

Scenario planning in public policy:

Understanding use, impacts and the role of

institutional context factors.

Volkery, A.;

Ribeiro, T.

Technological Forecasting


2010

The impact of foresight on innovation

policy-making: recent experiences and

future perspectives.

Havas, A.;

Schartinger, D.;

Weber, M.

Research Evaluation Strong

2012 Complexity science approaches to the

application foresight. Horton, A. Foresight Strong

2012 Facing the future: Scanning, synthesizing

and sense-making in horizon scanning.

Könnölä, T.; Salo, A.;

Cagnin, C.; Carabias, V.;

Vilkkuma, E.

Science & Public Policy Strong

2012 The strategic strength of weak signal

analysis.

Mendonça, S.;

Cardoso, G.;

Caraça, J.

Futures Strong

2012 A new roadmap for next-generation policy-

making.

Mureddu, F.; Osimo, D.;

Misuraca, G.;

Armenia, S.

6th ICEGOV Strong

2012 Barriers and levers to future exploration in

practice experiences in policy-making. Rijkens-Klomp, N. Futures Strong

2013

Towards improving the relevance of

scenarios for public policy questions: A

proposed methodological framework for

policy relevant low carbon scenarios.

Hughes, N. Technological Forecasting


2013

Foresight and long-term policy-making: An

analysis of anticipatory boundary work in

policy organizations in The Netherlands.

Van der Steen, M. A.;

Van Twist, M. J. W. Futures Strong

2004 Foresight in a multi-level governance

structure: Policy integration and

Lyall, C.;

Tait, J. Science & Public Policy Median

1 1 1

0

5

2

0

1

2

3

4

5

6

2008 2009 2010 2011 2012 2013

Art

icle

s

Year


Year Article Author(s) Journal Adherence

communication.

2010 Critical success factors for government-led

foresight.

Calof, J.;

Smith, J. E. Science & Public Policy Median

2010 Extending foresight: The case for and

nature of Foresight 2.0. Nelson, R. Futures Median


10. Final Considerations There are several policy analysis models, however, the use of strategic prospective as the analysis

methodological approach is still not widely explored for this purpose. A bibliometric study was

conducted in order to observe how prospective is used for the analysis of public policies, and the

identified researches are strongly related to the policy formulation stage. However, no studies

conducted with the purpose of using strategic prospective as a policy analysis approach on its broader

sense were identified.

Trying to understand the difficulties or obstacles for a more effective use of prospective studies

on the public sector, mainly in relation to public policies, may influence the modus operand of the

entire prospective process. This research intends to develop a Policy Prospective Analysis proposal,

whose actions may have a higher effectiveness, searching for a better alignment of the future trends to

the public policies.

In addition, the goal of this research is to suggest methods to improve policy management,

analyzing them under the perspective of the complexity that permeates the institutional, personal and

political relationships, contributing with the analysts, decision-makers and formulators of policies.

Therefore, regardless of the political sphere (municipal, state or federal sphere) or the underlying

sector, this methodological proposal aims at contributing to the adequate governance of the public

sector, through a more systemic, rational and sustainable decision-making, on its broader sense,

including environmental, social, economic, spatial and cultural variables on this process.

Referências [1] Afonso, M. H. F., Souza, J. V., Ensslin, S. R. and Ensslin, L. (2011), “Como construir

conhecimen to sobre o tema de pesquisa? Aplicação do processo Proknow-C na busca de

literatura sobre avaliação do desempenho sustentável”, Revista de Gestão Social e Ambiental,

Vol. 5 No. 2, pp.47-62.

[2] Ahlqvist, T., Valovirta, V. and Loikkanen, T. (2012), “Innovation policy roadmapping as a

systemic instrument for forward-looking policy design”, Science and Public Policy Journal,

Vol. 39 No. 2, pp. 178-190.

[3] Calof, J.and Smith, J. E. (2010), “Critical success factors for government-led foresight”,

Science and Public Policy Journal, Vol. 37 No. 1, pp. 31-40.

[4] Calof, J. and Smith, J. E. (2012),“Foresight impacts from around the world: a special

issue”,Foresight Journal, Vol. 14 No. 1, pp. 5-14.

[5] Cervo, A. L. and Bervian, P. A. (2002), Metodologia científica, Prentice Hall, São Paulo, SP.

[6] Coordenação de Aperfeiçoamento de Pessoal de Nível Superior – CAPES(2010), “Periódicos

Capes: institucional”,available at:

http://www.periodicos.capes.gov.br/index.php?option=com_pinstitucional&mn=69 (accessed

15 October 2013).

[7] Durance, P. (2004), “Les entretiens de la mémoire de la prospective: professeur Michel Godet,

titulaire de la chaire de prospective industrielle du CNAM”, available at:

http://www.laprospective.fr/dyn/francais/memoire/M_Godet_(entretien)_v1c.pdf (accessed 11

december 2013).


[8] Dye, T. R. (2009), “Mapeamento dos modelos de análise de políticas públicas”in Eidemann, F.

G.,and Salm, J. F. (Org.),Políticas públicas e desenvolvimen to,Editora UNB, Brasília,DF, pp.

98-129.

[9] Eriksson, E. A. and Weber, K. M. (2008), “Adaptive foresight: navigating the complex

landscape of policy strategies”, Technological Forecasting and Social Change, Vol. 75 No. 4,

pp. 462-482.

[10] Farah, M. F. S. (2011), “Administração pública e políticas públicas”, Revista de Administração

Pública – RAP, Vol. 45No. 3, pp. 813-836.

[11] Frey, K. (2000), “Políticas públicas: um debate conceitual e reflexões referentes à prática da

análise de políticas públicas no Brasil”, Revista Planejamento e Políticas Públicas, No. 21, pp.

211-259.

[12] Gerston, L. N. (2010), “Public policy making: process and principles”,M.E. Sharpe, New

York, NY.

[13] Godet, M. and Durance, P. (2011), La prospective stratégique: pour les entreprises et les

territoires, UNESCO / Dunod, Paris.

[14] Habegger, B. (2010), “Strategic foresight in public policy: reviewing the experiences of the

UK, Singapore, and the Netherlands”, Futures, Vol. 42 No. 1, pp. 49-58.

[15] Havas, A., Schartinger, D. and Weber, M. (2010), “The impact of foresight on innovation

policy-making: recent experiences and future perspectives”, Research Evaluation, Vol. 19 No.

2, pp. 91-104.

[16] Horton, A. (2012), “Complexity science approaches to the application foresight”, Foresight,

Vol. 14No. 4, pp. 294-303.

[17] Howlett, M. (2011), Designing public policies: principles and instruments, Routledge, New

York, NY.

[18] Hughes, N. (2013), “Towards improving the relevance of scenarios for public policy questions:

a proposed methodological framework for policy relevant low carbon scenarios”,

Technological Forecasting and Social Change, Vol. 80No. 4, pp. 687-698.

[19] Johnston, R. (2012), “Developing the capacity to assess the impact of foresight”, Foresight,

Vol. 14 No. 1, pp. 56-68.

[20] Kleinubing, L. S. (2010), “Análise bibliométrica da produção científica em gestão da

informação da base de dados LISA”, Revista Digital de Bibliotecnomia e Ciência da

Informação, Vol. 8No. 1, pp. 01-11.

[21] Könnölä, T., Salo, A., Cagnin, C., Carabias, V. and Vilkkuma, E. (2012), “Facing the future:

scanning, synthesizing and sense-making in horizon scanning”, Science & Public Policy, Vol.

39 No. 2, pp. 222-231.

[22] Lacerda, T. O., Ensslin, L. and Ensslin, S. R. (2012), “Uma análise bibliométrica da literatura

sobre estratégia e avaliação de desempenho”,Revista Gestão & Produção, Vol. 19 No. 1, pp.

59-78.

[23] Mendonça, S., Cardoso, G. and Caraça, J. (2012), “The strategic strength of weak signal

analysis”, Futures, Vol. 44 No. 3, pp. 218-228.

[24] Mureddu, F., Osimo, D., Misuraca, G. and Armenia, S. (2012), “A new roadmap for next-

generation policy-making”, in Proceedings of the 6thInternational Conference on Theory and

Practice of Electronic Governance – ICEGOV,New York, 2012, Association for Computing

Machinery – ACM, New York, pp. 62-66.

[25] Nelson, R. (2010), “Extending foresight: the case for and nature of Foresight 2.0”, Futures,

Vol. 42 No. 4, pp. 282-294.

[26] Rijkens-Klomp, N. (2012), “Barriers and levers to future exploration in practice experiences in

policy-making”, Futures, Vol. 44No. 5, pp. 431-439.

[27] Simon, H. A. (1979), “Rational decision making in business organizations”, The American

Economic Review, Vol. 69No. 4, pp. 493-513.


[28] Solem, K. E. (2011), “Integrating foresight into government. Is it possible? Is it

likely?”Foresight, Vol. 13 No. 2, pp. 18-30.

[29] Ughetto, E. (2007), “Foresight as a triple helix of industry, university and government

relations”, Foresight, Vol. 9 No. 5, pp. 14-22.

[30] Van Der Steen, M. A. and Van Twist, M. J. W. (2013), “Foresight and long-term policy-

making: an analysis of anticipatory boundary work in policy organizations in The Netherlands”,

Futures, Vol. 54 No. 0, pp. 33-42.

[31] Vaz, C. R., Tasca, J. E., Ensslin, L., Ensslin, S. R.and Selig, P. M. (2012), “Avaliação de

desempenho na gestão estratégica organizacional: seleção de um referencial teórico de pesquisa

e análise bibliométrica”, Revista Gestão Industrial, Vol. 8 No. 4, pp.121-153.

[32] Volkery, A. and Ribeiro, T. (2009), “Scenario planning in public policy: understanding use,

impacts and the role of institutional context factors”, Technological Forecasting and Social

Change Journal, Vol. 76No. 9, pp. 1198-1207.

[33] Walker, W. E. (2000), “Policy analysis: a systematic approach to supporting policymaking in

the public sector”, Journal of Multi-Criteria Decision Analysis, No. 9, pp. 11-27.

[34] Yoda, T. (2011), “Perceptions of domain experts on impact of foresight on policy making: the

case of Japan”, Technological Forecasting and Social Change Journal, Vol. 78 No. 3, pp. 431-

447.


Appendix A – Relationship among the Articles of the Final Portfolio


use of prospective studies on policy analysis · 2019. 2. 7. · use of prospective studies on...

Documents