HOW APPROPRIATE ARE LOCAL AUTHORITY SUSTAINABILITY INDICATORS?

by

Bernadette Cass

A thesis presented in part-fulfilment of the degree of Master of Science in accordance with the

regulations of the University of East Anglia

School of Environmental Sciences University of East Anglia University Plain Norwich NR4 7TJ

September 2008 2008

This copy of the dissertation has been supplied on condition that anyone who consults it is understood to recognise that its copyright rests with the author and that no quotation from the dissertation, nor any information derived there from, may be published without the authors prior written consent. Moreover, it is supplied on the understanding that it represents an internal University document and that neither the University nor the author are responsible for the factual or interpretative correctness of the dissertation.

Printed on recycled paper

ii

ABSTRACT

Sustainability, sustainable development and sustainability indicators have been

defined and the importance of operationalising these definitions with regards to

sustainability appraisal discussed. To improve local sustainability, Local Authorities in

England have each chosen a set of sustainability indicators to be used in sustainability

appraisal, in order to assess all options under consideration when choosing their

preferred option for their local development framework.

This dissertation explores a novel criteria-based method for deciding the

appropriateness of local sustainability indicators used in sustainability appraisals.

Over a medium period of time, for the author, this method exhibits stability when

using a reference set of indicators. The two extremes of the Defra classification of rural

and urban were used to ascertain whether, in England, there are differences between

the appropriateness of sustainability indicators chosen by Major Urban and Rural 80 Local Authorities in three regions.

To group the sustainability indicators into the three pillars of economic, environmental

and social sustainability, a classification framework was applied. A survey of Local

Authority officers and consultants involved in choosing sustainability indicators for the

three regions augmented the appropriateness method. Using these methods, an

attempt was made to suggest areas of improvement that would increase

appropriateness and possibly lower the number of local sustainability indicators used,

in order to make the communication of changes towards or away from sustainability

more transparent and manageable to all stakeholders that use the Local Authority

indicators.

iii

Table of Contents Page

Abstract ii

Table of Contents iii

List of Tables v

List of Figures vi

Abbreviations and Acronyms vii

Acknowledgements ix

CHAPTER ONE: INTRODUCTION 1

1.1 Introduction 1

1.2 Outline to Subsequent Chapters 2

CHAPTER TWO: SUSTAINABILITY AND THE ENGLISH PLANNING

SYSTEM 3

2.1 Sustainability and Sustainable Development 3

2.1.1 Sustainability or Sustainable Development? 3

2.1.2 Weak and Strong Sustainability 5

2.1.3 Is Sustainability Achievable? 6

2.2 The Planning System in England 6

2.2.1 Planning in England 7

2.2.2 Sustainability Appraisal 7

2.2.3 Sustainability Indicators 10

2.3 How Many Sustainability Indicators Should Local Authorities Use? 13

2.4 Classification of Sustainability Indicators 13

2.5 Classification of Rural and Urban Sustainability Challenges 14

2.5.1 Definitions of Rural and Urban 14

2.5.2 Different Sustainability Challenges in Rural and Urban Areas 17

2.6 Objectives 19

iv

CHAPTER THREE: METHODOLOGY 21

3.1 Choosing Local Authorities 21

3.2 Definition of a Sustainability Indicator 22

3.3 Appropriateness of Indicators 22

3.3.1 Choosing Criteria 22

3.3.2 Method for Using Criteria-based Assessment 23

3.4 Subjectivity, Compatibility and Reliability 24 3.4.1 Subjectivity of the Author 25 3.4.2 Subjectivity of the Sources Chosen to Produce the Criteria Method 26

3.4.3 Compatibility Matrix of Criteria 35

3.4.4 Trial Run of the Method 37

3.4.5 Reference Set of Indicators 38

3.4.6 Reliability of Appropriateness Method Using the Reference

Set Indicator 39

3.4.7 The Use of Weightings 41

3.5 Highest Scoring Individual Indicators - Top Ten Ranking 42

3.6 Number of Sustainability Indicators Used by Each Local Authority 42

3.7 Classifying Sustainability Indicators 42

3.8 Survey 42

3.9 Summary 43

CHAPTER FOUR: RESULTS AND ANALYSIS 44

4.1 Criteria-based Appropriateness of Sustainability Indicators 44

4.1.1 Individual Indicators 44

4.1.2 Top Ten ranking 45

4.1.3 Criteria 46

4.1.4 Super Criteria 51

4.1.5 Conclusion for the Appropriateness Method 51

4.2 Methods Used to Augment the Appropriateness Method 52

4.2.1 Defining Sustainability Indicators 52

4.2.2 Number of Indicators 54

v

4.2.3 Classification of Indicators 55

4.2.4 Survey Information 56

4.2.5 Conclusion From the Methods Used to Augment the

Appropriateness Method 56

4.3 Summary 57

CHAPTER FIVE: CONCLUSION, EVALUATION AND

RECOMMENDATIONS 58

5.1 Conclusion and Evaluation 58

5.2 Recommendations for Local Authorities 59

5.3 Recommendations for Research 60

REFERENCES 61

APPENDICES 85

Appendix 1 Scoring Appropriateness of Sustainability Indicators 86

Appendix 2 Local Authorities Used in this Study 90

Appendix 3 Appropriateness Scores, Numbers of Indicators and

Classification of Sustainability Indicators 91

Appendix 4 Justification of Classification of Indicators for all Local

Authorities 95

vi

List of Tables Page

Table 2.1 Areas for Consideration in the Development of Sustainable

Indicators 12

Table 3.1 Criteria and Super Criteria Used to Assess Appropriateness of SIs 23

Table 3.2 The Criterion Scoring Method for Reliability 24

Table 3.3 Compatibility Matrix of the Thirteen Criteria and Three Super

Criteria 36

Table 3.4 LAs Used in Trial Run of Criteria-based Appropriate Method 37

Table 3.5 Trial Run Scoring of Appropriateness: Some Results from Dacorum 38

Table 3.6 The Reference Indicator Set Total Number of Indicators in

Each Classification 39

Table 3.7 Differences in Total and Percentage Scoring for Dacorum LA 40

vii

List of Figures Page

Figure 1.1 Back Calculating Through the Cause-Effect Chain of Climate

Change 1

Figure 2.1 Sustainability Appraisal in LDFs 9

Figure 2.2 Rural Designations Where English Residents Live (2003) 16 Figure 2.3 The Heterogeneous Nature of MU and R80 Classifications 16

Figure 2.4 Estimates of End User CO2 Emissions for 2005 in England,

Using Defras Area Classification 18

Figure 3.1 Decision Tree Method for Choosing Regions and Local Authorities 21

Figure 3.2 Lead Authors and Organisations Used to Establish Criteria

to Assess Appropriateness 27

Figure 4.1 Stakeholders Involved in Choosing SIs in 2008 50

Figure 4.2 Groups and Organisations Involved in Local Agenda 21 (1998) 51 Figure 4.3 Sustainability Indicator Definitions Used by Local Authority

Officers and Found in Scoping Reports 53

Figure 4.4 Comparison of the Classification of Copeland Local

Authorities with the Average of the 38 Local Authorities 55

viii

ABBREVIATIONS AND ACRONYMS

ACCA Association of Chartered Certified Accountants A Levels Advanced Levels AONB Area of Outstanding Natural Beauty BBC British Broadcasting Company BREEAM Building Research Establishment Environmental Assessment Method BSA British Sociological Association BTO British Trust for Ornithology CAT Centre for Alternative Technology CIAT Chartered Institute of Architectural Technologists CRC Commission for Rural Communities CSD Commission on Sustainable Development DCLG Department for Communities and Local Government Defra Department for Environment Food and Rural Affairs Ec Economic Sustainability Indicator EE East of England EISs Environmental Impact Statements EMA Environmental Management and Auditing EMS Environmental Management System En Environmental Sustainability Indicator EU European Union FOE Friends of the Earth GCSE General Certificate of Secondary Education GDP Gross Domestic Product GHG Green House Gas HNC Higher National Certificate HND Higher National Diploma ICAEW Institute of Chartered Accountants in England and Wales ICT Information and Communications Technology IEMA Institute of Environmental Management and Assessment IMD Index of Multiple Deprivation IOW Isle of Wight LA Local Authority LADs Local Authority Districts

ix

LDF Local Development Framework LGBT Lesbian Gay Bisexual Transgender MU Major Urban NGOs Non Governmental Organisations NW North West ODPM Office of the Deputy Prime Minister ONS Office for National Statistics OS Ordnance Survey PCT Primary Care Trusts PPM Programme and Project Management PPS Planning Policy Statement R80 Rural 80 RCEP Royal Commission on Environmental Pollution RSC Royal Society of Chemistry RSPB Royal Society for the Protection of Birds RTPI Royal Town Planning Institute SA Sustainability Appraisal SCS Sustainable Communities Strategy SE South East SEA Strategic Environmental Assessment SD Sustainable Development SI Sustainability Indicator SIs Sustainability Indicators SMEs Small and Medium Size Enterprises So Social Sustainability Indicator SS Strong Sustainability UK United Kingdom UKCIP United Kingdom Climate Impacts Programme UNECE United Nations Economic Commission for Europe WCED World Commission on Environment and Development WS Weak Sustainability WWF World Wildlife Fund

x

ACKNOWLEDGEMENTS

My thanks to my wonderful husband, Colin, who travelled to Norwich from Somerset

every weekend for a year and had to endure some 152 outings and visit around 365

pubs (and one wine bar).

To Alan Bond, my supervisor, for divulging insider knowledge to demonstrate my

method could be valid and also for patiently showing me how to improve. Finally, my

thanks to Bill Sturges for advising me on how to make this look good.

1

CHAPTER ONE: INTRODUCTION

1.1 Introduction

The limited resources left in the world for the 6,720,701,504 humans (US Census Bureau, 2008) currently using the planets resources, and the opportunities for future generations to be able to use similar resources, are some of the significant issues facing

the world today. It is not just the resources that we use, but also the direct and indirect effects that are caused by the use of these resources that need to be considered. For

example, anthropogenic warming of the Earth caused by greenhouse gases (GHG) (IPCC, 2007:3), such as the carbon dioxide produced from the combustion of fossil fuels, is not a simple cause and effect situation (see Figure 1.1) and has long-term wide-reaching consequences that do not just include using up all the oil, gas and coal.

This dissertation aims to assess one method currently used by local authorities in

England, sustainability appraisal, which claims to lead towards stronger

sustainability.

Source: Gupta et al. (2006).

Figure 1.1: Back Calculating Through the Cause-Effect Chain of Climate Change

Different people and organisations have widely differing views on how to manage

these resources. The north and south of England have polarised views and it is really

the north of England that currently has this movement towards sustainable

2

development. The south of England seems just concerned with development. This dissertation concentrates on the views from the north of England, with a particular

focus on sustainability within the English local authorities.

1.2 Outline to Subsequent Chapters

This first chapter provides an introduction to the motivation behind this research.

Chapter Two frames the case for developing an appropriate method for sustainability

indicators. The third chapter then outlines the methods used to determine the

appropriateness of sustainability indicators, the results of which are discussed in

Chapter Four. Conclusions then follow in Chapter Five, outlining how well the overall

aims and objectives were addressed. A set of appendices provides additional specific details of the methodology not considered appropriate to the main text.

3

CHAPTER TWO: SUSTAINABILITY AND THE ENGLISH PLANNING SYSTEM

2.1 Sustainability and Sustainable Development

Firstly, we need to consider the appropriate definition of sustainability and sustainable

development, along with considering weak or strong sustainability and whether this is

really achievable.

2.1.1 Sustainability or Sustainable Development?

The wide variety of definitions for sustainability and sustainable development in

academia, English government and organisations can lead to drawbacks in comparing

research and initiatives within this field. A divergence of views (Hopwood et al., 2005; Counsell et al., 2006) and the availability of various information sources (Paris et al., 2003; Glavic et al., 2007) has led to a confusion of definitions (Glavic et al., 2007).

Academically, the terms sustainability and development are clearly defined to be

conceptually distinct, whereas activist, political and legal definitions have different

meanings (McNeill, 2000). Academics and NGOs are more prone to use the term sustainability in similar contexts, while government and private sector organisations

have tended to adopt the term sustainable development (Robinson, 2004). Tools are needed to measure sustainability (Fraser et al., 2005), especially new tools that transcend current conflicting views (Robinson, 2004). However, not one size fits all and meaningful comparisons need to occur (Phillips and Bridges, 2005; Roberts, 2006), therefore definitions become important. This much quoted definition is from Our

Common Future:

Humanity has the ability to make development sustainable, to ensure that it meets the needs of the present without compromising the ability of future generations to meet their own needs (WCED, 1987:8).

4

This definition is often the starting point where researchers begin to define

sustainability (Holland, 1997; Jackson, 2007; Morse, 2008). However, this definition reflects a managerial approach and is therefore more attractive to government and

business than a more radical definition (Robinson, 2004), although intragenerational and intergenerational equity does appear at the cornerstone of this WCED definition.

The terms sustainable development and sustainability have often been used

interchangeably and a wealth of definitions are available (Defra, 2005a; Dartford Borough Council, 2006; Moles et al., 2007; Zidansek, 2007; Wigan Council, 2007;

Morse, 2008), including the definition by Dahl (2007) that sustainability is the capacity of any system or process to maintain itself indefinitely. This definition

alludes to intragenerational and intergenerational equity, but does not refer to the

quality of the level maintained (i.e. using a quantified baseline). The following definition is from the Draft UK Sustainable Communities Bill 2007: By local sustainability we mean policies that work towards the long term well being of any

given area. That means promoting economic needs... (House of Commons, 2007). Unlike the WCED and Dahl, The House of Commons introduces both policy and

economics into the definition. The commonalities in all three definitions are the

temporal range, being long-term, and the omission of a stated baseline. Robinson

(2003) defines sustainability as being related to values and fundamental changes in individual attitudes towards nature (value changes) and sustainable development as being orientated towards efficiency gains and improvements in technology (technical fixes), with their ultimate goals being rather different. Of note is that a number of local, regional and national government and associated organisations, in England, have

chosen not to define sustainability or sustainable development (ODPM, 2005; Scott Wilson Business Consultancy, 2005; CRC, 2008; DCLG, 2008) within their sustainability documents.

Robinson (2003) concedes that it may be worth leaving the definition of sustainable development open, using the diplomats method of constructive ambiguity, and have

definitions emerge from attempts at implementing sustainable development. Moldan

and Dahl (2007) have the same opinion for the definition of sustainability. Because of

5

this lack of consensus in a definition, omitting a definition may be a viable alternative

to including a definition that is potentially unusable at the time.

2.1.2 Weak and Strong Sustainability

Academics further divide sustainability; this can then be applied to other organisations

and institutions. Researchers (Turner, 1993; Holland, 1997; Neumayer, 2003; Karlsson et al., 2007) suggest that sustainability can be measured in degrees of sustainability, termed weak and strong sustainability. Two (Holland, 1997; Karlsson et al., 2007), three (Neumayer, 2003) and four part (Turner, 1993) classifications have been suggested. These classifications are based upon what economists term natural capital,

which summarises the multiple and various services of nature benefiting human beings,

from natural resources to environmental amenities (Neumayer, 2007). Weak sustainability (WS) is built upon the unlimited substitutability of natural capital (Neumayer, 2003), whereas strong sustainability (SS) is more difficult to define (Holland, 1997).

Turners (1993:9-15) four part classification includes very weak sustainability, weak sustainability, strong sustainability and very strong sustainability, with the latter

suggested as being impossible to achieve by Turner. Neumayers (2003) three part classification also includes WS and suggests two interpretations of SS as available in

the literature. In one interpretation, SS is the paradigm that calls for preserving natural

capital itself in value terms. In the second interpretation, SS is not defined in value

terms but calls for the preservation of those forms of natural capital that are regarded as

non-substitutable (the so-called critical natural capital) (Neumayer, 2003:24-25).

Official UK conceptualisations of sustainability adopt a weak interpretation. The

attraction for politicians and policy makers is that it offers scope for claiming the

adoption of a given stance, not entailing the sacrifice of living standards (Jackson, 2007). However, SS requires more radical changes and therefore is more difficult to achieve and less attractive (Holland, 1997), requiring a sea of change in thinking (Glasson et al., 2005).

6

From the classifications available for sustainability, different actors choose definitions

from the weak or strong sustainability viewpoint. Ekins et al. (2003) suggest that the important point is that starting from a SS assumption of non-substitutability in general,

it is possible to shift to a WS position where that is shown to be appropriate. In this

dissertation, WS is taken to mean the same as SD, and SS will be defined as Neumayer

(2003) has concluded, unless stated otherwise; Elkins stance is used were applicable.

2.1.3 Is Sustainability Achievable?

It is suggested that a discussion of sustainability that only refers to definitions is

pointless without an understanding of how a definition is operationalised (Ozkaynak et al., 2004). Operationalisation includes three main elements: people, processes and outcomes (Oakland, 2000). Therefore, the three main factors that can determine the outcome (i.e. the degree of sustainability) are the definition applied, the people involved and the process used.

What shapes peoples pro-environmental behaviour is complex. Increased knowledge

and awareness, or greater affluence, in most cases does not lead to pro-environmental

behaviour (Kollmuss et al., 2002). For a process to emulate a sustainability definition, the pro-environmental stance (or not) of the person leading and managing the project is therefore important. Primarily, leaders and managers should have current pro-

environmental behaviour patterns rather than just a high level of environmental education (Kollmuss et al., 2002), as one of the variables contributing to achieving project outcomes. The processes that leaders develop should show that economic systems are underpinned by ecological systems and not vice versa (Holland, 1997). Sustainability appraisal (SA) is an example of how a definition can be operationalised, in this case within the English planning system.

2.2 The Planning System in England

The planning system in England, sustainability appraisal and indicators now need to be

considered in more detail.

7

2.2.1 Planning in England

A major culture shift in the English planning system has redefined the nature and purpose of planning from land use to spatial planning (Wong et al., 2006). England has been carrying out SA on development plans since 1992 and this SA has been

relatively effective at integrating environmental and sustainability considerations into

plans (Therivel et al., 2002). The 2004 Planning and Compulsory Purchase Act requires planning authorities in England and Wales to undertake SAs of Local

Development Frameworks, amongst other things, which are also intended to fulfil

Strategic Environmental Assessment (SEA) requirements (Jackson, 2007). Sustainable development is noted as being key to the reformed planning system (ODPM, 2005) with SA as an integral part of the planning system (Defra, 2005a).

2.2.2 Sustainability Appraisal

An academic definition of SA is that:

SA is committed to positive overall contributions to a more desirable and durable future, through the identification of Best Options (not just acceptable undertakings), and it is designed to achieve multiple reinforcing gains (rather than mere avoidance of problems and mitigation of adverse effects) (Gibson, 2006).

This definition states exactly what type of best option should be considered. It is not

linked to policy or economic considerations. The UK Revised PPS12 (2008) defines SA as an appraisal of the economic, social and environmental sustainability of the plan

(DCLG, 2008). Within this policy, SA is linked to the Sustainable Communities Strategy (SCS), where the current emphasis on sustainable development now lies. However, the Local Authorities (LAs) and consultancies used in this dissertation have used the older version of PPS12, and the definition is slightly different (as the current SCS was not in place then). Nevertheless, all government definitions are based on WS principles, due to the inclusion of economic values and omitting to define what best

option really means (ODPM, 2005; DCLG, 2006).

8

Local authorities and the consultants that they employ define SA in two ways. Firstly,

those who mention SA contributing towards the achievement of emulating SD (DCLG, 2006), such as Copeland Borough Council (2005), Elmbridge Borough Council (2005) and West Oxfordshire District Council (2008) (Gardner et al., 2006). Secondly, those who additionally incorporate an intergenerational view into their definition, such as

Liverpool City Council (2005) and Breckland Council (2008) (Costaras et al., 2006).

None of the SA definitions in this section introduce the non-substitutionability of

natural capital; therefore all err towards degrees of WS. The operationalisation of the

LA definitions for SA all function within the same process framework, set out in Figure

2.1. This process operates in parallel with the DPD process.

For the operationalisation of SA in LAs, the similarities are the definitions of SA, the

process which LAs use for SA, and the statutory stakeholders from whom they must

invite comment. Differences occur in the subject specialisms of the LA officers and their consultants, and the LAs choice of non-statutory stakeholders consulted for SA.

Gibson (2006) states that one should establish the SA contribution as the main test of proposed purpose, option, design and practice. The processes must put application of

these sustainability-based criteria at the centre of decision-making, not as one advisory

contribution among many (Gibson, 2006). In UK policy, sustainable development is only promoted by using SA. LAs are not required to justify national planning policy when conducting SAs (for example, by appraising alternatives to national policy), even if the non-policy alternative turns out to be the best option (ODPM, 2005); therefore, Gibsons stance is not followed. Recent research within English regions has obtained

the view that SA is a weak science with subjective outcomes, where the big issues are sidestepped (Counsell et al., 2006).

9

Source: abridged from ODPM (2005:38).

Figure 2.1: Sustainability Appraisal in LDFs

Preparing options

Choosing preferred option

Submission of LDF

Developing options

Preparing the SA report

Consulting on LDF and SA

reports

Examination of the LDF report

Evidence gathering

Monitoring the significant effects

Adoption and monitoring

Scoping/ baseline

1

2

3

Pre production

0

Production

Examination

Adoption

Tim

e (Y

ears

)

LDF process

SA process

10

Overall, SA is defined weakly in terms of sustainability by the UK government,

English LAs and associated consultancies. The operationalisation of these definitions

produces a weak degree of sustainability, as stronger sustainability (based on at least multiple reinforcing gains and best option based on sustainability alone) is not the central focus of a decision to choose a plan for an LDF.

2.2.3 Sustainability Indicators

The process of using SA to establish the best option for English LDFs employs the

objectives, targets and sustainability indicators approach (ODPM, 2005). Sustainability indicators (SIs) are derived from the objectives chosen, as one approach to gauging progress towards SD is to use SIs (Bell et al., 2001). England is now on its third generation of SIs, developed since the first UK SD summit, which was instigated after

the 1992 Rio Earth Summit (Hall, 2007). In addition to SIs having numerous definitions, there are alternative methods for choosing SIs, and also various techniques

to determine if SIs are appropriate.

Definitions vary depending on the actors who define SIs. Most academic and

government definitions contain an element of measurement (Astleithner et al., 2004; ODPM, 2005) and a change over either time, space or both (Smith et al., 2001; Astleithner et al., 2004; ODPM, 2005; Gasparatos et al., 2007). Cartwright et al.s (2000) survey of LAs showed that the majority of respondents (51%) indicated that the primary role of SIs was to help monitor progress towards SD, with raising awareness

and educating people acknowledged as key issues. Writing about the UK SIs, Hall

(2007) suggests that the principal role of indicators is communication, particularly to the public and to ministers who do not need a lot of detail. Some researchers consider

that SIs can be framed in terms of degrees of SS (Holland, 1997; Bastianoni et al., 2005) and WS (Holland, 1997), while Wackernagel et al. (2005) state that the ecological footprint indicator tracks core requirements from SS and identifies priority

areas for WS. However, other researchers do not agree with this (Ayres, 2000) and consider ecological footprints, due to methodological flaws, not to have any value for

policy evaluation or planning purposes (Neumayer, 2003:197).

11

The importance of community stakeholder involvement in the development of SIs has

been specifically identified by some researchers (Bond et al., 1998; Cartwright, 2000; Bell et al., 2001; Astleithner et al., 2004). However, the drawbacks are also represented in the literature (Astleithner et al., 2004; Fraser et al., 2005; Doak et al., 2005; Reed et al., 2005) and Morse (2008) states that participation in general has received remarkably little attention within SD literature compared to development literature.

Two methods for choosing SIs have been proposed by Reed (2005). The reductionist framework (which is expert led) and the bottom-up participatory philosophy, focussing on the importance of understanding local context (also known as the conversation paradigm) (Bell et al., 2001). According to Gasparatos et al. (2007), SA so far has relied on reductionist methodologies and tools. However, Bond et al.s (1998) survey responses to UK LA stakeholders involvement in Agenda 21 showed that it was clear

that there had been community involvement, but the extent of the involvement was

unclear. A participatory integrated method has been utilised (Gupta et al., 2006), using experts and a variety of stakeholders, not only to get stakeholders to identify indicators

but also to identify thresholds of acceptable and unacceptable risks (for dangerous climate change). A key tool in communicating difficult concepts to stakeholders was by using back calculation of cause and effect (related to climate change) (see Figure 1.1). The researchers considered the production of a simple usable visual communication

method of their results as an essential part of this process, even though this was

considered inappropriate by experts (Gupta et al., 2006). This method agrees with other researchers that communities need to be thinking through and deciding the kind of

future that they want to create (Robinson, 2004).

Different criteria are used for deciding the appropriateness of SIs, an example being

Donnelly et al. (2007) who propose criteria for the selection of four environmental indicator types (biodiversity, water, climate and air) used in SEA, which is now part of SA in England. Donnelly et al. (2007) consider it important to set criteria before a final list of indicators is agreed upon, to ensure the most pertinent environmental issues for

12

SEA are properly addressed, yet other researchers use criteria both pre and post

decision-making (Lin et al., 2007) to decide appropriateness.

There is no single method that is easily repeatable from the point of view of LAs. From

the literature, there is no clear agreement on a set of criteria or measurement of

appropriateness of local SIs, but there are areas that many researchers agree need

further consideration. Table 2.1 shows some areas for further consideration in the

development of SIs. However, there is seldom a perfect SI, so the design generally

involves some methodological tradeoffs between technical feasibility, societal usability

and systemic consistency (Moldan et al., 2007).

Disaggregation of data a, h, i, j

Small scales needed g

Averages shade issues l

Innovation needed b, c, d

Current data is not being acted on k

Relationships are not straight forward (linking cause and effect) f Indirect effects need consideration (climate, health and economy) e

Sources: (a) Coombes et al. (2004); (b) Robinson (2004); (c) Beveridge et al. (2005); (d) Defra (2005a); (e) Bosello et al. (2006); (f) Doran et al. (2006); (g) Weich et al. (2006); (h) Hajat et al. (2007); (i) Lin et al. (2007); (j) Warren (2007); (k) Hanratty et al. (2008); (l) Spilanis et al. (2008).

Table 2.1: Areas for Consideration in the Development of Sustainable Indicators

In conclusion, SIs have been defined by researchers who have suggested that some SIs

measure degrees of sustainability, although this is contested by other researchers.

Choosing SIs with their thresholds of acceptability in a participatory integrated way

may be a clearer way to communicate with stakeholders concerning the degree of

sustainability available from the best plan chosen for the LDF, via the process of SA.

The gaps in knowledge to decide on the appropriateness of SIs need to be considered in

future criteria-based appropriateness assessments.

13

2.3 How Many Sustainability Indicators Should Local Authorities Use?

There are numerous sets of SIs, each with widely varying numbers (Daniels, 2007), with sets of over 100 indicators being common (Rydin et al., 2003). Two examples of different researchers using a lower number of SIs are Fraser et al. (2005) who used 55 SIs in their Guernsey study and Spilanis et al. (2008) who used a total of 37 SIs in their Greek Islands study. Hall (2007) identified a total of 5000 SIs when developing the first generation of UK SIs and, from these, ideally wanted to reduce them to a set of fifty.

This reduction in numbers was so the set might be more manageable and

understandable. Reed et al. (2006) indicate that stakeholder involvement can lead to a large number of potential indicators and, when establishing the first generation of UK

SIs, Hall came away from a days stakeholder involvement with a potential set of 400

SIs (Hall, 2007) rather than the 50 that was planned. Using their integrated method of choosing indicators, Gupta et al. (2006) developed just 27 indicators of climate change after using their considered method of both expert and stakeholder involvement.

Bossels (2001) systems-based approach for deriving comprehensive indicator sets requires exactly 42 indicators. It turns the focus from an uncertain ad hoc search and

bargaining process to a much more systematic procedure with clear goals to find

indicators that represent all the important aspects of viability, sustainability and

performance (Reed et al., 2005). Currently, the UK's third generation set of SIs numbers 68 (Defra, 2007). No exact numbers of SIs are suggested for LA SAs; the only guidance is that the number of SIs needs to be manageable and developed with input

from relevant stakeholders (ODPM, 2005). However, LAs need to consider that decision-makers and the public rapidly lose interest if presented with more than just a few indicators (Moldan et al., 2007). Not only the number but also the types of SIs chosen by LAs will be considered next.

2.4 Classification of Sustainability Indicators

Sustainability indicators may be easier to understand and interpret when assembled in

some conceptual framework, perhaps with a hierarchical arrangement of sub-domains.

The three pillars (economic, environmental and social) are one such framework, used in Agenda 21 and, in a survey by Cartwright et al. (2000), 81% of LA respondents chose

14

this option - but many others are possible (Moldan and Dahl, 2007). A fourth pillar (institutional indicators) was included in the system of sustainability indicators adopted by the United Nations Commission on Sustainable Development (CSD) (Moldan et al., 2007; Zidansek, 2007) and institutional indicators are currently used by some researchers (Sheate et al., 2008). However, the institutional dimension is often subsumed into the social dimension (Spangenberg, 2007).

Most indicator sets have assembled indicators for each of the three pillars whilst

neglecting the links between them. Interlinkage indicators are also called decoupling

indicators, and a number of the UK government strategy indicators take the form of

decoupling indicators. Decoupling is defined as how successful we are in breaking the

link between economic growth and environmental damage (Defra, 2005a). From the classification systems available, the three pillars idea has been used by academics from

many disciplines in their research (Holland, 1997; Ekins et al., 2003; Rydin et al., 2003; Astleithner et al., 2004; Lehtonen, 2004; Counsell et al., 2006; Glavic et al.,

2007; Huby et al., 2007; Niemeijer et al., 2008; Sutherland et al., 2008).

The three pillars classification system will be used in this research, as the widespread

use of this approach and the advantage of ease of communication are considered to

outweigh the disadvantages of lack of linkage and decoupling, and subsumation of the

institutional dimension. This research will be compared to a previously used method by

Bond et al. (1998), who present a fully referenced classification of the three pillars system which will be adapted to classify LA SIs in 2008.

2.5 Classification of Rural and Urban Sustainability Challenges

2.5.1 Definitions of Rural and Urban

Definitions of rural and urban vary both within and between countries, and within

organisations worldwide a number of classification systems exist to divide rural and

urban (OECD, 1994; Reschovsky et al., 2002; Chomitz et al., 2005; Buckwell, 2006; Fotso, 2006; Gallego, 2006; Weich et al., 2006; Huby et al., 2007; Vickers et al., 2007;

Zonneveld et al., 2007), as different users have different needs (Champion et al., 2006).

15

However, there are unclear and contradictory usages of the term rural within England

(Haynes et al., 2000; Baird et al., 2006; Keirstead et al., 2007; Manthorpe et al., 2008). Sometimes a definition is not obvious in an academic paper (Ulubas-og lu et al., 2007) and some academics consider that in cultural, social and economic terms the notion of

rurality in a country such as the United Kingdom is outdated (Champion and Shepherd, 2006). In 2004, the Office for National Statistics (ONS) published a new definition of rural areas covering England and Wales, launched alongside Defras Rural

Strategy (Champion and Shepherd, 2006). Defra considers the new definition of rural areas to offer a distinctly different, potentially more useful and more transparent

approach to identifying LA districts (LADs), making it possible for policy makers, researchers and others to interpret their results against a known set of benchmarks

within the classification (Defra, 2005).

Englands 354 unitary authorities and LADs have been allocated to one of six main

types. Three (176 LADs in all) are overwhelmingly urban in nature and are called Major Urban, Large Urban and Other Urban (large market town). The rural types, of which there are 178, are called Significant Rural (rural town), Rural 50 and Rural 80, according to the proportion of people in rural settlements. Thus, Rural 80

LADs have between 80 and 100% of the people in rural settlements and Rural 50

LADs have more than 50%, while Significant Rural have more than the national

average of 26% (Champion and Shepherd, 2006). The percentage of residents in each of the six types (2003 data) is illustrated in Figure 2.2.

From this, it can be established that overall 63.5% of Englands residents live in urban

areas, while 36.5% live in a rural area (2003 statistics). Each LA is classified into one of the six designations. However, each LA can be further sub-classified (using the six designations) and, as can be seen in Figure 2.3, both Rural 80 (R80) and Major Urban (MU) are not homogeneous classifications. R80 can contain urban classifications and MU can contain rural classifications. However, this classification system is more

transparent than others that have been used previously in England and research findings

can be discussed with this classification in mind.

16

Source: derived from Champion et al. (2006).

Figure 2.2: Rural Designations Where English Residents Live (2003)

Source: Defra (2004).

Figure 2.3: The Heterogeneous Nature of MU and R80 Classifications

17

As there is some transparency in this classification system (Defra, 2005) and the classification system can be applied to all LAs in England, and also as a large amount

of administrative and other data is available only at this level (Champion et al., 2006), this classification system was used in this dissertation.

2.5.2 Different Sustainability Challenges in Rural and Urban Areas

A number of organisations now use Defras classification of rural and urban areas in

their more recently produced documents and research (CRC, 2008; RCEP, 2008). In England, researchers and organisations have suggested that different priorities are

needed for urban and rural areas in order to lead towards sustainability (Doran et al., 2006; Champion et al., 2006; CRC, 2008; RCEP, 2008).

Carbon dioxide (CO2) emissions will be considered for both urban and rural areas, bearing in mind that the UK has the goal of a 60% cut in CO2 emissions by 2050

(RCEP, 2008). The greatest new driver of public policy for rural communities, as for the nation as a whole, is climate change. Climate change poses particular challenges for

rural communities, both in terms of the sustainability of peoples car-reliant lifestyles

and in the way landscapes and biodiversity will adapt (CRC, 2008).

The density and infrastructure of urban areas helps make them more efficient in terms

of per capita energy consumption and emissions, which are lower in many of the UKs

major cities than the national average (RCEP, 2008). This point is agreed with by CRC (2008), as shown in Figure 2.4. CRC states that this is because people living in rural areas carry out much more of their travelling by car. In all three rural classifications,

transport has a significantly larger (approximately twice the value) CO2 emission value than all three urban areas.

18

Source: adapted from CRC, 2008:144).

Figure 2.4: Estimates of End User CO2 Emissions for 2005 in England, Using Defras Area Classification

Reducing the CO2 concentration is a local as well as global sustainability issue. This

may need to be achieved by different methods for rural and urban areas, so therefore

the significant differences need to be studied. In this example, car transport is a

significant aspect in rural areas. Indicators to monitor car usage in rural areas are

needed to increase sustainability. As previously stated (Defra 2005a), indicators do not always stand alone - they can link together. Currently, there are several sustainability

issues linked to the lack of public transport in rural areas, two of which are illustrated

below:

1. Increasing rurality is associated with the greater pace of growth in the number

of the oldest people (Buckwell, 2006; Champion et al., 2006), and the need to travel disproportionately affects these people in rural areas, particularly for

those without their own cars (Baird and Wright, 2006). 2. With regards to adult literacy and numeracy, lack of transport, access and

childcare are major barriers to learning in rural areas (Atkin et al., 2005).

19

RCEP (2008) recommends that before development plans are approved, the government should publish a clear assessment of the transport infrastructure needs for

all proposed housing growth, how they will be funded and the environmental and

health impacts of meeting those needs. This should be accompanied by a clear plan for

phasing in the necessary supporting infrastructure, ensuring that this new transport

provision is environmentally sustainable.

Overall, the 35.5% of English people who live in rural areas produced twice as much

carbon dioxide per person from personal transport use in 2005, than the 66.5% of

people who live in urban areas. This one piece of evidence indicates that there are real

differences between rural and urban sustainability. Rural and urban areas can face the

same issues (such as climate change) but have different scales of a problem, so they may need different solutions to achieve the same level of sustainability. Conducting SA

to choose the best option and the most appropriate choice of sustainability indicators

when choosing an LDF may help address this rural/urban difference to achieve a higher

level of sustainability within all LDFs.

The remainder of this dissertation examines how appropriate the choices of SIs are

when choosing the best option LDF.

2.6 Objectives

From the classifications available for sustainability, different actors will choose

different definitions from the WS or SS viewpoint. Recent research within English

Regions has shown that SA is a weak science with subjective outcomes, where the big issues are sidestepped (Counsell et al., 2006); although some authors do have the opinion that SIs can measure degrees of sustainability and therefore are a useful tool. A

guidance exists for SA (ODPM, 2005); however, there is a gap in this guidance on how to reliably assess the appropriateness of SIs. In order to fill this gap, the overall aim of

this project is to develop and apply a reliable criterion-based assessment that LAs can use to assess the appropriateness of SIs used when selecting the best option for an

LDF. This project aim will be addressed through three specific objectives:

20

1. To critically appraise sustainability literature and guidelines to enable the

development of a reliable criterion-based assessment of appropriateness for SI

used in LA SA of LDFs.

2. To apply the criterion-based assessment of appropriateness to LA SIs from three

regions, to enable an evaluation of similarities and differences of

appropriateness between rural and urban LAs, both within and between those

regions.

3. To examine other methods which augment the appropriateness method, by

applying a classification framework to indicators, investigating numbers of

indicators, and surveying LA officers and consultants about the choosing of

indicators.

21

CHAPTER THREE: METHODOLOGY

3.1 Choosing Local Authorities

To research the similarities and differences in appropriateness of SIs in rural and urban

areas of England, the two extremes of rurality, Major Urban (MU) and Rural 80 (R80) were used. There are nine relevant regions in England and, for this study, three regions

were chosen using the method shown in Figure 3.1.

Figure 3.1: Decision Tree Method for Choosing Regions and Local Authorities

22

3.2 Definition of a Sustainability Indicator

To locate the sustainability indicator definition used by individual LAs, the relevant LA

document containing the indicators was first consulted. If the indicators were found in a

separate appendix, then the scoping report was consulted. Using this method, it was

seen that sustainability and sustainable development were considered to have the

same meaning, with many government documents using these phrases interchangeably.

How LA officers and consultants define SIs is considered in the survey.

3.3 Appropriateness of Indicators

3.3.1 Choosing Criteria

The author followed the suggestion by Goodwin (2006) regarding the four key criteria involved in the use of documentary sources of qualitative data. Therefore, the

information sources used for this criterion-based method were checked by the author

for authenticity, were credibly recorded, representative of the literature search carried

out, and finally the author decided whether the source could be used in the literal sense

or not. A choice was made to use authors such as Lin et al. (2007), whose research contains evaluative criteria on gender equality and health for suites of indicators, but

does not explicitly name indicators in their research as SIs, but whose work would

nevertheless contribute towards the criteria-based method in a credible manner - in this

case, for disaggregation data. Authors such as Niemeijer and de Groot (2008) were also included as they met the definition that their environmental indicators became SIs with

the addition of time, limit or target (Rickard et al., 2007:75).

From a literature search of 121 sources, 27 were chosen and used, with sources being

equally weighted to show transparency in method (Moles et al., 2007).

The results from the above process were then used to construct the criteria. Quotes from each source were divided into groups according to their subject matter and thirteen criteria groups formed. A short title was given to each criterion to reflect the

subject matter of the material it contained. Each criterion was divided into two or three main areas, reflecting the complexity of the data in that group and a hierarchical

23

scoring system created, attaching a score of 0-3 for each level of hierarchy to reflect the

level of complexity in achieving that score. Finally, commonalities were established

between the thirteen criteria and three super criteria groups were formed. The full

criteria and scoring system is contained in Appendix 1. The criteria and super criteria

chosen are shown in Table 3.1.

Table 3.1: Criteria and Super Criteria Used to Assess Appropriateness of SIs

3.3.2 Method for Using Criteria-based Assessment

The following method was used for all thirteen criteria. All data obtained was easily

found within the main body of the LA scoping report or appendices. Each criterion was

scored as zero, one, two, or three. An example of how the criterion of reliability was

scored is presented in Table 3.2.

Each criterion has the possibility of a zero level score; this was scored where no

information pertaining to this criterion could be found.

To score one, evidence to support at least one of the statements, or partial

statements, was accessed. A score of one had to be achieved before proceeding

to decide if a score of two was possible.

Super Criteria Credibility Measurement Local Authority

Leads to strong sustainability Locality Relevant to plan

Academic credibility Disaggregation of data Actionable

Addressing uncertainty Measurability

Stakeholder involvement

Environmental receptors addressed Reliability Funding/cost

Criteria

Easily communicated

24

To score two, evidence to support at least one of the statements, or partial

statements to score two must be accessed. The scores of one and two had to be

achieved before deciding if a score of three was achievable.

To score three, evidence to support at least one of the statements, or partial

statements to score three had to be found.

Criterion Score Evidence Base for Score

0 No reliability

1

Repeatable and able to assess trends over space (Niemeijer and de Groot, 2008)and time (Reed et al., 2006), with the right spatial and temporal scales (Sustainable Seattle, 2005; Niemeijer and de Groot, 2008).

2 Sensitive and respond in a predictable manner to changes and stresses (Reed et al., 2006; Niemeijer and de Groot, 2008).

Reliability

3 Repeatable and reproducible in different contexts, that allows unambiguous interpretation (Niemeijer and de Groot, 2008).

Table 3.2: The Criterion Scoring Method for Reliability

3.4 Subjectivity, Compatibility and Reliability

To address different areas of subjectivity, compatibility and reliability in this method, seven areas were evaluated:

1. Subjectivity of the author. 2. Subjectivity of the sources used to produce this method. 3. Compatibility of the chosen criteria.

4. Trial run of the method.

5. Reference set of indicators.

25

6. Reliability of the criteria-based method.

7. Use of weightings.

These areas will now be considered in more detail.

3.4.1 Subjectivity of the Author

This criterion-based assessment of sustainability indicators was created and used by

one person. Three main author-based areas of subjectivity were considered:

1. Skills subjectivity of the author 2. Knowledge subjectivity of the author 3. Single researcher versus team approach

The authors skill base and knowledge base were assessed as to how subjective they would be towards scoring the thirteen criteria. Consideration was given as to how to

scale the authors subjectivity. Two options of matrix-rating scales were considered; Likert and Semantic differential scales (SurveyMonkey, 2008a). However, neither of these matrix-rating scales was considered suitable. Finding the correct wording for

subjectivity for five points on the Likert scale, or the semantic scale for either a balanced or unbalanced scale, did not work with ease when trialled. Whatever the

author self scored was unlikely to be objective, so the opposing ends, subjective and objective, of the semantic scale were not suitable. Therefore, a two point system of weakly subjective and highly subjective was used. A mid-point was avoided, as the author was marking herself and needed to be forced away from a neutral response.

Strongly subjective indicated that either the authors skills or knowledge needed upgrading to be equal to the same level as weakly subjective, which was nearer to objective, but not considered objective.

In relation to the chosen classification of indicators into the three pillars of

sustainability (economic, environmental and social), the following judgement of the authors knowledge and skills appertaining to these three areas was made. The authors

knowledge and skills of these three pillars was weakly subjective for environmental

26

indicators, weakly subjective for social indicators and strongly subjective for economic indicators.

The environmental impact statement (EIS) review package for assessing EISs uses a criteria-based method involving 92 criteria in eight sections, compared to the authors

13 criteria in three sections (super criteria). Subjectivity in the EIS review package is reduced, as the EIS is assessed by two independent reviewers on the basis of a double

blind approach. Here each reviewer assesses the EIS against the criteria and the

reviewers then compare results and agree grades (Glasson et al., 2005:395-407). It was not possible to use this approach in this study, although such an approach would have

been most suitable for this method.

The overall criteria that needed to be examined more closely in the analysis of the data

were addressing uncertainty, locality and funding/cost (for which the author was strongly subjective in both skills and knowledge) and scores for economic indicators also needed to be scrutinised.

3.4.2 Subjectivity of the Sources Chosen to Produce the Criteria Method

Once the criteria method had been created, the provenance of the data used was

partially analysed to assess whether the subject of the lead author or government document showed any preference to one subject specialism. The appropriateness criteria were created from 27 sources (see Appendix 1).

Lead authors from an environmental or scientific subject area (48%) dominated the sources used (see Figure 3.2). Economists formed the lowest percentage used, comprising just 8%. This should be considered when analysing appropriateness; however, most of the sources used had more than one lead author and their associated

subject specialisms are not considered further here.

27

Figure 3.2: Lead Authors and Organisations Used to Establish Criteria to Assess Appropriateness

Justification for choosing criteria and their hierarchical scoring

The thirteen criteria were then examined individually, as below.

(1) Leads to stronger sustainability: The definition of SS used is that of Neumayer (2003). The hierarchical scale created loosely represents the range between Neumayers two researched interpretations of SS.

A score of one was given when natural capital was preserved in value terms, ranging to

a score of three when the preservation of natural capital was regarded as non-

substitutable (with no value attached). Examples of the applied science needed to measure this criterion were also assessed in the scoring. Professional judgement was used by the author when a different applied measurement was stated in the indicator

that was absent in this criterions wording.

28

(2) Academic credibility: This criterion assesses the credibility or believability of the data used (Bryman, 2004:30) and how an academic audience taken from all disciplines involved with SIs would assess this. A hierarchy has been created starting with the believability of

accuracy and non-bias measurement scoring one, with wider parameters of the

individual measurement scoring two, then substantiation with suitable comparison to,

for example, national measurements, scoring three. Substantiation in research terms

refers to measurement validity (Bryman, 2004:72). It should be noted that this substantiation comparison does not apply to the measurement of rankings, such as the

Index of Multiple Deprivation (IMD) because, as stated in the DCLG (2007) guidance to using IMD:

The IMD 2007 is a relative measure of deprivation and therefore it cannot be used to determine how much more deprived one area is than another (DCLG, 2007).

Therivel and Ross (2007) concur with this statement, therefore it was concluded that rankings cannot be used in the substantiation of a measurement.

Low scores in academic credibility can be obtained by popular SIs. An example of a

popular indicator used by many LAs is population of wild birds (as, for example, used by Three Rivers) (Wooderson et al., 2006). Bird presence is often used as a proxy indicator for biodiversity (Moldan et al., 2007:1). Work by Tratalos et al. (2007) indicates a definite relationship between the abundance of bird species and household

density. The high density of housing suggested in current guidelines will result in a

lower overall avian abundance (Biggs et al., 2007). So temporal comparisons of bird populations, for example as found in LA annual monitoring reports, can only be made

if the housing density of an LA exhibits no change. More importantly, species richness

is a very insensitive indicator of biodiversity loss, and species richness does not

distinguish between native and introduced species (Biggs et al., 2007:254). Gregory et al. (2003), from the RSPB and the BTO, suggest using UK birds as indicators of biodiversity when disaggregated by habitat; however, the ecosystem approach to

biodiversity indicators is more widely espoused (Munda, 2006; Biggs et al., 2007;

29

RCEP, 2008; Tasser et al., 2008). The challenge is for LAs to find a complementary set of biodiversity indicators for their area, based on the type that they need for their policy

and not based on the type of abundant information available (Biggs et al., 2007), as in the case of populations of birds, which the author assesses as having no academic

credibility.

(3) Addressing uncertainty: Uncertainty in the context of this criterion is taken to mean unknown or open to

question. The intention is that this criterion is assessed in a positive way to show that an

LA is not playing safe and just measuring the indicators they feel comfortable with, but are taking into account change (e.g. the effects of climate change). Scoring three, when one is uncertain about the level, takes the measurement of uncertainty outside the

norm of system variability and can mean that this is not a tried and tested indicator in a

situation, or that events outside the current range of system variability can occur to give

the measure of uncertainty (e.g. the cost of oil suddenly rising). Variability occurs in all three pillars of sustainability: economic (Oakland, 2000; House of Commons, 2008), environmental (Defra, 2007; Tasser et al., 2008) and social (Lin et al., 2007). Therefore, this criterion has the possibility of scoring highly with all types of

indicators. Variability and uncertainty need to be transparent in this assessment, hence

the addition of this criterion.

(4) Environmental receptors addressed: The definition used for this criterion for named environmental receptors is taken from

both Annex 1 of the 2001 EU SEA directive (The European Parliament and the Council of the European Union, 2001) and ODPM 2005 SA guidelines (ODPM, 2005), as SEA is incorporated into SA in England. The hierarchy in this criterion is formed on the

basis that the interrelationship between receptors is of greater importance than just naming one receptor. The synergistic effect of interrelationships at higher ecological

levels, such as ecosystems, is not usually found within LA boundaries (ODPM, 2005; Reed et al., 2005). Tthis is a challenging area to be able to score three in, as LAs would need to consider the availability of extra resources associated with measuring the

30

transboundary effects, and also persuade policy makers of the importance of

transboundary effects on the LA.

(5) Locality: Locality is based on the definition of local being appropriate to the LA boundaries,

but not necessarily within the boundaries and can include any transboundary effects in

or out of the political boundaries. To assist with defining locality, the author used

physical maps of England, Ordnance Survey (OS) maps of some areas and the Excel table of definitions of the six rurality components (Defra, 2004) within the LA (as illustrated in Figure 2.3). However, there are some potential barriers such as the poor quality of databases, especially at a local level, which are a potential threat to the

quality of the related indicators (Moldan et al., 2007:10).

The overall marking scheme for appropriateness ranges from 03, but in reality the

score of three is not available for locality as insufficient hierarchical information was

obtained from the literature research. However, this criterion was kept as it was

considered important by three groups of researchers. A scale of zero to two was used,

yet in practice a score of two was infrequent due to the information not being easily

available in sufficient detail in the scoping report. A maximum score of two in this

criterion gives less internal reliability.

(6) Disaggregation of data: When undertaking the literature search for this dissertation, it became obvious that

many authors from various disciplines (Coombes et al., 2004; Doran et al., 2006; Biggs et al., 2007; Hajat et al., 2007; Huby et al., 2007; Lin et al., 2007; Warren, 2007; Manthorpe et al., 2008) see disaggregation of data as an essential tool of their analysis, but that the knowledge they gained was often lost when politicians or policy makers

averaged data or grouped data together in an inappropriate manner. One example is the

SI of homelessness, which is treated in LA SI tables as an integrated figure. Reports

differ in regard to the contribution to homelessness made by ex-service personnel.

Strachan (2003) reported that 25% of Britains homeless are ex-service personnel, whereas the BBC reported in 2008 that homeless ex-service personnel has gone up

31

from 5% to 6%. Although these are temporally different reports with different statistics,

it can be seen that a proportion of the homeless in England are ex-service personnel and

the local occurrence of this issue could be assessed for different plans if the SI of

homelessness is disaggregated appropriately.

The hierarchy scoring for this criterion is based firstly on the appropriate amount of

disaggregated data, for scores of zero to two, and then for a score of three the use of

current academic research describing areas where different groups of people can be

disadvantaged. Collecting evidence for this criterion was limited to the information

found in the indicator table used. The reason for this was that all who read the

document should be easily able to access indicators that they feel relevant to them and

see how improvement, via different plans and targets, are proposed.

(7) Measurability: This refers to how the values in an indicator are measured and the extent to which it

measures reality (Bauler et al., 2007:56). Like models, indicators can reflect reality only imperfectly; however, even within the measurable the quality of indicators is

determined largely by the way reality is changed into measures and data, be they

qualitative or quantitative. The quality of indicators inevitably depends on the

underlying data that are used to compose them (Moldan et al., 2007:9). The hierarchy chosen for this criterion at first glance appears to have a subjective view biased towards science and economics rather than social science. From bottom to top the hierarchy

goes from qualitative, then quantitative with the measurement being able to be adjusted to reflect individual situations. To balance the ways different disciplines measure in

qualitative and quantitative terms, a number of ways of scoring, by choosing different

statements within each score, have been made available to give all three pillars of

sustainability an equal chance of achieving all scores.

(8) Reliability: This criterion has the title of reliability (rather than reproducibility or repeatability) to enable both the general meaning of the word to be conveyed and the statistical

interpretation of this word to be considered in the discussion. The hierarchical scoring

32

was based on the three prominent factors involved when considering reliability:

stability, internal reliability and inter-observer consistency (Bryman, 2004:71). More detail on this theory can be found in section 3.4.6 of this dissertation.

(9) Relevant to plan: SIs generally are intended to target ongoing political processes, yet they often are

developed with surprisingly political naivet (Moldan et al., 2007). This criterion looks at one aspect, the set of targets from each LA, and how many suitable targets have been

used. The suitability of a target is defined as being either a visual target (up/down arrows, smiley faces) or a written target. Alternatively, if an LA clearly stated that a target was partially constructed, this was considered as suitable as a visual or written

target. However, if an LA had a column labelled targets, but the information

contained within the column appertaining to targets for individual indicators was either

partially or wholly lacking information about local targets, then this was deemed

unsuitable. When an LA left some or all of their target boxes blank, with no

information for future insertion of a target, these were also deemed unsuitable. Some

LAs used the not applicable stance a number of times without justification, and for the purposes of this research such data was deemed unsuitable; although the author

considers that this point requires further deliberation in future work. The data collected

was then given a percentage total of possible targets (with the total possible targets crudely assigned as the total number of indicators). A three tier systematic percentage marking scheme was created to score this criterion. The LA was given a single score of

0, 1, 2 or 3 and this was assigned to all indicators in that LA. The author recognises that

the method used to score this criterion needs to be improved, to link each indicator with

an individual target, in order to show if the average is masking important information.

(10) Actionable: Actionable is an important criterion for LAs as they are accountable to the people who

live there and need to be seen to be doing or actioning targets developed from SA

objectives and indicators which lead to more sustainable local living. However, it would be easy for LAs to choose SIs with targets that have quick fixes in order to

persuade policy makers and the public that sustainability is achievable (Astleithner et

33

al., 2004). The harder longer-term actions that could lead to stronger sustainability, often outside the temporal frame of the LDF, can be less visibly actionable so less

attractive to use. Therefore, there is conflict that needs to be addressed between what is

sustainable for the community and the political nature and timeframe of the LDF.

Crucially, effective action is much less common than cheerful visions and passionate

endorsement (Gibson, 2006), so an indicator scoring three in actionable does not ensure that effective action occurs. In this criterion the hierarchical scoring goes from

WS, the easiest indicators to action (scoring one), through to SS where greater knowledge and resources are needed to action indicators (scoring three).

(11) Stakeholder involvement: Two ways of participatory stakeholder involvement in choosing SIs are firstly

involving the public, and secondly involving users to increase the efficiency of the

decision-making process (audience targeted) (Bauler et al., 2007:62). Statutory stakeholders for SA are asked for information (ODPM, 2005), but Therivel et al. (2006) state this is not always provided and, in the case of SEA, almost two in three reports did not receive any response, so it should be kept in mind that an invitation to

consult about indicators is different to who has actually contributed towards choosing

SIs. Differentiating these two points is not easy from evidence currently available in

most scoping reports. Another barrier to obtaining evidence is that when LAs

demonstrate that they consult widely, such as with Wigan using the Wigan World

Summit with 250 consultees (WMBC, 2007:4), they then neglect to provide any easily accessible evidence of the detail of the consultation. Some information can be found in

additional documents, such as appendices, if they are appropriately labelled and easily

accessible (an example being Elmbridge Borough Council, Appendix 5: Amendments to Draft Scoping Report following Consultation) (EBC, 2005). The best practice to enable the author to ascertain the input of consultees was found where LAs had

documents containing the consultation responses with the exact wording given by the

consultee and also the LAs response to the consultee, such as Spelthorne (SBC, 2007:201-245). The hierarchy in this criterion is based upon both number and range of stakeholders. The score of three using the second method of audience targeted

participation (Bauler et al., 2007) was used as the preferred of the two participation

34

methods, in agreement with the views of Niemeijer and de Groot (2008) and Hall (2007).

(12) Funding and cost: LAs have limited budgets to spend on indicator development and monitoring, therefore

their decision to choose the most appropriate indicators will, to some extent, be based

on cost. There are a number of databases that are free for LAs to use, such as the Local

Authority Area Ecological Footprint (Audit Commission, 2005), but other data sets involve yearly subscription costs. The hardest areas to fund are the development of

local indicators that require a baseline to be established, or indicators that have high

costs involved in collecting the data (Biggs et al., 2007). If the LA has chosen a large number of SIs, this will mean that decision-making (concerning what to fund or which data to use for an indicator) could be more complex with finite funding available. This is the criterion that has the most potential incompatibilities with the other twelve

criteria (see Table 3.3).

(13) Easily communicated: Communicability is the extent to which indicators are understood and the effectiveness

with which they convey their purpose and meaning to the target audience (Bauler et al., 2007:57). An ideal indicator would be one that communicates for a specific purpose to a range of audiences. However, Bauler et al. (2007:63) consider this may not be achievable, given the diversity of stakeholders. Nevertheless, the UK Quality of Life Barometer has been described as the most single important development in

communicating sustainable development by Anne Power, the UK Sustainable

Development Commissioner (Hall, 2007:301), and editions of Sustainable Development Indicators in your Pocket have proved to be very popular and been

applauded by a variety of stakeholders (ibid.:302). Their popularity is due to a high percentage of the information being communicated simply and visually (traffic light evaluation method, charts and graphs), which links to the fact that 60% of all people prefer a way of learning that is visual (Gardner et al., 2003). The hierarchy used considers local community concerns being important (Holland, 1997) for a score of

35

one. The top score is given when the media starts to use SIs to communicate and, at a

higher level, analyse the changes over a period of time.

3.4.3 Compatibility Matrix of Criteria

A compatibility matrix was used to assess incompatibilities between the thirteen criteria

and the table of appropriateness in Appendix 1 was used to assess the level of

compatibility between the thirteen criteria. This serves to highlight potential conflicts

between pairs of criteria; Table 3.3 indicates where conflicts may arise. The results

show 28 potentially incompatible criterion pairings and, out of these, 26 are located

within the LA super criteria, one in measurement and one in credibility. The

criterion of funding/cost contains all 12 potential incompatibilities. The compatible

pairings are more evenly spread amongst the three super criteria, and for the uncertain

pairings the most uncertainty occurs in the credibility super criteria. It should be

noted that the criterion of funding/cost is also an area in which the author exhibits

strong subjectivity.

36

Table 3.3: Compatibility Matrix of the Thirteen Criteria and

Three Super Criteria

Credibility Measure-ment

Local Authority

R

e

l

e

v

a

n

t

t

o

P

l

a

n

Leads to Strong Sustainability Academic credibility Addressing uncertainty Credibility Environmental receptors addressed Locality Disaggregation of Data Measurability Measurement Reliability Relevant to Plan Actionable Stakeholder involvement Funding/cost

Local Authority

Easily Communicated

Key to Compatibility

Potentially incompatible Uncertain Compatible No Links

37

3.4.4 Trial Run of the Method

To establish if this method exhibited a wide range of values of SI appropriateness and

was also reliable, a trial run was completed using four LAs, as shown in Table 3.4.

Table 3.4: LAs Used in Trial Run of Criteria-based Appropriate Method

From trialling the four LAs containing a total of 381 SIs, it was concluded that:

A wide range of values was observed for computations, such as for individual

indicators from 030 out of a total of 38 - this could lead to appropriateness

being established.

Three criteria were identified that needed their scoring method adapted:

Locality, Stakeholder Involvement and Relevant to Plan (see Appendix 1 for final scoring system).

A method to ensure reliability of scoring needed to be created as there was

variation between similar indicators in different LAs. For example, Percentage

of affordable housing provided scored between 12 and 17.

See Table 3.5 for a trial run for appropriateness.

Local Authority Region Defra classification Number of indicators

Breckland East England Rural 80 59

Dacorum East England Major Urban 143 West Oxfordshire South East Rural 80 59

Mole Valley South East Major Urban 120

38

Dacorum Indicators

Lea

ds to

St

ron

g Su

sta

ina

bilit

y

Aca

dem

ic cr

edib

ility

Add

ress

ing

un

cert

ain

ty

Env

iro

nm

enta

l rec

epto

rs a

ddre

ssed

Cre

dibi

lity

Tota

l/12

Loca

lity

Disa

ggre

gatio

n o

f Da

ta

Mea

sura

bilit

y

Rel

iabi

lity

Mea

sure

men

t Tot

al/1

1

Rel

eva

nt t

o Pl

an

Act

ion

abl

e

Sta

keho

lder

in

vo

lvem

ent

Fun

din

g/co

st

Easil

y Co

mm

un

ica

ted

Loca

l Au

tho

rity

To

tal/1

5

Indi

cato

r To

tal/3

8

Populations of wild birds 1 0 1 1 3 0 0 2 0 2 0 1 1 1 3 6 11

Area of semi-natural habitat lost to development

1 0 3 1 5 1 0 2 0 3 0 1 1 1 2 5 13

Area of new semi-natural habitat created

1 0 3 1 5 1 0 2 0 3 0 1 1 1 2 5 13

Table 3.5: Trial Run Scoring of Appropriateness: Some Results from Dacorum

3.4.5 Reference Set of Indicators

A reference indicator set was created, which was built up throughout the scoring of the

38 sets of indicators, containing 2,970 individual indicators. For ease of use, 18

different pages on an Excel spreadsheet were grouped as the set was built up. Table 3.6

illustrates the grouping. Indicator scores from the reference indicator set were not

directly transferred to the LA score sheet, even if the indicator had exactly the same

wording. Firstly, four criteria were scored individually: locality, disaggregation of data,

relevant to plan and stakeholder involvement; all of which gave unique scores for each

LA. As such, the same indicator may have different scores in different LAs (range of difference 0-11). Similarly worded indicators were subjectively judged as to whether they had the same score as the reference indicator or were added as a unique indicator

to the reference indicator set. The total number of SIs in the reference set at the end of

the scoring period was 325.

39

Reference Indicator

Classification Category

Acc

ess

and

Com

mu

nity

Air

and

Clim

ate

Ben

efits

an

d W

ork

Bio

div

ersit

y

Bu

sines

s

Crim

e an

d Fi

re

Edu

catio

n

Ener

gy

Hea

lth

Her

itage

Ho

usin

g

Mat

eria

ls

Pollu

tion

Rec

reat

ion

Soil

and

Lan

d

Tran

spo

rt

Was

te

Wat

er

Number of Reference Indicators in category

30 9 16 26 41 8 16 16 18 12 27 6 10 9 14 29 11 27

Table 3.6: The Reference Indicator Set Total Number of Indicators in Each Classification

3.4.6 Reliability of Appropriateness Method Using the Reference Indicator Set

The appropriateness method needed to be checked for reliability over the time the data

analysis was carried out. Dacorum was chosen for this reliability test as it was the first

LA to be scored after the trial run was carried out and the reference indicator set was

created, and it was also temporally appropriate. It contained 143 SIs, which was at the

top end of the range of number of SIs per LA (range was 24151). See Table 3.7 for details.

Three prominent factors are involved when considering reliability: stability, internal

reliability and inter-observer consistency. Stability refers to administering a measure to

a group and then readministering it. If there is stability, then there will be little variation

over time in the results obtained. If it is a long span of time (a year or more), external variables can change (Bryman, 2004:71).

40

Table 3.7: Differences in Total and Percentage Scoring for Dacorum LA

In this test for stability, the time of 37 days was deemed to be a reasonable time span

and appropriate for the measurement being used. However, a consideration of the

actual change or perception of the change (for example, in the UK economy over this period of time) could be a variable that influences the authors viewpoint when scoring economic indicators. The results show that the change over 37 days between the two

Dacorum total scores was 2%. Two individual criteria outliers were academic

credibility and disaggregation of data. However, the super criteria credibility (3%) and measurement (4%) that these two criteria are in are still at acceptable levels of repeatability.

Internal reliability looks at whether the indicators that make up a scale are consistent

(Bryman, 2004:71). In relation to this method, it should be considered whether each criterions scoring of 1-3 is consistent between criteria (i.e. a score of two in the criterion addressing uncertainty should equal a score of two in the criterion locality). To address internal reliability, a statistical method could be applied to the data, either a

Cri

teri

a

Lea

ds to

St

ron

g Su

sta

ina

bilit

y

Aca

dem

ic cr

edib

ility

Add

ress

ing

un

cert

ain

ty

Env

iro

nm

enta

l rec

epto

rs a

ddre

ssed

Cre

dibi

lity

Tota

l/12

Loca

lity

Disa

ggre

gatio

n of D

ata

Mea

sura

bilit

y

Rel

iabi

lity

Mea

sure

men

t To

tal/1

1

Rel

eva

nt t

o Pl

an

Act

ion

abl

e

Sta

keho

lder

in

vo

lvem

ent

Fun

din

g/co

st

Easil

y C

om

mu

nic

ate

d

Loca

l Au

tho

rity

To

tal/1

5

Indi

cato

r To

tal/3

8

21.06.08 202 121 282 212 817 134 6 261 208 609 0 249 143 223 358 973 2399

27.07.08 205 133 276 219 833 136 17 267 215 635 0 261 143 221 365 990 2458

% change from June

to July

1 10 -2 3 2 1 183 2 3 4 0 5 0 -1 2 2 2%

41

spilt-half method or Cronbachs alpha. This was not undertaken and the author notes

this omission for future research.

The third factor involved with reliability, inter-observer consistency, is when there is

lack of consistency between judgements made by more than one observer. However, this was not present within this research as only one observer was used.

In conclusion, this method has stability, when using a reference set of indicators and

when measured by one researcher over a medium period of time (months) using one LA. Events that influence the degree of consistency were considered, but not calculated

within the measurement of stability. The method has the possibility to be tested further

to assess whether other researchers could use it and obtain stability and inter-observer

consistency. Internal reliability should be calculated.

3.4.7 The Use of Weightings

Glasson et al. (2005:145) and Moles et al. (2007) indicate that weighting seeks to identify the relative importance of criteria. The research by Malkina-Pykh and

Malkina-Pykh (2007) on quality of life indicators considered the main approach used to derive weightings, expert opinion, as a means to determine the list of criteria and

their significance, which is in agreement with the methods used by Moles et al. (2007) and Astleithner et al. (2004). Authors agree that panels of experts should decide weightings (Astleithner et al., 2004; Glasson et al., 2005; Moles et al., 2007), but weighting systems generate considerable debate (Glasson et al., 2005). In this study, the author decided to give all criteria equal weighting, so that no judgements were made as to which criterion was more or less important, and so as to make aggregation

as transparent as possible (Moles et al., 2007). As the author has strong subjectivity on three of the thirteen criteria, and was not part of an expert team, to add weighting as an

additional variable to change the internal reliability of the method was considered

unsuitable.

42

3.5 Highest Scoring Individual Indicators - Top Ten Ranking

The frequency of different groups of SIs within each LA was established using the

reference set of indicators. SIs in each LA were ranked by individual indicator score,

from highest to lowest. The top ten, including equal tenth, were then grouped using the

reference set of indicators. The frequency of each group was then measured within the

top ten and compared to similarly categorised rurality LAs. The stability of the

groupings was good, as the reference indicator set proved to be repeatable over time

when scoring for appropriateness.

3.6 Numbers of Sustainabili