risk of error on 2000-06 closure ljubljana, 12-13 october 2009

1

Ljubljana, 12-13 October 2009

Homologues Group Meeting

Slovenia, October 2009

Republika SlovenijaEuropean Union

Risk of Error on 2000-06 Closure Ljubljana, 12-13 October 2009

Risk of Error on 2000-06 Closure Ljubljana, 12-13 October 2009

Graeme Waterhouse ESF Audit Authority for England

& GibraltarRisk Assurance Division

Department for Work & Pensions

Graeme Waterhouse ESF Audit Authority for England

& GibraltarRisk Assurance Division

Department for Work & Pensions

2





IntroductionIntroduction

Our experience of trying to shut the 2000-06 programmes.

How we will communicate to the Commission:• Frequency of error• Materiality• Whether there is a high/low frequency of

error• Financial impact

3





BackgroundBackground• 12 programmes to shut in England & Gibraltar by

31 March 2010• ESF is delivered by the Department for Work &

Pensions• De-centralised system – delivery and A10 checks

through 9 regional Government OfficesCentral coordinating bodies:• Managing Authority in ESF Division• Paying Authority in ESF Division• Risk Assurance Division (for A10 & A15)

4





ApproachApproach

• Apply the Commission’s Guidelines on the Closure of Assistance for 2000-06

• Apply professional standards• Explore options• Minimise any potential financial damage• Keep it simple

5





GuidanceGuidanceLow frequency = “financial implications fall below a level of materiality

considered appropriate by the independent body”

High frequency = “confidence in entire management system is seriously affected and therefore no opinion can be given. To determine frequency a distinction must be drawn between the different categories of error”.

Materiality = “The materiality level should generally not exceed 2% in order to be consistent with the methodology of the ECA. Specific justification should be provided in case a higher level is applied”.

If high we must estimate the financial impact

6





Frequency of ErrorFrequency of Error

Article 10 Purpose:

To provide an independent assessment of the To provide an independent assessment of the effectiveness of the management and control effectiveness of the management and control systems operated by the Managing Authority & systems operated by the Managing Authority & Implementing Bodies for European Structural Implementing Bodies for European Structural FundsFunds

7





Frequency of ErrorFrequency of ErrorCALCULATE TWO ERROR RATES

Programme Error Rate:• Error that exists in the Managing Authority’s control environment• Before any audit intervention• Includes results of random & risk-based Article 10 checks & other Community &

national controls.• Cannot be used for extrapolation

Representative Error Rate:• Based on randomly selected Article 10 controls• Can be used for extrapolation

8





Programme Error RateProgramme Error Rate

Population/

Sample (£/Є)

Error Rate Weighting Weighted Error Rate

A10 – Risk Sample 40 10% 40 0.8%

Other Community & National Controls

10 10% 10 0.2%

A10 – Random Sample 50 2.5% 50 + 400=

4502.25%

PROGRAMME ERROR RATE

500 3.25%Representative of the unchecked population

10% x (40/500)

9





Programme Error RateProgramme Error Rate

Conclusion.

• The programme error rate is 3.25%

10





Representative Error RateRepresentative Error RateBased on randomly selected Article 10 checks only.

Representative:• Geographic spread• Time (annual coverage)• Type/size• Measure• Randomly selected

Can be used for extrapolation as it reflects the error that exists in the unchecked population

11





Representative Error RateRepresentative Error Rate

Population/

Sample

Error Rate

Weighting Weighted Error Rate

A10 – Risk Sample 40 10% 40 0.8%

Other Community & National Controls

10 10% 10 0.2%

A10 – Random Sample 50 2.5% 50 + 400=

4502.25%

PROGRAMME ERROR RATE

500 3.25%

12





Representative Error RateRepresentative Error Rate

Conclusion.

• The representative error rate is 2.5%

13





MaterialityMateriality

Materiality = “The materiality level should generally not exceed 2% in order to be consistent with the methodology of the ECA. Specific justification should be provided in case a higher level is applied”.

Conclusion:• The level of error in the programme is material if

the value of errors remaining in the claim exceeds 2% of the final claim value (total population)

14





Financial ImpactFinancial Impact

A Population (final claim value) £500

B Less A10, national & Community controls

-£100

C Unchecked population £400

D Representative Error Rate 2.5%

E Financial impact (CXD) £10

F Materiality Level (AX2%) £10

G Material Impact (E-F) £0

The checked population hasalready been corrected so

no error remains

The value of error inthe Final Claim

Are the errors in the final claim material?

15





ConclusionsConclusions

• The programme error rate is 3.25%• The representative error rate is 2.5%• The value of the errors that remain in the final

claim is within the Commission’s 2% materiality level

Conclusion – there is a low frequency of error

16





Another ExampleAnother Example

A Population (final claim value) £500

B Less A10, national & Community controls

-£100

C Unchecked population £400

D Representative Error Rate 3.5%

E Financial impact (CXD) £14

F Materiality Level (AX2%) £10

G Material Impact (E-F) £4

The checked population hasalready been corrected so

no error remains

The value of error inthe Final Claim

Are the errors in the final claim material?

17





ConclusionsConclusions

• The representative error rate is 3.5%• The value of the errors that remain in the

final claim exceeds the Commission’s 2% materiality level

Conclusion – there is a high frequency of error

18





Other ConsiderationsOther Considerations

• Stratifying the population?

• Unrepresentative Samples

• Off-Setting

19





StratifyingStratifying

• Basis of stratification must be set from the start of the programme

• Different strata types give different overall results

• Consistency across programmes

• Sample sizes often too small

20





Unrepresentative SamplesUnrepresentative Samples

Our definition:• An Article 10 check where the audit trail is completely lost (100% error)

as a result of insolvency or natural disaster (fire/flood).

Implication:• These can be removed from error rate and financial impact calculations

Assumptions:• These are circumstances beyond the Managing Authority’s control• The audit trail did exist at the time the project operated and the likely level

of error is in line with the representative error rate.

21





Off-SettingOff-Setting

Off –setting of positive and negative financial impacts

Argument:1)Commission may make corrections based on

the Article 15 body’s estimated financial impact2)The corrections should take account of the

Managing Authority's control environment as a whole (i.e. across all programmes)

22





Thank you for your attention!Thank you for your attention!

Email: [email protected]: [email protected]: (0044)113 2324777Tel: (0044)113 2324777

risk of error on 2000-06 closure ljubljana, 12-13 october 2009

Documents