managing the business risk of fraud

1. Managing the Business Risk of Fraud using Sampling and Data Mining Fall 2009 Mike Blakley Presented to:

Sharp rise in accounting fraud over the past 12 months

Accounting fraud had grown to 38 percent of the economic crimes in 2009

Employees face increased pressures to :

meet performance targets

keep their jobs

keep access to funding

Greater risk of fraud due to increased incentives or pressures

More opportunitiesto commit fraud, partially due toreductions in internal finance staff

While companies are expecting more fraud, theyhave not done much

People who look for fraud are more likely to find it

Understand the framework for managing the business risk of fraud

Plan, perform and explain statistical sampling in audits

Reduce audit costs using data mining, sequential sampling and other sampling techniques

Apply SAS 56, the new SAS suite and the revised (2007) Yellow Book.

Run, hands-on, the most productive analytic technique (regression analysis).

Use data mining to introduce greater efficiency into the audit process, without losing effectiveness.

Introduction and the Process for Managing the Business Risk of Fraud

Introductions All Around

Course Objectives

Framework of risk management for fraud

Fundamentals of data mining

Data mining: The Engine That Drives analysis

Analytics and Regression

Sources of Analytics Data

Basic and Intermediate ARTs

SAS 56

IIA Practice Advisory 2320

The Yellow Book (2007 revision)

The Guide Managing the Business Risk of Fraud

Sampling

The sampling process

Sampling methods

RAT-STATS

Random Numbers

Determining Sample Size

Case Study

Attribute sampling

Variable Sampling

Case study

Stratified Sampling

Obtaining and Interpreting the results

Other Sampling Approaches

DCAA Audit Package

Sequential Sampling

Overview of the process

Attribute Sampling

Variable Sampling

Regression Analysis

Overview

Statistical basis

Charting Regression Seeing Is Believing

Plotting Data

Inserting a Trend line

Statistical Intervals

Confidence Intervals

Prediction Intervals

Calculation of Statistical Confidence Bounds

Case Study - Wake County Schools Bus Maintenance

Overview

Statistical Basis

Data Conversion and Extraction

Data mining objectives

Classification

Trends

Identification of extremes

Major types of data analysis

Numeric

Macros

Tools Data Analysis

The Macro facility

Adding a little class to your audit

VBA friend or foe

CD with articles and software

PowerPoint presentation

More info at www.ezrstats.com

If you spot one roach.

There are probably 30 more that you dont see

Fraud patterns detectable with digital analysis

Basis for digital analysis approach

Usage examples

Continuous monitoring

Business analytics

Three brief examples

ACFE/IIA/AICPA Guidance Paper

Practice Advisory 2320-1

Auditors Top 10

Process Overview

Who, What, Why, When & Where

Supplier Kickback School Bus parts

$5 million

Jail sentences

Period of years

Understaffed internal audit

Software not used

Data on multiple platforms

Transaction volumes large

Need structured, objective approach

Let the data talk to you

Need efficient and effective approach

Stepwise to find relationships

Forwards

Backwards

Intervals

Confidence

Prediction

Sometimes an out and outLiar

But how do youdetectit?

Plot transportation costs vs. number of buses

Drill down on costs

Preventive maintenance

Inspection

CMS Report for 2005

South Florida -$2.2Billion

Rest of the country combined -$.1Billion

Serostim

Treat wasting syndrome, side effect of AIDS,OR

Used by body builders for recreational purposes

One physician prescribed $11.5 million worth (12% of the entire state)

A proposed implementation approach

Managing the Business Risk of Fraud: A Practical Guidehttp://tinyurl.com/3ldfza

Five Principles

Fraud Detection

Coordinated Investigation Approach

ACFE,IIA andAICPA Exposure draft issued 11/2007, final 5/2008

Section 4 Fraud Detection

Five Sections

Fraud Risk Governance

Fraud Risk Assessment

Fraud Prevention

Fraud Detection

Fraud Investigationand corrective action

Fraud risk management program

Written policy managements expectations regarding managing fraud risk

Periodic review and assessment of potential schemes and events

Need to mitigate risk

Establish prevention techniques

Mitigate possible impact on the organization

Establish detection techniques for fraud

Back stop where preventive measures fail, or

Unmitigated risks are realized

Reporting process to solicit input on fraud

Coordinated approach to investigation

Use of corrective action

2/15/09 Scott Pelley

Fraud Risk Governance one grand wink-wink, nod-nod

Fraud Risk Assessment - categorically false

Fraud Prevention my husband passed away

Fraud Detection -We didn't know? Never saw one.

Fraud Investigationand corrective action - Pick-A-Payment losses $36 billion

Detective Controls

Process Controls

Anonymous Reporting

Internal Auditing

Proactive Fraud Detection

Data Analysis to identify:

Anomalies

Trends

Risk indicators

Operate in the background

Not evident in everyday business environment

These techniques usually

Occur in ordinary course of business

Corroboration using external information

Automatically communicate deficiencies

Use results to enhance other controls

Whistleblower hot-lines (DHHS and OSA have them)

Process controls (Medicaid audits and edits)

Proactive fraud detection procedures

Data analysis

Benfords Law

Journal entries suspicious transactions

Identification of relationships

Benfords Law

Identify hidden relationships

Identify suspicious transactions

Assess effectiveness of internal controls

Monitor fraud threats

Analyze millions of transactions

Organization should develop ongoing monitoring and measurements

Establish measurement criteria (and communicate to Board)

Measurable criteria include:

fraud allegations

fraud investigations resolved

Employees attending annual ethics course

Whistle blower allegations

Messages supporting ethical behavior delivered by executives

Vendors signing ethical behavior standards

Each process owner should:

Evaluate effectiveness of technique regularly

Adjust technique as required

Document adjustments

Report modifications needed for techniques which become less effective

International standards for the professional practice of Internal Auditing

Analytical audit procedures

Efficient and effective

Useful in detecting

Differences that are not expected

Potential errors

Potential irregularities

May include

Study of relationships

Comparison of amounts with similar information in the organization

Comparison of amounts with similar information in the industry

Performed using monetary amounts, physical quantities, ratios or percentages

Ratio, trend andregression analysis

Period to period comparisons

Auditors should use analytical audit procedures in planning the engagement

Significance of the area being audited

Assessment of risk

Adequacy of system of internal control

Availability and reliability of information

Extent to which procedures provide support for engagement results

Traditionally, IT specialists

With appropriate tools, audit generalists (CAATs)

Growing trend of business analytics

Essential component of continuous monitoring

Using software to:

Classify

Quantify

Compare

Both numeric and non-numeric data

Basis is quantification

Software can do the leg work

Statistical measures of difference

Chi square

Kolmogorov-Smirnov

D-statistic

Specific approaches

Automated process

Handle large data populations

Objective, quantifiable metrics

Can be part of continuous monitoring

Can produce useful business analytics

100% testing is possible

Quantify risk

Repeatable process

Costly (time and software costs)

Learning curve

Requires specialized knowledge

Traditional intermittent (one off)

Trend is to use it as often as possible

Scheduled processing

Any organization with data in digital format, and especially if:

Volumes arelarge

Data structures arecomplex

Potential for fraudexists

Software

Training

Skills not widely available

Time consuming

Development costs

Testing resources

Three brief examples

CFE Guidance Paper

Top 10 Metrics

Process Overview

Who, What, Why, When & Where

Brief History / Timeline

Overview

Attribute Sampling Compliance

Variable Sampling Numeric Estimates

Basis is two laws/theorems of probability

Law of Large Numbers

Central Limit Theorem

Indian mathematician Bramagupta 600 AD

Italian mathematician Cardon 1500s

Statement without proof that empirical statistics improve with more trials

Jacob Bernoulli first to prove in 1713

Foundation for central limit theorem

See it in action today

Any population

Large number of samples

Average is normally distributed

French mathematician Abraham de Moivre

1733 approximate distribution from tossing coin (heads/tails)

Ho hum reaction

French Mathematician LaPlace expanded it

Ho hum reaction

Russian mathematician Lyapunov

Proof in 1901

Same reaction

Effective June, 1983

Exposure draft for revision in 2009

Buonaccorsi (1987)

Refined calculations

Few software packages use it

Sample size calculations

Attribute sampling

Variable sampling

Random number generators

Its a guess

Every package different answer

Need to know the population

But thats why youre taking a sample!

Understand what is attribute sampling and when to use it

Understand unrestricted populations

Overview of the process using RAT-STATS

Understand the formula behind the computations

Attribute

Compliance testing

Signatures on approval documents, attachment of supporting documentation, etc.

Recommended

Economical

Efficient

Requires determination of a sample size

Determine the sampling objective

Confidence

Precision

Determine required sample size

Identify samples to be selected based upon random numbers

Pull the sample and test

Compute the sampling results (i.e. estimate of range)

The sampling parameters are first developed by the auditor

RAT-STATS is used to compute sample size

RAT-STATS used to generate random numbers

Enter results in RAT-STATS to compute estimates

Size of population

Expected error rate

Required confidence

Required precision

Done by entering info into RAT-STATS

Output can be a variety of sources:

Text File

Microsoft Access

Print File

Each random number selected corresponds with an item

Put the selected item on a separate schedule

Generally requires reviewing documents

Enter summary information into RAT-STATS

Output can be in a variety of formats:

Microsoft Access

Text File

Print File

Printer

Now well see an actual demo using the RAT-STATS software

Excel population of 5,000 invoices

Results of test of attributes stored in the worksheet

Understand what variable sampling is and when to use it

Understand unrestricted populations

Variable

Estimating account balances

Estimating transaction totals

Recommended

Economical

Efficient

Requires determination of a sample size

Confidence

Precision

Probe sample

Statistical measure

Excel formula

Size of population

Average value

Standard deviation

Text File

Microsoft Access

Print File

Example data contains both examined and audited value.

Microsoft Access

Text File

Print File

Printer

Audited values stored in the worksheet

Understand what is stratification and when to use it

Strata

Homogenous

More efficient in some instances

Separation into strata

Confidence

Precision

Size of population

Expected error rate

Required confidence

Required precision

Text File

Microsoft Access

Print File

Microsoft Access

Text File

Print File

Printer

Results of test of attributes stored in the worksheet

Understand what stratified sampling is and when to use it

Populations benefiting from stratified sampling

Stratified

Variable

Estimating amounts

Narrower standard deviation

Confidence

Precision

Probe sample

Statistical measure

Excel formula

Number of strata

Size of population

Average value

Standard deviation

Multi-stage random numbers

Text File

Microsoft Access

Print File

Each random number selected corresponds with an item in a strata

Example data contains both examined and audited value.

Microsoft Access

Text File

Print File

Printer

Divided into three strata

Audited values stored in the worksheet

Optimize sample size (most bang for the buck)

Skip sampling review 100% of transactions usingc omputera ssisteda uditt echniques (CAATs)

Sequential sampling

Banana aphids

Provides complete coverage

Best practice

Basis for continuous monitoring

Repeatable process

3.43 Technical Knowledge and competence

The staff assigned to conduct an audit or attestation engagement under GAGAS must collectively possess the technical knowledge, skills, and experience necessary to be competent for the type of work being performed before beginning work on that assignment.

The staff assigned to a GAGAS audit or attestation

engagement should collectively possess:

Skills appropriate for the work being performed. For example, staff or specialist skills in

(1)statistical sampling if the work involves use of statistical sampling;

(2)information technology

Requiresuse of analytic review procedures for:

Audit planning

Overall review stages

Encouragesuse of analytical review

Provides guidance

Outliers

Stratification

Day of Week

Round Numbers

Made Up Numbers

Market basket

Trends

Duplicates

Linear regression

Sales areup , but cost of goods sold isdown

Spikes

Identify trend lines, slopes, etc.

Correlate trends

Identify anomalies

Key punch errors where amount is order of magnitude

Test relationships (e.g. invoice amount and sales tax)

Perform multi-variable analysis

Estimate linear trends using best fit

Measure variability (standard errors)

Measure slope

Sort descending by slope, variability, etc.

Analytical review

Isolate the significant few

Detection of errors

Quantifiedapproach

Quantified Approach

Population vs. Groups

Measuring the Difference

Stat 101 Counts, Totals, Chi Square and K-S

The metrics used

Based onmeasureable differences

Population vs. Group

Shotgun technique

Something isdifferentthan expected

Common theme something is different

Groups

Group pattern is different than overall population

Transactioncounts

Transactionamounts

Detailed knowledge of statistics not necessary

Software packages do the number-crunching

Statistics used only to highlight potential errors/frauds

Not used for quantification

Comparison of group with population as a whole

Can be based on either counts or amounts

Difference is measured

Groups can then be ranked using a selected measure

High difference = possible error/fraud

Based roughly on the Wake County Transportation fraud

Illustrates how the process works, using Excel

Attributes tallied and categorized into bins

Countsorsumsof amounts

Population and group

Two statistical measures of difference

Chi Squared (counts)

K-S (distribution)

Both yield a difference metric

Classic test on data in a table

Answers the question are the rows/columns different

Some limitations on when it can be applied

Table of Counts

Degrees of Freedom

Chi Squared Value

P-statistic

Computationally intensive

Two Russian mathematicians

Comparison of distributions

Metric is the d-statistic

Four step process

For each cluster element determine percentage

Then calculate cumulative percentage

Compare the differences in cumulative percentages

Identify the largest difference

Stratification

Day of week

Happens on holiday

Round numbers

Variability

Benfords Law

Trend lines

Relationships (market basket)

Duplicates

Outliers / Variability

Stratification

Day of Week

Round Numbers

Made Up Numbers

Market basket

Trends

Duplicates

Quantified Approach

Population vs. Groups

Measuring the Difference

Stat 101 Counts, Totals, Chi Square and K-S

The metrics used

Understand why and how

Understand statistical basis for quantifying differences

Identify ten general tools and techniques

Understand examples done using Excel

How pattern detection fits in

Overview

Explain Each Metric

Examples of what it can detect

How to assess results

Charting (visual)

Software to analyze z-scores

Top and Bottom 10, 20 etc.

High and low variability (coefficient of variation)

Basic statistics

Minimum, maximum and average

Variability

Sort by statistic of interest

Variability (coefficient of variation)

Maximum, etc.

Outliers

Stratification

Day of Week

Round Numbers

Made Up Numbers

Market basket

Trends

Duplicates

Charting (visual)

Chi Squared

Kolmogorov-Smirnov

By groups

Transactions out of the ordinary

Up-coding insurance claims

Skewed groupings

Based on either count or amount

Stratify the entire population into bins specified by auditor

Same stratification on each group (e.g. vendor)

Compare the group tested to the population

Obtain measure of difference for each group

Sort descending on difference measure

Outliers

Stratification

Day of Week

Round Numbers

Made Up Numbers

Market basket

Trends

Duplicates

Activity on weekdays

Activity on weekends

Peak activity mid to late week

Identify unusually high/low activity on one or more days of week

Dentist who only handled Medicaid on Tuesday

Office is empty on Friday

Programmatically check entire population

Obtain counts and sums by day of week (1-7)

Prepare histogram

For each group do the same procedure

Compare the two histograms

Sort descending by metric (chi square/d-stat)

Outliers

Stratification

Day of Week

Round Numbers

Made Up Numbers

Market basket

Trends

Duplicates

Isolate estimates

Highlight account numbers in journal entries with round numbers

Split purchases (under the radar)

Which groups have the most estimates

Classify population amounts

$1,375.23 is not round

$5,000 is a round number type 3 (3 zeros)

$10,200 is a round number type 2 (2 zeros)

Quantify expected vs. actual (d-statistic)

Generally represents anestimate

Journal entries

Outliers

Stratification

Day of Week

Round Numbers

Made Up Numbers

Market basket

Trends

Duplicates

Made up numbers e.g. falsified inventory counts, tax return schedules

Basic formula is =log(1+(1/N))

Workbook with formulae available athttp://tinyurl.com/4vmcfs

Obtain leading digits using Left function, e.g. left(Cell,1)

Benfords Law

Check Chi Square and d-statistic

First 1,2,3 digits

Last 1,2 digits

Second digit

Sources for more info

Decide type of test (first 1-3 digits, last 1-2 digit etc)

For each group, count number of observations for each digit pattern

Prepare histogram

Based on total count, compute expected values

For the group, compute Chi Square and d-stat

Outliers

Stratification

Day of Week

Round Numbers

Made Up Numbers

Market basket

Trends

Duplicates

Medical Ping ponging

Pattern associations

Apriori program

References at end of slides

Apriori Latin a (from) priori (former)

Deduction from the known

Unexpected patterns and associations

Based on market basket concept

Unusual combinations of diagnosis code on medical insurance claim

JE Accounts

JE Approvals

Credit card fraud in Japan taxi and ATM

First, identify groups, e.g. all medical providers for a patient

Next, for each provider, assign a unique integer value

Create a text file containing the values

Run apriori analysis

For each unique value, probability of other values

If you see Dr. Jones, you will also see Dr. Smith (80% probability)

If you see a JE to account ABC, there will also an entry to account XYZ (30%)

Outliers

Stratification

Day of Week

Round Numbers

Made Up Numbers

Market basket

Trends

Duplicates

Missing documents (sales, cash, etc.)

Inventory losses (missing receiving reports)

Items that walked off

Check any sequence of numbers supposed to be complete, e.g.

Cash receipts

Sales slips

Purchase orders

Excel sort and check

Excel formula

Sequential numbers and dates

Outliers

Stratification

Day of Week

Round Numbers

Made Up Numbers

Market basket

Trends

Duplicates

Sameitems samevendor,sameinvoice number,sameinvoice date,sameamount

Differentitems sameemployee name,samecity,differentsocial security number

High payback area

Fuzzy logic

Overriding software controls

Levenshtein distance

Soundex

Like clause in SQL

Regular expression testing in SQL

Vendor/employee situations

First, sort file in sequence for testing

Compare items in consecutive rows

Extract exceptions for follow-up

Outliers

Stratification

Day of Week

Round Numbers

Made Up Numbers

Market basket

Trends

Duplicates

Red Flag indicator

Difficult to determine

Floating holidays Friday, Saturday, Sunday, Monday

Journal entries

Employee expense reports

Business telephone calls

Invoices

Receiving reports

Purchase orders

Transactions when business is closed

Federal Office of Budget Management

An excellent fraud indicator in some cases

Identifying holiday dates:

Error prone

Tedious

U.S. only

Established by Law

Ten dates

Specific date (unless weekend),OR

Floating holiday

Office of Personnel Management

Example of specific date Independence Day, July 4 th(unless weekend)

Example of floating date Martin Luther Kings birthday (3 rdMonday in January)

Floating Thanksgiving 4 thThursday in November

Programmatically count holidays for entire population

For each group, count holidays

Compare the two histograms (group and population)

Overview

Explain Each Metric

Examples of what it can detect

How to assess results

How pattern detection fits in

Built-in functions

Add-ins

Macros

Database access

Variety of tests

Round numbers

Benfords Law

Outliers

Work with Ranges

=sum, =average, =stdevp

=largest(Range,1), =smallest(Range,1)

=min, =max, =count

Tools | Data Analysis | Descriptive Statistics

Tools | Data Analysis | Histogram

Bin Range

Data Range

Sort by sequential value

=if(thiscell-lastcell 1,thiscell-lastcell,0)

Copy/paste special

Sort by sort values

=if testing

=if(=and(thiscell=lastcell, etc.))

Repeatable process

Audit standardization

Learning curve

Streamlining of tests

Moreefficient and effective

Examples -http://ezrstats.com/Macros/home.html

Many built-in functions right off the shelf with SQL

Control totals

Exception identification

Drill down

Quantification

June 2008 article in the EDP Audit & Control Journal (EDPACS) SQL as an audit tool

http://ezrstats.com/doc/SQL_As_An_Audit_Tool.pdf

Built-in functions

Add-ins

Macros

Database access

How Pattern Detection fits in

Business Analytics

Fraud Pattern Detection

Business Analytics

Fraud Pattern Detection

Continuous fraud pattern detection

Continuous Monitoring

Fraud analytics -> business analytics

Business analytics -> fraud analytics

Fraud analytics can feed(CM)

Continuous fraud pattern detection

Use output from CM to tune fraud pattern detection

Kolmogorov-Smirnov

http://tinyurl.com/y49sec

Benfords Lawhttp://tinyurl.com/3qapzu

Chi Square testshttp://tinyurl.com/43nkdh

Continuous monitoringhttp://tinyurl.com/3pltdl

Apriori testing for ping ponging

Temple Universityhttp://tinyurl.com/5vax7r

Apriori program (open source)http://tinyurl.com/5qehd5

Article Medical ping ponginghttp://tinyurl.com/5pzbh4

Excel as an audit softwarehttp://tinyurl.com/6h3ye7

Selected macros -http://ezrstats.com/Macros/home.html

Spreadsheets forever -http://tinyurl.com/5ppl7t

Phone: (919)-219-1622

E-mail:[email_address]

Blog:http://blog.ezrstats.com

managing the business risk of fraud

Business

objective approach

statistical sampling

plotting data

data talk

sampling techniques

business risk of fraud

data mining fall

effective approach objective