A Discussant’s Comments on Continuous Monitoring of Business

Process Controls: A Pilot Implementation of a Continuous Auditing System at

Siemens by

Alles, Brennan, Kogan and Vasarhelyi

S. Michael GroomerIndiana University

Presented at the University of Waterloo-CICA Information Systems

Assurance SymposiumToronto, Canada, October 21, 2005

2

Introduction This paper reports on one of the first real

world attempts at implementing continuous auditing.

Goal of continuous auditing: To provide assurance on demand without constraints of location, time, and computing platform.

“A methodology for issuing audit reports simultaneously with, or a short period of time after, the occurrence of the related event” (CICA/AICPA, 1999).

3

The Paper Needs a Clearly Defined Purpose, Objectives and Motivation

Purpose: The purpose of this paper is to: Report on a pilot experiment involving the use

continuous auditing at Siemens Corporation. Objectives: The objectives of this paper are

to:1. Describe the necessary support environment for

continuous auditing and the related buy-in by management,

2. Compare two techniques that facilitate continuous auditing,

3. Report on a pilot experiment of continuous auditing at Siemens, and

4. Describe some of the issues, problems and any lessons learned from the pilot.

4

Motivation for the Paper

Motivation: The research is important because it reports on one of the first studies focused on a pilot implementation of continuous auditing.

5

Criteria/Requirements for Continuous Auditing

via Groomer and Murthy

The client must have highly reliable systems. √ The subject of the audit has suitable characteristics

necessary to conduct the audit. √ The auditor must have a high degree of proficiency

in information systems, computer technology, and the audited subject matter. √

Automated audit procedures will provide most of the necessary audit evidence. √

The auditor must have a reliable means of obtaining the necessary audit evidence so that an opinioncan reached. √

The auditor must have timely access to and control over any audit evidence. √

It is necessary to have a highly placed champion to support the adoption and use of continuous auditing. ?

6

Making Continuous Auditing A Reality (1)

Organizational Resistance – where is the win-win? The Bob Elliott (KPMG) Capital Markets Justification

Give me a lower loan rate if I can provide continuous assurance on a set of financial statements.

The Δ between the nominal and effective interest rates could be an amount substantial enough to justify continuous assurance.

The Buy-in at Siemens A value proposition - $100M savings or cost avoidance over 5

years. A process to directly facilitate Section 404 of Sarbanes-Oxley

or alternatively, to free others to work on Sarbanes-Oxley. The work at Siemens is clearly forward thinking and driven by

monetary concerns – “Its all in the numbers.” Is there a buy-in by top level management?

At the end of the day will the benefits out weight the costs?

Are there other measurable justifications for the use of CA?

7

Making Continuous Auditing A Reality (2)

Tool Building -- Tool Availability Substantial effort in building the tools for

Continuous Auditing (EAMs or the Alles, et. al. – MCL).

You need organizational cooperation regardless of the audit approach – you can’t do this work in a vacuum.

Limited availability of “off the shelf” packaged tool sets.

Little aid and comfort from ERP built-in’s like the SAP-AIS.

8

Making Continuous Auditing A Reality (3)

Intrusion into client application systems at some level is “part of the action.” While the read only – external data

transport MCL technique is less “intense” than EAMs, there is still a need for client involvement and some focus for systems intrusion.

Get into the game early – During systems design and not after implementation.

Make no mistake, ERP is likely the place to make continuous auditing a reality at least for now.

9

Making Continuous Auditing A Reality (4)

Internal Audit with CEO/CFO support can make it happen… The tone from the top must be a “we

will do this”! Capitalize on the Golden Age of

Internal Audit. Internal audit needs more

prominence inside the organization. Need to hire and retain skilled IT

Auditors.

10

Operational Concerns for Continuous Auditing – Systems Performance (1)

Frequency of polling the client’s data. Commonly held vision of polling is “real time.” Alles, et. al., address the impact of the real time polling

issue on systems performance. Groomer and Murthy offer a solution to the impact of

continuous auditing on systems performance with the use of Continuous Sampling (see, “Monitoring High Volume On-line Transaction Processing Systems Using a Continuous Sampling Approach.” International Journal of Auditing, Volume 7, No. 1, March 2003, pp. 3-19.).

This research involved the development of a working model for continuous sampling in the environment of Embedded Audit Modules. (See slide at the end of the handout for a vision of how continuous sampling works).

11

Operational Concerns for Continuous Auditing – Evaluating Evidence (2)

Concur with the authors -- There is a need to formalize the scoring processes.

Given the hierarchical control relationships, are there more informative scoring systems?

If you can determine the scoring system, can the scoring be mechanized/automated? The likely answer here is Yes.

Evaluation of the scoring system? What do the numbers mean?

12

Operational Concerns for Continuous Auditing – Materiality (3)

Materiality - The slippery slope. When are controls operating at an acceptable

level? If control events of interest are

(1) monitored on a real time basis and (2) exception reports summarize rule violations --

Then is the materiality issue a straight forward issue? The answer would seem to be YES as we have

audited the population! What is the error rate? If the observed error

rate =< some materiality threshold (the tolerable error rate) then .

13

Operational Concerns for Continuous Auditing – Audit Process (4)

Provide a clear indication of what role CA is playing in the audit process. What assertions or audit objectives

are being tested? Are you testing General Controls or

Application Controls?

14

Summary & Conclusion Clearly articulate the objectives of the research. The article has a bit of the committee flavor in the

exposition. Consider the Continuous Sampling work of

Groomer and Murthy in light of the penalty discussion on systems performance.

The MCL and EAM processes are essentially the same.

Both require significant effort and resources. For this research, error scoring and materiality

should remain issues of interest. I like this paper. Thank you for inviting me to

discuss this work. Keep pushing the envelope!

15

Questions??

Yes

No

No

Application System

Processing the Next Transaction

Have i Consecutive Transactions Been Free

of Defect?

Inspect A Randomly

Selected Fraction (f) of the

Transactions

Has A Defective

Transaction Been Found?

Start Figure 1The CSP-1 Sampling Plan in the

Environment of an Embedded Audit Module

Set i and f for CSP-1 Procedure

Application System Processes

Transaction

Begin (Continue) 100% Inspection and Turn Logging

on in the EAM

Application System

Processing the Next Transaction

Yes

S. Michael GroomerProfessor of Accounting and Information SystemsKelley School of BusinessIndiana University1309 East 10th StreetBloomington, IN 47405-1701812-855-4026groomer@indiana.edu

S. Michael Groomer is Professor of Accounting and Information Systems. He earned his doctoral degree in Accountancy from the University of Missouri at Columbia. He is a Certified Public Accountant (CPA). He is also certified as an Information Systems Auditor (CISA) and as an Information Technology Professional (CITP).

Mike has worked for the Marathon Oil Company, Ernst & Ernst and Touche Ross & Co. (Chicago). At Touche Ross he was employed as computer audit specialist. He has also served as a consultant to KPMG Peat Marwick (Montvale) where he participated in projects dealing with efforts to reengineer the audit process.

Mike teaches undergraduate auditing, as well as accounting systems and IT-Auditing in the Accounting Graduate Program at Indiana. He was the original designer of the information systems orientation that exists throughout this program. He is involved in the curricular use of SAP in the Kelly School. Mike has been recognized for teaching excellence at the national, state and local levels. He is a co-recipient of the American Accounting Association's Innovation in Accounting Education Award, the Indiana CPA Society’s Outstanding Educator award and a number of school and departmental teaching awards including a three-year recognition as the KPMG Peat Marwick Alumni Faculty Fellow. Mike is a co-author of Accounting Information Systems: A Database Approach (www.cybertext.com), the first electronic book in business. CyberText Publishing, Inc. facilitates this book, a company that Mike co-founded. This e-book initiative received the 1998 Innovative User of Technology Award from the Indiana CPA Society.

His research has appeared in the leading accounting journals including ABACUS, Accounting Horizons, Decision Sciences, The Journal of the American Taxation Association, Journal of Accounting Education, The Accounting Educators Journal, The Journal of Information Systems, The International Journal of Auditing, The International Journal of Accounting Information Systems and The Accounting Review. He currently serves as an ad-hoc reviewer on the editorial board of several accounting journals.

Mike is a black belt in Tae Kwon Do (4th Dan) and a black belt in Hapkido (3rd Dan). He makes infrequent attempts at collecting U.S. stamps. He enjoys music and in a past life, played drums in a number of jazz trios and big bands. During the past two summers Mike served as the principal snare drummer in a community concert band. Mike and wife Carolyn, along with daughter Emily (a student at Indiana) reside in Bloomington, Indiana.

17