auditor judgment about misstatement risk at the account level in a risk … · 2015-07-29 ·...

Auditor Judgment about Misstatement Risk at the Account Level in a Risk-Based Audit: A Descriptive Model and Experimental Evidence

Natalia Kotchetova Assistant Professor

University of Waterloo [email protected]

Ed O’Donnell Assistant Professor

University of Kansas [email protected]

R. Alan Webb Associate Professor

University of Waterloo [email protected]

mailto:[email protected]




Abstract

This paper investigates several factors that are likely to influence auditor judgment about the risk of material misstatement at the account level. We develop a descriptive model that includes the associations between account-level misstatement risk judgments and auditors’ selections of accounts for future investigation, and the following features of audit ecology: timing of strategic analysis; level of the client’s business risk communicated by strategic analysis; results of analytical procedures indicating inconsistency in account fluctuations; auditor task-specific experience; and several task structure interventions that affect how auditors may document their judgment during the business-risk audit planning process. We test links in the model in five separate laboratory experiments. Collectively, our findings indicate support for links depicted by the descriptive model, although in a number of cases we determine that associations are more complex that we had originally envisioned. Our results extend work by O’Donnell and Schultz (2005) and suggest that (a) knowledge of business risks can alter the way auditors evaluate account-level risk factors during analytical procedures, and (b) the extent to which business risk assessments moderate the association between account-level evidence and auditor judgment can be influenced by task knowledge and task structure.

1


I. INTRODUCTION

Identification and assessment of business risks based on understanding of the client’s

strategy and operations have become an integral part of the audit planning and evidence

collection process in recent years (ISA 315: IAASB 2005; Bell, Marrs, Solomon and Thomas

1997; Bell, Peecher and Solomon 2005). Although academic research in this area remains in its

infancy, some studies have documented that auditors’ assessments of client’s strategic viability

impact their assessments of misstatement risk at more detailed levels, such as the business

process level (Ballou, Earley, and Rich 2004) and the account level (O’Donnell and Schultz

2005).1 Auditing firms have been trying to find effective ways of integrating such assessments of

business risk into subsequent auditor judgments and decisions (Knechel 2005; Kotchetova,

Kozloski, and Messier 2006). This emphasis on business risk assessment by firms and by

standard setters has been largely inspired by the assumption that developing a better

understanding of conditions that could impair a client’s business model and specific business

processes helps auditors develop more effective knowledge structures for evaluating the risk of

misstatement in accounts (Eilifsen, Knechel, and Wallage 2001). The richer mental model that

auditors develop by learning about and assessing business risks is supposed to help them

recognize patterns of evidence that increase the likelihood of financial misstatement (Bell,

Peecher, and Solomon 2002).

1 Sometimes this effect of strategic assessment on subsequent auditor judgments is termed “halo effect” because it is viewed as an unintended consequence of carrying strategy-level judgments to the more detailed levels, to the point of disregarding additional, account- or process-level information (O’Donnell and Schultz 2005; Ballou et al. 2004). In this paper, we do not take a position on desirability of “halo effect” in an auditing context; we simply investigate what factors, other than strategic viability assessment, impact auditors’ judgments of misstatement risk at the account level.

2

In addition to the multi-level risk assessment process as mandated by the current audit

risk standards, auditors use analytical procedures to assess the likelihood that accounts are

misstated (ISA 315, 520: IAASB 2005). During analytical procedures, auditors use their

knowledge of client strategy and operations to analyze inter-period changes in account balances

and identify inconsistencies that increase the likelihood accounts could be misstated (Koonce

1993; ISA 520: IAASB 2005). Auditors rely on the risk assessments and results of analytical

procedures to help them design an effective program of substantive audit tests (Libby 1985).

Proponents of business risk assessment believe that developing a richer mental model of client

operations by learning about business risks before analyzing accounting details should help

auditors recognize inconsistencies in account-level evidence when they perform analytical

procedures (Bell et al. 1997; Bell et al. 2005).

Research on the association between business risk assessment and auditor judgment

about the risk of financial misstatement has been sparse. However, studies have found that

learning about business risks could hinder auditor ability to recognize and effectively respond to

diagnostic patterns of fluctuations in performance metrics (Ballou et al. 2004) and account

balances (O’Donnell and Schultz 2005). Although these findings contradict the assumption that

knowledge of business risks can improve auditor judgment, results from these studies are

difficult to interpret without a descriptive framework for integrating and evaluating experimental

evidence about the association between business risk assessments and auditor judgment about the

likelihood of misstatement in accounts.

In this study, we develop a model that describes how various factors, including entity-

level business risk assessments, account-level factors, auditors’ task-specific knowledge, and

firm-specific documentation procedures influence auditor judgments about account-level

3

misstatement risk. We test the associations specified in our model with evidence gathered from

five laboratory experiments. Consistent with our descriptive model, findings suggest that (a)

knowledge of business risks can alter the way auditors evaluate account-level risk factors during

analytical procedures and (b) the extent to which business risk assessments moderate the

association between account-level evidence and auditor judgment can be influenced by task

knowledge and task structure.

The remainder of this paper is organized into three sections. The next section presents our

descriptive model and explains the theoretical foundation on which it was built. Section three

describes the experiments we used to test associations specified in our descriptive model and

reports the results of each experiment. The fourth section summarizes the evidence provided by

our experiments, and discusses the implications of our findings for audit research and practice.

II. DESCRIPTIVE MODEL

Before auditors can use analytical procedures to evaluate account-level risk factors, they

develop a “mental model” of client business by integrating the client-specific information they

acquire during an audit engagement, including entity-level risk assessments, with the task-

specific knowledge, developed through training and experience (Koonce 1993; Choy and King

2005). This “mental model” provides auditors with a basis for evaluating account-level risk

factors and recognizing evidence of misstatement (Hylas and Ashton 1982). It also determines

the nature and extent of the influence that each piece of evidence has on auditor judgment. In the

context of analytical procedures, the mental model that auditors develop dictates whether, and to

what extent, observed patterns of fluctuations in related accounts increase the likelihood of

misstatement for a target account.

4

Based on findings from studies of auditor judgment, we believe variables that influence

auditors’ judgment of the risk of material misstatement in an account fall into four categories: (1)

account-level factors identified through analytical procedures; (2) task-specific knowledge; (3)

firm-specific documentation procedures and task structure; and (4) entity-level risk factors

identified through strategic analysis, identification and assessment of client business risk, and

their implications for the financial statements. Figure 1 illustrates how these variables can affect

the extent to which evidence of potential misstatement can influence auditor judgment about

account-level misstatement risk. Figure 1 also identifies the specific constructs from each

category that we examined in this study.2

Insert Figure 1 about here

Link One: Account-level Factors

Figure 1 suggests that the knowledge-based “mental model” that auditors develop by

learning about a client business dictates how they evaluate and integrate account-level evidence

into their judgment about the likelihood that accounts are misstated. While this “mental model”

can be influenced by several factors discussed below, the impact of account-level evidence on

account-level assessments of the risk of material misstatement is stipulated by standards (ISA

315, 500, 520: IAASB 2005) and is well documented by researchers (e.g., Asare and Wright

1997; Hirst and Koonce 1996). The account-level evidence can be represented by patterns in

account balances or account fluctuations that emerge during analytical procedures. Other

2 We acknowledge that some of the variables in each of these four categories could have a direct influence on auditor judgment during analytical procedures. Furthermore, in addition to the direct or moderating influence of these variables on account-level misstatement risk assessments, some of the variables in these four categories could moderate the influence that variables from other categories have on auditor judgment about misstatement risk or program design. Although a model that illustrates all potential associations among variables (e.g. main effects and two- or three-way interactions) would provide a more rigorous description, we have chosen to present a more parsimonious model of associations among the specific variables that were examined in this study.

5

examples of account-level information may include information about account-specific internal

controls, or inherent complexity of accounting rules related to an account, such as estimates and

fair value measurements. Therefore, Link 1 in our model suggests that account-level factors,

such as fluctuations in account balances determined through analytical procedures, influence the

assessment of the risk of material misstatement in the target account.

Link Two: Auditors’ Task-specific Knowledge

Auditors’ task-specific knowledge should influence how account-specific information is

incorporated into account-level misstatement risk assessments (Ashton and Ashton 1995; Bonner

1990; Knapp and Knapp 2001). In the context of misstatement risk assessment for a specific

account, the relevant task knowledge is developed through experience in audit planning, which

can be proxied by experience as a senior or in-charge auditor.3 For example, Bonner (1990)

demonstrates that task-specific knowledge aids the performance of experienced auditors in both

the cue selection and cue weighting components in an analytical risk assessment task. Also,

Tandy (1992) shows that that characteristics of the auditor, including the auditor's knowledge of

the client and the client's industry, the auditor's knowledge of analytical procedures, and the

auditor's perception of such techniques, are instrumental in the decision to use and rely upon

analytical procedures. In addition, auditing standards suggest that assignment of more

experienced personnel can be used as a response to client-specific factors that are deemed to

increase the risk of material misstatement in the financial statements (ISA 330: IAASB 2005).

Therefore, Link 2 in our model suggests that auditor’s task-specific knowledge, operationalized

3 See Abdolmohammadi (1999), and Abdolmohammadi and Usoff (2001) for inventory of audit tasks and what level of personnel typically performs them.

6

as experience as a senior or in-charge auditor, moderates the extent to which account-level

factors influence assessments of the risk of material misstatement in an account.4

Link Three: Firm-specific Documentation Procedures

Another factor that has been shown to impact auditor judgment is task structure. Given

the relatively recent adoption of the audit standards underlying the risk assessment process in a

risk-based audit (IAASB 2005, AICPA 2006) and variance in accounting firms’ particular

approaches to auditors’ understanding of the client business (e.g., Salterio, Knechel, and

Kotchetova 2006) and performing analytical procedures (e.g., Lin and Fraser 2003), it is

reasonable to expect some variation in documentation requirements associated with analytical

procedures and risk assessments. Some variants of such documentation requirements may, in

fact, work as interventions introducing additional structure in the risk assessment process. Prior

research suggests an inverse relationship between task complexity and task performance for

skill-intensive tasks (Campbell and Ilgen 1976; Iselin 1988; Paquette and Kida 1988; Bonner

1994).5 Human information processing research proposes the use of structuring techniques as a

means of overcoming cognitive overload in performing a complex task (Kleinmuntz 1990;

Hogarth 1991). Auditing studies have documented the performance-increasing effects of

structure in complex tasks (Kachelmeier and Messier 1990; McDaniel 1990; Zimbleman 1997;

Messier et al. 2001). More recently, Kotchetova et al. (2006) show that formal documentation of

the business process analysis influences auditors’ assessments of the business risk associated

with the process, which in turn, affect their assessments of the risk of material misstatement in

4 We do not investigate industry specialization as a task-specific knowledge proxy in this study because we used an experimental case requiring “generalist” auditor knowledge (Greenwood and Salterio 2002). However, based on prior research in auditing, we recognize that industry specialization and industry experience are very likely to have an effect on the auditor’s mental model developed during the risk assessment and planning process (e.g., Hammersley 2005; Owhoso et al. 2005). 5 Risk assessment is a task of medium-to-high complexity generally performed by audit seniors (Abdolmohammadi 1999; Abdolmohammadi and Usoff 2001).

7

accounts and transactions affected by the process. Therefore, Link 3 in our model suggests that

documentation requirements and other firm-specific procedures influencing structure of the risk

assessment process may have an effect on how auditors incorporate account-level factors

identified through analytical procedures into their assessments of the risk of material

misstatement for the target account.

Links Four, Five, and Six: Entity-level Risk Factors

Notwithstanding the associations depicted by Links 1-3, one of the main inputs into

account-level misstatement risk assessment is the auditor’s understanding of entity-level risk

factors (ISA 315: IAASB 2005; Bell et al. 1997; Bell et al. 2005). Entity-level risk factors

become known to the auditor through, but not limited to, strategic analysis of the client’s

business, analysis of its operations, and understanding of the entity-level controls, including the

strength of the control environment and governance mechanisms (ISA 315: IAASB 2006;

Messier, Glover, and Prawitt 2006). Research has already provided some evidence regarding

associations between the auditor’s assessment of the client’s business risk based on the strategic

analysis and risk assessments at the more detailed, account level. For example, O’Donnell and

Schultz (2005) (OS) examine how different levels of entity-level business risk moderate the

association between inconsistent patterns of account fluctuations and auditor judgment about the

likelihood of misstatement in those accounts. Results from their study provide evidence that

business risk assessments influence how auditors evaluate detailed evidence about client’s

financial performance.6

6 Also, Ballou et al. (2004) found that risk assessments that auditors developed by performing analysis of a core business process influenced the way they evaluated performance metrics for critical success factors when they performed analysis to assess risk for another, unrelated core business process. Results from their study provide evidence that business risk assessments influence how auditors evaluate detailed evidence about process performance.

8

Two findings from OS support associations presented by Links 4 and 5 in Figure 1. First,

auditors who assessed strategic business risks before they performed analytical procedures

evaluated patterns of changes in accounts differently from auditors who did not assess business

risks based on strategic analysis prior to performance of analytical procedures. Auditors who

evaluated business risks based on results of strategic analysis provided to them were less

concerned about the risk of misstatement associated with inconsistent patterns of fluctuations

than auditors who did not evaluate business risks. Second, for auditors who evaluated strategic

business risks, their rating of the likelihood that the account with the inconsistent fluctuation was

misstated increased as business risk assessments increased.

Using the OS experiment as a starting point, we suggest that auditors who evaluate

business risks before they perform analytical procedures would come to different conclusions

about anticipated audit effort than auditors who do not evaluate business risks. OS test whether

assessing strategic business risk influenced account-level risk assessments, i.e. Links 4 and 5.

We extend their logic and suggest Link 6, from account level risk of material misstatement to

audit program design. Consistent with auditing standards on the response to assessed risk of

material misstatement in an account, assessing strategic business risk should influence auditor

perceptions about the level of testing that would be needed to substantiate the account balance, a

decision that auditors make based on their misstatement risk assessments (ISA 330: IAASB

2006). Auditing research likewise suggests that audit programs are responsive to changes in

account level risks (Biggs, Mock, and Watkins 1988; Bedard, Mock, and Wright 1999; Mock

and Turner 2005). However, it is important to examine these linkages in the context of other

factors presented by Links 1, 2, and 3. The next section explains how we tested links in our

model using five different laboratory experiments.

9

III. METHOD AND RESULTS

We test various aspects of our theoretical model using five laboratory experiments. The

first three experiments are used to replicate and extend the results reported by OS. In particular,

we focus on linkages suggested by our model that were not considered by OS. The final two

experiments examine the effects of several interventions intended to influence the factors

auditors consider when making risk assessments. We build on our re-analysis of OS results in

designing these experiments and analyzing the results.

All experiments, except one portion of experiment two (discussed below), were

administered at national training sessions for entry-level and senior-level auditors for one Big 4

firm. Groups of about 30 auditors completed the experimental task in their classroom during a

one-hour time slot that the firm had reserved for research purposes. Each session was completed

under the supervision of one of the co-authors or a research proctor. Given the differences in

experimental conditions across the five experiments, completion time ranged from about 20

minutes (experiment one) to 50 minutes (experiment four). The extent of participants’ general

audit experience and experience as an in-charge senior also varies across the five experiments.

As reported in Table 1, auditors in study four had the least experience both as an auditor (1.8

years) and as an in-charge senior (.5 years) while those in study five had the most (respectively,

3.6 years and 1.8 years).7 We use this variation to test for the experience effects predicted in

Figure 1.

Insert Table 1 about here

We used a variant of the strategic auditing case employed by OS in each of our five

studies. The case was patterned after the Loblaw Companies case for a grocery store chain

7 Because the firm assigned participants to researchers conducting studies at the training sessions, we did not have control over the seniority level of the auditors who completed our materials.

10

(Greenwood and Salterio 2002). Each version of our materials contained information about: the

background of the company and the audit; the results of the auditors’ strategic analysis; client

operations grouped by strategic business processes; and comparative account balances for four

key accounts (sales, cost of sales, merchandise inventory and store expenses) used for analytical

procedures. Auditor judgments were based on the analytical procedures they performed in the

case and included: identifying whether specific accounts would be targeted for further

investigation; expected changes in audit effort for each of the four accounts; risk assessments at

the account level; and assessments of the overall risk of material misstatement. All cases

indicated that no significant changes had been made to unit sales price, sales mix or product

costs compared to the prior year. This suggests changes in cost of sales should be proportional to

any changes in total revenue. As per OS, this approach allowed us to manipulate whether the

fluctuations in sales and cost of sales provided to auditors were consistent with this information.8

The order and format in which some of the information was provided, the specific judgments

made by the auditors, and the consistency in account balance fluctuations varied across the five

experiments. We discuss the specifics of each experiment below and provide a summary of the

key research design features of each experiment in Table 1.

Experiment One

The purposes of experiment one are to assess: whether the results reported by OS

(experiment one) generalize to specific decisions about audit program design (Links 5 and 6 in

Figure 1); and the degree to which auditor experience influences these decisions (Link 2 in

Figure 1). The only manipulated variable is whether auditors received the strategic analysis

information before or after performing analytical procedures. The two dependent variables

8 See OS (p. 927) for a more complete description of the analytical procedures task used in the case and the approach used to create a seeded inconsistency in the data.

11

measured in the experiment are: auditors’ decision whether or not to target cost of sales for

further investigation (yes/no); and their assessment of the overall risk of material misstatement

(hereafter OverallRMM). The risk assessment was measured using a 9-point scale ranging from

“low risk of misstatement” (1) to “high risk of misstatement” (9).

Procedures

We use an abbreviated version of the OS materials for experiment one. Auditors in the

“received strategic analysis first” condition read information about the audit engagement, the

client’s background and the strategic analysis. The strategic analysis information consisted of

two paragraphs describing the client’s business strategy and five key performance indicators for

strategic objectives including current and prior year’s performance, and an industry best

benchmark. The pattern of performance on the key indicators was generally favorable. After

reading the strategic analysis information, auditors provided an assessment of the viability of the

client’s business strategy on a 9-point scale with endpoints “low likelihood of success” (1) and

“high likelihood of success” (9). Next, participants received information on key performance

indicators for operating activities including sales, cost of sales, merchandise inventory (current

and previous year, dollar change, percentage change). All cases contained an inconsistent

fluctuation between sales (increase of 5.2% over prior year) and cost of sales (increase of .9%

over prior year). After reviewing this information, participants indicated whether or not they

would target each of the four accounts for further investigation. Finally, participants answered

several debriefing questions related to their audit experience and reactions to the case materials.

Participants in the “received strategic analysis second” condition, completed the same

requirements as their counterparts, but received the strategic analysis information (and assessed

12

the viability of the strategy) after performing the analytical procedures and providing their

decisions on targeting the four accounts for further investigation.

Results

A total of 102 auditors with an average of 3.0 years of auditing experience and 1.6 years

as in-charge seniors completed experiment one. On average, the task took about 20 minutes to

complete. As shown in Table 2, Panel A, 42% (21 of 50) of auditors in the “received strategic

analysis first” condition indicated they would target cost of sales for further investigation

compared to 65% (34 of 52) in the “received strategic analysis second” condition. This pattern

of results indicates that, consistent with OS, receiving favorable information about strategic

performance prior to performing analytical procedures, impacts auditors’ judgments about

specific audit program features.


Panel B of Table 2 presents the results of a logistic regression used to evaluate the impact

of timing of strategic assessment and auditor experience (months as an in-charge senior) on

decisions to target the cost of sales account for further investigation. We use months as an in-

charge senior our proxy for auditor task-specific knowledge because, as discussed earlier with

respect to Link 2 in our model, senior associates frequently perform risk assessment and

planning tasks on audit engagements (Abdolmohammadi and Usoff 2001). Only the timing of

strategic analysis is significant (p = 0.017). The positive coefficient indicates that those who

received strategic analysis second (coded as “1”) were more likely to target cost of sales for

further investigation than those who received the analysis first (coded as “0”).9 The effect of

auditors’ experience is not significant effect (p = 0.747). Panel C of Table 2 summarizes the OLS

9 We use the same coding for this variable in all experiments in which it is manipulated.

13

regression used to assess the effects of strategic assessment timing and auditor experience on

OverallRMM. Results show that neither independent variable is significant.

Summary

The results of experiment one indicate that the key findings reported by OS (experiment

one) and depicted in Link 5 in Figure 1 generalize to decisions about specific features of the

audit program (Link 6 in Figure 1). Auditors that receive favorable information about strategic

performance are less likely to target accounts for further investigation, even though the pattern of

account balance fluctuations indicates misstatement is likely. We find no evidence that auditor

experience affects these audit program decisions (Link 2 in Figure 1). We also find no evidence

that the effects of strategic assessment timing generalize to overall misstatement risk

assessments.

Experiment Two

Our second experiment includes the results reported by OS for their experiment one (pp.

928-932) and further data we collected using more junior auditors as participants to get a wider

variation in experience and re-test Link 2 in Figure 1.10 All participants in OS were audit seniors;

none of our additional participants had any experience as in-charge seniors and averaged 9.5

months of audit experience. They were all graduate accounting students enrolled in a co-

operative education program. Their average audit experience is significantly less (p < 0.001) than

the 2.9 years of experience reported by OS for their sample of more senior auditors. Our

additional participants completed the same experiment described by OS during class time under

10 We also include data for 52 participants who completed an additional experimental condition not reported by OS. The condition presented client information in a traditional financial statement format as opposed to business process focus. Because our theoretical model has no predictions related to the focus of the information presentation, we include this data but collapse results across these conditions. Analysis (not tabulated) indicates no main or interaction effects involving information “format.”

14

the supervision of one of the co-authors. On average they took about 45 minutes to complete the

task.

The 2 x 2 research design is identical to that reported in OS (p. 927). The manipulated

variables are: timing of strategic analysis (before or after analytical procedures) (Links 4 and 5 in

Figure 1), and the presence or absence of a seeded inconsistency in the sales and cost of sales

fluctuations (Link 1 and in Figure 1). In the “seeded inconsistency” condition, sales increased by

5.2% and cost of sales by .9% while in the “no seeded inconsistency” condition, sales increased

by 5.2% and cost of sales 4.9%. The dependent variable is auditors’ misstatement risk

assessments for cost of sales in year two (hereafter COSRMM2).

Procedures

All procedures are as per OS (pp. 927-928) and only a brief summary is reported here.

After reading background information about the audit engagement, control risk, the client’s

industry and business operations, and key business processes, participants provided a baseline

misstatement risk assessment for sales, cost of sales, merchandise inventory, and store expenses

based on un-audited balances for the current year-end. Next, the independent variable

manipulations were presented with participants completing either the strategic analysis or the

analytical procedures phase.11 As part of the analytical procedures phase, current and prior year’s

balances for the four accounts (including cost of sales) were provided along with the dollar and

percent change for each. Completion of the analytical procedures phase required participants to

document their risk assessments for each of the four accounts. The final phase of the experiment

was the completion of a debriefing questionnaire.

Results

11 Details of the strategic analysis phase are the same as those reported above for our experiment one.

15

A total of 258 auditors completed experiment two, including an additional 113 less

experienced auditors described above. Table 3, Panel A presents descriptive statistics for the

complete set of participants and the pattern of means is consistent with OS (p. 929). When the

strategic analysis was presented before (after) analytical procedures, the mean for COSRMM2 is

4.1 (4.5) indicating that favorable information about strategy, when reviewed first, results in

lower risk assessments. We also find that COSRMM2 is higher in the presence of the seeded

inconsistency (4.5 versus 3.7).


Table 3, Panel B, extends the analysis reported included in OS by using OLS regression

to examine some of the relations among variables suggested by our model. Consistent with the

descriptive statistics reported in Panel A, both the timing of strategic assessments (SATiming, p =

0.058) and the seeded inconsistency (Seeded, p = 0.001) have a significant effect on COSRMM2

in the expected direction. Therefore, we find support for Links 1 and 5 in our model. We also

find that the auditors’ level of experience as an in-charge senior (SeniorMths, p = 0.068) and the

initial baseline cost of sales risk assessments (COSRMM1, p < 0.001), have a significant effect

on COSRMM2. Auditor experience and the baseline risk assessment (COSRMM1) (collected

before manipulation of the independent variables) are both positively associated with the year-

two risk assessments, indicating support for Link 2 in the descriptive model. Inclusion of

interaction effects involving either SATiming or Seeded causes multicollinearity problems.12

Accordingly, we run separate regressions for each of the SATiming conditions to examine

whether the overall model presented in Panel B, is comparable in each condition.

12 The variance inflation statistics are well in excess of the cut-off of 10 suggested by Stevens (1996) as indicative of multicollinearity problems.

16

Respectively, Panels C and D of Table 3 present results for the “received strategic

analysis first” and “received strategic analysis second” conditions. Comparison of the two

regression models indicates that the effects of both Seeded and SeniorMths are stronger when the

strategic analysis is received after auditors made their account-level risk assessments (Panel

D).13 This suggests that receiving information that strategic performance is generally favorable

makes auditors less sensitive to any unusual account fluctuations that may be present and reduces

the tendency for more experienced auditors to assess risk at higher levels. Hence, links 2 and 4

in Figure 1 receive experimental support.

Table 3, Panel E presents our analysis of the factors affecting auditors’ year-two

assessment of the overall risk that the financial statements were materially misstated

(OverallRMM2). Only the year one overall misstatement risk (OverallRMM1) assessment is

significant (p < 0.001).

Summary

Using an expanded data set we replicate the key findings reported by OS for their

experiment one. We extend their findings showing a significant positive association between

experience (SeniorMths) and COSRMM2. We also present preliminary evidence suggesting the

timing of strategic analysis affects account-level risk assessments by moderating the influence of

the seeded inconsistency. In terms of our descriptive model, we find support for Links 1, 2, 4,

and 5.

Experiment Three

Our third experiment includes the results reported by OS for their experiment two (p. 935),

as well as additional data they collected but did not report. Our objective in experiment three is

13 This does not appear to be a power issue as the total number of participants in the “received strategic analysis second” conditions is 90 compared to 166 in the “received strategic analysis first” conditions.

17

to develop a better understanding of the effects documented by OS in the context of our

theoretical model. Experiment three, including the additional data, is a 2 x 2 x 2 between subjects

design.14 The outcome of the strategic analysis provided to participants was either favorable (low

strategic risk) or unfavorable (high strategic risk), again testing Links 4 and 5 in Figure 1. The

presence or absence of a seeded inconsistency in the account balances is the second factor and

relates to Link 2. The final factor is the presence or absence of a requirement for participants to

document whether they expected selected account balances to increase or decrease in year two

over year one. This factor represents Link 3 in our descriptive model.

Procedures

Details of the experimental procedures are reported in OS (pp. 933-934), we present only

an overview here. All participants provided assessments of account-level misstatement risk and

overall risk of material misstatement for two consecutive years of the same audit client. The

experiment was conducted in three phases. During phase one, participants received background

information about the company including the overall engagement risk, the business strategy,

business process analyses and control risk assessments. They were then provided with

comparative balances for key performance indicators and documented their initial assessments of

account-level misstatement risk and the overall risk that the financial statements were materially

misstated.15 There was no seeded inconsistency in the year-one account balances. During phase

two participants received information about: strategic risk (high or low); engagement risk

(increase over year one); and control risk (increase over year one). Next, participants in the

"document expectations" condition were required to indicate expected changes in account

14 The design is not fully crossed; all participants in the “documented expectations” condition received information with a seeded inconsistency in the account balance fluctuations. 15 All account-level risk assessments in experiment three used a 101-point scale with 0 indicating “very low risk” and 100 indicating “very high risk.” The overall risk of material misstatement used a 7-point scale with 1 indicating “very low risk” and 7 “very high risk.”

18

balances (see above) using a 7-point scale (1 = decrease significantly; 7 = increase significantly).

Finally, all participants received year two account balances (seeded inconsistency present or

absent) and provided the second set of risk assessments (account-level and overall). Phase three

consisted of debriefing questions.

Results

The complete sample for experiment three consists of 145 auditors with an average of 2.6

years (1.0 years) of general audit (in-charge senior) experience. Descriptive results for all

conditions are presented in Table 4, Panel A. The top portion of Panel A summarizes the results

for participants required to document expected account balance changes prior to providing their

year two risk assessments.16 COSRMM2 is higher when strategic risk is high (49.4 versus 46.4)

but OverallRMM2 does not differ across the strategic risk conditions. The bottom portion of

Panel A summarizes the results for participants not required to document expectations prior to

providing year-two risk assessments.17 Neither COSRMM2 nor OverallRMM2 appear to have

been influenced by the strategic risk manipulation. COSRMM2 (OverallRMM2) is 39.8 (4.1)

when strategic risk is high compared to 40.2 (3.9) when it is low. As expected, the seeded

inconsistency affected both risk assessments. COSRMM2 (OverallRMM2) is 37.4 (3.9) in the

absence of a seeded inconsistency increasing to 42.5 (4.1) when it is present. Overall, these

results suggest that the requirement to document expectations (Link 3) was necessary to facilitate

the effects of strategic risk on account-level risk assessments.18 Conversely, the impact of the

seeded inconsistency does not appear to be dependent on the requirement to document

16 We also performed extensive analyses using OverallRMM2 as the dependent variable. The only significant factor influencing these risk assessments is the year one overall risk assessments. Given this, we decided to focus our results presentation and discussion on the cost of sales risk assessments. 17 Within this condition, strategic risk (high or low) and seeded inconsistency (present or absent) are fully crossed between subjects. 18 See footnote 2.

19

expectations.


Table 4, Panel B extends the analysis presented in OS by using OLS regression to examine

some of the relations among variables suggested by our theoretical model. Three key results

emerge from this analysis. First, consistent with results from experiment two, COSRMM1 has a

significant (p < 0.001) positive influence on COSRMM2. Second, there is a significant

interaction (p = 0.013) between the seeded inconsistency and SeniorMths. Finally, (as suggested

by the descriptive results in Panel A) there is a significant interaction (p = 0.018) between

strategic risk and the requirement to document expectations about account balances.

To examine the nature of the Seeded x SeniorMths interaction, we first performed a median

split based on months of experience as an in-charge senior. The higher experience auditors have

an average of 21 months of experience as a senior compared to about 3 months for the lower

experience auditors and this difference is significant (p < 0.001). For each experience group, we

then compare COSRMM2 in the two seeded inconsistency conditions (present versus absent).

Results (not tabulated) indicate that low experience auditors did not respond to the seeded

inconsistency manipulation. COSRMM2 averages 40.2 when the seeded inconsistency is present

and 40.9 when it is not; this difference is not significant (p = 0.88). Conversely, for the high

experience auditors, COSRMM2 averages about 32.5 when the seeded inconsistency is absent but

increases to 42.8 when it is present. This difference is significant (p =0.029).

Analysis of the significant StratRisk x DocExp interaction is consistent with the pattern of

means presented in Table 4, Panel A. Results (not tabulated) indicate that, adjusted for

COSRMM1, COSRMM2 differs significantly between the high and low strategic risk conditions

only when the requirement to document expectations is present. When the requirement to

20

document is present, COSRMM2 (adjusted for COSRMM1) averages 52.1 and 43.9 respectively

for the high and low strategic risk conditions and this difference is significant (p < 0.05). When

the requirement to document is absent, COSRMM2 means are 38.6 and 41.4 respectively for the

high and low strategic risk conditions and this difference is not significant.

Finally, to gain a better understanding of how strategic risk influences account level risk

assessments, we conduct a series of OLS regressions, summarized in Table 4, Panels C-E. We

include only the conditions where auditors documented expectations about account balance

changes since strategic risk has no effect in the absence of this requirement. Panel C results

indicate that both StratRisk (p = 0.039) and COSRMM1 (p < 0.001) affect COSRMM2. We then

use the regression models in Panels D and E to separately evaluate the impact of COSRMM1 in

the high and low strategic risk conditions.19 A comparison of the coefficients for COSRMM1

indicates that when strategic risk is high, auditors are influenced less by their initial risk

assessments (p = 0.019) than when strategic risk is low (p = 0.003) even though the seeded

inconsistency was present in both conditions. Accordingly, the results in Panels D and E indicate

that when strategic risk is low, the auditors relied more on their initial risk assessments than on

information indicating an unusual fluctuation in account balances.

Summary

Our re-analysis of the complete OS data set yields two substantive insights. First, we find

that task relevant experience moderates the effects of account-level risk factors on the related

risk assessments. More experienced auditors in our sample are more sensitive to the presence of

a seeded inconsistency in account balances. Therefore, we find support for the Links 1 and 2 in

19We initially included a StratRisk x COSRMM1 interaction term in the model but this resulted in multicollinearity problems (variance inflation factors > 10). This led to the use of separate regression models in the high and low strategic risk conditions.

21

our model. Second, strategic risk appears to moderate the effects of a seeded inconsistency on

account-level risk assessments by changing the extent to which auditors rely on initial

impressions of misstatement risk. This result suggests existence of Link 4, in addition to Link 5

that was tested in experiments one and two. When strategic risk is low, auditors account-level

risk assessments are less influenced by unusual fluctuations in the account balances.

Experiment Four

Based on the analysis reported above for experiments two and three, the objective of

experiment four is to identify and test interventions and documentation requirements intended to

alter the influence of strategic risk on account-level risk assessments (Link 3 in Figure 1). We

rely on Kennedy's framework (1993, 1995), which identifies ways in which judgments can be

improved. As discussed above, we focus on interventions aimed at improving the external data

available to the auditor (e.g., presentation format of the information that makes the task more

structured) or at improving the internal data used in making the risk assessments.

We use the same task as OS experiment two and have four conditions. In all conditions

strategic risk is low and there is a seeded inconsistency present in the account balances because

this is the scenario where our experiment three analysis indicates auditors may be less sensitive

to unusual fluctuations in accounts.20 We use no interventions in our base condition. We have

one condition with an external data related intervention, and one with an internal data related

intervention, both discussed below. Our fourth condition presents the strategic risk information

for the second year after the provision of account balance information for year-two, but prior to

the assessment of misstatement risk for those accounts. This differs from the design used in

experiments one and two reported above where strategic risk information was provided either

20 We use the same seeded inconsistency as OS experiment two. Sales increase by 3.2% year two over year one while cost of sales increase only .9%.

22

before the presentation of the year-two account balance information or after the year-two

account-level risk assessments had been made. This approach makes it impossible to conclude

whether the significant timing effects we report above are attributable to the effects of strategic

risk on the acquisition of account-level fluctuation evidence or on the evaluation of that

evidence.

Procedures

We employ procedures identical to those described above for experiment three.

Accordingly, we focus our discussion on details of the interventions. Our external data related

intervention (condition two) is designed to increase the salience of the inconsistent account

balance fluctuation present in our data. In the base condition the accounts containing the

inconsistent fluctuations (sales and cost of sales) are presented in two separate tables with a

paragraph of text between them (Figure 2, Panel A). Our intervention highlights the seeded

inconsistency by grouping all four accounts in one table and asking participants to document

their risk assessments in an adjacent column (Figure 2, Panel B). Our internal data related

intervention (condition three) is intended to encourage greater use of information from the

process analysis details in the case that is relevant to account-level risk assessments. We instruct

participants to document all information about client operations and account balances they

believe relevant to assessing account-level misstatement risk. By doing so, we believe

participants will rely less on their initial risk assessment (year one) and place greater emphasis

on risk increasing factors suggested by the seeded inconsistency. Our final intervention

(condition four) provides the strategic risk information after the account level details but before

they make their account-level risk assessments. The base case presents information on strategic

risk prior to the account-balance details. Thus, we test Link 3 of our model using three different

23

versions of interventions that can be seen as task structure and documentation requirements

which accounting firms potentially could apply to the risk assessment process.

Results

Descriptive results are presented in Table 5, Panel A. The pattern of means for COSRMM2

show little variation across conditions, and none of the pair-wise comparisons between

conditions are significant (all p values> .30). A similar pattern of results exists for

OverallRMM2. However, to test Link 3 suggested by our theoretical model, we use the OLS

regression reported in Panel Busing COSRMM2 as the dependent variable. The results show that

COSRMM1 (p < 0.001) and its interaction with our external data-related (ExtData) intervention

(p = 0.022) are both significant.21 To examine the nature of the significant interaction, we ran

separate regressions for the base condition (no intervention present) and the condition with our

external data related intervention. The results are presented in panels C and D respectively. A

comparison of the coefficients for COSRMM1 in the two panels indicates that when our

intervention is absent, the initial risk assessments have a significant influence (p = 0.001) on

COSRMM2 but not when it is present (p = 0.832).22


We perform similar analysis using OverallRMM2 as the dependent variable. The results are

reported in Table 6, Panels A-C. The coefficients for OverallRMM1, ExtData and the ExtData x

OverallRMM1 interaction are all significant (respectively, p < .001, p = 0.069 and p = 0.042).

We again use separate regressions in the base condition (panel B) and the external data-related

intervention condition (panel C) to examine the nature of the interaction. A comparison of the

21 In additional analyses not reported, we also included interactions involving SeniorMths. None of the interaction terms is significant so we have omitted them from Table 5 to simplify reporting. 22 It seems unlikely that differences in statistical power between the two sub-samples are the cause of this result since they are approximately equal in size (n = 21, n= 22).

24

coefficients for OverallRMM1 in the two models shows that in the absence of our intervention,

the initial risk assessments significantly influence (p = 0.002) OverallRMM2. However, when the

intervention is introduced, OverallRMM1 becomes non-significant (p = 0.209).


Summary

Results replicate a key finding from experiment three that in the absence of any

interventions, when strategic risk is low, auditors tend to rely on initial assessments of risk when

assessing risk in subsequent periods. This result holds despite the presence of a seeded

inconsistency. However, introducing an intervention designed to increase the salience of the

inconsistent account fluctuations significantly moderates the influence of the year-one risk

assessments (account-level and overall), supporting Link 3 in our model. Neither our internal

data related intervention nor our manipulation of the timing of strategic information presentation

affected risk assessments. This latter finding suggests that the influence of strategic risk is on

auditors’ evaluation, not acquisition of evidence.23

Experiment 5

Experiment five continues our examination of interventions intended to alter the effects

of strategic risk on auditors’ account-level risk assessments. Results from experiment four

indicate that clarifying account balance fluctuations can lead to less reliance on initial risk

assessments. However, requiring auditors to document all factors relevant to their risk

assessments was unsuccessful. We identified two reasons for this lack of success. First, our

participants may not have been willing to exert sufficient effort to document and evaluate all

23 If our timing manipulation had impacted risk assessments, this would suggest that strategic risk affects the acquisition of evidence. This is because in our base (timing intervention) condition strategic risk information was provided before (after) the year-two account-balance information. However, in both conditions, strategic risk information was presented before the year-two risk assessments were performed.

25

factors relevant to their risk assessment because the task is simply too long.24 To address this

issue, we significantly shortened the background information and required participants to

perform risk assessments for one year, instead of two. Second, consistent with OS (experiment

two) in experiment four our seeded inconsistency involved a sales increase of 3.2% and a cost of

sales increase of 0.9%. This inconsistency may not have been large enough to provide an

opportunity for our internal-data related intervention to work. Accordingly, in experiment five,

we revert to the larger seeded inconsistency used in experiment one reported by OS where the

sales increase is 5.2% and the cost of sales increase is 0.9%.

Experiment five has four conditions each of which contains the larger seeded

inconsistency described above (Link 1 in Figure 1). In two of the conditions, we do not use any

interventions and manipulate only strategic risk (low or high). Employing these two conditions

provides an opportunity to replicate the results reported in Table 4 (Panels C-E) that show

auditors rely more on initial risk assessments when strategic risk is low than when it is high. The

remaining two conditions each have low strategic risk and employ an internal data-related

intervention (Link 3 in Figure 1). Employing the interventions in the context of low strategic risk

is consistent with results reported above indicating this is the setting where over-reliance on

initial risk assessments is most likely to occur.

Procedures

All conditions used the same basic procedures in the following order: provision of

background information on the company and audit; description of strategic analysis where risk

was manipulated (low or high); provision of process analysis information; documentation of the

24 In monitoring experiment four, we observed several participants in the condition requiring risk factors to be documented who took in excess of 50 minutes to complete the task. Most participants in this condition took less than 40 minutes to complete all requirements and it may have been that they were not willing to spend more time evaluating the information.

26

overall risk of material misstatement by participants; presentation of account balance information

containing seeded inconsistency; documentation of account-level risk assessments; and

completion of debriefing questionnaire.25 We asked participants to document OverallRMM after

receiving information about strategic risk but before documenting their account-level

assessments. This differs from the experiments reported above where the overall assessment was

always performed after the account-level assessments. We adopted this approach because we

want to examine how the relation between initial and subsequent risk assessments is affected by

our interventions. However, since the account-level risk assessments are performed only once we

needed another way of capturing initial perceptions about risk.26

Each of our two interventions was introduced immediately prior to the documentation of

the account level risk assessments. Our first intervention required participants to “list all case

information and performance metrics that would be relevant for assessing misstatement risk”

separately for each of the four accounts. This is a simpler version of the internal data-related

intervention we used in experiment four where we asked participants to categorize all relevant

factors as either client operating information or related account balances. Based on Baltes and

Parker (2000), the objective is to increase the influence of information not consistent with the

“all is well” scenario established by the low strategic risk details.

Our second intervention asked participants to “list case information and performance

metrics that would be consistent with low misstatement risk for the account as well as

information and metrics that would be consistent with high misstatement risk for the account.”

Based on Mumma and Wilson (1995), we believe this intervention is stronger because it

25 Both the overall and the account-level risk assessments were measured using a 9-point scale with endpoints labeled “low risk” (1) and “high risk” (9). 26 Recall that in experiments two through four reported above, we used COSRMM1 as the initial measure of risk and examined its influence on the second cost of sales risk assessment.

27

specifically asks participants to consider risk-increasing factors (e.g., inconsistent fluctuations in

account balances) rather than just list all factors relevant to their risk assessments. We chose to

ask them to document both risk increasing and risk decreasing factors to avoid potential demand

effects.

Results

In total, 101 auditors with 3.6 (1.8) years of general (in-charge senior) experience took

part. Descriptive statistics are presented in Table 7, Panel A. As expected, OverallRMM is

highest when strategic risk is high (condition one). Simple t-tests show that the average

OverallRMM of 6.0 in condition one is significantly higher than each of the other three

conditions (all one-tailed p-values < 0.05).27 Within the three “low strategic risk” conditions, the

highest OverallRMM is in condition two where participants did not have an intervention intended

to reduce the effects of strategic risk. However, post-hoc comparisons indicate that none of the

pair-wise comparisons between the means differ across the three conditions (all p-values > 0.75).


Table 7, Panel B reports the results of the OLS regression used to examine the impact of

our strategic risk manipulation on OverallRMM in conditions one and two, neither of which

included our interventions. As predicted by our model, strategic risk has a significant positive

effect (p = 0.029) on OverallRMM. Neither auditor experience nor its interaction with strategic

risk is significant. Next, consistent with the analysis reported for experiment four, we use

regression to examine the factors affecting COSRMM in conditions one and two (no

interventions). The results in Panel C show that only OverallRMM has a significant and positive

effect (p = 0.08) on the account-level assessments. We also examine whether OverallRMM

27 All p-values for multiple comparisons use the Bonferonni procedure to correct for Type I error.

28

influences COSRMM similarly in the low and high strategic risk conditions.28 Results reported in

Table 7, Panel D show that when strategic risk is low, OverallRMM has a significant influence (p

= 0.078) on COSRMM. However, when strategic risk is high, the effect of the OverallRMM is

non-significant (Table 7, Panel E, p = 0.468). Thus, as was the case in experiment three, it

appears that auditors rely significantly more on initial impressions of risk in the presence of low

strategic risk, even when account balance fluctuations suggest there is a risk of material

misstatement.

To examine the effects of our interventions we use the regression model presented in

Table 8, Panel A. Consistent with results reported in our other experiments, SeniorMths has a

significant (p = 0.055) positive effect on COSRMM as does OverallRMM (p = 0.011). Because of

multicollinearity problems, we could not include interaction terms involving either of our

interventions and OverallRMM.29 Thus, we separately examine the association between

OverallRMM and COSRMM in each of our two conditions that used an intervention. The results

reported in Panels B and C of Table 8 indicate that in conditions three and four where our

interventions were used, the association between OverallRMM and COSRMM is not significant

(respectively, p = 0.361 and 0.272). Conversely, results in Table 7 (Panel D) indicate that in the

absence of our interventions, OverallRMM has a significant influence on COSRMM. Thus, it

appears that both of our interventions affect account-level risk assessments not directly, but by

moderating the degree to which auditors rely on initial assessments of risk.


Results for one of our debriefing questions provide further evidence about the nature of

the effects documented in Tables 7 and 8. We asked participants to rate the importance of

28 We initially included an interaction term, StratRisk x OverallRMM in our Table 7, Panel B model but significant collinearity problems resulted. Accordingly, we use separate regressions to test for the interaction. 29 Variance inflation factors > 10.

29

information about the client’s strategy in relation to their cost of sales risk assessments. We used

a 9-point scale with endpoints of “not important” (1) and “very important” (9). Means adjusted

for participants’ perception that the strategy would succeed are presented in Table 8, Panel D.30

Pair-wise comparisons show that participants required to list high and low risk factors rated the

strategy information as significantly less important (one-tailed, p = 0.065) than their counterparts

who did not receive an intervention (condition 2), and those asked to list all relevant risk factors

(one tailed, p = .085). This suggests, that as intended, our requirement to list both high and low

risk factors relevant to account-level risk assessments caused participants to focus less on the “all

is well” message conveyed in the low strategic risk information.

Summary

Two key findings emerge in experiment five. First, we were able to replicate the results

of experiment three showing that manipulating strategic risk affects account-level risk

assessments by moderating the extent to which auditors rely on initial impressions of risk (Link

5). The lower the strategic risk, the more initial risk impressions are relied upon, even in the

presence of inconsistent account fluctuations. Second, both of our interventions aimed at

increasing auditors’ reliance on the inconsistent account fluctuation information and decreasing

their reliance on initial risk assessments appear to have been successful (Link 3).

IV. DISCUSSION

In this paper, we have investigated several factors, suggested by prior research and by the

standard setters that are likely to influence auditor judgment about the risk of a material

misstatement at the account level. We organized these factors in a descriptive model presented in

30 We adjusted for perceptions of strategy success because it was unclear to us how they might affect the importance ratings.

30

Figure 1. Based on this model, we tested associations between account-level misstatement risk

judgments and auditors’ selections of accounts for future investigation, and the following

features of audit ecology: timing of strategic analysis; level of the client’s business risk

communicated by strategic analysis; results of analytical procedures indicating inconsistency in

account fluctuations; auditor task-specific experience (months as a senior auditor); and several

task structure interventions that affect how auditors may document their judgment during the

business-risk audit planning process. Collectively, our findings indicate support for links

depicted by the descriptive model, although in a number of cases we determine that associations

are more complex that we had originally envisioned. Our specific findings of interest are

summarized by experiment in Table 9.


First, we find that the timing of strategic analysis impacts the audit program design, i.e.,

the likelihood that the specific accounts will be targeted for further investigation. This result

extends OS findings with respect to account-level risk assessments to the decisions about the

features of the audit program. Auditors who receive favorable information about the client’s

strategic performance before results of analytical procedures not only assess misstatement risk

for the target account at lower levels, but also are less likely to propose further investigation of

the target and related accounts. We believe our study is the first to document such an extension

of OS findings. It also confirms that, as intended by the standard setters (ISA 330: IAASB 2005),

auditors take global business risk assessments into consideration when making decisions about

audit programs.

Second, when analytical procedures reveal inconsistent account fluctuations, auditors

tend to become less sensitive to such inconsistencies as account-level risk factors if they are

31

presented with results of strategic analysis indicating a low level of business risk. We find that

the timing of strategic analysis (relative to analytical procedures) moderates the effect of

account-level factors, proxied by inconsistent account fluctuations, on auditors’ assessments of

misstatement risk in a target account. We find that the mechanism by which strategic analysis

moderates the effect of account-level factors (seeded inconsistency in account fluctuations) on

account-level misstatement risk assessments, is through the change of the extent to which

auditors rely on their initial risk impressions. This finding is of importance to accounting

practitioners and standard setters because no specific guidance currently exists with respect to

the relative timing of analytical procedures and strategic analysis phases of risk assessment

process.

Third, auditors’ task-specific experience, measured by months as a senior associate

(which we considered a reasonable proxy for audit planning experience), is directly and

positively associated with account-level assessments of misstatement risk. In addition, more

experienced seniors were less likely to be “distracted” by the positive results of strategic analysis

in the face of inconsistent account fluctuations revealed by analytical procedures. Given that all

of our participants’ experience as senior auditors had been in the context of a risk-based audit,

this finding gives comfort to the auditing firms in that task-specific knowledge in a risk-based

audit is developed and applied similarly to other types of auditor task-specific knowledge. It also

gives comfort to audit researchers suggesting that prior findings on auditor task-specific

experience (e.g., Bonner 1990) are likely to be transferable to the risk-based audit contexts.

Finally, it suggests that so-called “halo” effect of strategic analysis (Ballou et al. 2004;

O’Donnell and Schultz 2005) may be successfully moderated by task-specific experience.

32

Fourth, we tested several documentation interventions that may be used by accounting

firms to affect the risk assessment task structure. We found that increasing the salience of

inconsistent fluctuations in the accounts via grouping all related accounts in one table and asking

participants to document their risk assessments in an adjacent column had the effect of reducing

the extent to which auditors relied on their initial impressions of risk when assessing both the

overall risk of material misstatement at the entity level and account-specific misstatement risk.

We found a similar effect when we set a documentation requirement to include all factors

relevant to account-level misstatement risk assessments, or to specifically identify risk-

increasing and risk-decreasing factors prior to making account-level misstatement risk

assessments. In summary, instead of observing a direct effect on account level risk assessments,

or a moderating effect of task structure/documentation requirements on the relationship between

such assessments and account-level factors (seeded inconsistency), as we originally suggested in

our model, we observed that our interventions encouraged auditors’ revision of initial beliefs

about account-level risks. We think this finding is important to accounting practitioners,

researchers, and standard setters because at a time when auditors operate in the environment of

ever-increasing documentation requirements (ISA 230R: IAASB 2005), it highlights the subtle

nature of the effect of various versions of such requirements on auditor risk judgments.

Our study is subject to a series of limitations. First, we are bound by the availability of

participants from only one of the Big 4 accounting firms. Second, we were constrained in the time

and numbers of participants allotted for our experiments and hence were not able to examine all

links in our model concurrently. Third, we would have liked to extend task-specific experience

tests to the manager level, but we had no opportunity to do so. Fourth, we used an experimental

case that does not require industry specialist knowledge and to that extent we are not able to

33

comment on the direct or moderating effects of such knowledge in the context of our model. Fifth,

in our experiments auditors made individual judgments about risks while in reality these

judgments would be made either at the group level or would be preceded by a discussion among

the engagement team members. These limitations should serve as a spring board for future

studies.

34

References Abdolmohammadi, M. J. 1999. A comprehensive taxonomy of audit task structure, professional

rank and decision aids for behavioral research. Behavioral Research in Accounting 11: 51-92.

Abdolmohammadi, M. and C. Usoff. 2000. A longitudinal study of applicable decision aids for

detailed tasks in a financial audit. Working paper. Bentley College, Waltham, MA.

Abdolomohammadi, M. and C. Usoff. 2001. The Assessment of Task Structure, Knowledge Base, and Decision Aids for a Comprehensive Inventory of Audit Tasks. Westport, CT: Quorum Books.

Arkes, H. R. 1991. Costs and benefits of judgment errors: Implications for debiasing.

Psychological Bulletin 110(3): 486-498. Asare, S. K., and A. Wright. 1997. Hypothesis revision strategies in conducting analytical

procedures. Accounting, Organizations and Society 22(8): 737-755. Ballou, B., C. E. Earley, and J. Rich. 2004. The impact of strategic-positioning information on

auditor judgments about business-process performance. Auditing: Journal of Practice and Theory 23(2): 71-88.

Baltes, B. B., and C. P.Parker. 2000. Reducing the effects of performance expectations on

behavioral ratings. Organizational Behavior and Human Decision Processes 82(2): 237-267.

Bedard, J. C., T. J. Mock, and A. M. Wright. 1999. Evidential planning in auditing: A review of

the empirical research. Journal of Accounting Literature 18: 96-142. Bell, T., F. Marrs, I. Solomon, and H. Thomas. 1997. Auditing Organizations through a

Strategic Systems Lens: The KPMG Business Measurement Process. KPMG Peat Marwick LLP.

Bell, T. B., M. E. Peecher, and I. Solomon. 2002. The Strategic-Systems Approach to Auditing. In

Cases in Strategic-Systems Auditing. Bell, T.B., and I. Solomon (Eds.). KPMG LLP. __________. 2005. The 21st Century Public Company Audit: Conceptual Elements of KPMG’s

Global Audit Methodology. KPMG LLP. Biggs, S. F., T. J. Mock, and P. R. Watkins. 1988. Auditors use of analytical review in audit

program design. The Accounting Review 63(1): 148-162. Bonner, S. E. 1990. Experience effects in auditing: The Role of task-specific knowledge. The

Accounting Review 65(1): 72-93.

35

Bonner, S. E. 1994. A model of the effects of audit task complexity. Accounting, Organizations, and Society 19(3): 213-234.

Campbell, D. J., and D. R. Ilgen. 1976. Additive effects of task difficulty and goal-setting on

subsequent task performance. Journal of Applied Psychology 61(3): 319-324.

Choy, A. K., and R. R. King. 2005. An experimental investigation of approaches to audit decision making: An evaluation using systems-mediated mental models. Contemporary Accounting Research 22(2): 311-350.

Eilifsen, A., W. R. Knechel, and P. Wallage. 2001. Application of the business risk audit model:

A Field study. Accounting Horizons 15(3): 193-207.

Greenwood, R., and S. Salterio. 2002. Loblaw Companies Ltd. In Cases in Strategic-Systems Auditing. Eds., Bell, T. B., and I. Solomon. U.S.A.: KPMG LLP.

Hirst, D. E., and L. Koonce. 1996. Audit analytical procedures: A field investigation.

Contemporary Accounting Research 13(2): 457-487. Hogarth, R. M. 1991. A Perspective on cognitive research in auditing. The Accounting Review

66(2): 277-290. Hogarth, R. M., and H. J. Einhorn. 1992. Order effects in belief updating: The belief-adjustment

model. Cognitive Psychology 24:1-55. Hylas, R. E., and R. H. Ashton. 1982. Audit detection of financial statement errors. The

Accounting Review 57(4): 751-766. IAASB. 2005. Handbook of International Auditing, Assurance, and Ethics Pronouncements.

International Federation of Accountants: New York.

Iselin, E. R. 1988. The effects of information load and information diversity on decision quality in a structured decision task. Accounting, Organizations, and Society 13(2): 147-164.

Kachelmeier, S. J., and W. F. Messier, Jr. 1990. An investigation of the influence of a

nonstatistical decision aid on auditor sample size decisions. The Accounting Review 65: 209-226.

Kennedy, J. 1993. Debiasing audit judgment with accountability: A framework and experimental

results. Journal of Accounting Research 31(2): 231-245. Kennedy, J. 1995. Debiasing the curse of knowledge in audit judgment. The Accounting Review

70(2): 249-273. Kleinmuntz, B. 1990. Why we still use our heads instead of formulas: toward an integrative

approach. Psychological Bulletin 107(May): 296-310.

36

Knapp, C. A., and M. C. Knapp. 2001. The effects of experience and explicit fraud risk

assessment in detecting fraud with analytical procedures. Accounting, Organizations and Society 26(1): 25-37.

Koonce, L. 1992. Explanation and counterexplanation during audit analytical review. The

Accounting Review 67(1): 59-76.

Kotchetova, N., T. M. Kozloski, and W. F. Messier, Jr. 2006. Linkages between auditors’ risk assessments in a risk-based audit. Working paper. University of Waterloo.

Kotchetova, N., and W.F. Messier, Jr. 2006. Strategic analysis and auditor’s risk assessment.

Working paper. University of Waterloo. Libby, R. 1985. Availability and the generation of hypotheses in analytical review. Journal of

Accounting Research 23(2): 648-668. McDaniel, L. S. 1990. The effects of time pressure and audit program structure on audit

performance. Journal of Accounting Research 28: 267-285. McDaniel, L. S, and W. R. Kinney, Jr. 1995. Expectation-formation guidance in the auditor's

review of interim financial information. Journal of Accounting Research 1995 33 1, p. 59 Messier, W.F., Jr., Glover, S. M., and D. F. Prawitt. 2006. Auditing and Assurance Services: A

Systematic Approach. 4th Ed. McGraw-Hill/Irwin: New York, NY. Messier, W.F., Jr., S. J. Kachelmeier, and K. L. Jensen. 2001. An experimental assessment of

recent professional developments in non-statistical audit sampling guidance. Auditing: Journal of Practice and Theory 20(Spring): 81-96.

Mock, T. J, and J. L Turner. 2005. Auditor Identification of Fraud Risk Factors and their Impact

on Audit Programs. International Journal of Auditing 9(1): 59 Mumma, G. H., and S. B. Wilson. 1995. Procedural debiasing of primacy/anchoring effects in

clinical-like judgments. Journal of Clinical Psychology 51(6): 841-853. O’Donnell, E. and J. Schultz. 2005. The halo effect in business risk audits: Can strategic

assessment bias auditor judgment about accounting details? The Accounting Review 80(3): 921-939.

Salterio, S.E., W. R. Knechel, and N. Kotchetova. 2006. Performance measurement systems and

strategic analysis extensiveness: Auditor’s usage of balanced scorecards and performance benchmarks. Working paper. Queen’s University.

Tandy, Paulette R. 1992. The influence of auditor and client characteristics on auditor use of

analytical procedures. Journal of Applied Business Research 8(4): 87-97.

37

Zimbleman, M. F. 1997. The effects of SAS No. 82 on auditors’ attention to fraud risk factors

and audit planning decisions. Journal of Accounting Research 35(Supplement): 75-97.

38

Table 1: Key Features of the Five Experiments

Experience in Years (standard deviation)

Experiment

# of Participants

General Audit

In-Charge Senior

Manipulated Variables1

Key Dependent Variables2

1 102 3.0 (1.6)

1.6 (1.2)

SATiming AccountsTargeted, OverallRMM

2 2583 2.0 (1.4)

.7 (1.0)

SATiming, Seeded COSRMM, OverallRMM

3 1454 2.6 (1.1)

1.0 (1.0)

StratRisk, Seeded, DocExp

COSRMM, OverallRMM

4 85 1.8 (.8)

.5 (.6)

Interventions COSRMM, OverallRMM

5 101 3.6 (1.4)

1.8 (1.3)

StratRisk, Interventions

COSRMM, OverallRMM

1SATiming: timing of strategic analysis (before or after analytical procedures); Seeded: presence or absence of seeded inconsistency; StratRisk: strategic risk (low or high); DocExp: participants were required (not required) to document expectations for year 2 account balances; Interventions: manipulations intended to weaken relation between strategic assessments and account-level risk assessments. 2AccountsTargeted: decision (yes/no) to target specific accounts for further investigation; COSRMM: assessment that cost of sales is materially misstated; OverallRMM: overall risk that financial statements are materially misstated. 3 Results for 90 of these participants were reported in OS, experiment 1. 4 Results for 48 of these participants were reported in OS, experiment 2.

39

Table 2: Experiment 1 Results and Analysis Panel A: Descriptive Statistics

Target cost of sales for investigation1

Strategic Analysis Timing (SATiming2) No Yes TotalsBefore analytical procedures 18 34 52 After analytical procedures 29 21 50 47 55 102

Panel B: Logistic Regression Analysis of Factors Affecting Investigation Decisions

Model: Investigate (yes/no)i = ββ1SATimingi + β2SeniorMthsi3 + ε (n=102)

Constant (Wald-statistic) 0.556 (2.131)

Coefficients (Wald-statistic) SATiming

0.996

(5.647)**

SeniorMths 0.005 (0.104) Cox and Snell R2 5.8%

Panel C: Ordinary Least Squares (OLS) Regression of Factors Affecting Overall Risk Assessments Model: OverallRMMi

4 = δ δ1 SATimingi + δ2SeniorMthsi5 + ε (n=102)

Constant (t-value) 4.821 (20.792)***

Standardized Coefficients (t-value) SATiming

0.011

(0.107)

SeniorMths 0.135 (1.315) Adjusted R2 1.8%

1After performing analytical procedures, participants indicated (yes/no) whether they would target cost of sales for further investigation. Yes (no) answers were coded “1” (“0”). 2Timing of strategic analysis: before (coded “0”) or after (coded “1”) performing analytical procedures. 3Months as an in-charge senior. 4Auditors’ overall assessment of the risk that the financial statements were materially misstated. Measured on a 9-point scale with 1 = very low risk, 9 = very high risk. 5Number of months as an in-charge senior (self-reported by participants). * Significant at the 0.10 level. ** Significant at the 0.05 level. *** Significant at the 0.01 level.

40

Table 3: Experiment Two Results and Analysis

Panel A: Means (standard deviations) for Dependent Variables

Seeded Inconsistency1

SATiming

Yes

No

Condition Means

Before analytical procedures COSRMM22

OverallRMM23

4.3 (1.5) n=116 4.6 (1.1) n=113

3.7 (1.3) n=51 4.1 (1.3) n=51

4.1 (1.5) n=167 4.4 (1.2) n=164

After analytical procedures COSRMM2

OverallRMM2

4.8 (1.3) n=67 4.6 (1.1) n=68

3.8 (1.7) n=24 4.3 (1.4) n=25

4.5 (1.5) n=91 4.5 (1.2) n=93

Condition Means COSRMM2

OverallRMM2

4.5 (1.4) n=183 4.6 (1.1) n=181

3.7 (1.4) n=75 4.1 (1.3) n=76

Panel B: OLS Regression of Factors Affecting COSRMM2

Model: COSRMM2i = ββ1SATimingi +β2Seededi + β3SeniorMthsi + β4 COSRMM1i4 + ε (n=258)

Constant (t-value) 1.601 (4.896)***


0.106

(1.907)*

Seeded 0.209 (3.433)*** SeniorMths 0.110 (1.834)* COSRMM1 0.384 (6.806)*** Adjusted R2 21.5%

Panel C: OLS Regression of Factors Affecting COSRMM2 (Strategy analysis provided before analytical procedures) Model: COSRMM2i = δδ1Seededi + δ2 SeniorMthsi + δ3 COSRMM1i + ε (n=167)


Standardized Coefficients (t-value)

Seeded 0.160 (2.048)** SeniorMths 0.066 (0.858) COSRMM1 0.394 (5.492)*** Adjusted R2 18.0%

41

Table 3 continued Panel D: OLS Regression of Factors Affecting COSRMM2 (Strategy analysis provided after analytical procedures) Model: COSRMM2i = φφ1Seededi + φ2SeniorMthsi +φ3COSRMM1i + ε (n=91)



Seeded 0.296 (2.964)*** SeniorMths 0.176 (1.800)* COSRMM1 0.369 (3.923)*** Adjusted R2 23.2%

Panel E: OLS Regression of Factors Affecting OverallRMM2 Model: OverallRMM2i = γγ1SATimingi +γ2Seededi +γ3SeniorMthsi + γ4 OverallRMM1i

5 +ε (n=257)



0.024

(0.436)

Seeded 0.092 (1.542) SeniorMths -0.008 (-0.136) OverallRMM1 0.459 (8.220)*** Adjusted R2 22.4%

1When the seeded inconsistency was present (absent) the sales increase was 5.2% (5.2%) and the cost of sales increase was 0.9% (4.9%). Conditions were coded “0” (“1”) when the inconsistency was absent (present). 2Auditors’ assessment COS is materially misstated (made after completion of analytical procedures). Measured on a 9-point scale with 1 = very low risk, 9 = very high risk. 3Auditors’ overall assessment of the risk that the financial statements are materially misstated (made after completion of analytical procedures). Measured on a 9-point scale with 1 = very low risk, 9 = very high risk. 4Auditors’ assessment COS is materially misstated (baseline measure taken prior to manipulation of independent variables). Measured on a 9-point scale with 1 = very low risk, 9 = very high risk. 5Auditors’ overall assessment of the risk that the financial statements are materially misstated (baseline measure taken prior to manipulation of independent variables). Measured on a 9-point scale with 1 = very low risk, 9 = very high risk.

42

Table 4: Experiment Three Results and Analysis Panel A: Means (Standard Deviations) for Dependent Variables

(i) Results for conditions that required participants to document expected changes in account balances.

Strategic Risk1

Low

High

COSRMM2 46.4 (17.3) n=25 49.4 (15.6) n=23OverallRMM2 4.1 (1.2) n=25 4.1 (0.7) n=23

(ii) Results for conditions that did not require participants to document expected changes in

account balances.

Strategic Risk

Condition Means

Low

High

Seeded Inconsistency Present2

COSRMM2 46.1 (17.2) n=26

38.8 (18.1) n=26

42.5 (17.8) n=51

OverallRMM2 4.2 (1.0) n=26 4.1 (1.0) n=25 4.1 (1.0) n=51 Seeded Inconsistency Absent COSRMM2

34.1 (14.9) n=25

41.0 (13.9) n=23

37.4 (14.7) n=48

OverallRMM2 3.7 (0.9) n=25 4.1 (0.7) n=23 3.9 (0.8) n=48 Condition Means COSRMM2

40.2 (17.1) n=51

39.8 (16.1) n=49

OverallRMM2 3.9 (1.0) n=51 4.1 (1.3) n=23 Panel B: OLS Regression of Factors Affecting COSRMM2 Model: COSRMM2i = ββ1StratRiski +β2Seededi + β3DocExpi

3 + β4COSRMM1i + β5SeniorMthsi + β6Seeded x SeniorMths+ β7StratRisk x DocExp + �� (n=145)

Constant (t-value) 18.791 (4.621)***

Standardized Coefficients (t-value) StratRisk

0.070

(0.858)

Seeded -0.028 (-0.283) DocExp 0.268 (2.624)*** COSRMM1 0.558 (8.074)*** SeniorMths -0.191 (-1.627) Seeded x SeniorMths 0.343 (2.531)** StratRisk x DocExp -0.259 (-2.391)** Adjusted R2 37.7%

43

Table 4 continued Panel C: OLS Regression of Factors Affecting COSRMM2 when Auditors were required to Document Expectations about Account Balances Model: COSRMM2i = δδ1StratRiski + δ2COSRMM1i + δ3SeniorMthsi + ε (n=47)

Constant (t-value) 28.344 (4.723)***


StratRisk 0.278 (2.130)** COSRMM1 0.568 (4.323)*** SeniorMths 0.138 (1.087) Adjusted R2 31.9%

Panel D: OLS Regression of Factors Affecting COSRMM2 when Auditors were required to Document Expectations about Account Balances (Strategic Risk = High) Model: COSRMM2i = φφ1COSRMM1i + δ2SeniorMthsi + ε (n=22)

Constant (t-value) 23.305 (2.123)**


COSRMM1 0.513 (2.558)* SeniorMths 0.181 (0.904) Adjusted R2 18.4%

Panel E: OLS Regression of Factors Affecting COSRMM2 when Auditors were required to Document Expectations about Account Balances (Strategic Risk = Low) Model: COSRMM2i = γγ1COSRMM1i + γ2SeniorMthsi + ε (n=25)



COSRMM1 0.584 (3.373)** SeniorMths 0.118 (0.680) Adjusted R2 28.5%

1Strategic risk was described as either low (company was meeting its strategic objectives) or high (company was not meeting its strategic objectives). Low (high) risk condition was coded “0” (“1”). 2When the seeded inconsistency was present (absent) the sales increase was 3.2% (3.2%) and the cost of sales increase was 0.9% (2.9%). Conditions were coded “0” (“1”) when the inconsistency was absent (present). 3Some participants were required to document expected changes in account balances prior to providing year-two risk assessments. The documentation required (not required) condition was coded “1” (“0”).

44

Table 5: Experiment Four Results and Analysis Panel A: Means (Standard Deviations) for Dependent Variables Condition

COSRMM21

OverallRMM22

1. Low strategic risk 39.2 (18.5) n=21 3.7 (1.1) n=21 2. Low strategic risk/external data intervention 32.9 (13.2) n=22 3.9 (1.0) n=23 3. Low strategic risk/internal data intervention 40.8 (24.0) n=19 4.0 (1.0) n=21 4. Low strategic risk/strategy analysis after account balance details 34.5 (13.1) n=20 3.4 (0.7) n=20

Panel B: OLS Regression of Factors Affecting COSRMM2

Model: COSRMM2i = ββ1ExtDatai

3 +β2IntDatai4 +β3SATimingi + β4SeniorMthsi + β5COSRMM1i

+β6ExtData x COSRMM1 + β7IntData x COSRMM1 + β8 SATiming x COSRMM1 + ε (n=82)

Constant (t-value) 8.595 (1.132)

Standardized Coefficients (t-value) ExtData

0.440

(1.644)*

IntData -0.104 (-0.438) SATiming 0.163 (0.656) SeniorMths 0.085 (0.968) COSRMM1 0.916 (4.167)*** ExtData x COSRMM1 -0.652 (-2.341)** IntData x COSRMM1 -0.146 (.455) SATiming x COSRMM1 -0.271 (-1.050) Adjusted R2 44.1%

Panel C: OLS Regression of Factors Affecting COSRMM2 (Base Condition) Model: COSRMM2i = δδ1SeniorMthsi + δ2COSRMM1i + ε (n=21)



SeniorMths 0.110 (0.635) COSRMM1 0.665 (3.854)*** Adjusted R2 40.8%

45

Table 5 continued Panel D: OLS Regression of Factors Affecting COSRMM2 (External Data Intervention Condition) Model: COSRMM2i = φφ1SeniorMonthsi + φ2COSRMM1i + ε (n=22)

Constant (t-value) 35.458 (3.429)***


SeniorMths -0.268 (-1.103) COSRMM1 0.052 (0.215) Adjusted R2 8.7%

1COSRMM2: year two assessment of risk that cost of sales is materially misstated. 101-point scale with 0 = very low risk and 100 = very high risk. COSRMM1 was measured using the same scale. 2OverallRMM2: year two overall assessment of risk that financial statements are materially misstated. 7-point scale with 1 = very low risk and 7 = very high risk. OverallRMM1 was measured using the same scale. 3ExtData: external data related intervention (0 = not used, 1 = used). 4IntData: internal data related intervention (0 = not used, 1 = used). 5SATiming: order strategic analysis information was provided (0 = before account balances, 1 = after account balances).

46

Table 6: Experiment Four Analysis of Overall Risk of Material Misstatement Panel A: OLS Regression of Factors Affecting OverallRMM2

Model: OverallRMM2i = ββ1ExtDatai +β2IntDatai +β3SATimingi + β4SeniorMthsi + β5OverallRMM1i +β6ExtData x OverallRMM1 + β7IntData x OverallRMM1 + β8SATiming x OverallRMM1 + ε (n=81)


Standardized Coefficients (t-value) ExtData

0.637

(1.847)*

IntData 0.151 (0.415) SATiming 0.310 (0.798) SeniorMths 0.032 (0.306) OverallRMM1 0.835 (3.793)*** ExtData x OverallRMM1 -0.821 (-2.067)** IntData x OverallRMM1 -0.342 (-.801) SATiming x OverallRMM1 -0.622 (-1.484) Adjusted R2 23.5%

Panel B: OLS Regression of Factors Affecting OverallRMM2 (Base Condition only) Model: OverallRMM2i = δδ1SeniorMonthsi + δ2OverallRMM1i + ε (n=21)



SeniorMths -0.065 (-0.365) OverallRMM1 0.649 (3.635)*** Adjusted R2 36.4%

Panel C: OLS Regression of Factors Affecting OverallRMM2 (External Data Intervention Condition only) Model: OverallRMM2i = φφ1 SeniorMths i + φ2OverallRMM1i + ε (n=21)



SeniorMths 0.135 (0.539) OverallRMM1 0.327 (1.304) Adjusted R2 8.6%

47

Table 7: Experiment Five Results and Analysis Panel A: Means (Standard Deviations) of Dependent Variables

Condition

OverallRMM1

COSRMM2

1. High strategic risk 6.0 (1.1) n=24 4.7 (1.6) n=242. Low strategic risk 4.2 (1.3) n=25 5.0 (1.3) n=253. Low strategic risk/list relevant risk factors 4.4 (1.4) n=26 4.6 (1.7) n=264. Low strategic risk/list factors increasing & decreasing risk 5.0 (1.6) n=26 4.7 (2.3) n=26

Panel B: OLS Regression of Factors Affecting OverallRMM (Conditions 1 and 2 only) Model: OverallRMMi = ββ1StratRiski +β2SeniorMthsi +β3StratRiski x SeniorMthsi + ε (n=49)



StratRisk 0.511 (2.252)** SeniorMths -0.568 (-0.952) StratRisk x SeniorMths 0.361 (0.559) Adjusted R2 41.7%

Panel C: OLS Regression of Factors Affecting COSRMM (Conditions 1 and 2) Model: COSRMMi = δδ1StratRiski + δ2OverallRMMi + δ3SeniorMthsi + �� (n=49)



StratRisk -0.261 (-1.424) OverallRMM 0.328 (1.765)* SeniorMths -0.065 (-0.435) Adjusted R2 2.8%

Panel D: OLS Regression of Factors Affecting COSRMM (Condition 2) Model: COSRMMi = φ φ1OverallRMMi + φ2SeniorMonthsi + ε (n=25)



OverallRMM 0.377 (1.847)* SeniorMths 0.042 (0.204) Adjusted R2 5.8%

48

Table 7 continued Panel E: OLS Regression of Factors Affecting COSRMM (Condition 1) Model: COSRMMi = γ γ1OverallRMM1i + γ2SeniorMonthsi + ε (n=24)



OverallRMM1 0.170 (0.739) SeniorMonths -0.116 (-0.504) Adjusted R2 3.2%

1OverallRMM: assessment of overall risk that financial statements are materially misstated. A 9-point scale was used with endpoints “low risk” (1) and “high risk.” 2COSRMM: assessment of risk that cost of sales is materially misstated. A 9-point scale was used with endpoints “low risk” (1) and “high risk.”

49

Table 8: Experiment Five Analysis of the Effectiveness of the Interventions Panel A: OLS Regression of Factors Affecting COSRMM (Conditions 2-4) Model: COSRMMi = ββ1OverallRMMi +β2ListFactorsi

1 +β3LowHighi2 + β4SeniorMthsi + ε (n=74)



OverallRMM 0.303 (2.252)** ListFactors -0.085 (-0.666) Low/High -0.116 (-0.884) SeniorMths 0.218 (1.953)* Adjusted R2 10.4%

Panel B: OLS Regression of Factors Affecting COSRMM (Condition 3 only: List all risk factors) Model: COSRMMi = δδ1OverallRMMi + δ2SeniorMthsi + ε (n=25)



OverallRMM 0.192 (0.933) SeniorMths 0.192 (0.932) Adjusted R2 1.6%

Panel C: OLS Regression of Factors Affecting COSRMM (Condition 4 only: Identify risk increasing/decreasing factors) Model: COSRMMi = φφ1OverallRMMi + φ2SeniorMthsi + ε (n=24)



OverallRMM 0.247 (1.129) SeniorMths 0.306 (1.398) Adjusted R2 15.3%

Panel D: Adjusted Means (Standard Errors) for Question on Importance of Strategy to Risk Assessments

Condition

StratImport3

2. Low strategic risk 6.6 (0.3) n=25 3. Low strategic risk/list relevant risk factors 6.6 (0.3) n=26 4. Low strategic risk/list factors increasing & decreasing risk 6.0 (0.3) n=26

1ListFactors: Intervention requiring participants to list all factors relevant to COSRMM assessments. Coded 1 (0) if intervention was present (absent). 2Low/High: Intervention requiring participants to list and classify all factors that would increase or decrease COSRMM assessments. Coded 1 (0) if intervention was present (absent).

50

Table 8 continued 3Debriefing question asking participants to rate the importance of information about the client’s business strategy to conclusions about account-level misstatement risk. Measured on a 9-point scale with endpoints labeled “not important” (1) and “very important” (9). We adjusted the means for participants’ assessments that the company’s strategy would succeed as it seemed plausible to assume this could affect their importance ratings.

51

Table 9: Summary of Key Results by Experiment

Experiment

Table Reference

Key Findings

1 Table 2 • Performing strategy analysis before analytical procedures lowers the likelihood that specific accounts will be targeted for investigation.

2 Table 3 • Auditor experience as in-charge seniors is positively associated with

account-level risk assessments. • Timing of strategy analysis affects account-level risk assessments

by moderating the effects of inconsistent account fluctuations. Seeded inconsistencies have less influence on cost of sales risk assessments when strategy analysis occurs prior to review of account balances.

3 Table 4 • Auditor experience as in-charge seniors moderates the effects of

account-level risk factors on cost of sales risk assessments. More experienced auditors were influenced more by the seeded inconsistency than less experienced auditors.

• Strategic risk affects account-level risk assessment by moderating the influence of initial risk impressions. When strategic risk is low (high) initial impressions of risk have more (less) influence in the presence of a seeded inconsistency.

4 Tables 5-6 • Replication of the Experiment 3 finding that strategic risk affects

account-level risk assessments by moderating the influence of initial risk impressions.

• Increasing the salience of the inconsistent fluctuations in account balances reduced the extent to which auditors relied on initial impressions of risk when assessing both overall risk and cost of sales risk.

5 Tables 7-8 • Replication of the Experiment 3 finding that strategic risk affects

account-level risk assessments by moderating the influence of initial risk impressions.

• Requiring auditor to document all factors relevant to account level risk assessments or to specifically identify risk increasing and risk decreasing factors reduces their reliance on initial risk assessments.

• When strategic risk is low, auditors experience as in-charge seniors is positive associated with account-level risk assessments.

52

Figure 1: Framework of Factors Influencing Auditor Judgment of Misstatement Risk at the Account Level in a Risk-based Audit

Surrogate/ Underlying Operationalization1 Variable Assessment of risk of material misstatement for Cost of Goods Sold (COS RMM) Accounts targeted for

Link 4 E2, E3

Link 5 E1, E2, E3

Link 3 E4, E5

Link 2 E1, E2, E32

Auditor risk assessment and planning experience (“experience as a senior”)

Task-specific knowledge

Documentation requirements; other “de-biaser” interventions

Firm-specific Procedures

Business risk assessments based on strategic analysis of the client entity (“timing of strategic analysis,” “strategic risk”)

Entity-level Risk Factors

Fluctuations in account balances determined via analytical procedures (“seeded inconsistency”)

Account-level Misstatement Risk

Assessment

Account-level Factors

Link 1 E2, E5

Link 6 E1

Audit Program Design further investigation

53

1The solid arrows represent relationships between theoretical constructs in the model. The dashed lines represent what proxies we used in our experiments to operationalize theoretical constructs. 2 E1-E5 notations indicate the experiments testing corresponding links in the model.

54

Figure 2: Experiment Four External Data-Related Manipulation Panel A: Presentation of Account Balance Information without Intervention

There have been no significant changes in category management processes during 2004, unit sales prices have not changed significantly since the end of 2003, and there have been no major changes in the product mix.

IndustryBest

Current (2004)

Previous (2003) Change Percent

Sales revenue (in $ thousands) 29,782 28,859 923 3.2% Store operating expense (in $ thousands) 5,386 5,244 142 2.7% Sales per square foot $10.82 $10.25 $9.94 $0.31 3.1% Average customer transaction $23.67 $21.63 $22.05 $<0.42> <1.9>% Operating expenses to sales 17.6% 18.1% 18.2% <0.1>% <0.5>%

There have been no significant changes in supply chain management processes during 2004 and there have been no significant changes in unit costs for inventory. During 2004, ABC installed a significant upgrade to their supply chain management software. The computer risk management group has evaluated ABC’s enterprise system for 2004 and is comfortable that the new software is functioning effectively. However, because the new system has altered a number of inventory management procedures, the audit team has increased control risk for 2004 from 20 to 25 (on a 100-point scale).

IndustryBest

Current (2004)

Previous (2003) Change Percent

Merchandise inventory (in $ thousands) 2,785 2,736 49 1.8% Cost of goods sold (in $ thousands) 20,532 20,349 183 0.9% Average inventory turnover 10.5 10.3 10.1 0.2 2.0% Average distribution costs per case $16.44 $16.44 $16.62 $<0.18> <1.1>% On-time deliveries from warehouse 95.5% 95.5% 95.7% <0.2>% <0.2>%

55

Figure 2 continued Panel B: Presentation of Account Balance Information with Intervention1

Document your 2003 misstatement risk assessment for each of the following accounts on a 100-point scale where 0 represents very low risk and 100 represents very high risk:

Account Balance (in thousands of dollars) Current

(2003) Previous

(2002) Change Percent

Misstatement Risk from

low (0) to high (100)

Merchandise inventory 2,736 2,760 <24> <0.9>% __________

Sales revenue (a) 28,859 28,349 510 1.8% __________

Cost of goods sold (b) 20,349 19,891 458 2.3% __________

Store operating expense 5,244 5,181 63 1.2% __________

(a) Unit sales prices reduced on selected items; no change in product mix from 2002 to 2003. (b) Product costs and product mix did not change from 2002 to 2003.

1The information presented in Panel A preceded this summarized version of the account balance fluctuations.

56

auditor judgment about misstatement risk at the account level in a risk … · 2015-07-29 ·...

Documents