(rutgers business school - transportation.org...big data analytics •the right tools and skills can...
TRANSCRIPT
Emerging Audit Technologies
Hussein Issa(Rutgers Business School)
Outline• What is Artificial Intelligence (AI)
• Technological Process Reframing (TPR)
• Research Questions:
– 1. How will the field of artificial intelligence change the audit process through TPR?
– 2. How to analyze the cost and benefit of the investment in AI?
– 3. How to make auditors, who lack data mining and AI knowledge and skill, master AI techniques and tools?
– 4. What are the differences and similarities between expert system and deep learning?
– 5. What can be learned from the research of the application of expert system to auditing to support the application of deep learning to auditing?
– 6. Is it appropriate to directly employ existing deep learning trained with nonfinancial data, to analyze financial content?
– 7. What are the changes in audit conceptualization that will be facilitated by sensing, archiving, and predictive technologies?
– 8. What are the lines of defense in the modern continuous and (partially) intelligent audit?
– 9. What are the modules of the modern (intelligent) assurance process?
– 10. What are these more detailed “modern assertions”?
– 11. What data will be evidence?
– 12. What is an adequate way to taxonomize audit judgments that is appropriate for intelligent automation?
– 13. To what degree can audit judgment be automated?
– 14. Are audit populations a large enough sample for deep learning?
– 15. How can you do deep learning of financial statement fraud identification if the known frauds & restatements are very limited?
– 16. If less independence can result in better assurance, should the standards be modified in that direction?
– 17. What are the parts of auditing that can be divided into a series of automatable production processes?
– 18. Would a different organization of the audit process be more appropriate to the AI enabled audit (AIEA)?
– 19. What are the subcategories of audit judgments?
– 20. Which of these can be formalized?
– 21. Which of these can be supported by expert systems / neural networks / deep learning methodologies?
– 22. How does the evolution of technology and its adoption affects the audit process? Is there a substantive amount of TPR?
– 23. Will automation cause workforce replacement or supplementation in the auditing field?
• Exogenous measurement and quality of measurement
• How will AI affect the Auditing Profession
• Formalization of Audit through Automation
• Workforce replacement or supplementation
• List of additional Research Questions
7/26/2019AASHTO Audit Annual Meeting 2019 2
Outline• What is Artificial Intelligence (AI)
• Technological Process Reframing (TPR)
• Research Questions:
– 1. How will the field of artificial intelligence change the audit process through TPR?
– 2. How to analyze the cost and benefit of the investment in AI?
– 3. How to make auditors, who lack data mining and AI knowledge and skill, master AI techniques and tools?
– 4. What are the differences and similarities between expert system and deep learning?
– 5. What can be learned from the research of the application of expert system to auditing to support the application of deep learning to auditing?
– 6. Is it appropriate to directly employ existing deep learning trained with nonfinancial data, to analyze financial content?
– 7. What are the changes in audit conceptualization that will be facilitated by sensing, archiving, and predictive technologies?
– 8. What are the lines of defense in the modern continuous and (partially) intelligent audit?
– 9. What are the modules of the modern (intelligent) assurance process?
– 10. What are these more detailed “modern assertions”?
– 11. What data will be evidence?
– 12. What is an adequate way to taxonomize audit judgments that is appropriate for intelligent automation?
– 13. To what degree can audit judgment be automated?
– 14. Are audit populations a large enough sample for deep learning?
– 15. How can you do deep learning of financial statement fraud identification if the known frauds & restatements are very limited?
– 16. If less independence can result in better assurance, should the standards be modified in that direction?
– 17. What are the parts of auditing that can be divided into a series of automatable production processes?
– 18. Would a different organization of the audit process be more appropriate to the AI enabled audit (AIEA)?
– 19. What are the subcategories of audit judgments?
– 20. Which of these can be formalized?
– 21. Which of these can be supported by expert systems / neural networks / deep learning methodologies?
– 22. How does the evolution of technology and its adoption affects the audit process? Is there a substantive amount of TPR?
– 23. Will automation cause workforce replacement or supplementation in the auditing field?
• Exogenous measurement and quality of measurement
• How will AI affect the Auditing Profession
• Formalization of Audit through Automation
• Workforce replacement or supplementation
• List of additional Research Questions
7/26/2019AASHTO Audit Annual Meeting 2019 3
Outline
• Big Data & Its Implications
• Big Data & Audit Analytics
• Artificial Intelligence
• Impact of AI On the Auditing Profession
• The Emerging Technological Landscape
• The Future!
7/26/2019AASHTO Audit Annual Meeting 2019 4
DISRUPTIVE TECHNOLOGY
A technology that significantly alters the way that businesses
operate. A disruptive technology may force companies to alter
the way that they approach their business, risk losing market
share or risk becoming irrelevant (ISACA 2017).
7/26/2019AASHTO Audit Annual Meeting 2019 6
• Current Technological Landscape
– Big Data
– Data Analytics
– Artificial Intelligence
• Emerging Technological Landscape
– Drones
– Blockchain
– Visualization
– Audit Production Line
BIG DATA & ITS IMPLICATIONS
What is Big Data?
• No unified definition:
– Data exceeding the level of efficient manageability within traditional DB
(Harris 2013)
– Process of analyzing a large volume of diverse data, in any variety of form,
using ground-breaking apparatus to identify opportunities to improve overall
value (Miller 2012; Moore et al. 2013; Wyner 2013)
• Common trait: large population of data
• Components: volume, variety, velocity, veracity
• New to accounting and audit industry: no formal means to evaluate it, has not applied it in assessments.
• Correlations vs. causation
The power of Big Data lies in the ability to find patterns, which drives the way in which Big
Data is analyzed (Alles 2013)7/26/2019AASHTO Audit Annual Meeting 2019 8
Big Data explosion
AASHTO Audit Annual Meeting 2019
12 ZBDATA CREATED
IN 2018
0.5 %OF AVAILABLE DATA
USED BY BUSINESSES
© Copyright 2019, MindBridge Analytics Inc.
7/26/2019 9
EXAMPLES OF EXOGENOUS (BIG) DATA
• GPS receiver in your cell phone
• Cash registers when you make a purchase
• Cameras in public places
• Your car
• Your digital photos
• Your IoT devices
• Sensors
7/26/2019AASHTO Audit Annual Meeting 2019 10
Challenges of Auditing Big Data
• Pattern recognition using unstructured data vs. deriving intelligence from
benchmarks and models derived from structured data
• Identified exceptions and anomalies are expected to increase dramatically
• Lack of adequate training and necessary skills to analyze Big Data
• Increasing complexity of Big Data leads to increased cost for companies
(hiring data scientists and investing in additional software)
• Some analytical tools are like a black box to auditors (e.g. Neural Networks),
leading to decreased popularity.
7/26/2019AASHTO Audit Annual Meeting 2019 11
BEHAVIORAL IMPLICATIONS
• Audit by Exception (Vasarhelyi and Halper, 1991)
• Continuous Auditing literature is rich with studies that propose statistical and
machine learning techniques to identify exceptions (Dull et al., 2006; Groomer
& Murthy, 1989a; Alexander Kogan et al., 1999; Vasarhelyi & Halper, 1991)
• Problem?
• Result?
7/26/2019AASHTO Audit Annual Meeting 2019 12
Large numbers of Exceptions!
-Information overload!
-Pattern recognition!
-Ambiguity!
-Information relevance!
BEHAVIORAL IMPLICATIONS
• Audit by Exception (Vasarhelyi and Halper, 1991)
• Continuous Auditing literature is rich with studies that propose statistical and
machine learning techniques to identify exceptions (Dull et al., 2006; Groomer
& Murthy, 1989a; Alexander Kogan et al., 1999; Vasarhelyi & Halper, 1991)
• Problem?
• Result?
7/26/2019AASHTO Audit Annual Meeting 2019 13
Large numbers of Exceptions!
-Information overload!
-Pattern recognition!
-Ambiguity!
-Information relevance!
Behavioral Implications-Information Overload
7/26/2019AASHTO Audit Annual Meeting 2019 14
Behavioral Implications-Information Relevance
7/26/2019AASHTO Audit Annual Meeting 2019 15
Behavioral Implications-Pattern Recognition
7/26/2019AASHTO Audit Annual Meeting 2019 16
Behavioral Implications-Ambiguity
7/26/2019AASHTO Audit Annual Meeting 2019 17
Exceptional Exceptions Framework
7/26/2019AASHTO Audit Annual Meeting 2019 18
Exceptional Exceptions Framework
7/26/2019AASHTO Audit Annual Meeting 2019 19
How will the audit approach change?
Sampling
Retroactive – Point in Time
Traditional Audit Evidence
Processing Few Notable Items
Full Population
Predictive – More Frequent
Non-Traditional Audit Evidence
Processing Numerous Notable
Items
7/26/2019AASHTO Audit Annual Meeting 2019 20
BIG DATA AND AUDIT ANALYTICS
What is Audit Data Analytics (ADA)?
Audit data analytics (ADA) is data analytics applied to the audit
process to produce audit evidence and assist in auditor judgements.
Audit Evidence Auditor Judgements
Firm-Relevant
Data
7/26/2019AASHTO Audit Annual Meeting 2019 22
Ratio
Analysis
Trend
Analysis
Descriptive
Statistics
SamplingLinear
Regression
Logistic
Regression
Machine
Learning
Expert
SystemsClustering
Content
Analysis
Visualization
Robotic Process
Automation
DATA ANALYTICS TECHNIQUES
• Descriptive – provides an analysis of past
performance: “what happened”
– Example: traditional financial reporting
Newer techniques for
audit data analytics
7/26/2019AASHTO Audit Annual Meeting 2019 23
Predictive – provides an
estimate of future performance:
“what may happen”
Example: management or
analyst earnings forecasts
Prescriptive – provides a specific action to take:
“what to do”
Example: if inventory levels reach “x” amount,
purchase more
Example: if variance greater than threshold, flag
as an exception
Retroactive vs. Predictive
• Auditing emphasizes a retroactive, backward-looking approach
• With technology, close to real-time (predictive) assurance is
possible
• Reduce expectation gap between auditors and financial statement
users
7/26/2019AASHTO Audit Annual Meeting 2019 24
Predictive Analytic (cont.) Clustering Using Store Sales by Peer Group
7/26/2019AASHTO Audit Annual Meeting 2019 25
Few vs. Numerous Notable Items
• Sampling methodologies limit the number of items to test
• Full population expands coverage, but generates numerous
notable items
• How to process these items?
– Human involvement
– Technology
7/26/2019AASHTO Audit Annual Meeting 2019 26
Challenges to Applications of ADA
• Data capture – need a streamlined process
– IT capabilities – will determine manual vs. automatic capture
– Data standards – all data fields must be in the same format
– Client approval / privacy concerns
• Data validation – verifying the completeness and accuracy of
data extracts
• Data volume – need capacity to process and store big data
– Size of data set
– Computational complexity
7/26/2019AASHTO Audit Annual Meeting 2019 27
Challenges to Applications of ADA
• Data exceptions – too many exceptions in full population testing
– Must develop proper filters for exceptions
– Acceptable level of precision
• Audit evidence – documentation and support concerns
– “Black Box” of data analytics – how to document ?
– Hierarchy of evidence – where does ADA fit in?
• Compliance – existing audit standards and regulations
– Current standards do not provide explicit guidance on analytical procedures
– Standards may need to be transformed
7/26/2019AASHTO Audit Annual Meeting 2019 28
How Can ADA Impact Audit Quality?
• Assist in risk assessment during the acceptance and planning stages of
the audit
• Increase efficiency and effectiveness of controls and substantive testing
• Provide increased coverage through full population testing vs. sampling
methodologies
• More timely analysis through continuous auditing methodologies vs.
year-end audit approach
7/26/2019AASHTO Audit Annual Meeting 2019 29
Big Data Analytics
• The right tools and skills can help the auditor take advantage of Big Data, such as
– Predictive analytics
– Data visualization
– Text mining
– Dashboards (real-time analytics)
– Data warehouses and DBMS
– Expert systems
• Technology is available and being used by other industries
– Audit organizations should look to some of these companies to evaluate how they may be
able to leverage these technologies to integrate Big Data in their audit process.
7/26/2019AASHTO Audit Annual Meeting 2019 30
ARTIFICIAL INTELLIGENCE
What is Artificial Intelligence?
Definition:
“Intelligence exhibited by machines. A
flexible rational agent that perceives its
environment and takes actions that
maximize its chance of success at some
goal. the term ‘artificial intelligence’ is
applied when a machine mimics ‘cognitive’
functions that humans associate with other
human minds, such as ‘learning’ and
‘problem solving’”
7/26/2019AASHTO Audit Annual Meeting 2019 32
Introduction
The increased use of automation and artificial intelligence in
auditing will result in a shift in the auditor’s roles and level of
involvement in the audit, but not the auditor’s responsibility.
“Hiring of auditors and accountants could fall by as much as 50% by 2020 due to the impact of artificial
intelligence” - Steven B. Harris, Board Member PCAOB ~ quoting Big 4 executive
“30% of corporate audits [will be] performed by AI” by 2025 - World Economic Forum survey of 800
executives and experts
“AI technologies are rapidly outpacing the organizational governance and controls” – EY Assurance and
Advisory Leaders
7/26/2019AASHTO Audit Annual Meeting 2019 33
Computerization of Occupations
7/26/2019AASHTO Audit Annual Meeting 2019 34
Adapted from: “The Future of Employment: How Susceptible are Jobs to Computerisation?” (Frey and Osborne, 2013)
History of AI
7/26/2019AASHTO Audit Annual Meeting 2019 35
© Copyright 2019, MindBridge Analytics Inc.
Supervised learning
Human expert feeds the computer with training
data. From that data the computer should learn
the pattern.
Unsupervised learning
No expert input, the computer identifies
pattern in data and looks for outliers. Particularly useful where the human
expert doesn’t know what to look for.
Reinforced learning
Reinforced learning algorithm continuously
learns from the environment in an iterative fashion.
What is machine learning?
AASHTO Audit Annual Meeting 2019 7/26/2019 36
Machine Learning
37
https://www.linkedin.com/pulse/building-machine-learning-infrastructure-pat-alvarado/
7/26/2019
AI Enablers
– Faster technology
– Larger yet cheaper storage
– Computerization
– High level of investments by industry (Google, Baidu, Microsoft, etc)
Deepmind developed AlphaGo
IBM Watson uses in healthcare
Deloitte and Kira systems in contract analysis
7/26/2019AASHTO Audit Annual Meeting 2019 38
Examples of What Machine Learning can do
INPUT A RESPONSE B APPLICATION
Picture Are there human faces? (0 or 1) Photo tagging
Loan Application Will they repay the loan? (0 or 1) Loan approvals
Ad plus user information Will user click on ad? (0 or 1) Targeted online ads
Audio clip Transcript of audio clip Speech recognition
English Sentence French Sentence Language translation
Sensor from plane engine, etc Is it about to fail? Preventive maintenance
Car camera and other sensors Position of other cars Self-driving cars
Source: Andrew Ng
7/26/2019AASHTO Audit Annual Meeting 2019 39
IMPACT OF AI ON THE AUDITING PROFESSION
Artificial Intelligence and Auditing
• What will it enable?– Deep learning in Image recognition (Inventory checks, security footage)
– Natural Language analysis (text mining)
– Speech recognition
– Data from videos (e.g. drones, security footage)
– Sensor data (e.g. RFID)
• What will it impact?– Sampling
– Auditor independence
– Manual preprocessing and examination of certain documents
– Current training and accounting education
7/26/2019AASHTO Audit Annual Meeting 2019 41
Exogenous (Secondary) Evidence Integration
Measurements Measurement variables Assurance of Quality compared with
traditional
Facebook/twitter/news mentions Name mentions
Positive / negatives
Sentiment
Text meaning
Risk faced
Product popularity
Sales level
Different
Calls / mails to customer
services
Classification of type and outcome by
agent
Reserve for product replacement
Bad debt estimates
Different
Internet of Things (IoT) records
of equipment usage
Sensor data (e.g. weather data) External Verification Better
Face recognition of clients Metadata of videos and pictures: time,
location, identity of the person
Fraud Less accurate but exogenous
so it is not intrusive
Video footage Number of cars in parking lots Estimates of sales revenue Less accurate, but more
difficult (costlier) to falsify
Geo-locational data GPS coordinates
Zip codes
Efficiency
Fraud (collision)
FCPA (kickbacks)
Accurate
7/26/2019AASHTO Audit Annual Meeting 2019 42
What data will be considered evidence?
THE EMERGING TECHNOLOGICAL LANDSCAPE
• Inspection
• Damage assessment
• Surveillance
• Bridges
Drones
7/26/2019AASHTO Audit Annual Meeting 2019 44
• Inspection
• Damage assessment
• Surveillance
• Bridges
Drones
7/26/2019AASHTO Audit Annual Meeting 2019 45
• Inspection
• Damage assessment
• Surveillance
• Bridges
Drones
7/26/2019AASHTO Audit Annual Meeting 2019 46
• Inspection
• Damage assessment
• Surveillance
• Bridges
Drones
7/26/2019AASHTO Audit Annual Meeting 2019 47
E&Y University Drones for Inventory Case Studies! Bryan’s
Amazing Animals
7/26/2019AASHTO Audit Annual Meeting 2019 48
Blockchain & Smart contracts
• Distributed ledger technology
• Ability to share databases and processes
– MLS database
– Title records
• Smart contracts:
– Real estate contracts
– Escrows
– Property records
– Money7/26/2019AASHTO Audit Annual Meeting 2019 49
Blockchain & Smart contracts
• Distributed ledger technology
• Ability to share databases and processes
– MLS database
– Title records
• Smart contracts:
– Real estate contracts
– Escrows
– Property records
– Money7/26/2019AASHTO Audit Annual Meeting 2019 50
Process Map
Purchase-to-Pay Data
Source:
(1) Purchase-to-Pay: Mieke Jans
7/26/2019 51
Visualization
7/26/2019AASHTO Audit Annual Meeting 2019 52
Audit as a production line
7/26/2019AASHTO Audit Annual Meeting 2019 53
Formalization of Audit through Automation
Issues to be considered
• Billing by hour
• Rigidity of the standards
• Formalization of audit steps:1. Pre-planning Phase
2. Contracting Phase
3. Understanding Internal Controls and Identifying Risk Factors
4. Control Risk Assessment
5. Substantive Tests
6. Evaluation of Evidence
7. Audit Report
7/26/2019AASHTO Audit Annual Meeting 2019 54
Audit Production Line
7/26/2019AASHTO Audit Annual Meeting 2019 55
Phase AI-Enabled Automated Audit Process Traditional Audit Process
Pre-planning -AI collects and analyzes Big Data (exogenous)
-Data related to the client’s organizational
structure, operational methods, and accounting
and financial systems feed into AI system
-Auditors examines client’s industry
-Auditor examines client’s organizational structure,
operational methods, and accounting and financial
systems
Contracting -AI uses the estimate of the risk level (from phase
1) and calculates audit fees, number of hours
-AI analyzes a database of contracts and prepares
the contract
-Auditor and Client sign contract
-Engagement Letter prepared by the auditor based on
the estimated Client risk
-Auditor and client sign contract
Understanding Internal
Controls and Identifying
Risk Factors
-Feed flowcharts, questionnaire answers,
narratives, into AI and use image recognition and
text mining to analyze them
-Use Drones to conduct the walkthrough, then
use AI to analyze the generated video
-Use visualization and pattern recognition to
identify Risk factors
-AI aggregates all this data to Identify Fraud and
illegal acts risk factors
-Document understanding (flowcharts,
questionnaires, narratives, walkthrough)
-Auditor aggregates this information and uses their
judgment to identify risks factors
-Understanding of IC to determine the scope, nature,
and timing of substantive tests.
Control Risk Assessment -Continuous Control Monitoring Systems examine
controls continuously
-AI runs Process mining to verify proper IC
implementation
-Logs are automatically generated to ensure their
integrity.
-Examination of the client’s IC policies and
procedures
-Risk assessment for each attribute
-Test of controls
-Reassess risk
-Document testing of controls.
Substantive tests -Continuous Data Quality Assurance to ensure
quality of data and evidence
-AI examines data provenance
-Continuous test of details of transactions on
100% of the population
-Continuous test of details of balances (at all
times)
-Continuous pattern recognition, outlier
detection, benchmarks, visualization
-Periodical Sampling-based tests, and nature, extent,
and timing depend on IC tests
-Tests of details of a sample of transactions
-Test of details of balances (at a certain point in time)
-Analytical procedures
Evaluation of Evidence -This becomes part of the previous phase -Auditor must evaluate the sufficiency, clarity, and
acceptability of collected evidence. Accordingly, the
auditor may either collect more evidence, or
withdraw from engagement.
Audit Report -AI uses a predictive model to estimate the
various risks identified
-Audit report can be continuous (graded 1-00 for
example) rather than categorical (clean, qualified,
adverse, etc.)
-Auditor aggregates previous information to issue a
report
-Report is categorical: Clean, qualified, adverse, etc.
Audit Production Line
7/26/2019AASHTO Audit Annual Meeting 2019 56
Phase AI-Enabled Automated Audit Process Traditional Audit Process
Pre-planning -AI collects and analyzes Big Data (exogenous)
-Data related to the client’s organizational
structure, operational methods, and accounting
and financial systems feed into AI system
-Auditors examines client’s industry
-Auditor examines client’s organizational structure,
operational methods, and accounting and financial
systems
Contracting -AI uses the estimate of the risk level (from phase
1) and calculates audit fees, number of hours
-AI analyzes a database of contracts and prepares
the contract
-Auditor and Client sign contract
-Engagement Letter prepared by the auditor based on
the estimated Client risk
-Auditor and client sign contract
Understanding Internal
Controls and Identifying
Risk Factors
-Feed flowcharts, questionnaire answers,
narratives, into AI and use image recognition and
text mining to analyze them
-Use Drones to conduct the walkthrough, then
use AI to analyze the generated video
-Use visualization and pattern recognition to
identify Risk factors
-AI aggregates all this data to Identify Fraud and
illegal acts risk factors
-Document understanding (flowcharts,
questionnaires, narratives, walkthrough)
-Auditor aggregates this information and uses their
judgment to identify risks factors
-Understanding of IC to determine the scope, nature,
and timing of substantive tests.
Control Risk Assessment -Continuous Control Monitoring Systems examine
controls continuously
-AI runs Process mining to verify proper IC
implementation
-Logs are automatically generated to ensure their
integrity.
-Examination of the client’s IC policies and
procedures
-Risk assessment for each attribute
-Test of controls
-Reassess risk
-Document testing of controls.
Substantive tests -Continuous Data Quality Assurance to ensure
quality of data and evidence
-AI examines data provenance
-Continuous test of details of transactions on
100% of the population
-Continuous test of details of balances (at all
times)
-Continuous pattern recognition, outlier
detection, benchmarks, visualization
-Periodical Sampling-based tests, and nature, extent,
and timing depend on IC tests
-Tests of details of a sample of transactions
-Test of details of balances (at a certain point in time)
-Analytical procedures
Evaluation of Evidence -This becomes part of the previous phase -Auditor must evaluate the sufficiency, clarity, and
acceptability of collected evidence. Accordingly, the
auditor may either collect more evidence, or
withdraw from engagement.
Audit Report -AI uses a predictive model to estimate the
various risks identified
-Audit report can be continuous (graded 1-00 for
example) rather than categorical (clean, qualified,
adverse, etc.)
-Auditor aggregates previous information to issue a
report
-Report is categorical: Clean, qualified, adverse, etc.
Audit Production Line
7/26/2019AASHTO Audit Annual Meeting 2019 57
Phase AI-Enabled Automated Audit Process Traditional Audit Process
Pre-planning -AI collects and analyzes Big Data (exogenous)
-Data related to the client’s organizational structure,
operational methods, and accounting and financial systems
feed into AI system
-Auditor examines client’s industry
-Auditor examines client’s organizational
structure, operational methods, and
accounting and financial systems
Contracting -AI uses the estimate of the risk level (from phase 1) and
calculates audit fees, number of hours
-AI analyzes a database of contracts & prepares contract
-Auditor and Client sign contract
-Engagement Letter prepared by the
auditor based on the estimated Client risk
-Auditor and client sign contract
Understanding
Internal
Controls and
Identifying Risk
Factors
-Feed flowcharts, questionnaire answers, narratives, into AI
and use image recognition and text mining to analyze them
-Use Drones to conduct the walkthrough, then use AI to
analyze the generated video
-Use visualization and pattern recognition to identify Risk
factors
-AI aggregates all this data to Identify Fraud and illegal acts
risk factors
-Document understanding (flowcharts,
questionnaires, narratives, walkthrough)
-Auditor aggregates this information and
uses their judgment to identify risk factors
-Understanding of IC to determine the
scope, nature, and timing of substantive
tests.
Control Risk
Assessment
-Continuous Control Monitoring Systems examine controls
continuously
-AI runs Process mining to verify proper IC implementation
-Logs are automatically generated to ensure their integrity.
-Examination of the client’s IC policies and
procedures
-Risk assessment for each attribute
-Test of controls
-Reassess risk
-Document testing of controls.
1
2
3
4
Audit Production Line (Continued)
7/26/2019AASHTO Audit Annual Meeting 2019 58
Phase AI-Enabled Automated Audit Process Traditional Audit Process
Substantive tests -Continuous Data Quality Assurance to
ensure quality of data and evidence
-AI examines data provenance
-Continuous test of details of transactions
on 100% of the population
-Continuous test of details of balances (at
all times)
-Continuous pattern recognition, outlier
detection, benchmarks, visualization
-Periodical Sampling-based tests, and nature,
extent, and timing depend on IC tests
-Tests of details of a sample of transactions
-Test of details of balances (at a certain point
in time)
-Analytical procedures
Evaluation of
Evidence
-This becomes part of the previous phase -Auditor must evaluate the sufficiency, clarity,
and acceptability of collected evidence.
Accordingly, the auditor may either collect
more evidence, or withdraw from
engagement.
Audit Report -AI uses a predictive model to estimate the
various risks identified
-Audit report can be continuous (graded
from 1-100 for example) rather than
categorical (clean, qualified, adverse, etc.)
-Auditor aggregates previous information to
issue a report
-Report is categorical: Clean, qualified,
adverse, etc.
5
6
7
THE FUTURE!
Actor or Assistant?
7/26/2019AASHTO Audit Annual Meeting 2019 60
Actor or Assistant?
7/26/2019AASHTO Audit Annual Meeting 2019 61
What about the impact on Accounting Education?
7/26/2019AASHTO Audit Annual Meeting 2019 62
•AACSB’s New A5 Standard!–“Accounting degree programs include learning
experiences that develop skills and knowledge
related to the integration of information technology in
accounting and business. This includes the ability of
both faculty and students to adapt to emerging
technologies as well as the mastery of current
technology.”
Carlab Research (Rutgers Accounting Department)
7/26/2019AASHTO Audit Annual Meeting 2019 63
Hussein Issa