iisc - proficience - module 6

SQAM Course, Proficience, IIScSQAM Course, Proficience, IISc 11

Module 6Module 6 Software Quality Audit Software Quality Audit


BasicsBasics

An independent examination of a work An independent examination of a work product or set of work products to assess product or set of work products to assess compliance with specifications, standards, compliance with specifications, standards, contractual agreements, or other criteriacontractual agreements, or other criteria – – IEEE IEEE Std 610.12-1990Std 610.12-1990

The goal of a software audit is to provide The goal of a software audit is to provide an independent determination as to an independent determination as to whether the software, its documentation, whether the software, its documentation, and/or the development and maintenance and/or the development and maintenance processes meet stated requirements. processes meet stated requirements.


Audit Value to the businessAudit Value to the business Managing risksManaging risks Strengthening internal controls Strengthening internal controls Measuring operational effectivenessMeasuring operational effectiveness Reducing costsReducing costs Eliminating wasteEliminating waste Assuring stakeholders business Assuring stakeholders business

requirements are satisfied.requirements are satisfied. Periodic assessments also provide trend Periodic assessments also provide trend

line data to determine baseline and line data to determine baseline and benchmark performance improvement. benchmark performance improvement.


A Little storyA Little story

This is a story about four people named Everybody, This is a story about four people named Everybody, Somebody, Anybody, and Nobody. There was an Somebody, Anybody, and Nobody. There was an important job to be done and Everybody was sure important job to be done and Everybody was sure that Somebody would do it. Anybody could have that Somebody would do it. Anybody could have done it, but Nobody did it. Somebody got angry done it, but Nobody did it. Somebody got angry about that, because it was Everybody's job. about that, because it was Everybody's job. Everybody thought Anybody could do it, but Everybody thought Anybody could do it, but Nobody realized that Everybody wouldn't do it. It Nobody realized that Everybody wouldn't do it. It ended up that Everybody blamed Somebody when ended up that Everybody blamed Somebody when Nobody did what Anybody could have done! Nobody did what Anybody could have done!

Lesson: Responsibilities and authorities are to be Lesson: Responsibilities and authorities are to be defined and communicated well before the auditdefined and communicated well before the audit


Roles and ResponsibilitiesRoles and Responsibilities

The client, person or organization which requests the audit The client, person or organization which requests the audit

The auditor or team who performs the auditThe auditor or team who performs the audit

The auditee whose work is being examined.The auditee whose work is being examined.

Audit can include interested observers and regulatory Audit can include interested observers and regulatory agenciesagencies

Lead Auditor and Audit teamLead Auditor and Audit team

- Audit Team Training- Audit Team Training

- Technical Expertise of the audit- Technical Expertise of the audit

- Team assists LA in checklist - Team assists LA in checklist preparation, preparation, back ground work back ground work

- Team conducts the audit and - Team conducts the audit and prepare the prepare the audit reportsaudit reports


Lead Auditor Lead Auditor ResponsibilitiesResponsibilities

Overall responsible to organize and direct the audit, Co-Overall responsible to organize and direct the audit, Co-ordinate the preparation and issuance of the audit reportordinate the preparation and issuance of the audit report

Determine the team sizeDetermine the team size Brief the team members on the audit scope and areas to be Brief the team members on the audit scope and areas to be

auditedaudited Provide the background about the organization Provide the background about the organization

being auditedbeing audited Assign the workload of who will audit what areasAssign the workload of who will audit what areas Determine the audit scheduleDetermine the audit schedule Notify and brief the audited organization on the scope of the Notify and brief the audited organization on the scope of the

audit and materials that need to be providedaudit and materials that need to be provided Ensure that the audit team is prepared to conduct the auditEnsure that the audit team is prepared to conduct the audit Ensure that the audit plan or procedures are performedEnsure that the audit plan or procedures are performed Issue reports in accordance with the audit plan or procedures. Issue reports in accordance with the audit plan or procedures.


Auditee ResponsibilitiesAuditee Responsibilities

Establish a professional, positive attitude Establish a professional, positive attitude about the audit among the members of the about the audit among the members of the audited groupaudited group

Participate well in the auditParticipate well in the audit Provide all relevant materials and Provide all relevant materials and

resources to the audit teamresources to the audit team Understand the concerns of the auditors Understand the concerns of the auditors Provide a response to the audit report, Provide a response to the audit report,

andand Correcting or resolving deficiencies cited Correcting or resolving deficiencies cited

by the audit team.by the audit team.


Arguing with an auditor is like wrestling with a pig in mud . . . Sooner or later you realize the pig enjoys it!


Audit ProcessAudit Process

An audit should be performed in accordance with An audit should be performed in accordance with documented plans and proceduresdocumented plans and procedures

Four Phases - planning, performance, reporting , Four Phases - planning, performance, reporting , follow-up follow-up


PlanningPlanning

What is the audit's scope?What is the audit's scope?

What should the audit achieve?What should the audit achieve?

Does it cover the total system or Does it cover the total system or part of the system?part of the system?

What is the authority for the What is the authority for the audit? audit?

What background information is What background information is needed?needed?


Planning activitiesPlanning activities Client Requests an auditClient Requests an audit Scope and purpose of the audit are agreed upon by the client Scope and purpose of the audit are agreed upon by the client

and auditor.and auditor. The auditor forms an appropriate team and contacts the The auditor forms an appropriate team and contacts the

auditee. auditee. The auditors convey to the audited organizations the audit's The auditors convey to the audited organizations the audit's

purpose, scope, and authority purpose, scope, and authority The auditor will then request preliminary documentation The auditor will then request preliminary documentation

needed for the audit needed for the audit The auditor and auditee agree on the audit schedule, audit The auditor and auditee agree on the audit schedule, audit

procedures or requirements, responsible people, and content procedures or requirements, responsible people, and content of the audit. of the audit.

An audit plan is developed and documented. An audit plan is developed and documented. The auditor then reviews the available information, including The auditor then reviews the available information, including

previous audits and corrective actionsprevious audits and corrective actions


Planning-PreparationPlanning-Preparation

Audit Coordinator will make Audit Coordinator will make arrangements for the audit. arrangements for the audit.

People are selected to be principal People are selected to be principal points of contact for each task to be points of contact for each task to be audited audited

Escorts are assigned to accompany Escorts are assigned to accompany the auditor during the audit. the auditor during the audit.

The auditee conducts a self-evaluation The auditee conducts a self-evaluation to prepare the employees for the audit to prepare the employees for the audit


Performance Performance

Consists of auditors interviewing, Consists of auditors interviewing, reviewing records, observing operations reviewing records, observing operations and collecting informationand collecting information

Opening meetingOpening meeting

Performance of the AuditPerformance of the Audit

Closing MeetingClosing Meeting


Opening MeetingOpening Meeting Scope of the audit is reviewedScope of the audit is reviewed Schedules are determinedSchedules are determined Auditor and auditee personnel are introducedAuditor and auditee personnel are introduced Logistics and the time for the closing meeting are Logistics and the time for the closing meeting are

determined. determined. Communicate to the auditee the audit's objectives, areas of Communicate to the auditee the audit's objectives, areas of

concentration concentration LA will establish the audit's tone, sense of cooperation, and LA will establish the audit's tone, sense of cooperation, and

act as a seeker of information and facts. act as a seeker of information and facts. Describe the audit process, clarify any administrative Describe the audit process, clarify any administrative

matters and solicit the auditee's input matters and solicit the auditee's input


Performance of the auditPerformance of the audit Auditors check compliance with requirements by Auditors check compliance with requirements by

- reviewing written instructions and - reviewing written instructions and procedures, procedures,

- conducting interviews, checking records, - conducting interviews, checking records, and observing work activities.and observing work activities.

factual evidence of the auditee's compliance factual evidence of the auditee's compliance The audit records include The audit records include

- auditors' notes from interviews and observations- auditors' notes from interviews and observations- photocopies of examples from the record - photocopies of examples from the record reviews. reviews.

The facts noted in the audit are reviewed by the The facts noted in the audit are reviewed by the lead auditor and conclusions are drawnlead auditor and conclusions are drawn


Closing MeetingClosing Meeting

The performance phase of an audit The performance phase of an audit ends with the closing meeting or exit ends with the closing meeting or exit interview where the lead auditor interview where the lead auditor reports the audit team's conclusion. reports the audit team's conclusion.

This is the last opportunity for the This is the last opportunity for the auditee to provide input to the audit.auditee to provide input to the audit.


ReportingReporting The lead auditor is responsible for generating the The lead auditor is responsible for generating the

audit report that is the product of the audit. audit report that is the product of the audit. The lead auditor should start the report the first The lead auditor should start the report the first

day of the audit day of the audit The lead auditor will provide a summary of the The lead auditor will provide a summary of the

written report that allows for factual corrections written report that allows for factual corrections and explanations. and explanations.

The report usually consists of an introduction, The report usually consists of an introduction, purpose, scope, findings, observations, exemplary purpose, scope, findings, observations, exemplary practices, and response requirements.practices, and response requirements.

The report is mailed to the client, the auditee and The report is mailed to the client, the auditee and the audit team the audit team


Follow Up Follow Up The auditee proposes corrective actions, which may be The auditee proposes corrective actions, which may be

reviewed by the client or auditor, if there are any reviewed by the client or auditor, if there are any problems identifiedproblems identified

Resolution requiresResolution requires -correction of the specific deficiency found-correction of the specific deficiency found -resolution of the root cause of the problem-resolution of the root cause of the problem

-setting a date when corrective action will be in -setting a date when corrective action will be in place to prevent a recurrence. place to prevent a recurrence.

The follow-up activities include: evaluation of the The follow-up activities include: evaluation of the response, re-audit, closing and documentation response, re-audit, closing and documentation

The auditor is responsible for requesting a timely The auditor is responsible for requesting a timely response from the auditee. response from the auditee.

When all the findings have been resolved, the auditee is When all the findings have been resolved, the auditee is notified that the audit is closed notified that the audit is closed


Auditors TrainingAuditors Training Listen activelyListen actively Observe body language Observe body language Take notes and explain whyTake notes and explain why Start with open-ended questions- why, when, Start with open-ended questions- why, when,

how, who, what, where, to what extent. how, who, what, where, to what extent. Keep questions short and to the point. Keep questions short and to the point. Move to close-ended questions, answered by yes Move to close-ended questions, answered by yes

or no, to start the clarification process or no, to start the clarification process Use follow-up questions for more information Use follow-up questions for more information Use paraphrasing and repeating Use paraphrasing and repeating


Effective AuditorEffective Auditor

Establish a rapport with the interviewee,Establish a rapport with the interviewee,

Avoid nit-picking or judgmental comments about Avoid nit-picking or judgmental comments about individuals,individuals,

Avoid placing blame or fault for problemAvoid placing blame or fault for problem

Always operate ethicallyAlways operate ethically

Rely upon objective evidence and maintain objectivityRely upon objective evidence and maintain objectivity

Use random sampling to get representative results.Use random sampling to get representative results.

Document results and retain notes.Document results and retain notes.

Report known problems and avoid opinions.Report known problems and avoid opinions.

Avoid surprises: keep your contacts informed.Avoid surprises: keep your contacts informed.


Audit ResultsAudit Results

Best Practice - Best Practice - A practice, procedure, or A practice, procedure, or instruction that is well above the expected norm instruction that is well above the expected norm of performanceof performance

Deviation - IDeviation - Inadequacy which results in a product nadequacy which results in a product nonconformance to a specified requirementnonconformance to a specified requirement , , lack lack of a system or controls to satisfy a customer or of a system or controls to satisfy a customer or system requirement,system requirement, any nonconformance to a any nonconformance to a procedural requirement or inadequate procedure procedural requirement or inadequate procedure

Observation - An opinion regarding a condition Observation - An opinion regarding a condition not covered by a specific requirement; or a not covered by a specific requirement; or a procedure, practice, or instruction whose procedure, practice, or instruction whose effectiveness could be improved. effectiveness could be improved.


NC and CARNC and CAR

Major – Systems failureMajor – Systems failure Minor – Impacts the product quality in Minor – Impacts the product quality in

short periodshort period Corrective Action Report Corrective Action Report

- - Corrective action to correct the Corrective action to correct the unresolved deviations identifiedunresolved deviations identified

- - Cause identification.Cause identification.- - Actions to prevent recurrenceActions to prevent recurrence- Lessons Learnt- Lessons Learnt- Actions taken for improvement- Actions taken for improvement


CA and PACA and PA

Corrective Action – Non Conformities Corrective Action – Non Conformities encounteredencountered

Preventive Action – Potential Non Preventive Action – Potential Non ConformitiesConformities


ReferencesReferences

Mills,Charles A.; Mills,Charles A.; The Quality Audit, A The Quality Audit, A Management Evaluation Tool. Management Evaluation Tool. USA: USA: McGraw-Hill, 1989.McGraw-Hill, 1989.

Burr John T.; Burr John T.; Keys to a Successful Keys to a Successful Internal Audit.Internal Audit.Quality ProgressQuality Progress, Vol. , Vol. 30, No. 4, April 1997.30, No. 4, April 1997.

iisc - proficience - module 6

Documents