q.1. what is qms? explain the elements (clauses) of qms ... web viewexplain in brief qms terms used...

IT Quality & Assurance, Information Systems & Audit

QB Answers 2016

Contents

Q.1. What is QMS? Explain the elements (clauses) of QMS process model using ISO 9001:2015? Map these to the software development phases or activities. Identify at least two major challenges that arise in each of these phases and show how QMS help us meet these challenges?................................................................................................................................................5

Q.2. Explain in brief QMS terms used in ISO 9001:2015:...................................................................7

a. Quality....................................................................................................................................................7

b. Quality Management System..............................................................................................................7

c. Interested Parties..................................................................................................................................7

d. Quality Objectives................................................................................................................................7

e. Risk and Opportunities under Planning.............................................................................................7

f. Context of an Organization..................................................................................................................7

Q.3. Name the seven quality management principles and explain each one of the following with suitable example.....................................................................................................................................10

a. Evidence based decision making.....................................................................................................10

b. Relationship management................................................................................................................10

c. Customer focus...................................................................................................................................10

d. Leadership...........................................................................................................................................10

e. Engagement of people.......................................................................................................................10

f. Process approach................................................................................................................................10

g. Improvement.......................................................................................................................................10

a. Evidence based decision making.....................................................................................................10

b. Relationship management................................................................................................................11

c. Customer focus...................................................................................................................................12

d. Leadership...........................................................................................................................................13

e. Engagement of people.......................................................................................................................13

f. Process approach................................................................................................................................14

g. Improvement.......................................................................................................................................15

1

Q.4. What is the importance of Quality in Software Development process? Explain briefly its role in software quality criteria with suitable examples.............................................................................17

Q.5 Draw the following quality process improvement tools and give two points on how each of these tool help in Process Improvement.............................................................................................18

a) Flowchart..........................................................................................................................................18

b) Check sheet.....................................................................................................................................18

c) Histogram.........................................................................................................................................18

d) Pareto Chart....................................................................................................................................18

e) Scatter diagram...............................................................................................................................18

f) Control chart....................................................................................................................................18

g) Cause-and-effect diagram.............................................................................................................18

h) Run chart..........................................................................................................................................18

Q.6 What are the primary differences between quality process improvement tools and quality management system (QMS) tools? Explain the following QMS tools in brief:...............................24

Internal Quality Audit......................................................................................................................24

Review by Steering Committee.....................................................................................................24

Customer Satisfaction....................................................................................................................24

Third Party or External Certification.............................................................................................24

(For Question no. 7 & 8 – First part of question is same – Name the maturity levels described in CMMi? Explain in brief these levels.)...................................................................................................30

Q.7 Name the maturity levels described in CMMi? Explain in brief these levels and in your own words describe any “generic goals” and “specific goals” in a software development organization that is ranked “Performed” or “Optimised” as its capability maturity level.......................................36

Q.8 Describe briefly all maturity levels in CMMi? Explain in brief these levels and in your own words describe any “generic goals” and “specific goals” in a software development organization that is ranked “Managed” as its capability maturity level...................................................................38

Q.9 What is your understanding about the terms “Quality Assurance” and “Quality Control”? Name any two measures for each for QA and QC. Explain the same with suitable example......40

Q.10 Explain briefly any 10 of 14 management principles of Deming and relate each principle to an information management or software development industry.......................................................43

Q.11 “Do not have unrealistic targets” OR “Eliminate quotas and numerical targets”. Explain with suitable example in real time to demonstrate how this principle is to be implemented and practiced..................................................................................................................................................46

Q.12 What is the meaning of the term (i) measure and (ii) metrics? Name and explain atleast four software quality metrics? Name any two software attributes that are normally measured?

2

List and explain any two metrics for each of these two software attributes. (2 metrics for 2 attributes = 4 measurements)...............................................................................................................48

Q.13 Write a brief note on “Benchmarking”. (Points expected: What is it? Who is it for? How to implement? And Benefit realization)....................................................................................................49

Q.14 What do you understand by the term Information Systems Audit?........................................54

You have been asked to conduct an IS audit for 3 locations of JB Technologies Ltd, a software development company, at multi-cities within India (17 locations), United Kingdom (3 locations), and the United States of America (13 locations)................................................................................54

What is the basis on which you will choose your 3 locations?.........................................................54

What areas (at least 2 main and 2 support functions) of organization’s business practices will need to be covered? List at least 3 things that you will cover in each of these business practices. Explain briefly justifying your stand taken above..............................................................54

Q.15 Describe briefly with suitable examples as to how introduction of Information System Audit can improve the organizations:.............................................................................................................57

a. Safeguard of assets?.........................................................................................................................57

b. System effectiveness?.......................................................................................................................57

c. Data integrity?.....................................................................................................................................57

Q.16 Define Risk.....................................................................................................................................58

Risk management constitutes risk assessment and risk treatment. Following are some of the activities under risk management. Classify them under either risk assessment or risk treatment and explain each of these activities in brief.........................................................................................58

a. Risk identification................................................................................................................................58

b. Risk analysis.......................................................................................................................................58

c. Risk evaluation....................................................................................................................................58

d. Risk response.....................................................................................................................................58

e. Risk transfer........................................................................................................................................58

f. Risk mitigation......................................................................................................................................58

Q.17 Describe briefly the four major activities of Information System Audit Process. (Planning of an audit activities taking into consideration the various audit risks, Conducting of Audit that includes both compliance testing and substantive testing, Reporting of Audit findings, and Follow-up)................................................................................................................................................64

Q.18 Explain the terms: (i) IT Service Management (ITSM) and (ii) IT Service Management System (ITSMS)? Describe briefly all 13 IT service processes of ISO 20000-1:2011 and map the same with software service activities.............................................................................................68

Q.19 How does control of “Change” and “Configuration” help in controlling Quality of Services in Business Application Releases?...........................................................................................................72

3

Q.20 What are the steps involved in risk management detailing risk assessment and risk treatment processes?.............................................................................................................................74

Q-21 Explain the terms: (i) Information Security (IS), (ii) Information Security Management (ISM) and Information Security Management System (ISMS)? Describe briefly the 13 control areas or domains of ISO 27001:2013.................................................................................................................77

Q.22 Explain the terms: (i) Business Continuity, (ii) Disaster Management and (iii) Business Continuity Management System (BCMS)? What are the roles of suppliers and contractors during the disruption of business service activities?..........................................................................80

Q.23 Name the five major areas of COBIT5 and explain your understanding on each one of them..........................................................................................................................................................81

1. EDM - Evaluate, Direct and Monitor,...............................................................................................81

2. APO - Align, Plan and Organise,......................................................................................................81

3. BAI - Build, Acquire and Implement,................................................................................................81

4. DSS - Deliver, Service and Support,...............................................................................................81

5. MEA - Monitor, Evaluate and Assess..............................................................................................81

Q.24 Name and explain in brief (one or two sentences) the five COBIT 5 Principles and seven enterprise Enablers referred in COBIT 5 framework..........................................................................82

Q.25 What is your understanding about (i) risk and (ii) risk management with respect to an IT enterprise? Explain how risk is related to the terms threat, likelihood and impact. Explain briefly the activities that are carried out during risk identification, risk estimation, risk evaluation and risk treatment in the overall risk management process. Explain “Heat Chart” or “Severity Chart” in brief.......................................................................................................................................................84

Q.26 What constitute an eCommerce activity(ies)? What will you audit in an eCommerce environment? Describe the content of an audit report with respect to the objective of the audit, outcome or audit findings for an eCommerce business....................................................................88

4

Q.1. What is QMS? Explain the elements (clauses) of QMS process model using ISO 9001:2015? Map these to the software development phases or activities. Identify at least two major challenges that arise in each of these phases and show how QMS help us meet these challenges?Answer:

The Quality Management System (QMS) is a collection of policies, processes, documented procedures and records that defines the set of internal rules that governs how a company creates and delivers product or service to its customers.

The QMS must be tailored to the needs of a company and the product or service it provides.

But ISO 9001 standard provides a set of guidelines to help make sure that it does not miss any important elements that a QMS needs to be successful.

5

Software development phases or activities:

Challenges could be different based on what software development model one uses and hence can be written on their own.

6

Q.2. Explain in brief QMS terms used in ISO 9001:2015:a. Qualityb. Quality Management Systemc. Interested Partiesd. Quality Objectivese. Risk and Opportunities under Planningf. Context of an Organization

Answers:

(A) QualityDegree to which a set of inherent characteristics (3.10.1) of an object (3.6.1) fulfils requirements (3.6.4)The term “quality” can be used with adjectives such as poor, good or excellent. “Inherent”, as opposed to “assigned”, means existing in the object (3.6.1).

(B) Quality Management System

set of interrelated or interacting elements of an organization (3.2.1) to establish policies (3.5.8) and objectives (3.7.1), and processes (3.4.1) to achieve those objectives

A management system can address a single discipline or several disciplines, e.g. quality management (3.3.4), financial management or environmental management.

The management system elements establish the organization’s structure, roles and responsibilities, planning, operation, policies, practices, rules, beliefs, objectives and processes to achieve those objectives.

The scope of a management system can include the whole of the organization, specific and identified functions of the organization, specific and identified sections of the organization, or one or more functions across a group of organizations.

(C) Interested Parties interested party person or organization (3.2.1) that can affect, be affected by, or perceive itself to be affected by a

decision or activity EXAMPLE: Customers (3.2.4), owners, people in an organization, providers (3.2.5), bankers, regulators,

unions, partners or society that can include competitors or opposing pressure groups.

(D) Quality Objectives

Quality objectives are generally based on the organization's (3.2.1) quality policy (3.5.9). Quality objectives are generally specified for relevant functions, levels and processes (3.4.1) in

the organization (3.2.1).

Generally the quality policy is consistent with the overall policy of the organization (3.2.1), can be aligned with the organization’s vision (3.5.10) and mission (3.5.11) and provides a framework for the setting of quality objectives (3.7.2).

Process : set of interrelated or interacting activities that use inputs to deliver an intended result Note 1 to entry: Whether the “intended result” of a process is called output (3.7.5), product (3.7.6)

or service (3.7.7) depends on the context of the reference. Inputs to a process are generally the outputs of other processes and outputs of a process are

generally the inputs to other processes. Two or more interrelated and interacting processes in series can also be referred to as a process.

7

Processes in an organization (3.2.1) are generally planned and carried out under controlled conditions to add value.

A process where the conformity (3.6.11) of the resulting output cannot be readily or economically validated is frequently referred to as a “special process”.

(E) Risk and Opportunities under Planning

Quality Planning part of quality management (3.3.4) focused on setting quality objectives (3.7.2) and specifying necessary operational processes (3.4.1), and related resources to achieve the quality objectives.

Establishing quality plans (3.8.9) can be part of quality planning.

Quality plan specification (3.8.7) of the procedures (3.4.5) and associated resources to be applied when and

by whom to a specific object (3.6.1) These procedures generally include those referring to quality management (3.3.4)processes

(3.4.1) and to product (3.7.6) and service (3.7.7) realization processes. A quality plan often makes reference to parts of the quality manual (3.8.8) or to procedure

documents (3.8.5). A quality plan is generally one of the results of quality planning (3.3.5).

The “Planning” clause has three sub clauses ie Clause 6.1 Actions to address risks and opportunities Clause 6.2 Quality Objectives and Planning to Achieve Them Clause 6.3 Planning of Changes

6.1.1

When planning for the quality management system, the organization shall consider the issues referred to in Understanding the organization and it context (4.1) and the requirements referred to in Understanding the needs and expectations of interested parties(4.2) and determine the risks and opportunities that need to be addressed to give assurance that the quality management system can achieve its intended result(s); prevent, or reduce, undesired effects; and to achieve continual improvement.

6.1.2

The organization must plan actions to address the risks and opportunities determine in clause 6.1.1. The organization must also plan on how to integrate and implement the actions into its quality management system processes and evaluate the effectiveness of these actions. Actions taken to address risks and opportunities must be proportionate to the potential impact on the conformity of products and services. Options to address risks can include but not limited to avoiding, risk, taking risk in order to pursue an opportunity, eliminating the risk source, changing the likelihood or consequences, sharing the risk, retaining risk by informed decision or implementing standards like ISO 31000. It is the prerogative of the management to adopt any one of the practices. Opportunities can lead to the adoption of new practices, launching of new products, opening new markets, addressing new customers, building partnerships, using new technology and other desirable and viable possibilities to address the organization’s or its customer’s needs.

(F) Context of an Organization

Combination of internal and external issues that can have an effect on an organization’s (3.2.1) approach to developing and achieving its objectives (3.7.1)

An objective can be strategic, tactical, or operational.

8

The organization’s objectives can be related to its products (3.7.6) and services (3.7.7), investments and behaviour towards its interested parties (3.2.3).

The concept of context of the organization is equally applicable to not-for-profit or public service organizations as it is to those seeking profits.

In English, this concept is often referred to by other terms such as “business environment”, “organizational environment” or “ecosystem of an organization”.

Understanding the infrastructure (3.5.2) can help to define the context of the organization. <organization> system (3.5.1) of facilities, equipment and services (3.7.7) needed for the

operation of an organization (3.2.1)

9

Q.3. Name the seven quality management principles and explain each one of the following with suitable example.

a. Evidence based decision making

b. Relationship management

c. Customer focus

d. Leadership

e. Engagement of people

f. Process approach

g. Improvement

Answers:

a. Evidence based decision makingStatement

Decisions based on the analysis and evaluation of data and information are more likely to produce desired results.

Rationale

Decision making can be a complex process, and it always involves some uncertainty. It often involves multiple types and sources of inputs, as well as their interpretation, which can be subjective. It is important to understand cause-and-effect relationships and potential unintended consequences. Facts, evidence and data analysis lead to greater objectivity and confidence in decision making.

Key benefits

•Improved decision-making processes

•Improved assessment of process performance and ability to achieve objectives

•Improved operational effectiveness and efficiency

10

•Increased ability to review, challenge and change opinions and decisions

•Increased ability to demonstrate the effectiveness of past decisions

Actions you can take

•Determine, measure and monitor key indicators to demonstrate the organization’s performance.

•Make all data needed available to the relevant people.

•Ensure that data and information are sufficiently accurate, reliable and secure.

•Analyze and evaluate data and information using suitable methods.

•Ensure people are competent to analyze and evaluate data as needed.

•Make decisions and take actions based on evidence, balanced with experience and intuition.

b. Relationship managementStatement

For sustained success, an organization manages its relationships with interested parties, such as suppliers.

Rationale

Interested parties influence the performance of an organization. Sustained success is more likely to be achieved when the organization manages relationships with all of its interested parties to optimize their impact on its performance. Relationship management with its supplier and partner networks is of particular importance.

Key benefits

•Enhanced performance of the organization and its interested parties through responding to the opportunities and constraints related to each interested party

•Common understanding of goals and values among interested parties

•Increased capability to create value for interested parties by sharing resources and competence and managing quality-related risks

•A well-managed supply chain that provides a stable flow of goods and services


•Determine relevant interested parties (such as suppliers, partners, customers, investors, employees, and society as a whole) and their relationship with the organization.

•Determine and prioritize interested party relationships that need to be managed.

•Establish relationships that balance short-term gains with long-term considerations.

•Pool and share information, expertise and resources with relevant interested parties.

11

•Measure performance and provide performance feedback to interested parties, as appropriate, to enhance improvement initiatives.

•Establish collaborative development and improvement activities with suppliers, partners and other interested parties.

•Encourage and recognize improvements and achievements by suppliers and partners.

c. Customer focusStatement

The primary focus of quality management is to meet customer requirements and to strive to exceed customer expectations.

Rationale

Sustained success is achieved when an organization attracts and retains the confidence of customers and other interested parties.

Every aspect of customer interaction provides an opportunity to create more value for the customer. Understanding current and future needs of customers and other interested parties contributes to sustained success of the organization.

Key benefits

•Increased customer value

•Increased customer satisfaction

•Improved customer loyalty

•Enhanced repeat business

•Enhanced reputation of the organization

•Expanded customer base

•Increased revenue and market share


•Recognize direct and indirect customers as those who receive value from the organization.

•Understand customers’ current and future needs and expectations.

•Link the organization’s objectives to customer needs and expectations.

•Communicate customer needs and expectations throughout the organization.

•Plan, design, develop, produce, deliver and support goods and services to meet customer needs and expectations.

•Measure and monitor customer satisfaction and take appropriate actions.

12

•Determine and take actions on interested parties’ needs and expectations that can affect customer satisfaction.

•Actively manage relationships with customers to achieve sustained success.

d. LeadershipStatement

Leaders at all levels establish unity of purpose and direction and create conditions in which people are engaged in achieving the organization’s quality objectives.

Rationale

Creation of unity of purpose and direction and engagement of people enable an organization to align its strategies, policies, processes and resources to achieve its objectives.

Key benefits

•Increased effectiveness and efficiency in meeting the organization’s quality objectives

•Better coordination of the organization’s processes

•Improved communication between levels and functions of the organization

•Development and improvement of the capability of the organization and its people to deliver desired results


•Communicate the organization’s mission, vision, strategy, policies and processes throughout the organization.

•Create and sustain shared values, fairness and ethical models for behavior at all levels of the organization.

•Establish a culture of trust and integrity.

•Encourage an organization-wide commitment to quality.

•Ensure that leaders at all levels are positive examples to people in the organization.

•Provide people with the required resources, training and authority to act with accountability.

•Inspire, encourage and recognize people’s contribution.

e. Engagement of peopleStatement

Competent, empowered and engaged people at all levels throughout the organization are essential to enhance its capability to create and deliver value.

Rationale

13

To manage an organization effectively and efficiently, it is important to involve all people at all levels and to respect them as individuals. Recognition, empowerment and enhancement of competence facilitate the engagement of people in achieving the organization’s quality objectives.

Key benefits

•Improved understanding of the organization’s quality objectives by people in the organization and increased motivation to achieve them

•Enhanced involvement of people in improvement activities

•Enhanced personal development, initiatives and creativity

•Enhanced people satisfaction

•Enhanced trust and collaboration throughout the organization

•Increased attention to shared values and culture throughout the organization


•Communicate with people to promote understanding of the importance of their individual contribution.

•Promote collaboration throughout the organization.

•Facilitate open discussion and sharing of knowledge and experience.

•Empower people to determine constraints to performance and to take initiatives without fear.

•Recognize and acknowledge people’s contribution, learning andimprovement.

•Enable self-evaluation of performance against personal objectives.

•Conduct surveys to assess people’s satisfaction, communicate the results, and take appropriate actions.

f. Process approachStatement

Consistent and predictable results are achieved more effectively and efficiently when activities are understood and managed as interrelated processes that function as a coherent system.

Rationale

The quality management system consists of inter-related processes. Understanding how results are produced by this system enables an organization to optimize the system and its performance.

Key benefits

•Enhanced ability to focus effort on key processes and opportunities for improvement

•Consistent and predictable outcomes through a system of aligned processes

14

•Optimized performance through effective process management, efficient use of resources, and reduced cross-functional barriers

•Enabling the organization to provide confidence to interested parties as to its consistency, effectiveness and efficiency


•Define objectives of the system and processes necessary to achieve them.

•Establish authority, responsibility and accountability for managing processes.

•Understand the organization’s capabilities and determine resource constraints prior to action.

•Determine process interdependencies and analyze the effect of modifications to individual processes on the system as a whole.

•Manage processes and their interrelations as a system to achieve the organization’s quality objectives effectively and efficiently.

•Ensure the necessary information is available to operate and improve the processes and to monitor, analyze and evaluate the performance of the overall system.

•Manage risks that can affect outputs of the processes and overall outcomes of the quality management system.

g. ImprovementStatement

Successful organizations have an ongoing focus on improvement.

Rationale

Improvement is essential for an organization to maintain current levels of performance, to react to changes in its internal and external conditions and to create new opportunities.

Key benefits

•Improved process performance, organizational capabilities and customer satisfaction

•Enhanced focus on root-cause investigation and determination, followed by prevention and corrective actions

•Enhanced ability to anticipate and react to internal and external risks and opportunities

•Enhanced consideration of both incremental and breakthrough improvement

•Improved use of learning for improvement

•Enhanced drive for innovation


15

•Promote establishment of improvement objectives at all levels of the organization.

•Educate and train people at all levels on how to apply basic tools and methodologies to achieve improvement objectives.

•Ensure people are competent to successfully promote and complete improvement projects.

•Develop and deploy processes to implement improvement projects throughout the organization.

•Track, review and audit the planning, implementation, completion and results of improvement projects.

•Integrate improvement considerations into the development of new or modified goods, services and processes.

•Recognize and acknowledge improvement.

16

Customer Focus The primary focus of quality management is to meet customer requirements and to strive to exceed customer expectations.

LeadershipLeaders at all levels establish unity of purpose and direction and create conditions in which people are engaged in achieving the organization’s quality objectives.

Engagement of peopleCompetent, empowered and engaged people at all levels throughout the organization are essential to enhance its capability to create and deliver value.

Process approachConsistent and predictable results are achieved more effectively and efficiently when activities are understood and managed as interrelated processes that function as a coherent system.

Improvement Successful organizations have an ongoing focus on improvement.

Evidence-based decision making

Decisions based on the analysis and evaluation of data and information are more likely to produce desired results.

Relationship management

For sustained success, an organization manages its relationships with interested parties, such as suppliers.

Q.4. What is the importance of Quality in Software Development process? Explain briefly its role in software quality criteria with suitable examples.Answer:

Quality will be of concern at all stages of project planning and execution.

We would expect quality to be concern of all procedures of goods and services.

Increasing criticality of software:

The end user of software generally anxious about the quality of software especially about the reliability. They are concern about the safety because of their dependency on the software system such as aircraft control system are more safety critical systems

Intangibility of software:

This makes it difficult to know that a particular task in project has been completed satisfactorily. The results of these tasks can be made tangible by demanding that the developer produces deliverables that can be examined for quality

Accumulating errors during software development:

As computer system develops, the error in the earlier deliverables will be added to those in the later steps leading to accumulating determined effects.In general the later in a project that an error is found the more expensive it will be to fix.Since the number of errors in the system is unknown, the debugging phases of a project are particularly difficult to control.

Role in Software Quality Criteria- Requirements Management

◦ Establish common understanding of customer requirements between the customer and the software project

◦ Requirements is basis for planning and managing the software project◦ Not working backwards from a given release date!

Software Project Planning◦ Establish reasonable plans for performing the software engineering activities and for

managing the software project Software Project Tracking and Oversight

◦ Establish adequate visibility into actual progress◦ Take effective actions when project’s performance deviates significantly from planned

Software Subcontract Management◦ Manage projects outsourced to subcontractors

Software Quality Assurance◦ Provide management with appropriate visibility into

process being used by the software projects work products

Software Configuration Management◦ Establish and maintain the integrity of work products

17

◦ Product baseline◦ Baseline authority

18

Q.5 Draw the following quality process improvement tools and give two points on how each of these tool help in Process Improvement.

a) Flowchartb) Check sheetc) Histogramd) Pareto Charte) Scatter diagramf) Control chartg) Cause-and-effect diagramh) Run chart

Answer: -

a) FlowchartA flow chart is a graphical or symbolic representation of a process. Each step is represented by a different

symbol and contains a short description of the process step. The flow chart symbols are linked together

with arrows showing the process flow direction. Flowcharts are used in analysing, designing, documenting

or managing a process or program in various fields. Flowcharts are used in designing and documenting

complex processes or programs. Like other types of diagrams, they help visualize what is going on and

thereby help the people to understand a process, and perhaps also find flaws, bottlenecks, and other

less-obvious features within it.

Basic Flowchart Symbols

For most flowcharts, these five basic symbols are all you will need.

19

b) Check sheetThe check sheet is a form or document used to collect data in real time at the location where the data is

generated. The data it captures can be quantitative or qualitative. When the information is quantitative,

the check sheet is sometimes called a tally sheet. The defining characteristic of a check sheet is that data

are recorded by making marks ("checks") on it. A typical check sheet is divided into regions, and marks

made in different regions have different significance. Data are read by observing the location and number

of marks on the sheet.

It is a simple data collection form consisting of multiple categories with definitions. Data are entered on

the form with a simple tally mark each time one of the categories occurs.

The most straightforward check sheet is simply to make a list of items that you expect will appear in a

process and to mark a check beside each item when it does appear. This type of data collection can be

used for almost anything, from checking off the occurrence of particular types of defects to the counting of

expected items (e.g., the number of times the telephone rings before being answered).

c) HistogramA histogram is a graphical representation of the distribution of data. It is a display of statistical information

that uses rectangles to show the frequency of data items in successive numerical intervals of equal size.

In the most common form of histogram, the independent variable is plotted along the horizontal axis and

the dependent variable is plotted along the vertical axis. The data appears as coloured or shaded

rectangles of variable area.

20

This tool will be used during the Analysis stage of DMAIC. The project team will review data collected

during the Measure stage of DMAIC. It is often suggested that the data be organized into graphs or charts

to more easily understand what the data is saying about the process. Data is of two types - Discrete data

(go or no go, fail or pass) and continuous data (time, height etc.). For continuous data presentation, the

best tool to use is the histogram.

The illustration, below, is a histogram showing the results of a final exam given to a hypothetical class of

students. Each score range is denoted by a bar of a certain colour. Conclusions might also be drawn

concerning the improvement or decline of the professor's teaching ability with the passage of time. If this

histogram were compared with those of other classes in the same semester who had received the same

final exam but who had taken the course from different professors, one might draw conclusions about the

relative competence of the professors.

d) Pareto ChartA Pareto Chart is “a series of bars whose heights reflect the frequency or impact of problems. The bars

are arranged in descending order of height from left to right. This means the categories represented by

the tall bars on the left are relatively more significant than those on the right”. The chart gets its name

from the Pareto Principle, which postulates that 80 percent of the trouble comes from 20 percent of the

problems.

21

A Pareto chart is a bar graph. The lengths of the bars represent frequency or cost (time or money), and

are arranged with longest bars on the left and the shortest to the right. In this way the chart visually

depicts which situations are more significant. The bars are arranged in descending order of height from

left to right.

e) Scatter diagramA scatter diagram is a tool for analysing relationships between two variables. One variable is plotted on

the horizontal axis and the other is plotted on the vertical axis. The pattern of their intersecting points can

graphically show relationship patterns. Most often scatter diagram is used to prove or disprove cause-

and-effect relationships. While the diagram shows relationships, it does not by itself prove that one

variable causes the other.

A scatter diagram is a type of plot or mathematical diagram using Cartesian coordinates to display values

for typically two variables for a set of data. If the points are color-coded, you can increase the number of

displayed variables to three. The data is displayed as a collection of points, each having the value of one

variable determining the position on the horizontal axis and the value of the other variable determining the

position on the vertical axis.

22

f) Control chartThe control chart is a graph used to study how a process changes over time. Data are plotted in time

order. A control chart always has a central line for the average, an upper line for the upper control limit

and a lower line for the lower control limit. These lines are determined from historical data. By comparing

current data to these lines, you can draw conclusions about whether the process variation is consistent (in

control) or is unpredictable (out of control, affected by special causes of variation).

Similar to a run chart, a control chart uses the data from a run chart to determine the upper and lower

control limits. Control limits are the expected limits of variation above and below the average of the data.

These limits are mathematically calculated and indicated by dotted lines.

It is used to determine whether or not a process is stable or has predictable performance. Typically,

control charts identify upper and lower control limits to determine the acceptable range of test results.

Control charts commonly have three types of lines:

(a) Upper and lower specification limits

(b) Upper and lower control limits

(c) Planned or goal value

Control charts illustrate how a process behaves over time and defines the acceptable range of results.

When a process is outside the acceptable limits, the process is adjusted. Control charts can be used for

both project and product life cycle processes. For example, for project processes a control chart can be

used to determine whether cost variances or schedule variances are outside of acceptable limits.

23

g) Cause-and-effect diagramCause-and-effect diagram also known as the Fishbone Diagram or the Ishikawa Diagram is a tool used

for systematically identifying and presenting all the possible causes of a particular problem in graphical

format. The possible causes are presented at various levels of detail in connected branches, with the

level of detail increasing as the branch goes outward, i.e., an outer branch is a cause of the inner branch

it is attached to. Thus, the outermost branches usually indicate the root causes of the problem. The

Ishikawa Diagram resembles a fishbone (hence the alternative name “Fishbone Diagram”) – it has a box

(the ‘fish head’) that contains the statement of the problem at one end of the diagram. From this box

originates the main branch (the ‘fish spine’) of the diagram. Sticking out of this main branch are major

branches that categorize the causes according to their nature. In semiconductor manufacturing, 4 major

branches are often used by beginners, referred to as the ’4 M’s', corresponding to ‘Man’, ‘Machine’,

‘Materials’, and ‘Methods’. Sometimes 5 branches are used (’5 M’s'), with the fifth branch standing for

‘Measurement’, or even ‘M-ironmen.’ These ‘M’s’ or problem cause categories are used to classify each

cause identified for easier analysis of data. Of course, one is not constrained to use these categories in

fishbone diagram. Experienced users of the diagram add more branches and/or use different categories,

depending on what would be more effective in dealing with the problem.

h) Run chartWe have discussed the histogram and Pareto chart. Think of both of these tools as similar to a camera

where a snapshot of the process has been taken. But the run chart is similar to a camcorder, recording

some process element over time.

A run chart is a line graph that shows data points over time. Run charts are helpful in identifying trends

and predicting future performance. Run charts are similar to control charts, plotting data results over time;

however, there are no defined control limits.

Example

A control chart may be used for a pharmaceutical company that is testing a new pain medication. The

drug must stay effective in the system for a minimum of three hours but last no more than five hours, to

prevent accidental overdose.

The mean time or goal efficacy duration would be four hours, with three hours the lower control limit and

five hours the upper control limit.

A run chart may be used to plot the temperature within the manufacturing plan every day for a month to

determine a trend.

24

Q.6 What are the primary differences between quality process improvement tools and quality management system (QMS) tools? Explain the following QMS tools in brief:

Internal Quality Audit Review by Steering Committee Customer Satisfaction Third Party or External Certification

Quality process improvement tools –

Understanding processes so that they can be improved by means of a systematic approach requires the knowledge of a simple kit of tools or techniques. The effective use of these tools and techniques requires their application by the people who actually work on the processes, and their commitment to this will only be possible if they are assured that management cares about improving quality. Managers must show they are committed by providing the training and implementation support necessary.

Process improvement efforts should entail a systematic approach that adheres to a certain methodology, where the specific approaches to accomplish this task may differ.When undertaking a process improvement endeavor, more efficient outcomes are expected. Process improvement may involve a sequence of actions to attain new objectives and goals, like improving performance, reducing costs and elevating profits. Such actions may follow a particular technique or methodology to increase the odds of achieving successful results.

The tools and techniques most commonly used in process improvement are:•Problem solving methodology, such as DRIVE•Process mapping•Process flowcharting •Force field analysis•Cause & effect diagrams•CEDAC (Cause and Effect Diagram with the Addition of Cards)•Brainstorming•Pareto analysis•Statistical process control (SPC)•Control charts•Check sheets•Bar charts•Scatter diagrams•Matrix analysis•Dot plot or tally chart•Histograms

DRIVE is an approach to problem solving and analysis that can be used as part of process improvement.Define the scope of the problem the criteria by which success will be measured and agree the deliverables and success factorsReview the current situation, understand the background, identify and collect information, including performance, identify problem areas, improvements and “quick wins”Identify improvements or solutions to the problem, required changes to enable and sustain the improvementsVerify check that the improvements will bring about benefits that meet the defined success criteria, prioritize and pilot the improvementsExecute plan the implementation of the solutions and improvements, agree and implement them, plan a review, gather feedback and review

25

Quality Process Improvement Tools

PDSA (Plan-Do-Study-Act Cycles)

Lean Six Sigma (DMAIC)

FADE (Focus-Analyse-Develop-Execute Cycles)

CQI: Continuous Quality Improvement

Quality management system (QMS) tools-

Quality Management Tools can be Indispensable, Both to Speed-to-Market and Quality

Quality management tools such as CAPA, Deviations, and Nonconformance, Customer Complaints, Employee Training, etc., that integrate with the rest of a company's system for quality management are critical. They ensure not only quality products, but they can also make processes more efficient and accelerate the time it takes for a product to get to market. With appropriate QMS tools, a company’s quality management program can almost manage itself.

As a result of implementing QMS tools, companies are able to produce more, faster, and at a much lower cost. Please take a moment to look at the videos provided below. You may also want to download the literature about a particular quality management tool, or the full array of quality management tools that are available from MasterControl.

Quality Management Tools are Critical for Effective Quality Management

A quality management tool that is automated and connects all departments is essential for a regulated or ISO-compliant company. Any company with business operations that require communication between several departments is aware of the criticality of an effective quality management program in the system. Master Control’s QMS tools are specifically designed to cater to the business needs of companies in regulated environments. FDA regulations clearly emphasize that companies implement GxP quality management solutions to improve quality of processes. Successful culmination of any process requires automation and collaboration between operations of various departments. MasterControl’s suite of QMS tools such as CAPA, Deviations, Nonconformance, Customer Complaints, and Employee Training ensure that employees work closely together in operations.

QMS Tools Guard Against Human Oversight and Error

Process automation--or a TQM (total quality management) system--can connect each phase in a product's development with every department in a company. This is requisite because building quality into products requires a collaborative effort that can be achieved through the quality management tools offered by MasterControl.

Automated routing, with escalation of tasks that languish, prevents bottlenecks by ensuring a rapid response to queries or inputs needed to progress forward.

Quality Management Tools Enhance Production Efficiency and Effectiveness of QA Departments

QMS tools that consist of CAPA, Deviations, Non Conformances, Training, etc., and the types of quality management tools discussed above literally build quality into products. This reduces the burden on the Quality Department while simultaneously speeding up a company's production and time-to-market.

26

QMS Tools Lead to Easy Quality Control

Companies that are severely dependent on QA departments to report bugs, issues, non-conformances, or deviations in the product are aware of how difficult the recursive rounds of testing can be. For starters, the entire responsibility of manufacturing a safe product that conforms to ISO regulations is on the shoulders of the QA department. Any carelessness or inability to test products thoroughly can lead to serious consequences. As a result, many companies are now focusing on employing QMS tools that help their quality control department test and verify product batches without any chances of slippage. By building quality into products via quality management tools (as opposed to forcing QA to bear the brunt of the responsibility), everyone is happy--engineering, regulatory, QA, manufacturing, sales and marketing, and, most of all, upper management.

QMS Tools Lead to Automated Processes and Quality Products

Automated processes for assuring quality products and accelerating time-to-market can be tailored to meet the needs of any company, regardless of size. Without quality management tools, companies will find it difficult to automate processes in the system. Any process is a series of events that requires collaboration between different departments. In order to execute a process to its entirety, the chain of events should be automated. If QMS tools are not used, this becomes a difficult task for companies to execute processes on a timely basis. Lack of automation causes a chain reaction in the system. The first hit comes in the form of the delay it takes in producing quality products. The second hit is the delay in the time it takes for the company to market products. The final blow comes when the competition takes the lead in the market and the company faces heavy losses. All these factors prove the necessity of incorporating QMS tools for automating processes and developing quality products.

Quality Management System (QMS) tools: -

Internal Quality Audit

Review by Steering Committee

Customer Satisfaction

Third Party or External Certification

QMS tools: -

Internal Quality Audito Internal Quality Audits, is the mechanism by which the company monitors adherence to the

documented quality system. Its confirmation that the quality management system is operating

correctly.

o Internal Quality Audits provides objective data used to highlight the potential for improvement and

a basis on which to plan improvements.

o One of the most important objectives of an internal quality audit is measuring the effectiveness of

an organization's quality management system.

o The audit is carried out by people who understand both the company and its activities.

o Internal Quality Audits checks Quality Management System against requirements of all the

applicable standards.

o Internal Quality Audits checks effectiveness of Quality Management System meeting specified

quality objectives.

27

o Compliance and change in requirements

o Meeting external audit objectives and requirements

o The Internal Quality Audit System is a documented system covering the planning, execution and

follow-up of quality audits carried out within the scope of the quality management system. It’s part

of the quality management system itself.

o A typical IQAS will combine three functions (required bythe ISO 9000):

Management Representative (usually the Quality Manager)

Internal Quality Auditors

Management Review Body

Review by Steering Committeeo The steering committee is a project governance body with authority for larger project decisions. A

steering committee is formed at the beginning of the project, with the blessing of the company

leadership. The purpose of the monthly meeting is to review the health of the project with the

steering committee.

o Steering Committee reviews: -

Quality activities performed

Quality audits results

Corrective/preventing actions & customer complaints

Metrics analysis

Project acceptance level

Management of resources

Delivery schedules

Complaints (functional)

Budget and cost overruns

Satisfaction review (optional)

Customer Satisfactiono What is Customer Satisfaction?

ISO 9001 defines customer satisfaction as a customer’s perception of the degree to which the

customer’s requirements have been fulfilled. This definition clearly shows that customer

satisfaction is a subjective judgment of whether their requirements (not the contractual

obligations) are fulfilled. It also points out that customer satisfaction is not a yes/no issue, but a

range of different levels of satisfaction.

o The organization shall monitor customers’ perceptions of the degree to which their needs and

expectations have been fulfilled. The organization shall determine the methods for obtaining,

monitoring and reviewing this information.

28

o Why we need Customer Satisfaction?

It’s not always the case that the client who does not complain is a satisfied client, and the one

who complains is a dissatisfied customer. On the contrary, very often unsatisfied customers don’t

complain directly to the provider, but complain to people around them, which constructs negative

image for company. On the other side, there are clients who might complain directly to the

company even in cases when they are generally satisfied by the service/product, but they want to

get something more, such as additional features. If these complaints are handled properly, these

customers, or even the most dissatisfied customers, can end up having a positive attitude

regarding company. Additionally, company can use the complaints from the customers as

feedback for improving their business (through improving operations, employees, suppliers, etc.).

o Integrated framework for enhancing customer satisfaction

Analysing customer requirements and expectations.

Managing client expectations

Monitoring and measuring customer satisfaction.

o Tools for measuring Customer Satisfaction

Surveys

Geographical Index information (GI)

Effects of product transition

Repeat Orders

CRM tools

Focus meetings with Customer

Complaints and Compliments monitoring

Third Party or External Certificationo External certification ensuring the safety and quality of products and services, facilitating

international trade and improving the environment.

o External certifications are strategic tools and guidelines to help companies tackle some of the

most demanding challenges of modern business.

o External Certifications helps organisations to

Meet Customer requirements

Get More Revenue and Business from New Customers

Improve Company and Product Quality

Increase Customer Satisfaction with Products

Describe, Understand, and Communicate, Company Processes

Develop a Professional Culture and Better Employee Morale

Improve the Consistency of Company’s Operations

Better process integration

29

Focus Management and Employees

Improve Efficiency, Reduce Waste, and Save Money

Achieve International Quality Recognition

o Below is list of few Third Party or External Certification

SEI CMM

ISO 9001:2015

ISO 20000-1:2011 (and ITIL)

Malcolm Balridge Quality Award

ISO 27001:2013

ISO 15408 (CC)

30

(For Question no. 7 & 8 – First part of question is same – Name the maturity levels described in CMMi? Explain in brief these levels.)Answer (Name the maturity levels described in CMMi? Explain in brief these levels.): -

Maturity levels of CMMi: -

Capability Maturity Model Integration (CMMI) is a process improvement training and

appraisal program and service administered and marketed by Carnegie Mellon University

and required by many DOD and U.S. Government contracts, especially in software

development. Carnegie Mellon University claims CMMI can be used to guide process

improvement across a project, division, or an entire organization. Under the CMMI

methodology, processes are rated according to their maturity levels.

A maturity level is a well-defined evolutionary plateau toward achieving a mature software

process. Each maturity level provides a layer in the foundation for continuous process

improvement

In CMMI models with a staged representation, there are five maturity levels designated by

the numbers 1 through 5

1. Initial

2. Managed

3. Defined

4. Quantitatively Managed

5. Optimizing

Characteristics of the CMMi Maturity levels: -

31

Maturity Level 1 - Initialo Processes are usually ad hoc and chaotic. The organization usually does not provide

a stable environment. Success in these organizations depends on the competence

and heroics of the people in the organization and not on the use of proven

processes.

o Organizations often produce products and services that work; however, they

frequently exceed the budget and schedule of their projects.

o Organizations are characterized by a tendency to over commit, abandon processes

in the time of crisis, and not be able to repeat their past successes.

Maturity Level 2 - Managedo An organization has achieved all the specific and generic goals of the maturity level 2

process areas. In other words, the projects of the organization have ensured that

requirements are managed and that processes are planned performed, measured,

and controlled.

o The process discipline reflected by maturity level 2 helps to ensure that existing

practices are retained during times of stress. When these practices are in place,

projects are performed and managed according to their documented plans.

32

o Requirements, processes, work products, and services are managed. The status of

the work products and the delivery of services are visible to management at defined

points.

o Commitments are established among relevant stakeholders and are revised as

needed. Work products are reviewed with stakeholders and are controlled.

o The work products and services satisfy their specified requirements, standards, and

objectives.

Maturity Level 3 - Definedo An organization has achieved all the specific and generic goals of the process areas

assigned to maturity levels 2 and 3.

o Processes are well characterized and understood, and are described in standards,

procedures, tools, and methods.

o A critical distinction between level 2 and level 3 is the scope of standards, process

descriptions, and procedures. At level 2, the standards, process descriptions, and

procedures may be quite different in each specific instance of the process (for

example, on a particular project). At maturity level 3, the standards, process

descriptions, and procedures for a project are tailored from the organization's set of

standard processes to suit a particular project or organizational unit. The

organization's set of standard processes includes the processes addressed at

maturity level 2 and maturity level 3. As a result, the processes that are performed

across the organization are consistent except for the differences allowed by the

tailoring guidelines.

o Processes are typically described in more detail and more rigorously than at maturity

level 2. At maturity level 3, processes are managed more proactively using an

understanding of the interrelationships of the process activities and detailed

measures of the process, its work products, and its services.

Maturity Level 4 - Quantitatively Managedo An organization has achieved all the specific goals of the process areas assigned to

maturity levels 2, 3, and 4 and the generic goals assigned to maturity levels 2 and 3.

o At level 4 Sub processes are selected that significantly contribute to overall process

performance. These selected sub processes are controlled using statistical and other

quantitative techniques.

o Quantitative objectives for quality and process performance are established and

used as criteria in managing processes. Quantitative objectives are based on the

33

needs of the customer, end users, organization, and process implementers. Quality

and process performances are understood in statistical terms and are managed

throughout the life of the processes.

o Special causes of process variation are identified and, where appropriate, the

sources of special causes are corrected to prevent future occurrences.

o Quality and process performance measures are incorporated into the organization’s

measurement repository to support fact-based decision making in the future.

o A critical distinction between maturity level 3 and maturity level 4 is the predictability

of process performance. At maturity level 4, the performance of processes is

controlled using statistical and other quantitative techniques, and is quantitatively

predictable. At maturity level 3, processes are only qualitatively predictable.

Maturity Level 5 - Optimizingo At maturity level 5, an organization has achieved all the specific goals of the process

areas assigned to maturity levels 2, 3, 4, and 5 and the generic goals assigned to

maturity levels 2 and 3.

o Processes are continually improved based on a quantitative understanding of the

common causes of variation inherent in processes.

o Level 5 focuses on continually improving process performance through both

incremental and innovative technological improvements.

o Quantitative process-improvement objectives for the organization are established,

continually revised to reflect changing business objectives, and used as criteria in

managing process improvement.

o The effects of deployed process improvements are measured and evaluated against

the quantitative process-improvement objectives. Both the defined processes and

the organization's set of standard processes are targets of measurable improvement

activities.

o Optimizing processes that are agile and innovative depends on the participation of

an empowered workforce aligned with the business values and objectives of the

organization. The organization's ability to rapidly respond to changes and

opportunities is enhanced by finding ways to accelerate and share learning.

Improvement of the processes is inherently part of everybody's role, resulting in a

cycle of continual improvement.

o A critical distinction between maturity level 4 and maturity level 5 is the type of

process variation addressed. At maturity level 4, processes are concerned with

34

addressing special causes of process variation and providing statistical predictability

of the results. Though processes may produce predictable results, the results may

be insufficient to achieve the established objectives. At maturity level 5, processes

are concerned with addressing common causes of process variation and changing

the process (that is, shifting the mean of the process performance) to improve

process performance (while maintaining statistical predictability) to achieve the

established quantitative process-improvement objectives.

Maturity Levels and Process Areas:

Below is a list of all the corresponding process areas defined for software organisations.

Level Focus Key Process Area Result

5 – Optimizing Continuous

Process

Improvement

Organizational Innovation and

Deployment

Causal Analysis and Resolution

Highest

Quality /

Lowest Risk

4 –

Quantitatively

Managed

Quantitatively

Managed

Organizational Process

Performance

Quantitative Project Management

Highest

Quality /

Lowest Risk

3 – Defined Process

Standardizatio

n

Requirements Development

Technical Solution

Product Integration

Verification

Validation

Organizational Process Focus

Organizational Process Definition

Organizational Training

Integrated Project Mgmt (with

IPPD extras)

Risk Management

Decision Analysis and Resolution

Integrated Teaming (IPPD only)

Org. Environment for Integration

(IPPD only)

Integrated Supplier Management

Medium Quality

/

Medium Risk

35

(SS only)

2 – Managed Basic Project

Management

Requirements Management

Project Planning

Project Monitoring and Control

Supplier Agreement Management

Measurement and Analysis

Process and Product Quality

Assurance

Configuration Management

Low Quality /

High Risk

1 – Initial Process is

informal and

Adhoc

Lowest Quality /

Highest Risk

36

Q.7 Name the maturity levels described in CMMi? Explain in brief these levels and in your own words describe any “generic goals” and “specific goals” in a software development organization that is ranked “Performed” or “Optimised” as its capability maturity level.Answer: -

Generic goals in maturity level “Optimised”: -

1. Ensure Continuous Process Improvement

2. Correct Root Causes of Problems

Ensure Continuous Process Improvement:

The purpose of this generic role is to select and systematically deploy process and technology

improvements that contribute to meeting established quality and process-performance objectives.

Optimizing processes that are agile and innovative depends on the participation of an empowered

workforce aligned with the business values and objectives of the organization. The ability to rapidly

respond to changes and opportunities is enhanced by finding ways to accelerate and share learning.

Improvement of the processes is inherently part of everyone’s role, resulting in a cycle of continual

improvement.

o Establish and maintain quantitative process-improvement objectives that support the

business' objectives.

The quantitative process-improvement objectives may be specific to the individual

process or they may be defined for a broader scope (i.e., for a set of processes), with the

individual processes contributing to achieving these objectives. Objectives that are

specific to the individual process are typically allocated from quantitative objectives

established for a broader scope.

These process-improvement objectives are primarily derived from the business

objectives and from a detailed understanding of process capability. These objectives are

the criteria used to judge whether the process performance is quantitatively improving the

ability to meet its business objectives. These process-improvement objectives are often

set to values beyond the current process performance, and both incremental and

innovative technological improvements may be needed to achieve these objectives.

These objectives may also be revised frequently to continue to drive the improvement of

the process (i.e., when an objective is achieved, it may be set to a new value that is again

beyond the new process performance).

These process-improvement objectives may be the same as, or a refinement of, the

objectives established in the Establish Quantitative Objectives for the Process generic

37

practice, as long as they can serve as both drivers and criteria for successful process

improvement.

o Identify process improvements that would result in measurable improvements to process

performance. Process improvements include both incremental changes and innovative

technological improvements. The innovative technological improvements are typically

pursued as efforts that are separately planned, performed, and managed. Piloting is often

performed. These efforts often address specific areas of the processes that are

determined by analyzing the process performance and identifying specific opportunities

for significant measurable improvement.

o Define strategies and manage deployment of selected process improvements based on

the quantified expected benefits, the estimated costs and impacts, and the measured

change to process performance. The costs and benefits of these improvements are

estimated quantitatively, and the actual costs and benefits are measured. Benefits are

primarily considered relative to the quantitative process-improvement objectives.

Improvements are made to both the set of standard processes and the defined

processes.

Managing deployment of the process improvements includes piloting of changes and

implementing adjustments where appropriate, addressing potential and real barriers to

the deployment, minimizing disruption to ongoing efforts, and managing risks

38

Q.8 Describe briefly all maturity levels in CMMi? Explain in brief these levels and in your own words describe any “generic goals” and “specific goals” in a software development organization that is ranked “Managed” as its capability maturity level.Answer: -

Generic goals in maturity level “Managed”: -

Establish an Organizational Policy

Plan the Process

Provide Resources

Assign Responsibility

Train People

Manage Configurations

Identify and Involve Relevant Stakeholders

Monitor and Control the Process

Objectively Evaluate Adherence

Review Status with Higher Level Management

Plan the Process Requirements for the process's specified work products and for performing the work may be derived

from other requirements. In the case of a project’s processes, they may come from that project’s

requirements management process; in the case of an process, they may come from organizational

sources.

The objectives for the process may be derived from other plans (e.g., the project plans). Included are

objectives for the specific situation, including quality, cost, and schedule objectives. For example, an

objective might be to reduce the cost of performing a process for this implementation over the

previous implementation.

Establishing a plan includes documenting the plan and providing a process description. Maintaining

the plan includes changing it as necessary, in response to either corrective actions or to changes in

requirements and objectives for the process.

o Obtain management sponsorship for performing the process.

o Define and document the process description. The process description, which includes

relevant standards and procedures, may be included as part of the plan for performing

the process or may be included in the plan by reference.

o Define and document the plan for performing the process. This plan may be a stand-

alone document, embedded in a more comprehensive document, or distributed across

multiple documents. In the case of the plan being distributed across multiple documents,

39

ensure that a coherent picture is preserved of who does what. Documents may be

hardcopy or softcopy.

o Review the plan with relevant stakeholders and get their agreement. This includes

reviewing that the planned process satisfies the applicable policies, plans, requirements,

and standards to provide assurance to relevant stakeholders.

o Revise the plan as necessary.

40

Q.9 What is your understanding about the terms “Quality Assurance” and “Quality Control”? Name any two measures for each for QA and QC. Explain the same with suitable example.

Quality control (QC) is a procedure or set of procedures intended to ensure that a manufactured product or performed service adheres to a defined set of quality criteria or meets the requirements of the client or customer. QC is similar to, but not identical with, quality assurance (QA)

QA is defined as a procedure or set of procedures intended to ensure that a product or service under development (before work is complete, as opposed to afterwards) meets specified requirements. QA is sometimes expressed together with QC as a single expression, quality assurance and control (QA/QC).

Additional Reference:

Quality Assurance Quality ControlDefinition QA is a set of activities for ensuring

quality in the processes by which products are developed.

QC is a set of activities for ensuring quality in products. The activities focus on identifying defects in the actual products produced.

Focus on QA aims to prevent defects with a focus on the process used to make the product. It is a proactive quality process.

QC aims to identify (and correct) defects in the finished product. Quality control, therefore, is a reactive process.

Goal The goal of QA is to improve development and test processes so that defects do not arise when the product is being developed.

The goal of QC is to identify defects after a product is developed and before it's released.

How Establish a good quality management system and the assessment of its adequacy. Periodic conformance audits of the operations of the system.

Finding & eliminating sources of quality problems through tools & equipment so that customer's requirements are continually met.

What Prevention of quality problems through planned and systematic activities including documentation.

The activities or techniques used to achieve and maintain the product quality, process and service.

Responsibility Everyone on the team involved in developing the product is responsible for quality assurance.

Quality control is usually the responsibility of a specific team that tests the product for defects.

Example Verification is an example of QA Validation/Software Testing is an example of QC

Statistical Techniques

Statistical Tools & Techniques can be applied in both QA & QC. When they are applied to processes (process inputs & operational parameters), they are called Statistical Process Control (SPC); & it becomes the part of QA.

When statistical tools & techniques are applied to finished products (process outputs), they are called as Statistical Quality Control (SQC) & comes under QC.

As a tool QA is a managerial tool QC is a corrective toolOrientation QA is process oriented QC is product oriented

Measures for Quality Assurance

41

Benchmarking and Process Improvement

Benchmarking is practical with minimal data, namely the time, total effort and software size for main development phase. Defects are an optional extra. This data is collected for recently completed projects. Purchasing team requests the same data to build benchmark measures of supplier performance and uses the results to negotiate current and future developments. SQA verify this data is being collected and is used to build the benchmark database.

As developments complete a review examines the detailed monthly progress data collected by the Software Control Office (SCO). SQA participates in the review to verify the SCO has assembled the complete history for each development. The final data is added to the benchmark database of projects. Over time this growing database provides concrete evidence of development process productivity and quality improvement.

A regular procedure quantifies improvement benefits. At intervals, say every 6 months, a report is made showing benefits from recent projects through initiatives such as CMMI. This includes calculating the Return on Investment (ROI) based on productivity improvement and investments made to improve.

The Estimating and Risk Assessment Process

In a development group SQA is performed on the documented estimating procedure to check input data is used that quantifies:• The software product size and uncertainty• The development process productivity• The development constraints: time, effort, staff, and reliability• The risk levels for each constraint

Software acquisition requires the supplier to provide this data using a formal questionnaire. Software size is quantified using the estimated size range. The range reflects specification uncertainty that reduces as progress is made through the feasibility and specification phases. Greater detail is practical as feature specifications are refined. Each software module is estimated in terms of the smallest, most likely and largest size. This size range uses the most practical sizing units such as logical input statements, function points or objects

The time, effort, resources, costs and reliability constraints for the development are risk assessed taking in to account the quantified uncertainties such as the size range. Each constraint is associated with a risk level and estimates are evaluated against these risk levels.

Frequently it is found that specific constraints cannot be met since the risk is too high. The estimating procedure evaluates alternatives, each of which is logged and documented. This may mean allowing additional time, adding staff and/or reducing features (size). The alternative “What If” estimates document how the final baseline plan is determined, risk assessed and agreed.Measures for Quality Control

Acceptance Sampling:

Acceptance sampling is done on sample’s post production to check for quality parameters as decided by the organization covering both attributes as well as variables. If the sample does not meet the required parameters of quality than that given lot is rejected, and further analysis is done to identify the source and rectify the defects. Acceptance sampling is done on the basis of inspection, which includes physical verification of color, size, shape, etc.

The major objectives of inspection are:

To detect and prevent defects in products and process.

42

To identify defected parts or product and prevent it from further consumption or usage. To highlight the product or process defect to appropriate authorities for necessary and corrective

actions.

Scope of inspection covers input materials, finished material, plant, machinery etc. To sustain quality of product and services it is important to have in place robust quality control techniques.

Statistical process control charts

Statistical process control uses sampling and statistical methods to monitor the quality of an ongoing process such as a production operation. A graphical display referred to as a control chart provides a basis for deciding whether the variation in the output of a process is due to common causes (randomly occurring variations) or to out-of-the-ordinary assignable causes.

Whenever assignable causes are identified, a decision can be made to adjust the process in order to bring the output back to acceptable quality levels.

Control charts can be classified by the type of data they contain. For instance, an x̄-chart is employed in situations where a sample mean is used to measure the quality of the output. Quantitative data such as length, weight, and temperature can be monitored with an x̄-chart.

Process variability can be monitored using a range or R-chart. In cases in which the quality of output is measured in terms of the number of defectives or the proportion of defectives in the sample, an np-chart or a p-chart can be used.

All control charts are constructed in a similar fashion. For example, the centre line of an x̄-chart corresponds to the mean of the process when the process is in control and producing output of acceptable quality. The vertical axis of the control chart identifies the scale of measurement for the variable of interest. The upper horizontal line of the control chart, referred to as the upper control limit, and the lower horizontal line, referred to as the lower control limit, are chosen so that when the process is in control there will be a high probability that the value of a sample mean will fall between the two control limits.

Standard practice is to set the control limits at three standard deviations above and below the process mean. The process can be sampled periodically. As each sample is selected, the value of the sample mean is plotted on the control chart. If the value of a sample mean is within the control limits, the process can be continued under the assumption that the quality standards are being maintained. If the value of the sample mean is outside the control limits, an out-of-control conclusion points to the need for corrective action in order to return the process to acceptable quality levels.

43

Q.10 Explain briefly any 10 of 14 management principles of Deming and relate each principle to an information management or software development industry.

W. Edwards Deming’s 14 points are the basis for transformation of industry. Adoption and action on the 14 points are a signal that the management intends to stay in business and aims to protect investors and jobs. Such a system formed the basis for lessons for top management in Japan in 1950 and in subsequent years.The 14 points apply anywhere, to small organizations as well as to large ones, to the service industry as well as to manufacturing. They equally apply to any division within a company and to its suppliers. Deming’s fourteen points of management approach provide guidelines for implementing the TQM concept. These fourteen points can be applied to managing software development processes.

1) Create constancy of purpose for improvement of product and service.Software development process traditionally ends when the completed system is handed over to the support group and put into production mode. Under the TQM culture, there is no finish line for the development team. Maybe there is a shift of focus from one project to another. The development team should be responsible for what they delivered, not the support group. Any quality problem occurs during the production should be addressed to the development team. Management must [Zultner, 1988]: • Establish operational definitions for each step in the software development process. • Define what is meant by “service to the customer.” • Define standards of development, maintenance, and service for the next year and five years ahead. • Define the internal and external customer. • Develop ways to provide better systems and services in less time, using fewer resources. • Invest in tools and techniques for better software development.

2) Adopt the new philosophy of total quality. Quality is everyone’s business. Not just the worker, management is part of the quality team. Under the TQM culture, quality comes first and everyone must join in. Corporate management, from top to bottom, must embrace the TQM concept and clearly communicate their support of this concept to all members in the software development team.

3) Cease dependence on mass inspection to achieve quality.Quality is built in, not added on. It is better to prevent errors in code, rather than reworking the code to remove the errors. Inspection or testing cannot prevent errors from happening, only experience and knowledge can. Management must install programs to continually improve software development processes. Examples of such programs are job training and job incentive programs.

4) End the practice of awarding business based on price tag alone.Many software organizations today are outsourcing their projects to subcontractors. It is important not to award a software contract based on price tag alone. Quality is more important than the difference in costs. Low quality in the long run will result in high total cost. It is better to create long-term relationship with a few loyal and trustworthy suppliers who can produce quality code for your system.

5) Improve constantly and forever the systems of production and service.System development processes must be constantly improved by introducing new and working methodology, paradigm, standards, practices, techniques, tools, policies, and procedures. All these require the organization to constantly keep tracking the best practice in the field of management information system (MIS)—the so-called learning organization. Each individual staff member is required to improve oneself by updating or even expanding one’s skill set.

6) Institute training on the job. To build quality into the software, the development team must have appropriate experience and knowledge. On-the-job training program is an effective means of obtaining such experience and knowledge. In the broadest sense, all MIS staff members must know what their jobs entail and how to do

44

their work. Management must assess the skill level of an employee before he or she is assigned to a software project. Different skill levels can play different roles and assume different responsibilities in a project.

7) Institute leadership. Management must lead, not punish. It is manager's job to help MIS staff do a better job and create a better system. Project managers must be trained in basic interpersonal and analytical skills. They must have a solid understanding of statistical process control. They should know that in any software development team whose performance is in statistical control, half of them would always be below average. They should focus on those members whose performance is out of statistical control.

8) Drive out fear of job insecurity. Employees must feel secure before they are willing to ask questions, make suggestions, or even expose their weaknesses by asking for help. The policy of long-term employment could easily drive out the fear of job insecurity. Moreover, any MIS staff whose performance is out of statistical control should be offered help in retraining or reassignment. However, if one consistently rejects helps from one's co-workers or supervisors, a layoff may be the last resort.

9) Break down barriers between departments or staff areas. Software development requires collaborative effort between users and IS staff. For as long as we can remember, communication gap has been the major factor to many MIS implementation failures. Furthermore, today's business system projects would most likely involve different functional areas and require expertise in database processing, client-server computing, and network installation, etc. Therefore, open communication among functional areas and general knowledge across disciplines are necessary for a successful system implementation. This requires appropriate education and training for team members to change their behavior and improve their knowledge.

10) Eliminate slogans, exhortations, and targets for the workforce. Slogans do not build quality systems. MIS management should not ask for impossible target or schedule, or unrealistic level of productivity. Instead, they should post their progress in responding to suggestions and in helping the staff improve quality. Let the employees put up their own signs and slogans [Zultner, 1988].

11) Eliminate numerical quotas, and work standards. Quotas (such as metrics), goals (such as schedules), and work standards (such as unit times) address numbers, not quality. A software development project that causes haste and non-conformities accomplishes nothing and services no one. Let the project members put up their own goals. Managers should concentrate on helping people do a better job by reducing rework, errors, and waste. Everyone must work toward constant improvement, not the achievement of some arbitrary, short-term goals [Zultner, 1988].

12) Remove barriers to pride of workmanship. All people are motivated. They would like to make quality products. However, a good workmanship relies on good materials, good tools, good methods, and good timing. Poor materials, broken tools, ineffective methods, or belated schedule are all barriers to pride of workmanship and should be eliminated. Let the software development team put its group identity or team members' names on the software product to take the credit (or the responsibility) of their work.

13) Institute a vigorous program of education and retraining for everyone. On-the-job training is effective, but slow, for an employee to acquire skill set for a particular type of job. In today's MIS arena, technology is changing so fast that new skill set is needed for the same type of job in a short period of time. Management must set aside enough budgets to execute a generous education and retraining program for everyone to improve oneself. Under the TQM culture, all employees must know enough statistical method to understand the nature of variation, to manage the special causes of variation. Support for training employees to acquire necessary statistical method should be institutionalized.

45

14) Put everyone to work to accomplish the transformation. The TQM transformation is everyone's job. Everyone has a customer. Ask yourself who is the person receiving your work? All of us must identify our customers in order to determine precisely what our jobs are. Everyone belongs to a team, to work in the Plan-Do-Check-Act cycle, to address one or more specific issues, to find special causes detected by statistical signals. Moreover, we must put management to work. Only management can change the culture and environment that dominate any individual's performance. Management must agree on their meaning and on the direction to take. They must acknowledge their mistakes, if any, and have the courage to change. They must explain to a critical mass of people in the organization why change is necessary and that the change will involve everybody. Obviously, people must understand the Fourteen Points to know what to do and how to do it [Walton, 1986].

Total quality management is not only a philosophy of work but also an ethic of workers. It is coming from the wisdom and the teachings of many quality improvement gurus. It has helped many companies to improve quality of products and processes, and in turn, increase the productivity and the profitability. Any software organization that is planning to implement the TQM must have the critical mass of its employees embrace the TQM philosophy and methods before jumping onto the bandwagon. That is, all employees regardless of their ranks must fully understand (or be trained with) and internalize the TQM concept and tools. To increase the chance of success, a TQM-implementation project should start from the top management and unfold it downward to lower-level management and workers with a goal to benefit the critical mass of employees. Specifically, the goal is to improve the quality of work life for the employees through improving work conditions, work methods, work compensation, work relations, and providing the employees with opportunities for professional development. Only with this goal could we gain the full cooperation from the employees and bring about successful TQM implementation.

Source: http://www.cob.calpoly.edu/~eli/pdf/jqai-00.pdf

46

http://www.cob.calpoly.edu/~eli/pdf/jqai-00.pdf

Q.11 “Do not have unrealistic targets” OR “Eliminate quotas and numerical targets”. Explain with suitable example in real time to demonstrate how this principle is to be implemented and practiced.

“Eliminate work standards that prescribe numerical quotas for the workforce and numerical goals for

people in management. Substitute aids and helpful leadership; use statistical methods for continual

improvement of quality and productivity.”

Dr. Deming argued that pursuit of arbitrary quantity goals had nothing whatsoever to do with the quality of output. Indeed, in the pursuit of quantity, the worker would routinely sacrifice quality, taking short-cuts along the way. This would in turn lead to rework, rejects, and demoralization.

Reasons for Targets/Quotas

Management creates numerical quotas or targets for their employees to meet because they believe these targets set a standard and provide a basis for employee performance reviews. Management wants employees to understand and meet their performance requirements. Targets can be easily communicated and shared with the employees. If the employee meets the target then they perform accordingly and they will not encounter repercussions.

Management believes quotas are a win-win-win situation. Employees win because they know the expected their performance expectations, management wins because they receive predictable quantities of units and customers win because they receive expected quantities in a given amount of time.

What are some examples for quotas?

Production worker must make a minimum of 100 units an hour. Phone marketer must call a minimum of 15 people an hour. Customer service must input 10 orders every 4 hours.

There are four necessary items for creating a quota.

State who is responsible Determine the measurable unit Determine the targeted quantity Determine the precise time

Why quotas are not effective?

Let’s consider the example of an airline reservations clerk who has a quota of 25 customer calls per hour to process. What happens if her customers on a given day have some difficult problems? What happens if customers are slow in providing information? Her job then becomes taking 25 calls per day rather than satisfying the customer.

Dr. Deming does not promote the concept of quotas and goals because they focus on the outcome rather than the process. He argues that half the workers will be above average and half will be below – no matter what you do.

47

If there exists a stable system, then there is no use specifying a goal. The output will be whatever the system will deliver. If a goal is set beyond the capacity of the system, it will not be reached. If the system is not stable there is no reason to set a goal, because there is absolutely no way of telling what the system will produce.

Let’s also see this with the context of production unit:

Let’s consider the Management makes the quota equal to the average production rate of all the employees.

Employee A, works hard, fast and smart. Employee A exceeds the target. If the target is 100 units an hour, employee A makes 120 units. However, overtime, employee A changes his works habits. With peer pressure, he realizes the extra effort doesn't help him. He scales back only to meet the quota. Because of the quota, the company loses out of his extra available production. Also management loses out on understanding and implementing his methods for the improved production beyond the goal.

Employee B, works hard and tries his best. Employee B performs less than the target and makes 80 units an hour. However every unit produce meets the quality standards. Management, not realizing people perform differently from each other, reprimands the employee for not meeting the target. Management feels the reprimand will motivate the employee. Instead it demoralizes employee B. Initially, his performance may slightly improve, but overtime, due to the negative feelings, employee B production decreases. Overall, this causes employee dissatisfaction and employee turnover. Because of training and hiring expense, employee turnover is costly for the company. In addition new employees tend to make defects at a higher percentage, which adds to the cost.

The above example based the production target on the average production. Where can a target be set that makes management and employees happy? This is not possible. Based on normal distribution, If you lower the target to the start of the bell shape curve then everybody will produce at that lower rate and not try to increase production. If you set the target to the end of the bell shape curve then moral will be extremely low because 99% of the employees cannot meet this requirement.

Companies that use production targets eventually lose focus on production improvement. Lets say all employees meet the production target. What will drive management to improve the production rate? Deming Point 11 argues that management becomes less involve in that area. Targets tend to create an invisible barrier between management and production improvement.

So what is Deming’s solution?

He advocates replacing work standards with leadership – understanding the work that you and the workers are responsible for, understanding who the customers are, and how to better serve them. Deming does not say, “Don’t measure.” He is an advocate of measuring, but not as a way to define a job. “Is your job to make 25 calls per hour, or to give callers courteous satisfaction?” Measure and then improve the system and address those that fall outside the limits of performance variation.

Focus on the process and system details that make the production. Don't focus on the end results, place your efforts into the details. Improvement comes with being hands on within the given area.

Measurement is key to improvement; but instead of monitoring employees performance to targets, focus on improving the current output. In order to do this, management must be hands-on and study the details for the given target. Dr. Deming point wants management to follow the PDCA cycle and use quality improvement tools to improve parameters like production rates.

48

Summing-up: Eliminate work standards and numerical quotas. This focuses on quantity rather than quality of product.

Q.12 What is the meaning of the term (i) measure and (ii) metrics? Name and explain atleast four software quality metrics? Name any two software attributes that are normally measured? List and explain any two metrics for each of these two software attributes. (2 metrics for 2 attributes = 4 measurements)

Metrics" and "measurements" can be viewed as different entities. A measures is a numerical value assigned to an attribute according to defined criteria, for example, one's current temperature in degrees FahrenheitA metric is a mathematical set of relevant, quantifiable, attributes (measures) taken over time. In this example, the metric would be a table or graph of one's temperature taken over a period of time, which would be helpful for identifying trends.

Schedule Variance = (Actual Dt- Planned Dt)/Est.Calender Days Defect Density = Total Defects Found /Size

o Low Defect Density need not be good o may indicate poor reviews

Defect removal Efficiencyo Pre-Delivery Defects/Total No. of Defectso Ideally 100% target could be 95%o Indicates what % of defects are fixed before delivery.

Review Effectiveness o (Defects found during Review)/tot No. of Defectso Indicates the quality of Reviews

Common software metrics include: Bugs per line of code Code coverage Cohesion Coupling Cyclomatic complexity Function point analysis Number of classes and interfaces Number of lines of customer requirements Order of growth Source lines of code Robert Cecil Martin’s software package metrics

Software Quality Metrics focus on the process, project and product. By analyzing the metrics the organization can take corrective action to fix those areas in the process, project or product which are the cause of the software defects.

The de-facto definition of software quality consists of the two major attributes based on intrinsic product quality and the user acceptability. The software quality metric encapsulates the above two attributes, addressing the mean time to failure and defect density within the software components. Finally it assesses user requirements and acceptability of the software. The intrinsic quality of a software product is generally measured by the number of functional defects in the software, often referred to as bugs, or by testing the software in run time mode for inherent vulnerability to determine the software "crash"

49

scenarios. In operational terms, the two metrics are often described by terms namely the defect density (rate) and mean time to failure (MTTF).

Although there are many measures of software quality, correctness, maintainability, integrity and usability provide useful insight.

CorrectnessA program must operate correctly. Correctness is the degree to which the software performs the required functions accurately. One of the most common measures is Defects per KLOC. KLOC means thousands (Kilo) Of Lines of Code.) KLOC is a way of measuring the size of a computer program by counting the number of lines of source code a program has.

MaintainabilityMaintainability is the ease with which a program can be correct if an error occurs. Since there is no direct way of measuring an indirect way has been used to measure this. MTTC (Mean time to change) is one such measure. It measures when an error is found, how much time it takes to analyze the change, design the modification, implement it and test it

IntegrityThis measure the system’s ability to with stand attacks to its security. In order to measure integrity two additional parameters are threat and security need to be defined. Threat – probability that an attack of certain type will happen over a period of time. Security – probability that an attack of certain type will be removed over a period of time. Integrity = Summation [(1 - threat) X (1 - security)]

UsabilityHow usable is your software application? This important characteristic of your application is measured in terms of the following characteristics:

Physical / Intellectual skill required to learn the system Time required becoming moderately efficient in the system. The net increase in productivity by use of the new system. Subjective assessment(usually in the form of questionnaire on the new system)

Measure Metrics1. Customer satisfaction index

Number of system enhancement requests per year Number of maintenance fix requests per year User friendliness: call volume to customer service hotline User friendliness: training time per new user Number of product recalls or fix releases (software vendors) Number of production re-runs (in-house information systems groups)

2. Delivered defect quantities

Normalized per function point (or per LOC) At product delivery (first 3 months or first year of operation) Ongoing (per year of operation) By level of severity By category or cause, e.g.: requirements defect, design defect, code defect, documentation/on-line help defect, defect introduced by fixes, etc.

3. Reliability Availability (percentage of time a system is available, versus the time the system is needed to be available) Mean time between failure (MTBF) Mean time to repair (MTTR) Reliability ratio (MTBF / MTTR) Number of product recalls or fix releases Number of production re-runs as a ratio of production runs

Q.13 Write a brief note on “Benchmarking”. (Points expected: What is it? Who is it for? How to implement? And Benefit realization)

What is Benchmarking?

50

Benchmarking is the process of comparing one's business processes and performance metrics to industry bests or best practices from other companies. It is a systematic process for identifying and implementing best or better practices. Although experts break benchmarking into several types, there exist two main types; “Informal" and "Formal" Benchmarking.

What is Informal Benchmarking? This is a type of benchmarking that most of us do unconsciously at work and in our home life. We constantly compare and learn from the behavior and practices of others. In the context of work, most learning from informal benchmarking comes from the following:

Talking to work colleagues and learning from their experience Consulting with experts Networking with other people from other organizations at conferences, seminars, and Internet

forums. On-line databases/web sites

What is Formal Benchmarking?

There are two types of Formal Benchmarking - Performance and Best Practice Benchmarking.

Performance benchmarking: This involves comparing the performance levels of organizations for a specific process. Performance levels of other organizations are normally called benchmarks and the ideal benchmark is one that originates from an organization recognized as being a leader in the related area. Performance benchmarking may involve the comparison of financial measures (such as expenditure, cost of labor, cost of buildings/equipment, cost of energy, adherence to budget, cash flow, revenue collected) or non-financial measures (such as absenteeism, staff turnover, the percentage of administrative staff to front-line staff, budget processing time, complaints, environmental impact or call center performance).

Best practice benchmarking: This is where organizations search for and study organizations that are high performers in particular areas of interest. The processes themselves of these organizations are studied rather than just the associated performance levels, normally through some mutually beneficial agreement that follows a benchmarking code of conduct.

Who uses Benchmarking?

Most large and highly successful organizations use best practice benchmarking as a tool to continually learn and improve. The resources needed to carry out repeated best practice benchmarking projects properly and in a way that maximizes the learning to be gained from the experiences can be considerable; hence it is used more frequently within large organizations.The growth from year to year in membership of the Global Benchmarking Network is increasing which now has representatives from over 20 countries.

The growth in business excellence is likely to be correlated to the growth in benchmarking as a central part of business excellence is benchmarking with as much as 50% of the points associated with these models attributed to benchmarking.

How to conduct Benchmarking?

A benchmarking process looks like:

51

Defining and Planning the Project

You need to define the project in precise terms and develop a complete, yet simple, project plan. Start with a preliminary plan and build it over time to the appropriate level of preciseness. Such a plan should include a way to measure your success. A project like benchmarking is like (and should probably be managed like) any other project you undertake. Be sure to include in your project plan items such as project objectives, scope, approach, timeline, and budget.

Understanding Where You AreIn order to utilize information about how others are doing, you need to first understand how you are doing or, at least, how you would like to be doing. This requires that you have performance measures or Metrics (see How to Measure Success -- Uncovering the Secrets of Effective Metrics) so that you can judge how you are doing.Given these measures, you can use them to help organize your project and to select your benchmarking partners. You can use these measures to guide your search for secondary data, to help generate your preliminary questionnaire, and to conduct a preliminary survey to narrow the field in your search for potential partners.

Understanding Where You Can BeBased your preliminary studies, you need to select potential partners, ascertain their willingness to participate, and develop your final questionnaire. The questions should help you focus on the specifics of what you want to learn.To get the most out of an exercise like this you have to have the "right" people participate, both from your team, as well as those of your partners. The right people means the best combination of technical and people skills so that you can both elicit and understand the information you are gathering.

52

Once you have your team, you can proceed to schedule and conduct the information exchanges with the several partners you've identified.Two points to remember:Benchmarking is a search for how, as well as how much. To replicate results in your organization you need to understand how they have been achieved by others, andBenchmarking need not require you to visit others. You can achieve the results in many ways, depending on the time and resources available to you. The following chart outlines several alternatives for conducting exchanges. As more time and resources are available and as the need increases, you can elect to use the more sophisticated and time-consuming processes.

It is through these processes that you gather the data to determine where you can be.And the next question is, "How soon can I expect to see some results?" The following table gives some ideas of time frame, based on our experience. How Soon You Need Results Benchmarking Alternatives Within a week Reading library research

Surfing the web Telephone interviews

One to two weeks Research by a professional librarian Hire a consultant

Three to six weeks Rapid Benchmarking* Traditional site visit (2 or 3 sites only)

Two or more months Traditional benchmarking Identifying Lessons LearnedNow that you know how others are doing, you can use the data to understand how you can improve. The most straight-forward way is to assess where there are gaps between your performance and that of your benchmarking partners. Further, you can use these assessments to identify best practices, in particular ones you'd like your organization to adopt.Applying the Lessons LearnedYou are ready to begin implementing what you've learned. This is the "next step."

53

This is where the rubber hits the road. You've learned what others are doing and how they are doing it. You need to ensure that all relevant staff in you reorganization is aware of and can make use of what you've learned. Your report and your presentations may in fact be one of the most important activities in your project.

Benefits of Benchmarking

Benchmarking is a common practice and sensible exercise to establish baselines, define best practices, identify improvement opportunities and create a competitive environment within the organization. Benchmarking helps companies:

Lowering Labour CostsOne advantage of benchmarking may be lower labour costs. For example, a small manufacturing company may study how a top competitor uses robots for several basic plant functions. These robots may help the competitor save a significant amount of money on labour costs. Company managers may obtain information on these robotics systems through the competitor's website or online articles. They may also identify the company that sold the competitor the robots. Subsequently, the company using benchmarking may call the robot manufacturer to help set up its own system.

Improving Product QualityCompanies may also use benchmarking to improve product quality. Engineers sometimes purchase leading competitors' products. They may then take them apart, study them and determine how the competitors' products outlast or outperform others in the industry. Chemical engineers may study food or cleaning products in a similar manner. They can then compare various elements contained in competitive products to their own product line. Subsequently, improvements can be made to product quality.

Increasing Sales and ProfitsA company that uses benchmarking to improve its functions, operations, products and services may enjoy increases in sales and profits. Customers are likely to notice these improvements. The benchmarking company may also promote is improvements through company brochures, its sales reps, magazine and television ads. These efforts are likely to increase sales, especially among core customers. Companies that operate more efficiently due to benchmarking can drastically lower their expenses. These savings can be lead to greater profits.

Considerations

Some organizations use internal benchmarking to improve performance in different departments. Department managers may study and emulate the best practices of one particular department. These changes may spark improvements among all departments. Internal benchmarking has its limitations, however. The company's top department may not be functioning as efficiently as others in the industry. This means the other departments were not truly benchmarking against the best departments out there.

54

Q.14 What do you understand by the term Information Systems Audit?You have been asked to conduct an IS audit for 3 locations of JB Technologies Ltd, a software development company, at multi-cities within India (17 locations), United Kingdom (3 locations), and the United States of America (13 locations). What is the basis on which you will choose your 3 locations?What areas (at least 2 main and 2 support functions) of organization’s business practices will need to be covered? List at least 3 things that you will cover in each of these business practices. Explain briefly justifying your stand taken above.Answer:

Introduction

Today, organisations are adapting to the technological changes in order to survive competition. They are adopting technology for different reasons, such as to seek competitive advantage, to achieve operational excellence and sometimes to network their different branches. This increases their operational efficiency. Application of technology by the organisation gives rise to many risks, which may be detrimental to the business. Thus, there is a need for Information System Auditing (ISA).

Information System Auditing – Explained

Information Systems Auditing involves using technical tools and expertise to evaluate the adequacy and effectiveness of Information Systems in an organisation. Further, it involves working with management to identify weak controls and risk, which arises due to the application of technology in a business. It also suggests ways to enhance these weak controls to increase the reliability of IS, which will help an organisation to achieve its strategic objectives.

Three elements in ISA that evaluate the reliability of a particular system are--

Exposures: These refer to the adverse affects that an organisation may encounter by using IS. Examples of exposures are business interruption, fraud, embezzlement, and so on. It is measured as the financial effect of any cause multiplied by the probable frequency of its occurrence.

Causes: They are the activities that adversely affect a business. A cause usually precedes exposure; a cause may generate more than one type of exposures.

Controls: They act upon causes in order to reduce exposures. They tend to reduce/eliminate the causes that lead to exposures rather than directly affecting the exposure. Controls are of different types:

i) Preventive controls

ii) Detective controls

iii) Corrective controls

Example:

Exposure—Destruction that fire may cause

55

Preventive Control—Inspection

Detective Control—Fire Alarm

Corrective controls—Fire extinguisher and sprinklers.

IS Auditing Methodology

Step 1: Define objectives of the audit.

Step 2: Obtain basic understanding of systems and flow of transactions.

Step 3: Detailed information gathering.

Step 4: Search for exposures that exist under the system and suggest the control to eliminate that exposure

Step 5: Define Auditing procedures to verify controls.

Step 6: Perform audit tests using various techniques and tools.

Step 7: Evaluation of findings.

Step 8: Generation of Report.

We need to enumerate the various functions in an organization like JBL

So there would be 3 core processes

1. Operations2. Management3. Support

Operations: Since JBL is a Software development firm, the operations would include sub processes like(these are high-level, detailing is welcome

1. Research2. Design3. Development(Coding)4. Testing5. Release6. Support

Management Process: Some of the sub-process for Management include

1. Corporate2. Finance3. Project-Management4. Audit5. Business Development6. Sales7. Communications(PR)

56

Support Process: Mostly cost-center processes

1. IT2. Quality3. HR4. Admin(Facilities, Security, Transport, vendor Management)

(Please elaborate more if you can)

Sites will be chosen with the objective to maximize the audit scope on critical processes. The locations where the most number of processes can be conveniently audited are preferable.

Make your choice of process and list 3 points for audit in each

The choice of the sites is largely based with the objective to maximize the Audit scope and obtain samples from location where the maximum number of critical processes are operational. So for example we assume the US offices may be BD(Business Development) and sales and marketing locations, the Indian Offices may have the off-shore Development centers and the datacenters and DR locations. The UK offices could be other BD and minor development. It is essential to select processes which are most critical to the business to have the highest value derived from the audit. We also assume that all locations across US would have similar and standardized setups in IS infrastructure. Hence auditing one location would be similar to auditing all locations. Location in India would be chosen based on their criticality, the location with the Datacenter needs to be selected to audit the security and safeguarding controls implemented, the remaining location can be a center where other operations originate

Functions which can be chosen for audit can be

1. Operations Processesa. Development functions

i. Segregation of development environments from the testing and Productions one ii. Security measures in place for the code libraryiii. Has adequate training been provided for developers

2. Management Processesa. Internal Audit Team

i. Findings of the last internal auditii. Risk reportsiii. The status of the actions taken to mitigate risks

3. Support Processesa. IT Team(Datacenter)

i. Location of Datacenter and protection measures from environmental threats(Fire,flooding, heating, pests etc)

ii. It team members who have access to the datacenter and logical access to the systems

iii. Change management procedures employed by the IT teamb. Admin

i. Vendor Contracts(SLA’s,)ii. Physical Securityiii. Evacuation procedures in times of disasters

57

Q.15 Describe briefly with suitable examples as to how introduction of Information System Audit can improve the organizations:a. Safeguard of assets?b. System effectiveness?c. Data integrity?Answer:

1.2.1 Safeguarding IS assets

The Information systems assets of the organization must be protected by a system of internal controls. It includes protection of hardware, software, facilities, people, data, technology, system documentation and supplies. This is because hardware can be damaged maliciously, software and data files may be stolen, deleted or altered and supplies of negotiable forms can be used for unauthorized purposes. The IS auditor will be require to review the physical security over the facilities, the security over the systems software and the adequacy of the internal controls. The IT facilities must be protected against all hazards. The hazards can be accidental hazards or intentional hazards.

Examples: need to find

1.2.2 Maintenance of Data Integrity

Data integrity includes the safeguarding of the information against unauthorized addition, deletion, modification or alteration. The desired features of the data are described here under:

a. Accuracy: Data should be accurate. Inaccurate data may lead to wrong decisions and thereby hindering the business development process.

b. Confidentiality: Information should not lose its confidentiality. It should be protected from being read or copied by anyone who is not authorized to do so.

c. Completeness: Data should be complete

d. Reliability: Data should be reliable because all business decision are taken on the basis of the current database.

e. Efficiency: The ratio of the output to the input is known as efficiency. If output is more with the same or less actual input, system efficiency is achieved, or else system is inefficient. If computerization results in the degradation of efficiency, the effort for making the process automated stands defeated. IS auditors are responsible to examine how efficient the application in relation to the users and workload.

Examples: need to find

System Effectiveness:

Effectiveness of a system is evaluated by auditing the characteristics and objectives of the system to meet substantial user requirements

58

Q.16 Define Risk.Risk management constitutes risk assessment and risk treatment. Following are some of the activities under risk management. Classify them under either risk assessment or risk treatment and explain each of these activities in brief.a. Risk identificationb. Risk analysisc. Risk evaluationd. Risk responsee. Risk transferf. Risk mitigationAnswer:

Risk is a probability or threat of damage, injury, liability, loss or any other negative occurrence that is caused by external or internal vulnerabilities.

Risk management is the application of the principles of management of IT Risks inorder to manage the risks associated with the field. IT risk management aims to manage the risks that come with the ownership, involvement, operation, influence, adoption and use of IT as part of a larger enterprise. IT risk management is a process done by IT managers to allow them to balance economic and operational costs related to using protective measures to achieve nominal gains in capability brought about by protecting the data and information systems that support an organization’s operations.

From the IT security perspective, risk management is the process of understanding and responding to factors that may lead to a failure in the confidentiality, integrity or availability of an information system. IT security risk is the harm to a process or the related information resulting from some purposeful or accidental event that negatively impacts the process or the related information. Risk is a function of the likelihood of a given threat-source’s exercising a particular potential vulnerability, and the resulting impact of that adverse event on the organization.i

(a) Risk Identification

Risks are about events that, when triggered, cause problems. Hence, risk identification can start with the source of our problems and those of our competitors (benefit), or with the problem itself.

• Source analysis- Risk sources may be internal or external to the system that is the target of risk management (use mitigation instead of management since by its own definition risk deals with factors of decision-making that cannot be managed).

59

Examples of risk sources are: stakeholders of a project, employees of a company or the eather over an airport.

Problem analysis - Risks are related to identified threats. For example: the threat of losing money, the threat of abuse of confidential information or the threat of human errors, accidents and casualties. The threats may exist with various entities, most important with shareholders, customers and legislative bodies such as the government.

When either source or problem is known, the events that a source may trigger or the events that can lead to a problem can be investigated. For example: stakeholders withdrawing during a project may endanger funding of the project; confidential information may be stolen by employees even within a closed network; lightning striking an aircraft during takeoff may make all people on board immediate casualties.

The chosen method of identifying risks may depend on culture, industry practice and compliance. The identification methods are formed by templates or the development of templates for identifying source, problem or event. Common risk identification methods are:

• Objectives-based risk identification - Organizations and project teams have objectives. Any event that may endanger achieving an objective partly or completely is identified as risk.

• Scenario-based risk identification - In scenario analysis different scenarios are created. The scenarios may be the alternative ways to achieve an objective, or an analysis of the interaction of forces in, for example, a market or battle. Any event that triggers an undesired scenario alternative is identified as risk – see Futures Studies for methodology used by Futurists.

• Taxonomy-based risk identification - The taxonomy in taxonomy-based risk identification is a breakdown of possible risk sources. Based on the taxonomy and knowledge of best practices, a questionnaire is compiled. The answers to the questions reveal risks.

• Common-risk checking- In several industries, lists with known risks are available.

Each risk in the list can be checked for application to a particular situation.

• Risk charting - This method combines the above approaches by listing resources at risk, threats to those resources, modifying factors which may increase or decrease the risk and consequences it is wished to avoid. Creating a matrix under these headings enables a variety of approaches. One can begin with resources and consider the threats they are exposed to and the consequences of each. Alternatively one can start with the threats and examine which resources they would affect, or one can begin with the consequences and determine which combination of threats and resources would be involved to bring them about.

(b) Risk Analysis

IT risk analysis is a crucial part of any IT department's job as it helps identify and manage potential problems that could affect an organization's IT infrastructure. In order to perform an IT risk analysis, IT professionals must identify any potential threats to their organization and then estimate the likelihood they will occur.

Performing an IT Risk Analysis

60

IT risk analysis is typically performed as a part of IT risk assessment and involves identifying threats, estimating risks and determining how to manage them.

Identifying Threats

There are many different types of threats that can affect IT infrastructure. For your risk analysis, you will need to list all of these possible types of threats. These can include:

Technical threats — disruption caused by technological advances or failures Structural threats — anything related to the building that houses your IT infrastructure that could

cause it to be harmed Financial threats — If the business loses funding or experiences another significant

financial change Human threats — human error or loss of important individual Natural threats — weather and natural disasters such as earthquakes, tornadoes, and floods

Estimating Risks

Estimating risk involves calculating the likelihood the risks you listed will occur. You can do this by using a risk impact/probability chart or by estimating the probability of an event happening, then using this equation to calculate the risk value: Risk Value = Probability of Event x Cost of Event.

Managing Risks

Once you've estimated how likely your listed risks are to occur, you can begin to consider management procedures. There are four types of management procedures to consider based on your determined risks.

1. Protective measures: Protective measures reduce the chance that one of your listed risks will occur.

2. Mitigation measures: Mitigation measures decreases the severity of a disaster after it has occurred.

3. Recovery activities: Recovery activities restore systems and infrastructure so that the affected organization can return to normal business operations.

4. Contingency plans: Contingency plans tell you what to do after a disruptive event or disaster occurs.

(c) Risk Evaluation

Risk evaluation is defined by the Business Dictionary as: “Determination of risk management priorities through establishment of qualitative and/or quantitative relationships between benefits and associated risks.”

So how does that relate to managed service providers or IT administrators?

Anyone responsible for a company’s data, server, network or software must perform a risk evaluation. A risk evaluation can help determine if those assets are at risk for a cyber attack, virus, data loss through natural disaster or any other threat.

The benefit of a risk evaluation is simple — it provides IT professionals with knowledge of where and how their business and reputation are at risk.

Performing a Risk Evaluation

A risk evaluation can be performed in five simple steps.

61

http://www.businessdictionary.com/definition/risk-evaluation.html

1. Identify and prioritize assets. Consider all the different types of data, software applications, servers and other assets that are managed. Determine which of these is the most sensitive or would be the most damaging to the company if compromised.

2. Locate assets. Find and list the source of those assets. Be it desktop office computers, mobile devices, internal servers or anything else, you’ll want to trace each asset back to its source.

3. Classify assets. Categorize each asset as either public information, sensitive internal information, non-sensitive internal information, compartmentalized internal information and regulated information.

4. Perform a threat modeling exercise. Identify and rate all the threats faced by your top-rated assets. Microsoft’s STRIDE method is a popular one.

5. Finalize data and make a plan. Once you have your evaluation, it’s time to start tackling those risks, beginning with the most critical.

Additional Measures

A risk evaluation is just one way IT professionals can be proactive. Another way is to use SolarWinds MSP backup and remote management software. With risk-combating features like task status alerts, hybrid cloud backups and continuous recovery.

(d) Risk Response

Risk response is the process of developing strategic options, and determining actions, to enhance opportunities and reduce threats to the project’s objectives. A project team member is assigned to take responsibility for each risk response. This process ensures that each risk requiring a response has an owner monitoring the responses, although the owner may delegate implementation of a response to someone else

Risk Response Strategies

For Threats For Opportunities

Avoid. Risk can be avoided by removing the cause of the risk or executing the project in a different way while still aiming to achieve project objectives. Not all risks can be avoided or eliminated, and for others, this approach may be too expensive or time‐consuming. However, this should be the first strategy considered.

Exploit. The aim is to ensure that the opportunity is realized. This strategy seeks to eliminate the uncertainty associated with a particular upside risk by making the opportunity definitely happen. Exploit is an aggressive response strategy, best reserved for those “golden opportunities” having high probability and impacts.

62

https://msdn.microsoft.com/en-us/library/ee823878(v=cs.20).aspx

Transfer. Transferring risk involves finding another party who is willing to take responsibility for its management, and who will bear the liability of the risk should it occur. The aim is to ensure that the risk is owned and managed by the party best able to deal with it effectively. Risk transfer usually involves payment of a premium, and thecost‐effectiveness of this must be considered when deciding whether to adopt a transfer strategy.

Share. Allocate risk ownership of an opportunity to another party who is best able to maximize its probability of occurrence and increase the potential benefits if it does occur. Transferring threats and sharing opportunities are similar in that a third party is used. Those to whom threats are transferred take on the liability and those to whom opportunities are allocated should be allowed to share in the potential benefits.

Mitigate. Risk mitigation reduces the probability and/or impact of an adverse risk event to an acceptable threshold. Taking early action to reduce the probability and/or impact of a risk is often more effective than trying to repair thedamage after the risk has occurred. Risk mitigation may require resources or time and thus presents a tradeoff between doing nothing versus the cost of mitigating the risk.

Enhance. This response aims to modify the “size” of the positive risk. The opportunity is enhanced by increasing its probability and/or impact, thereby maximizing benefits realized for the project. If the probability can be increased to 100 percent, this is effectively an exploit response.

Acceptance. This strategy is adopted when it is not possible or practical to respond to the risk by the other strategies, or a response is not warranted by the importance of the risk. When the projectmanager and the project team decide to accept a risk, they are agreeing to address the risk if and when it occurs. A contingency plan, workaround plan and/or contingency reserve may be developed for that eventuality.

Responding to Risks

Following identification and analysis of project risks, the PRMT (Project Risk Management Team) takes action to improve the odds in favor of project success. Ultimately, it is not possible to eliminate all threats or take advantage of all opportunities – but they will be documented to provide awareness that they exist and have been identified. Successful risk response will change the risk profile through the project life cycle, and risk exposure will diminish.

Risk response involves:

The PRMT determining which risks warrant a response and identifying which strategy is best for each risk.

Assigning an action to the Risk Owner to identify options for reducing the probability or impacts of each risk. The Risk Owner takes the lead and can involve experts available to the project.

Evaluating each option for potential reduction in the risk and cost of implementing the option. Selecting the best option for the project. Requesting additional contingency, if needed. Assigning an action to the Risk Owner to execute the selected response action. The Risk

Owner is the lead and may assign specific tasks to other resources to have the response implemented and documented.

63

If the PRMT judges that a risk should be accepted, it may assign an action to the Risk Owner to prepare a contingency plan if deemed necessary.

(e) Risk Mitigation

Mitigation is the most commonly considered risk management strategy. Mitigation involves fixing the flaw or providing some type of compensatory control to reduce the likelihood or impact associated with the flaw. A common mitigation for a technical security flaw is to install a patch provided by the vendor. Sometimes the process of determining mitigation strategies is called control analysis.

(f) Risk Transfer

Transference is the process of allowing another party to accept the risk on your behalf. This is not widely done for IT systems, but everyone does it all the time in their personal lives. Car, health and life insurance are all ways to transfer risk. In these cases, risk is transferred from the individual to a pool of insurance holders, including the insurance company. Note that this does not decrease the likelihood or fix any flaws, but it does reduce the overall impact (primarily financial) on the organization.

64

Q.17 Describe briefly the four major activities of Information System Audit Process. (Planning of an audit activities taking into consideration the various audit risks, Conducting of Audit that includes both compliance testing and substantive testing, Reporting of Audit findings, and Follow-up)

ANS:

1. Planning:

Planning is one of the primary & important phase of IS Audit. Planning develops the annual audit schedule to perform the individual audits. It includes budget of time and costs, and state priorities according to organizational goals and policies. The objective of audit planning is to optimize the audit resources. Important points for an audit planning:

The extent of planning will vary according to the size of the entity, complexity of the audit etc. The auditor should develop and document an overall audit plan and underlying audit program and

methodology. Changes to audit plan should follow a change management procedure.

The planning phase consists of five key steps.

Determine audit subject. Define audit objective. Set audit scope. Perform pre-audit planning. Determine audit procedures and steps for data gathering.

Risk assessment in Planning:

To correctly and completely assess the risk that is related to the complete scope of the IS audit area, professionals should consider the following elements when developing the IS audit plan:

Full coverage of all areas within the scope of the IS audit universe, which represents the range of all possible audit activities

Reliability and suitability of the risk assessment provided by management The processes followed by management to supervise, examine and report possible risk or issues Cover risk in related activities relevant to the activities under review

Various Risks:

Inherent Risk: Inherent risk is the susceptibility of an audit area to err in a way that could be material, individually or in combination with other errors, assuming that there were no related internal controls. For example, the inherent risk associated with operating systems without appropriate controls is ordinarily high, since changes to, or even disclosure of, data or programs through operating system security weaknesses could result in false management information or competitive disadvantage. By contrast, the inherent risk associated with security for a stand-alone PC without controls, when a proper analysis demonstrates it is not used for business-critical purposes, ordinarily is low. Inherent risk for most IS audit areas is high since the potential effects of errors ordinarily spans several business systems and many users.

65

Control Risk: Control risk is the risk that an error that could occur in an audit area and could be material, individually or in combination with other errors, will not be prevented or detected and corrected on a timely basis by the internal control system. For example, the control risk associated with manual reviews of computer logs can be high because of the volume of logged information. The control risk associated with computerized data validation procedures ordinarily is low because the processes are applied consistently.Professionals should assess the control risk as high unless relevant internal controls are:

Identified Evaluated as effective Tested and proved to be operating appropriately

Detection Risk: Detection risk is the risk that professionals’ substantive procedures will not detect an error that could be material, individually or in combination with other errors. For example, the detection risk associated with identifying breaches of security in an application system ordinarily is high because logs for the whole period of the audit are not available at the time of the audit. The detection risk associated with identifying a lack of disaster recovery plans ordinarily is low, since existence is verified easily. In determining the level of substantive testing required, the professionals should consider the:

Assessment of inherent risk Conclusion reached on control risk following compliance testing

2. Conducting an Audit:

In the performance of Audit Work the Information Systems Audit Standards require us to provide supervision, gather audit evidence and document our audit work. We achieve this objective through:

Establishing an Internal Review Process where the work of one person is reviewed by another,

preferably a more senior person.

We obtain sufficient, reliable and relevant evidence to be obtained through Inspection,

Observation, Inquiry, Confirmation and recomputation of calculations

We document our work by describing audit work done and audit evidence gathered to support the

auditors’ findings.

The compliance testing phase

The objective of this phase is to determine whether or not the system of internal controls operates as it is supposed to operate. The auditor checks whether all internal controls exist and are working reliably. The auditor makes use of both manual sources of information mentioned above and computer-assisted evidence collection techniques to gather inputs for evaluation.

At the conclusion of this phase, the auditor must evaluate the internal control system in the light of the evidence collected on the reliability of individual controls.

The substantive testing phase

The objective of this phase is to obtain sufficient evidence to enable the auditor make a final judgement on whether or not material losses have occurred during computer data processing. The external and the internal auditor express the results of this phase differently. The former expresses his judgement in the

66

form of an opinion as to whether any misstatement of accounts really exists. The latter however, is concerned with a broader perspective i.e. given the state of the internal control system, have the losses occurred or could they occur in future due to the weaknesses in control systems used to safeguard assets.

The following are the five types of substantive tests that can be used within a data processing installation:

1. Tests to identify erroneous processing2. Tests to assess the quality of data3. Tests to identify inconsistent data4. Tests to compare data with physical counts5. Confirmation of data with outside sources6.

3. Reporting of Audit Findings: Upon the performance of the audit test, the Information Systems Auditor is required to produce and appropriate report communicating the results of the IS Audit. The audit report contains the conclusions of audit work or an opinion that is related to the objectives of the audit. Auditing standards stipulate that reports contain certain information; the order and structure within which that content is presented is driven by relevant practices and the need to make reports readable and understandable. A good audit report contains precise and concise facts that are easily understood by the readers. In addition to terminology, language, report structure, content requirements and protocol, sentence structure and punctuation are also important considerations. An IS Audit report should:

Identify an organization, intended recipients and any restrictions on circulation

State the scope, objectives, period of coverage, nature, timing and the extend of the audit work

State findings, conclusions, recommendations and any reservations, qualifications and limitations

Provide audit evidence

4. Follow up:

Follow-up is performed by professionals is a process by which they determine the adequacy, effectiveness and timeliness of actions taken by management on reported observations and recommendations, including those made by external auditors and others.

A follow-up process should be established to help provide reasonable assurance that each review conducted by professionals provides optimal benefit to the enterprise by requiring that agreed-on outcomes arising from reviews are implemented in accordance with management undertakings or that (executive) management recognizes and acknowledges the risk of delaying or not implementing proposed outcomes and/or recommendations.

Procedures for follow-up activities should be established and should include:

• The recording of a time frame within which management should respond to agreed-on recommendations.

• An evaluation of management’s response• A verification of the response, if appropriate • Follow-up work, if appropriate

67

• A communication procedure that escalates outstanding and unsatisfactory responses and/or actions to the appropriate levels of management and to those charged with governance

• A process for obtaining management’s assumption of associated risk, in the event that corrective action is delayed or not proposed to be implemented.

68

Q.18 Explain the terms: (i) IT Service Management (ITSM) and (ii) IT Service Management System (ITSMS)? Describe briefly all 13 IT service processes of ISO 20000-1:2011 and map the same with software service activities.

IT service management (ITSM) refers to the entirety of activities – directed by policies, organized and structured in processes and supporting procedures – that are performed by an organization to plan, design, deliver, operate and control information technology (IT) services offered to customers. It is thus concerned with the implementation of IT services that meet customers' needs, and it is performed by the IT service provider through an appropriate mix of people, process and information technology.

ITSMS: An IT Service Management System (ITSMS) is a systematic approach to managing the IT services delivered to customers (internal or external). It encompasses people, processes and IT systems.

ISO 20000 is the first international standard for Information Technology Service Management and is fully compatible and supportive of the ITIL (IT Infrastructure Library) framework. ISO/IEC 20000-1:2011 specifies four key service management processes broken into 13 IT processes:1. Service Delivery Processes – includes Service Level Management, Availability Management, and

Capacity Management, Service Reporting, Information Security Management, Budgeting and Accounting

2. Control Processes – involves managing changes, assets, and configurations3. Relationship Processes – involves interfaces between the service provider and customers and

suppliers4. Resolution Processes – focuses on incidents being resolved or prevented

1. Service Delivery Process:-

a. Capacity Management: Adjustment of the capacity of a resource (equipment, machine, or system) to meet a planned demand or load. In general, manufacturing capacity may be adjusted by working overtime or redeploying the manpower. In many organizations 24*7 support is required. So the capacity of the resource is adjusted to work in 3 shifts of 8 hours each so that the support is given for 24 hours.

b. Service Level Management: Implementing Service Level Management can only be completely successful when the other ITIL processes are implemented as well. The main aim of SLM is to ensure the quality of the IT services provided, at a cost acceptable to the business/customer. The goal for SLM is to maintain and improve on service quality through a constant cycle of agreeing, monitoring, reporting and improving the current levels of service. Software Example:- In software industry for various projects we have batch running for generating reports or processing large data for warehouses. Such batches have defined SLA which define the time within which the data or reports are expected.

c. Availability and Service Continuity Management: The two processes, availability and service continuity management, must ensure that the agreed objectives of availability and continuity for the customer can be met in every case. It is vital that all activities and expenditure, as well as their sources assigned for the implementation of the continuity and availability targets, should be coordinated with the requirements of the business. The availability must be recorded for monitoring in

69

order to identify and document deviations from the defined targets. We also recommend that the effectiveness of improvement measures which have been introduced should then be reviewed. The availabilities and planned maintenance windows must be forecast in advance and communicated to all those involved. This will enable preventative maintenance to be carried out on a targeted basis.The service continuity strategy must be reviewed jointly with the business representatives on a continual basis, at least however annually.

d. Service Reporting: A clear definition must be provided for all reports as to the intention and purpose of the report, its target groups and, in particular, the data sources. Reporting needs identified from customer requirements must be met. The success of all service management processes depends upon the utilization of the information from the service reports. The management decisions, together with corrective action, must be based on the results of the service reports and communicated to all relevant parties

e. Budgeting and Accounting: The aim of budgeting and accounting for IT services is to budget for and provide documentary evidence of the costs for service provision. The costs expended for the budgeting and accounting processes must be determined according to customer, service provider and supplier demand. The benefits of recording operational data must justify the expense.

f. Information Security Management: The objective of the information security management is to provide effective control and monitoring of the information security for all service activities. Information security is a system of guidelines and procedures for identifying, controlling and protecting information and all operating materials associated with its/their storage, transfer and processing.

2. Control Process:-

a. Change Management: The aim of change management is to ensure that all changes are evaluated, approved, introduced and reviewed using stipulated methods. In this context the focus is on the efficient and prompt implementation with minimal risk to the operational business.The change management processes and procedures are intended to ensure that changes have a clearly defined and documented scope. Only changes which have an identified business benefit will be authorized. Changes should be planned on the basis of priority and potential risk. Changes to configurations must be verified during the implementation of the change.The status of the changes and planned dates for implementation form the basis for change and release planning. Information on dates should be communicated to the persons affected by the change.

b. Release Management: Whereas change management concentrates on controlling changes, release management prepares the planned changes for distribution. Release management should be integrated into the configuration and change management processes in order to ensure that the releases and implemented changes are coordinated. Release management coordinates the activities of the service provider, suppliers and business cycles. The outcome of this is a plan for the supply of a release to the operational IT environment. The aim of the release management is to deliver, distribute and monitor one or more changes in a release to the operational environment. One of the key tasks of release management process is to coordinate all the participating resources in order to hand over a release to a shared environment. In this context good planning and management is a basic prerequisite for packaging releases, their successful distribution as well as for having the associated impact and risks for the business and the IT under control.

70

c. Configuration Management: The aim of configuration management is to define and control the components of the service and infrastructure and to manage precise configuration information.All key assets and configurations should be assigned to the responsibility of a manager who ensures appropriate security and control. This is intended to guarantee, amongst other things, that approval is obtained before changes to the CI are implemented. The following recommendations for meeting the specifications for the configuration management process have become established practice:1. Planning and implementation2. Configuration identification3. Configuration control4. Proof of status5. Verification and audit

3. Relationship Process:-The relationship processes describe the two aspects of business relationship management and supplier management. In this context the standard focuses on the role of the service provider (frequently a company’s IT organization) which is logically positioned between customer and supplier. Both customers as well as suppliers can be part of the service provider’s organization or external. A fundamental distinction is drawn between the following three levels for the contracts:• The agreements between the customer and service provider are known as service level agreements

(SLA).• External support (suppliers) required for the agreed IT services are formalized with underpinning

contracts.• Operational level agreements govern the relationships within the IT organization for the service delivery.

In order to create good relationships between the participating parties’ clear agreements must be in place. In this context, all parties should have the same understanding of the business requirements, service capacity as well as the framework conditions and the respective responsibilities and obligations. This is the only way in which each party can meet its performance obligation.a. Supplier Management: The aim of supplier management is to control suppliers in order to ensure a smooth delivery of high quality services .As a general rule there are a number of suppliers involved. These are often also subdivided into main suppliers and subcontract suppliers. It is therefore necessary to clearly define whether the service provider is to negotiate directly with all suppliers or whether a main supplier is to take over the responsibility for the subcontract suppliers.The supplier management process must ensure that the supplier understands its obligations to the service provider. The requirements must therefore be clearly defined and agreed. It is also necessary to ensure that all changes to these agreements are monitored by the change management process. In order to avoid conflicts we recommend that records be created of all official business transactions between all the parties. The services of the supplier must be continually monitored and an appropriate response taken as required.

b. Business Relationship Management: The aim of business relationship management is to understand the customer and the business process drivers and based on this to establish and maintain a good relationship between the service provider and the customer.Three key aspects must be anchored within the organization in order to meet the requirements demanded of business relationship management:• Regular service reviews• Service complaints procedure• Measurement of customer satisfaction

71

There is no separate business relationship management process in ITIL V3.

4. Resolution Process:-The resolution processes include the incident and problem management processes. These are standalone processes even if they are closely interlinked. Incident management deals with the restoration of the service for the service user. Problem management by contrast deals with the identification and elimination of root causes in the case of major or repeat disruptions and therefore ensures a permanent and stable service infrastructure

a. Incidence Management: The aim of incident management is to restore the agreed service for the business and respond to service enquiries as quickly as possible. In order to fulfill the specification requirements it is necessary to ensure that the incident management is designed as a reactive and proactive process that responds to error messages. The process must focus on the restoration of the IT service concerned and consciously not deal with the identification of the root cause.The incident process (incidents and service requests) comprises receiving calls, recording, prioritization, taking account of security provisions as well as following up on the incident processing status. It should also govern the agreement on fault processing with the customer as well as any escalation procedures. All incidents must be recorded in such a way as to enable the relevant information to rectify the error to be ascertained and analyzed. The progress of work should be reported to the current and any potential personnel affected. All activities must be fully recorded in the incident ticket.Wherever possible, customers must be able to continue their business in the appropriate way. Workarounds can also be utilized for this purpose

b. Problem Management: The aim of problem management is to minimize the disruption to and impact on the business by proactively identifying and analyzing the root causes of service incidents and by managing problems until these are rectified.Problem management must identify the root causes of the incidents on a reactive basis and proactively prevent incidents reoccurring. Problems are to be classified as known errors as soon as the root cause of the incident is known and a solution method for avoiding such incidents has been found.

For incident management to receive an optimum supply of information, all known errors and IT services affected must be documented and the associated configuration items identified. Known errors should only be closed once a definitive, successful solution has been found.

Once the root cause has been identified and a decision reached on the solution, this solution must be dealt with by the change management process. Information on the progress, potential workarounds or permanent solutions must be sent to all parties involved.

The closure of problem tickets should always be carried out in accordance with the following reviews:• Has the solution been precisely documented?• Has the root cause been categorized in order to provide support for future further analyses?• Have the customers and support employees affected been informed of the solution?• Has the customer confirmed that he/she accepts the solution?• Has the customer been informed if no solution has been found?The effectiveness of completed solutions to problems must be reviewed. In particular, trends such as for example reoccurring problems and incidents, defects, errors, known errors in planned releases or resource commitments must be identified by employees.

72

Q.19 How does control of “Change” and “Configuration” help in controlling Quality of Services in Business Application Releases?Change and configuration management are fundamental to product development. The process of taking market requirements, customer requests, and innovative ideas and turning them into viable products cannot be accomplished without managing product change. While informal changes are accepted early in the design cycle, changes become formal as change complexity and cost increase, and as product matures. The change and configuration management process-controls how informal and formal changes are proposed, analyzed, planned, implemented, and released, as the product evolves from conception to retirement.

Understanding the Need for Change and Configuration Management

Many companies face challenges around the use of change and configuration management procedures stemming primarily from the fact that the process is typically manual and paper-based. As a result, it is very slow and bureaucratic.

An overly complex process coupled with a lack of users’ knowledge, results in low process adoption. Changes are either avoided or the process is bypassed, and as a result the downstream documentation is either incomplete or incorrect, and valuable design and configuration history is lost, which causes problems in downstream product development stages.

The process is further complicated due to companies using multiple systems managing product information and different applications to aid in the change process execution. As a result it is difficult to find, analyze, monitor and provide status of change information.

An increased number of global and external contributors in the product development process complicates change coordination and information access across multiple organizations. Companies must consider how change and process is executed in a highly distributed environment and how product data and intellectual property is secured as change information is delivered to and from contributors.

As product complexity, variants, and options increase, so too does the need to manage change and product configurations. Inadequate configuration management practices make it difficult to capture important product milestones, track incremental product updates and make updates to configurations impacted by change.

Benefits of an Optimized Process for Change and Configuration Management

A flexible, responsive and efficient change and configuration management process improve a companys’ ability to compete. Typical benefits from improving the change and configuration management process may include:

1. Improve Business Decisions

• Replace legacy systems and applications with single enterprise process

• Enable reliable best practices process approaches supporting minor and major changes

2. Increase Innovation

• Increase overall engineering time by reducing the change related administrative workload

73

• Minimize rework and work task coordination across the extended enterprise by automating the process

3. Reduce Product Cost

• Early knowledge of changes allows manufactures to better prepare for change and minimize inventory

4. Attain Higher Quality

• Ensure product quality across the product lifecycle by improving process adoption

• Reduce manufacturing related issues by incorporating quality improvement changes early in design process

5. Reduce Time-To-Market

• Reduce product development cycle time and downtime awaiting change decisions by providing accurate documentation of changes

74

Q.20 What are the steps involved in risk management detailing risk assessment and risk treatment processes?

Risk is a probability or threat of damage, injury, liability, loss, or any other negative occurrence that is caused by external or internal vulnerabilities.

Risk management is the application of the principles of management to IT Risks in order to manage the risks associated with the field. IT risk management aims to manage the risks that come with the ownership, involvement, operation, influence, adoption and use of IT as part of a larger enterprise. IT risk management is a process done by IT managers to allow them to balance economic and operational costs related to using protective measures to achieve nominal gains in capability brought about by protecting the data and information systems that support an organization’s operations.

Fig. Risk Management Process ISO 31000:2009

The risk management process should be:

An integral part of management, Embedded in the culture and practices, and Tailored to the business processes of the organization.

1. Communication and consultation with external and internal stakeholders should take place during all stages of the risk management process. Effective external and internal communication and consultation should take place to ensure that those accountable for implementing the risk management process and stakeholders understand the basis on which decisions are made, and the reasons why particular actions are required.

75

2. Establishing the context: By establishing the context, the organization articulates its objectives, defines the external and internal parameters to be taken into account when managing risk, and sets the scope and risk criteria for the remaining process.

Risk assessment is the overall process of risk identification, risk analysis and risk evaluation.

3. Risk Identification: The organization should identify sources of risk, areas of impacts, events (including changes in circumstances) and their causes and their potential consequences. The aim of this step is to generate a comprehensive list of risks based on those events that might create, enhance, prevent, degrade, accelerate or delay the achievement of objectives. It is important to identify the risks associated with not pursuing an opportunity. Comprehensive identification is critical, because a risk that is not identified at this stage will not be included in further analysis.

4. Risk analysis: Risk analysis involves developing an understanding of the risk. Risk analysis provides an input to risk evaluation and to decisions on whether risks need to be treated, and on the most appropriate risk treatment strategies and methods. Risk analysis can also provide an input into making decisions where choices must be made and the options involve different types and levels of risk. Risk analysis involves consideration of the causes and sources of risk, their positive and negative consequences, and the likelihood that those consequences can occur. Factors that affect consequences and likelihood should be identified. Risk is analyzed by determining consequences and their likelihood, and other attributes of the risk. An event can have multiple consequences and can affect multiple objectives. Existing controls and their effectiveness and efficiency should also be taken into account.

5. Risk evaluation: The purpose of risk evaluation is to assist in making decisions, based on the outcomes of risk analysis, about which risks need treatment and the priority for treatment implementation. Risk evaluation involves comparing the level of risk found during the analysis process with risk criteria established when the context was considered. Based on this comparison, the need for treatment can be considered. Decisions should take account of the wider context of the risk and include consideration of the tolerance of the risks borne by parties other than the organization that benefits from the risk. Decisions should be made in accordance with legal, regulatory and other requirements.

6. Risk treatment: Risk treatment involves selecting one or more options for modifying risks, and implementing those options. Once implemented, treatments provide or modify the controls.

Selecting the most appropriate risk treatment option involves balancing the costs and efforts of implementation against the benefits derived, with regard to legal, regulatory, and other requirements such as social responsibility and the protection of the natural environment. Decisions should also take into account risks which can warrant risk treatment that is not justifiable on economic grounds, e.g. severe (high negative consequence) but rare (low likelihood) risks.

The purpose of risk treatment plans is to document how the chosen treatment options will be implemented. The information provided in treatment plans should include:

The reasons for selection of treatment options, including expected benefits to be gained; Those who are accountable for approving the plan and those responsible for implementing the

plan; Proposed actions; Resource requirements including contingencies; Performance measures and constraints; Reporting and monitoring requirements; and Timing and schedule.

76

Treatment plans should be integrated with the management processes of the organization and discussed with appropriate stakeholders.

7. Monitoring and review: Both monitoring and review should be a planned part of the risk management process and involve regular checking or surveillance. It can be periodic or ad hoc. Progress in implementing risk treatment plans provides a performance measure. The results can be incorporated into the organization's overall performance management, measurement and external and internal reporting activities. The results of monitoring and review should be recorded and externally and internally reported as appropriate.

77

Q-21 Explain the terms: (i) Information Security (IS), (ii) Information Security Management (ISM) and Information Security Management System (ISMS)? Describe briefly the 13 control areas or domains of ISO 27001:2013.

Information Security, sometimes shortened to InfoSec, is the practice of defending information from unauthorized access, use, disclosure, disruption, modification, perusal, inspection, recording or destruction. It is a general term that can be used regardless of the form the data may take (electronic, physical, etc.)

“It is preservation of confidentiality, integrity and availability of information. In addition, other properties, such as authenticity, accountability, non-repudiation and reliability can also be involved." Information is an asset to all individuals and businesses. Information Security refers to the protection of these assets in order to achieve C - I - A:

Confidentiality - protecting information from being disclosed to unauthorized parties.

Integrity - protecting information from being changed by unauthorized parties.

Availability - to the availability of information to authorized parties only when requested.

Information Security Management (ISM) describes controls that an organization needs to implement to ensure that it is sensibly managing these risks.

The risks to these assets can be calculated by analysis of the following issues:

Threats to your assets. These are unwanted events that could cause the deliberate or accidental loss, damage or misuse of the assets

Vulnerabilities. How susceptible your assets are to attack

Impact. The magnitude of the potential loss or the seriousness of the event.

Standards that are available to assist organizations implement the appropriate programs and controls to mitigate these risks are for example BS7799/ISO 17799, Information Technology Infrastructure Library and COBIT.

Information Security Management System (ISMS)

An information security management system (ISMS) is a set of policies and procedures for systematically managing an organization's sensitive data. The goal of an ISMS is to minimize risk and ensure business continuity by pro-actively limiting the impact of a security breach. It is a systematic approach to managing sensitive company information so that it remains secure. It includes people, processes and IT systems by applying a risk management process. It can help small, medium and large businesses in any sector keep information assets secure. Using this family of standards will help your organization manage the security of assets such as financial information, intellectual property, employee details or information entrusted to you by third parties.

78

14 control areas or domains of ISO 27001:2013

1. Information Security Policies: The Information Security Policies clause addresses the need to define, publish and review different types of policies required for information security management.

2. Organization of Information Security: The Organization of Information Security clause addresses the need to define and allocate the necessary roles and responsibilities for information security management processes and activities. This includes controls related to the definition of information security roles and responsibilities, segregation of duties, contact with authorities, contact with special interest groups, information security in project management and mobile devices and teleworking.

3. Human Resource Security: The Human Resource Security clause addresses the required controls for processes related to staff recruiting, their job during employment and after the termination of their contracts. These considerations should include information security coordination, allocation of information security responsibilities, authorization processes for information processing facilities, confidentiality agreements, contact with authorities, contact with special interest groups, independent review of information security, identification of risks related to external parties, addressing security when dealing with customers, addressing security on contractors’ agreements, etc.

4. Asset Management: The Asset Management clause addresses the required responsibilities to be defined and allocated for the asset management processes and procedures. The owner of the assets and other parts involved in this matter should be identified to be held accountable for assets’ security, including classification, labelling, and handling of information; and information processing facilities should be identified and maintained. Moreover, this clause addresses controls on management of removable media, disposal of media, and physical media transfer.

5. Access Control: The Access controls clause addresses requirements to control access to information assets and information processing facilities. The controls are focused on the protection against accidental damage or loss, overheating, threats, etc. This requires a documented control policy and procedures, registration, removal and review of user access rights, including here physical access, network access and the control over privileged utilities and restriction of access to program source code.

6. Cryptography: The Cryptography clause addresses policies on cryptographic controls for protection of information to ensure proper and effective use of cryptography in order to protect the confidentiality, authenticity, integrity, non-repudiation and authentication of the information. It also includes the need for digital signatures and message authentication codes, and cryptographic key management.

7. Physical and Environmental Security: The Physical and Environmental Security clause addresses the need to prevent unauthorized physical access, damage and interference to the organization’s information and information processing facilities. Controls cover: to physically secure the perimeter of office rooms and facilities, protection against external and environmental threats, prevent loss, damage, theft or compromise of assets, protect the equipment from power failures, cabling should be protected from interception or damage, maintenance of equipment, etc.

79

8. Operations Security: The Operations security clause addresses the organization’s ability to ensure correct and secure operations. The controls cover the need for operational procedures and responsibilities, protection from malware, backup, logging and monitoring, control of operational software, technical vulnerability management, information systems audit considerations.

9. Communication Security: The Communication Security clause addresses the organization’s ability to ensure protection of information in systems and applications in networks and its supporting information processing facilities. Controls cover security of information in networks and connected services from unauthorized access, transfer policies and procedures, secure transfer of business information between the organization and external parties, information involved in electronic messaging, the need for confidentiality or non-disclosure agreements.

10. System Acquisition, Development and Maintenance: The System Acquisition, Development and Maintenance clause covers controls for identification, analyses and specification of information security requirements, securing application services in development and support processes, technical review restrictions on changes to software packages, secure system engineering principles, secure development environment, outsourced development, system security testing, system acceptance testing and protection of test data.

11. Supplier Relationships: The Supplier Relationships clause addresses controls for supplier’s relationship issues, including here information security policies and procedures, addressing security within supplier agreements, communication and awareness about technology supply chain and service delivery management.

12. Information Security Incident Management: The Information Security Incident Management clause covers controls for responsibilities and procedures, reporting information and security weaknesses, assessment of and decision on information security events, response to information security incidents, learning from information security incidents, and collection of evidence.

13. Information Security Aspects of Business Continuity Management: The Business Continuity Management clause addresses the organization’s ability to counteract interruptions to normal operations, including availability of information processing facilities, verify, review and evaluate information security continuity, implementing information security continuity, and planning information security continuity.

14. Compliance: The Compliance clause addresses the organization’s ability to remain in compliance with regulatory, statutory, contractual, and security requirements, including: identification of applicable legislation and contractual requirements, intellectual property rights, protection of records, privacy and protection of personally identifiable information, regulation of cryptographic controls, independent review of information security, compliance with security policies and standards, and technical compliance review.

80

Q.22 Explain the terms: (i) Business Continuity, (ii) Disaster Management and (iii) Business Continuity Management System (BCMS)? What are the roles of suppliers and contractors during the disruption of business service activities?

(i) Business Continuity (BC):The capability of the organization to continuedelivery of products or services at acceptablepredefined levels following a disruptiveincidentBC is about building and improvingresilience in your business; it’s about identifying your key products and services and the mosturgent activities that underpin them and then, once that ‘analysis’ is complete, it is about devisingplans and strategies that will enable you to continue your business operations and enable you to recoverquickly and effectively from any type disruption whatever its size or cause. It gives you a solid framework to lean on in times of crisis and provides stability and security. In fact, embedding BC into your business is proven to bring business benefits.

(ii) Disaster ManagementDisaster management (or emergency management) is the creation of plans through which communities reduce vulnerability to hazards and cope with disasters. Disaster management does not avert or eliminate the threats; instead, it focuses on creating plans to decrease the effect of disasters. Failure to create a plan could lead to human mortality, lost revenue, and damage to assets. Currently in the United States 60 percent of businesses do not have emergency management plans. Events covered by disaster management include acts of terrorism, industrial sabotage, fire, natural disasters (such as earthquakes, hurricanes, etc.), public disorder, industrial accidents, and communication failures.

(iii) Business Continuity Management System (BCMS):Business Continuity Management (BCM) is defined as a holistic management process that identifies potential threats to an organization and the impacts to business operations those threats, if realized, might cause, and which provides a framework for building organizational resilience with the capability of an effectiveresponse that safeguards the interests of its key stakeholders, reputation, brand and value-creating activities

Roles of suppliers and contractors during the disruption of business service activities

1. PunctualityVendors are responsible for providing the products they offer in a time frame that is consistent with the expectations they create. A vendor who fails to deliver a product on time can create trouble for business customers whose sales depend on the promised materials; a vendor who is late in delivering a product to a private customer can cause inconvenience or waste his customer's time.

2. HonestyThe responsibility of a vendor to be honest with her customers extends from her claims about her product's effectiveness to her billing practices. Vendors should never make false claims about the materials in products or about their own qualifications for providing these products. 3. SafetyVendors have a responsibility to create products that, if used properly, will not harm customers. Equipment manufacturers are responsible for testing equipment to ensure that it holds up under typical working conditions and even when roughly used. Vendors of food products have a responsibility to handle their products at safe temperatures and away from contaminants in order to minimize the risk of foodborne illness.

81

https://en.wikipedia.org/wiki/Public_disorder

https://en.wikipedia.org/wiki/Hurricanes

https://en.wikipedia.org/wiki/Earthquakes

https://en.wikipedia.org/wiki/Natural_disaster

https://en.wikipedia.org/wiki/Fire

https://en.wikipedia.org/wiki/Sabotage

https://en.wikipedia.org/wiki/Terrorism

Q.23 Name the five major areas of COBIT5 and explain your understanding on each one of them.

1. EDM - Evaluate, Direct and Monitor,The EDM domain covers setting the governance framework, establishing responsibilities in terms of value (e.g., investment criteria), risk factors (e.g., risk appetite) and resources (e.g., resource optimization), and maintaining transparency on IT to stakeholders.

2. APO - Align, Plan and Organise,The APO domain concerns the identification of how IT can best contribute to the achievement of the business objectives. Specific processes within the APO domain relate to IT strategy and tactics, enterprise architecture, innovation and portfolio management. Other important processes address the management of budgets and costs, human resources, relationships, service agreements, suppliers, quality, risk, and security.

3. BAI - Build, Acquire and Implement,The BAI domain makes IT strategy concrete by identifying the requirements for IT and managing the IT investment program and projects within that program. This domain also addresses the management of capacity; organizational change; IT change management; acceptance and transitioning; and knowledge, asset and configuration management.

4. DSS - Deliver, Service and Support,The DSS domain refers to the actual delivery of the IT services required to meet strategic and tactical plans. The DSS domain includes processes to manage operations, service requests and incidents, as well as the management of problems, continuity, security services and business process controls.

5. MEA - Monitor, Evaluate and AssessMEA, includes processes that are responsible for the assessment of process performance and conformance, evaluation of internal control adequacy, and monitoring of regulatory compliance

82

Q.24 Name and explain in brief (one or two sentences) the five COBIT 5 Principles and seven enterprise Enablers referred in COBIT 5 framework.

Principle 1: Meeting Stakeholder Needs: Enterprises have many stakeholders, and ‘creating value’ means different—and sometimes

conflicting—things to each of them. Governance is about negotiating and deciding amongst different stakeholders’ value interests. The governance system should consider all stakeholders when making benefit, resource and risk

assessment decisions. For each decision, the following can and should be asked:

o Who receives the benefits? o Who bears the risk? o What resources are required?

The COBIT 5 goals cascade translates stakeholder needs into specific, practical and customised goals within the context of the enterprise, IT-related goals and enabler goals.

Principle 2: Covering the Enterprise End-to-end: COBIT 5 addresses the governance and management of information and related technology from

an enterprisewide, end-to-end perspective. This means that COBIT 5:

Integrates governance of enterprise IT into enterprise governance, i.e., the governance system for enterprise IT proposed by COBIT 5 integrates seamlessly in any governance system because COBIT 5 aligns with the latest views on governance.

Covers all functions and processes within the enterprise; COBIT 5 does not focus only on the ‘IT function’, but treats information and related technologies as assets that need to be dealt with just like any other asset by everyone in the enterprise.

Principle 3: Applying a Single Integrated Framework: COBIT 5 aligns with the latest relevant other standards and frameworks used by enterprises:

Enterprise: COSO, COSO ERM, ISO/IEC 9000, ISO/IEC 31000 IT-related: ISO/IEC 38500, ITIL, ISO/IEC 27000 series, TOGAF, PMBOK/PRINCE2,

CMMI This allows the enterprise to use COBIT 5 as the overarching governance and management

framework integrator.Principle 4: Enabling a Holistic ApproachCOBIT 5 enablers are:

Factors that, individually and collectively, influence whether something will work—in the case of COBIT, governance and management over enterprise IT

Driven by the goals cascade, i.e., higher-level IT-related goals define what the different enablers should achieve

Systemic governance and management through interconnected enablers—To achieve the main objectives of the enterprise, it must always consider an interconnected set of enablers, i.e., each enabler:

o Needs the input of other enablers to be fully effective, e.g., processes need information, organisational structures need skills and behaviour

o Delivers output to the benefit of other enablers, e.g., processes deliver information, skills and behaviour make processes efficient

Principle 5: Separating Governance from Management: The COBIT 5 framework makes a clear distinction between governance and management. These two disciplines:

83

Encompass different types of activities Require different organisational structures Serve different purposes

Governance—In most enterprises, governance is the responsibility of the board of directors under the leadership of the chairperson.

Management—In most enterprises, management is the responsibility of the executive management under the leadership of the CEO.

Governance ensures that stakeholders needs, conditions and options are evaluated to determine balanced, agreed-on enterprise objectives to be achieved; setting direction through prioritisation and decision making; and monitoring performance and compliance against agreed-on direction and objectives (EDM).

Management plans, builds, runs and monitors activities in alignment with the direction set by the governance body to achieve the enterprise objectives (PBRM).

7 enterprise enablers in Cobit 5 Processes—Describe an organised set of practices and activities to achieve certain objectives

and produce a set of outputs in support of achieving overall IT-related goals Organisational structures—Are the key decision-making entities in an organisation Culture, ethics and behaviour—Of individuals and of the organisation; very often

underestimated as a success factor in governance and management activities Principles, policies and frameworks—Are the vehicles to translate the desired behaviour into

practical guidance for day-to-day management Information—Is pervasive throughout any organisation, i.e., deals with all information produced

and used by the enterprise. Information is required for keeping the organisation running and well governed, but at the operational level, information is very often the key product of the enterprise itself.

Services, infrastructure and applications—Include the infrastructure, technology and applications that provide the enterprise with information technology processing and services

People, skills and competencies—Are linked to people and are required for successful completion of all activities and for making correct decisions and taking corrective actions

84

Q.25 What is your understanding about (i) risk and (ii) risk management with respect to an IT enterprise? Explain how risk is related to the terms threat, likelihood and impact. Explain briefly the activities that are carried out during risk identification, risk estimation, risk evaluation and risk treatment in the overall risk management process. Explain “Heat Chart” or “Severity Chart” in brief.

Risk is a probability or threat of damage, injury, liability, loss, or any other negative occurrence that is caused by external or internal vulnerabilities

Risk management is the application of the principles of management to IT Risks in order to manage the risks associated with the field. IT risk management aims to manage the risks that come with the ownership, involvement, operation, influence, adoption and use of IT as part of a larger enterprise. IT risk management is a process done by IT managers to allow them to balance economic and operational costs related to using protective measures to achieve nominal gains in capability brought about by protecting the data and information systems that support an organization’s operations.

IdentificationRisks are about events that, when triggered, cause problems. Hence, risk identification can start with the source of our problems and those of our competitors (benefit), or with the problem itself.

Source analysis- Risk sources may be internal or external to the system that is the target of risk management (use mitigation instead of management since by its own definition risk deals with factors of decision-making that cannot be managed).

Examples of risk sources are: stakeholders of a project, employees of a company or the weather over an airport.

Problem analysis - Risks are related to identified threats. For example: the threat of losing money, the threat of abuse of confidential information or the threat of human errors, accidents and casualties. The threats may exist with various entities, most important with shareholders, customers and legislative bodies such as the government.

When either source or problem is known, the events that a source may trigger or the events that can lead to a problem can be investigated. For example: stakeholders withdrawing during a project may endanger funding of the project; confidential information may be stolen by employees even within a closed network; lightning striking an aircraft during takeoff may make all people on board immediate casualties.The chosen method of identifying risks may depend on culture, industry practice and compliance. The identification methods are formed by templates or the development of templates for identifying source, problem or event.

Common risk identification methods are:Risk Assessment

Objectives-based risk identification - Organizations and project teams have objectives. Any event that may endanger achieving an objective partly or completely is identified as risk.

Scenario-based risk identification - In scenario analysis different scenarios are created. The scenarios may be the alternative ways to achieve an objective, or an analysis of the interaction of

85

forces in, for example, a market or battle. Any event that triggers an undesired scenario alternative is identified as risk – see Futures Studies for methodology used by Futurists.

Taxonomy-based risk identification - The taxonomy in taxonomy-based risk identification is a breakdown of possible risk sources. Based on the taxonomy and knowledge of best practices, a questionnaire is compiled. The answers to the questions reveal risks.

Common-risk checking - In several industries, lists with known risks are available. Each risk in the list can be checked for application to a particular situation.

Risk charting - This method combines the above approaches by listing resources at risk, threats to those resources, modifying factors which may increase or decrease the risk and consequences it is wished to avoid. Creating a matrix under these headings enables a variety of approaches. One can begin with resources and consider the threats they are exposed to and the consequences of each. Alternatively one can start with the threats and examine which resources they would affect, or one can begin with the consequences and determine which combination of threats and resources would be involved to bring them about.

Risk Assessment Once risks have been identified, they must then be assessed as to their potential severity of

impact (generally a negative impact, such as damage or loss) and to the probability of occurrence. These quantities can be either simple to measure, in the case of the value of a lost building, or impossible to know for sure in the case of the probability of an unlikely event occurring. Therefore, in the assessment process it is critical to make the best educated decisions in order to properly prioritize the implementation of the risk management plan.

The fundamental difficulty in risk assessment is determining the rate of occurrence since statistical information is not available on all kinds of past incidents. Furthermore, evaluating the severity of the consequences (impact) is often quite difficult for intangible assets. Asset valuation is another question that needs to be addressed. Thus, best educated opinions and available statistics are the primary sources of information. Nevertheless, risk assessment should produce such information for the management of the organization that the primary risks are easy to understand and that the risk management decisions may be prioritized. Thus, there have been several theories and attempts to quantify risks. Numerous different risk formulae exist, but perhaps the most widely accepted formula for risk quantification is:

Rate (or probability) of occurrence multiplied by the impact of the event equals risk magnitude

Risk TreatmentAccording to its definition, Risk Treatment is the process of selecting and implementing of measures to modify risk. Risk treatment measures can include avoiding, optimizing, transferring or retaining risk. The measures (i.e. security measurements) can be selected out of sets of security measurements that are used within the The options available for the treatment of risks include:

Retain/accept the risk - if, after controls are put in place, the remaining risk is deemed acceptable to the organization, the risk can be retained. However, plans should be put in place to manage/fund the consequences of the risk should it occur.

Reduce the Likelihood of the risk occurring - by preventative maintenance, audit & compliance programs, supervision, contract conditions, policies & procedures, testing, investment & portfolio management, training of staff, technical controls and quality assurance programs etc.

Reduce the Consequences of the risk occurring - through contingency planning, contract conditions, disaster recovery & business continuity plans, off-site back-up, public relations, emergency procedures and staff training etc.

Transfer the risk - this involves another party bearing or sharing some part of the risk by the use of contracts, insurance, outsourcing, joint ventures or partnerships etc.

Avoid the risk - decide not to proceed with the activity likely to generate the risk, where this is practicable.

86

“Heat Chart” or “Severity Chart”

A risk heat map is a tool used to present the results of a risk assessment process visually and in a meaningful and concise way.

Whether conducted as part of a broad-based enterprise risk management process or more narrowly focused internal control process, risk assessment is a critical step in risk management. It involves evaluating the likelihood and potential impact of identified risks.

Heat maps are a way of representing the resulting qualitative and quantitative evaluations of the probability of risk occurrence and the impact on the organisation in the event that a particular risk is experienced.

The development of an effective heat map has several critical elements – a common understanding of the risk appetite of the company, the level of impact that would be material to the company, and a common language for assigning probabilities and potential impacts.

The 5x5 heat map diagram below provides an illustration of how organisations can map probability ranges to common qualitative characterisations of risk event likelihood, and a ranking scheme for potential impacts. They can also rank impacts on the basis of what is material in financial terms, or in relation to the

87

achievement of strategic objectives. In this example, risks are prioritised using a simple multiplication formula.

Organisations generally map risks on a heat map using a ‘residual risk’ basis that considers the extent to which risks are mitigated or reduced by internal controls or other risk response strategies.

88

Q.26 What constitute an eCommerce activity(ies)? What will you audit in an eCommerce environment? Describe the content of an audit report with respect to the objective of the audit, outcome or audit findings for an eCommerce business.

Electronic commerce, commonly known as e-commerce or eCommerce, is a type of industry where the buying and selling of products or services is conducted over electronic systems such as the Internet and other computer networks. Modern electronic commerce typically uses the World Wide Web at least at one point in the transaction's life-cycle, although it may encompass a wider range of technologies such as e-mail, mobile devices social media, and telephones as well.Components of an E-commerce environment:

o Networko Datao Databaseo E-commerce softwareo Server softwareo Server Operating systemo Server hardware

While auditing an ecommerce environment the focus will be on:a. Security: The system needs to be protected from unauthorized access, both logically and physically. With e-commerce specifically, information is to be made available only to those who need the access to complete the transaction or services, or follow up on questions and issues that may arise.

b. Availability. The system is available for operation and use as committed or agreed. This in itself does not set a minimum acceptable performance level for system availability - that is established through commitments made by mutual agreements between the related parties within the e-commerce business.

c. Processing integrity: The system processing is complete, accurate, timely, and authorized. It should be performing its intended function in an unimpaired manner that is free from unauthorized or inadvertent manipulation.

d.Confidentiality. Information that is being communicated and exchanged is protected as committed or agreed by the partners.

e. Privacy. Personal information collected from the client’s customers, employees, and other individuals is used, retained, and disclosed in conformity with the commitments in the entity’s privacy notice and with criteria set forth in Generally Accepted Privacy Principles issued by the AICPA/CICA

Content of an audit report:The auditor's report is a formal opinion, or disclaimer thereof, issued by either an internal auditor or an independent external auditor as a result of an internal or external audit or evaluation performed

The primary purpose of the audit for an ecommerce environment would be to assess the effectiveness and efficiency of security measures and their compliance with any of the accepted security standards and Operational Standards. The objectives follow the audit of Security and Audit Guide to Information Technology Security and include the assurances that: •A management control framework exists; •An effective security program is in place; •Security education and training is adequate;

89

•Information/communications is appropriately classified and protected; •An effective personnel screening program is enforced; •Security breaches are dealt with; •Physical safeguards are in place for the protection of personnel and assets; •Contingency management has been developed; •Security requirements are met in contract management; and •Threat and risk assessments are conducted on a regular basis and prior to major system, application and telecommunication changes.

The Audit Findings are based on written documentation, and are sometimes more difficult to prioritize, hence, there is a need to prioritize the findings into categories. Throughout industry, audit findings are generally placed into one of four categories.

The first category is “Major” or “serious” findings. These are findings that if not addressed will lead or will very possibly lead to critical negative business impact. These are the first priority for an Ecommerce establishment to address. An example of a serious finding is that when customer data has not been secured and have high risk of data leakage or loss. This can invite legal fines and penalties as well which highly damage the brand and reputation of the business

The second category for findings is “somewhat serious or somewhat major”. These types of findings can be serious if something out of the ordinary or of a non-routine nature happens. Frequently these types of findings are based on “what if” scenarios. What are the measures taken to prepare for a downtime, for unexpecected changes, or even a DDos attack on the site.Often these types of situations and “what if” scenarios are addressed by redundant measures in a process or on a piece of equipment, but these secondary safety features must frequently be in place, as human error, equipment failures do occur.

The third category is “Minor” findings. These are frequently small items that are easily overlooked. Examples include items where the documentation does not exactly match what the is operational. During an audit, this difference would be noted. Frequently, administrations have already come up with solutions to address a concern, but the documentation was never changed or updated. A minor finding might be that not every employee was provided their annual refresher training for security

The fourth category is “Scope for improvement” findings. These are simple errors, usually in documentation (spelling, grammar, references that do not take you directly to where you need to be, or incorrect dates). Awareness findings could also be errors where the auditors or inspectors feel that management should be aware of something.

◦

90