analytic vs enumerative studies

Presented by:

Mike HendersonBusiness Performance Consultant

[email protected]

Data Out of Context:Enumerative vs. Analytic Studies

Adapted from Dr. D. Wheeler

2

To keep data in context, to avoid inappropriate and costly action,

one must appreciate the distinction between enumerative

and analytic studies.


Enumerative studies are often misapplied to analytic problems.

3

The most significant problems in business involve improving performance in the future, thus obtaining information from the system or process under study

in order to take appropriate action.

This kind of study is known as an “analytic study”

Analytic Studies

4

• An enumerative study is focused on obtaining information about, and taking action on, specific items contained in a frame, which is a well-defined group of physical items. (e.g. sampling from a batch of product to answer the question, “Should the batch be rejected or accepted?”)

• The statistical inference made from the data is applied to the remaining units in the frame.

• The goal is not to characterize the process that produced the frame, but to describe and act on the frame.

Enumerative Studies

5

• A 100% sample of the units in a frame (i.e. a complete census) will eliminate all uncertainty and provide a complete answer to the question posed in an enumerative study. (e.g. Does the batch contain less than 1% defective?)

• In an enumerative study, the method of choice to reduce uncertainty is to increase sample size (reduce standard error).

• Therefore, within enumerative studies there is a rational basis to utilize confidence intervals, significance levels, analysis of variance, etc.

Enumerative Studies

6

• A coal train is heading from the a mine in Wyoming to a generating station in Wisconsin.

• According to the Coal Contract, the plant can reject the shipment of coal if the Coal Specifications are not met (no more than 6% ash, no more than 1.1lb Sulfur per ton, no less than 8700 Btu per lb).

• A sample (which was taken from the train at the mine) is processed and the information is given to the plant – 6.2% ash, .91 lb sulfur per ton, 8750 Btu per lb.

• Question posed: “Should the plant reject this coal shipment?”

Note: the statistical inference is from the sample back to the remaining coal already on the train (to the frame).

Enumerative Study Example

7

• However, an analytic study is focused on obtaining information from the system or process under study and taking action on the cause system to improve performance in the future. (e.g. sampling from a batch of product to answer the question, “Has the process or system changed as a result of our actions?” or “Is the process consistently producing acceptable product?”)

• The statistical inference made from the data is applied to the process.

• The goal is to characterize the process that produced the frame, not to describe and act on the frame.

Analytic Studies

8

• A 100% sample of the units in a frame will be inconclusive concerning the future performance of the process.

• In an analytic study, the major source of uncertainty is the dynamics of the process or system under study (i.e. the physics of the process, effects of entropy, human will, and other assignable causes). The method of choice to reduce uncertainty is to study the process over time (control charts), increasing knowledge of the cause system and reducing variability and improving predictability.

• Therefore, within analytic studies there is no rational basis to utilize confidence intervals, significance levels, analysis of variance, etc. Increasing sample size is of little help and can be costly.

• The most significant problems in business are analytic problems.

Analytic Studies

9

• Does electric Generating Unit # 7 consistently convert chemical energy (Btu/lb coal) to Electric Energy (Btu/KWH)?

Question: Can you see the statistical inference in the above graph?

Jan-

05

Feb-

05

Mar

-05

Apr

-05

May

-05

Jun-

05

Jul-0

5

Aug

-05

Sep

-05

Oct

-05

Nov

-05

Dec

-05

Jan-

06

Feb-

06

Mar

-06

Apr

-06

May

-06

Jun-

06

Jul-0

6

Aug

-06

Sep

-06

Oct

-06

Nov

-06

Dec

-06

Jan-

07

Feb-

07

Mar

-07

Apr

-07

May

-07

Jun-

07

Jul-0

7

Aug

-07

Sep

-07

Oct

-07

Nov

-07

Dec

-07

10,500

10,700

10,900

11,100

11,300

11,500

11,700

11,900

12,100

12,300

Generating Unit #7: On Line Heat Rate

BTU

/ K

WH

Analytic Study Example

10

The primary goal of analysis of data is the objective creation of information, evidence, and new knowledge that forms the rational basis for action.

The distinctions between enumerative and analytic studies are important because the applicable theories of probability, sampling, estimation, and prediction are not the same for both studies.

Misapplication of mathematical or statistical theory can result in a false sense of confidence in predictions and conclusions that are reached from analysis of the data, leading to incorrect decisions and costly actions.
