dr. serhat eren 1 res 341 research and evaluation workshop 1 by dr. serhat eren university of...

Dr. Serhat Eren1

RES 341RES 341RESEARCH AND RESEARCH AND

EVALUATIONEVALUATION

WORKSHOP 1

By Dr. Serhat Eren

University OF PHOENIXSpring 2002

Dr. Serhat Eren2

CHAPTER 1CHAPTER 1

THE ROLE OF STATISTICAL THINKING IN BUSINESS RESEARCH

Dr. Serhat Eren3

CHAPTER I CHAPTER I THE ROLE OF STATISTICAL THINKING IN BUSINESS RESEARCHTHE ROLE OF STATISTICAL THINKING IN BUSINESS RESEARCH

1.1 CHAPTER OBJECTIVES1.1 CHAPTER OBJECTIVES

What Managers Should Know About Research and Statistics Statistical Thinking-A New Paradigm for Management

Some Situations That Call for Research and Statistical Thinking Types of Research

Steps in Conducting Business Research

Key Components of Research and Statistical Thinking Organization of This Course

Dr. Serhat Eren4


1.3 WHAT MANAGERS SHOULD KNOW ABOUT 1.3 WHAT MANAGERS SHOULD KNOW ABOUT RESEARCH AND STATISTICSRESEARCH AND STATISTICS

As a manager you will be making several decisions each day at work. – What would help you to make the right

decisions? – Will it be your experience on the job, your sixth

sense or hunch, or will you just trust to good luck?

For sure, all these will play a part after you have thoroughly investigated or researched the problem, analyzed some data, and generated some alternative solutions to choose from.

Dr. Serhat Eren5



Some of the main tools of research are statistical techniques because in attempting to understand a problem you will almost always gather and analyze data.

Statistical techniques allow you to see the information in the data. Thus, an understanding of statistics is needed to do the research necessary for you to be a good manager.

The use of statistical techniques has long played an important role in quality control and quality improvement in business and industry.

Dr. Serhat Eren6



Unfortunately, for too long quality has been relegated to the "quality department" and not integrated into the whole organization.

The purchasing department would purchase the raw materials needed at the cheapest price without regard to quality and then would throw the material "over the wall" to the engineers.

One of the reasons that people prefer to have the quality department handle quality issues is that thinking about quality leads to thinking about data analysis.

Dr. Serhat Eren7



Data analysis requires the use of statistical techniques, which are often viewed as difficult.

Four issues must be addressed if statistics is to become an integral part of management:

– Managers must understand why they need to possess statistical knowledge.

– Current and future managers (that's you) must have this knowledge.

– Steps must be taken to ensure that the knowledge is used.

– The payoff from the knowledge and its application must be measured.

Dr. Serhat Eren8


1.4 Statistical Thinking- A New Paradigm and Statistics1.4 Statistical Thinking- A New Paradigm and Statistics Look at the Figure 1.1 below. What do you see?

Some of you will see two faces and some of you will see a vase. You probably can see both of these things, but which one did you see first?

There is no right or wrong answer here. What you see first depends on your viewpoint, your lens, and your glasses. This is your paradigm; this is how you see things.

Dr. Serhat Eren9


1.6 TYPES OF RESEARCH1.6 TYPES OF RESEARCH

Research is simply the process of finding solutions to a problem after a thorough study and analysis of the situation and data.

The term business research simply tells us that the problem to be investigated is in the work setting. Business research comprises a series of steps with the goal of finding answers to the issues that are of concern to the manager.

Business research is process of finding solutions to a specific problem encountered in the work setting.

Dr. Serhat Eren10



Business research can be undertaken for two different purposes. One is to solve a current problem faced by the manager.

For example, a particular product may not be selling well and the manager might want to find the reasons for this in order to take corrective action. Such research is called applied research.

The results are applicable only to this particular product in this particular company at this particular time.

Dr. Serhat Eren11



Applied research is research, done with the intention of applying the results of the findings to solve specific problems being experienced in the organization.

Basic research is research done mainly to increase the understanding of certain problems that commonly occur in organizational settings.

Dr. Serhat Eren12


1.7 STEPS IN CONDUCTING BUSINESS RESEARCH1.7 STEPS IN CONDUCTING BUSINESS RESEARCH

As you gain some experience with doing business research, you will find that although the situations and problems are varied, the steps necessary to successfully complete the research can be generalized to apply in any situation.

Many authors have summarized these steps and you may find some procedures with 5 steps, some with 6 or 8 steps.

Dr. Serhat Eren13


1.7 STEPS IN CONDUCTING BUSINESS RESEARCH1.7 STEPS IN CONDUCTING BUSINESS RESEARCH

These procedures simply differ in how the necessary work is divided up into manageable and logical steps.

– Step 1. Problem Identification

– Step 2. Statement of Desired Goal or Outcome

– Step 3. Research Evidence and Hard Data

– Step 4. Outcomes

– Step 5. Identification of Possible Cause(s) – Step 6. Proposed Solutions

Dr. Serhat Eren14


1.8 KEY COMPONENTS OF STATISTICAL THINKING1.8 KEY COMPONENTS OF STATISTICAL THINKING

It is important to understand the components of this new paradigm that we have called statistical thinking. There are three key components:

– We must use data whenever possible to guide us. – We must look for connections and relationships. – We must understand why data values differ from

each other.

Statistics is a branch of mathematics dealing with the analysis and interpretation of masses of data.

Dr. Serhat Eren15

CHAPTER 11CHAPTER 11

NEEDS ASSESSMENT

Dr. Serhat Eren16

CHAPTER 11CHAPTER 11NEEDS ASSESSMENT ANALYSIS AND EVALUATIONNEEDS ASSESSMENT ANALYSIS AND EVALUATION

11.1 CHAPTER OBJECTIVES11.1 CHAPTER OBJECTIVES

The three-level approach to needs assessment

Data collection methods used for needs assessment

Four types of needs assessment$, classified according to the type of problem being addressed and the conditions under which the problem occurs

Some guidelines that may be applied to almost any needs assessment

Dr. Serhat Eren17


11.2 LEVELS OF ANALYSIS11.2 LEVELS OF ANALYSIS

Needs assessments have traditionally been classified as focusing on three levels of analysis: organization, job or task, and individual or person?

The assessors should start by analyzing the organization to determine what results are not occurring and should be, and what organizational factors are contributing to that condition. The assessors should then analyze work, jobs, or tasks to determine what performance should be occurring.

Dr. Serhat Eren18



Finally, assessors should study individuals to determine who needs learning to accomplish those jobs or tasks.

Figure 11.1 shows how these corresponding levels can be viewed as a U-shaped process of needs assessment leading to evaluation.

Dr. Serhat Eren19

Dr. Serhat Eren20



By starting at the organizational level, needs assessment is most likely to lead to well-designed interventions with a very good chance of solving real performance problems.

Evaluation after the program is easy because the criteria for each level were determined before the interventions were designed.

Dr. Serhat Eren21



Assessors are usually cautioned to be sure that a needs assessment encompasses these three traditional levels (organization, work or task, and individual).

Failure to collect assessment data at all levels is likely to lead to misdirected resources and low-impact interventions that are difficult to evaluate.

Dr. Serhat Eren22



It should be pointed out, however, that not all needs assessors are given the opportunity to perform an all-encompassing three-level analysis.

The levels of analysis included in an assessment depend on such factors as the status of the assessor in the organization, available time and resources, management policy, and the assessor's biases.

Dr. Serhat Eren23


11.3 METHODS EMPLOYED11.3 METHODS EMPLOYED

Needs assessors have long been known for employing a wide variety of methods?

Table 11.1 summarizes the methods used.

Dr. Serhat Eren24

Dr. Serhat Eren25



Some notable and encouraging trends evident in needs assessment include the following:Multi-method strategies-

Although surveys and interviews are the most popular methods, many others are used as well.

Dr. Serhat Eren26



Integration of qualitative methods-Interviews and focus groups provide open-ended information that enriches an assessment. True revelations tend to occur more frequently with qualitative methods, largely because quantitative data collection instruments constrain the range of responses.

Dr. Serhat Eren27



Appropriate use of surveys- The message to assessors is this: Surveys are powerful tools, but they are best used in conjunction with other methods, and they are not a necessary part of needs assessment, even with a large group.

Dr. Serhat Eren28


11.4 FOCUS OF NEEDS ASSESSMENT11.4 FOCUS OF NEEDS ASSESSMENT

Traditionally, needs assessment had been deficiency oriented, designed to identify and address existing deficiencies or gaps in performance. A deficiency approach is, by definition, focused in the present.

Opportunity-oriented needs assessment is future oriented, identifying performance gaps likely to occur in the future and proactively implementing solutions to prevent them.

Dr. Serhat Eren29



And if organizations are to handle today's fast-changing business environments, needs assessments must be more focused on the future.

R. Revans has suggested that organizations face four types of change, varying along two dimensions.

Dr. Serhat Eren30



One dimension is the type of problem-known or unknown.

The other dimension is the conditions under which the problem occurs-also divided into known and unknown.

The four-cell matrix that results, Figure 11.2, can be used to describe the following four types of needs assessments:

Dr. Serhat Eren31

Dr. Serhat Eren32



Corrective needs assessments are those that analyze existing problems in existing circumstances to identify performance problems.

Adaptive needs assessments occur in organizations that find themselves performing under new conditions, but facing the same job demands as in the past.

Dr. Serhat Eren33



Developmental needs assessments are those designed to improve an organization's ability to deal with additional problems in the existing environment and conditions.

Strategic needs assessments require the anticipation of unknown problems or deficiencies that are likely to occur in the future under changing, but unknown, conditions.

Dr. Serhat Eren34



Traditional needs assessment and performance analysis methodologies are strong tools for corrective needs assessments, limited tools for adaptive and developmental needs assessments, and weak tools for strategic needs assessments.

Needs assessors have traditionally avoided strategic needs assessments because they require a great deal of "crystal ball" analysis.

Dr. Serhat Eren35



However, needs assessors must develop the capability to assist organizations with non-corrective assessments.

Corrective assessments, by definition, place the human resource development (HRD) person conducting the assessment in a reactive mode. But no organization can afford to wait for performance problems to occur in order to figure out how to prevent them.

Dr. Serhat Eren36



Experience indicates that the basic assessment methods appear to work in strategic assessments with minimal adaptation.

Perhaps the barriers lie more in a lack of strategic thinking in HRD, not in the lack of methodologies.

Dr. Serhat Eren37


11. 5 11. 5 PLANNING AHEADPLANNING AHEAD

11.5.1 Plan to use data Someone once said, "Planning is a

substitute for good luck." The data you collect in a needs assessment can be useful long after you've used it to determine training needs.

You should plan to make your needs-assessment data an integral part of your whole training project. The data can keep you from having to "reinvent the wheel" at every stage of the training, as new needs arise.

Dr. Serhat Eren38



11.5.2 Establish criteria or goals What do you expect your needs

assessment to find?

List your expectations and make a note of conditions prior to the assessment.

Dr. Serhat Eren39



11.5.3 Research the topic, task, or focus

Become familiar with the subject that is targeted by your needs assessment.

Interview people in the field.

Review books, journals, and reports on the topic.

Dr. Serhat Eren40



11.5.4 Create a guidance group Get together a group of people

whose function is to make decisions and keep the needs assessment on track.

The group should include some of the people who requested the needs assessment, some subject matter experts, and some stakeholders-people who may be affected by the outcome.

Dr. Serhat Eren41



11.5.5 Identify your data sources With the group's help, determine

the best sources for the data you'll need.

Even while collecting data, continue looking for other sources of information. Ask the people who provide you with information to recommend additional resources.

Dr. Serhat Eren42



11.5.5 Identify your data sources There are two kinds of data: hard and

soft.

Hard data are factual. They can take the form of production reports, defective-parts reports, recall reports, and absentee reports.

Such data provide real numbers that you can count, analyze, and translate into statistics.

Dr. Serhat Eren43



11.5.5 Identify your data sources Soft data are pieces of information

that are obtained through such means as group discussions, interviews, questionnaires, and literature reviews.

Soft data are subjective-similar to opinions and beliefs-but they can often be supported by hard data.

Dr. Serhat Eren44



11.5.6 Design an interview format A trial run can help you refine the

content of your interviews with the people on your list of sources and stakeholders.

The more refined an interview is, the more information you're likely to get. Carefully planned interviews also take less time to conduct.

Dr. Serhat Eren45



11.5.6 Design an interview format Construct a questionnaire-either on

paper or on a computer disk-to use as the basis for your interviews.

For the trial run, ask several people in your guidance group to answer the questionnaire.

Dr. Serhat Eren46



11.5.6 Design an interview format In both the trial and actual interviews,

describe for participants the goals of your needs assessment. Tell them who authorized it.

You also should tell them why they were selected, how you plan to use the information they provide, how much you value their participation, what kinds of feedback they'll receive, and how they might benefit from the outcome of the assessment.

Dr. Serhat Eren47


11. 6 COLLECTING DATA11. 6 COLLECTING DATA

11.6.1 Interviews Interviewing is an excellent way to get

information. The best interview subjects tend to be those who are professionals in the area you're assessing and who may be directly affected by the outcome of the assessment.

Let an interviewee know in advance how much time you expect the interview to take. You should have some idea from your trial interviews.

Dr. Serhat Eren48



11.6.1 Interviews When you begin an interview, ask

for permission to record the session. Warn people that you may ask probing questions. Invite them to ask questions as well.

Feel free to stray from your present format.

Dr. Serhat Eren49



11.6.1 Interviews For example, interviewees may

elaborate immediately on some points that you'd planned to ask later. Just glance through the questions at the end to ensure that all of them have been addressed.

When you can't interview people in person, consider sending them questionnaires to fill out instead.

Dr. Serhat Eren50



11.6.1 Interviews One study shows that disk" based

surveys reap better response rates than written surveys.

Afterwards, be sure to send interviewees thank-you notes that emphasize how valuable their input will be to the outcome of the needs assessment.

Dr. Serhat Eren51



11.6.2 Group discussion A group discussion can bring out all

kinds of data; an interactive environment tends to elicit information that individuals might not bring up on their own.

It's important to select a diverse group of people, to follow an agenda, and to keep the number of members manageable.

Dr. Serhat Eren52



11.6.2 Group discussion Group size can affect the amount and

kinds of information you get. If a group is too large, some people may not join the discussion. If a group is too small, it may not provide the richness or variety of information you seek.

Don't let one or two people in the group dominate. And don't let anyone stray from the topic.

Dr. Serhat Eren53



11.6.3 Hard data If your needs assessment requires

hard data, make sure the data are relevant to and accurately reflect work conditions that are observed during the assessment.

Also, it's important that the data address the concerns of people who requested the needs assessment.

Dr. Serhat Eren54


11. 7 ANALYZING THE DATA11. 7 ANALYZING THE DATA

11.7.1 Compile the data By now, you probably have a lot of

information.

What are you going to do with it? It's up to you and the group to decide which data is relevant.

Dr. Serhat Eren55


11. 7 ANALYZING THE DATA11. 7 ANALYZING THE DATA

11.7.2 Provide statistics If any of the hard data include

statistics, a statistical analysis can help clarify the results. And sometimes, it may be necessary to provide statistical documentation in order to obtain resources for projects.

The data used for statistics also may be used to monitor any changes that are implemented.

Dr. Serhat Eren56


11.7 ANALYZING THE DATA11.7 ANALYZING THE DATA

11.7.3 Prepare a report Your report should flow. It should

lead readers step-by-step through the needs assessment to the results.

The report should contain a statement of purpose and goals, dates, methodologies, data sources, analysis techniques, and results-as well as a list of the needs you and the group identified.

Dr. Serhat Eren57


11.7 ANALYZING THE DATA11.7 ANALYZING THE DATA

11.7.3 Prepare a report Include charts and graphs for

clarification. Use the data you have collected as documentation for the needs you list.

The report provides a solid basis for making changes and for measuring the effectiveness of those changes. You may even want to go ahead and make some recommendations at this point.

Dr. Serhat Eren58

CHAPTER 8CHAPTER 8

THE BUSINESS RESEARCH PROPOSAL

Dr. Serhat Eren59

CHAPTER VIIICHAPTER VIIITHE BUSINESS RESEARCH PROPOSALTHE BUSINESS RESEARCH PROPOSAL

8.3 Ethics in Business Research

Dr. Serhat Eren60

CHAPTER 8CHAPTER 8THE BUSINESS RESEARCH PROPOSALTHE BUSINESS RESEARCH PROPOSAL

8.3 ETHICS IN BUSINESS RESEARCH8.3 ETHICS IN BUSINESS RESEARCH

8.3.1 Ethical Issues in the Preliminary Stages of Investigation

Once a problem is recognized and an investigation is decided on, it is important to inform all employees, particularly those who will be interviewed for preliminary data gathering through structured and unstructured interviews, of the proposed study.

It is also necessary to assure them that their responses will be kept confidential by the interviewer(s) and that individual responses will not be divulged to anyone in the organization.

Dr. Serhat Eren61



8.3.2 Ethics in Handling Information Technology PC technology imposes certain obligations on the part of

its users. – First, it is important that the privacy of all individuals is protected

whether they are consumers, suppliers, employees, or others. – Second, companies also need to ensure that confidential

information relating to individuals is protected and does not find its way to unscrupulous vendors.

– Third, care should be taken to ensure that incorrect information is not distributed across the many different files of the company.

– Fourth, those who collect data for the company should be honest, trustworthy, and careful in obtaining and recording the data.

Dr. Serhat Eren62



8.3.3 Ethical Issues in Experimental Design Research The following practices are considered unethical:

– Pressuring individuals to participate in experiments through coercion, or applying social pressure.

– Giving menial tasks and asking demeaning questions that diminish their self respect.

– Deceiving subjects by deliberately misleading them as to the true purpose of the research.

– Exposing participants to physical or mental stress.– Not allowing them to withdraw from the research when

they want to.

Dr. Serhat Eren63



8.3.3 Ethical Issues in Experimental Design Research

– Using the research results to disadvantage the participants, or for purposes that they would not like.

– Not explaining the procedures to be followed in the experiment.

– Exposing respondents to hazardous and unsafe environments.– Not debriefing participants fully and accurately after the

experiment is over.– Not preserving the privacy and confidentiality of the

information given by the participants.– Withholding benefits from control groups.

Dr. Serhat Eren64



8.3.4 Ethics in Data Collection8.3.4.1 Ethics and the Researcher– Treating the information given by the respondent as strictly

confidential and guarding his or her privacy is one of the primary responsibilities of the researcher.

– Researchers should not misrepresent the nature of the study to subjects, especially in lab experiments.

– Personal or seemingly intrusive information should not be solicited, and if it is absolutely necessary for the project, it should be tapped with high sensitivity to the respondent, offering specific reasons therefor.

Dr. Serhat Eren65



8.3.4 Ethics in Data Collection8.3.4.1 Ethics and the Researcher– Whatever be the nature of the data collection method,

the self-esteem and self-respect of the subjects should never be violated.

– No one should be forced to respond to the survey, and if someone does not want to avail of the opportunity to participate, the individual's desire should be respected.

– Nonparticipant observers should be as nonintrusive as possible.

Dr. Serhat Eren66



8.3.4 Ethics in Data Collection8.3.4.1 Ethics and the Researcher– In lab studies, the subjects should be debriefed with full

disclosure of the reason for the experiment after they have participated in the study.

– Subjects should never be exposed to situations where they could be subject to physical or mental harm.

– There should be absolutely no misrepresentation or distortion in reporting the data collected during the study.

Dr. Serhat Eren67



8.3.4 Ethics in Data Collection8.3.4.2 Ethical Behaviors of Respondents

– The subject, once having exercised the choice to participate in a study, should cooperate fully in the tasks ahead, like responding to a surveyor taking part in an experiment.

– The respondent also has an obligation to be truthful and honest in the responses.

dr. serhat eren 1 res 341 research and evaluation workshop 1 by dr. serhat eren university of...

Documents

role of statistical

statistical knowledge

research necessary

use of statistical techniques

serhat erenchapter

main tools of research

business research1

business researchdr