research design and instrument development ensuring what you want is what you get

Research Design and Instrument Development

Ensuring what you want is what you get

Objectives

• to review some key research design issues

• to highlight key elements of instrument design

Research Technique

• Technique is determined by objectives

• Ensure that the data collected is the right type and quality to answer the research question

Research Techniques

• Experiments

• Surveys

• Secondary data

• Observation

Selecting the Appropriate Survey Design

• Personal interviewing• Telephone interviewing

– Speed, cost, impersonal, representative sample– Greater tendency to get “no answer” and

“incomplete answers”– No longer than ten minutes; uniform administration

• Mail survey– No longer than six pages

• Electronic survey

Personal Interviews

• Open-ended– ask for the facts of a matter as well as for

opinions about events or insights

• Focused– interviews may still be open-ended and

assume a conversational manner, but you’re likely following a set of questions

• Survey– structured questions

General Design Considerations

• Time frame

• Explicit and consistent frame of reference

• Standardization (for comparability)

• Descriptive and/or evaluative data

Designing Survey Instruments

• Collect only the data needed to address the research question

• Be clear about what concepts you want to collect data on and how you are going to ensure that this data is what you think it is

• Ask the “right” questions about the concepts in the study

Major Decisions

• What should be asked?• How should each question be phrased?• In what sequence should the questions be

arranged?• What questionnaire layout will best serve the

research objectives?• How should the questionnaire be pretested?• Does the questionnaire need to be revised?

Item Phrasing

• Dependent upon method of administration

• Nonpersonal administration requires items and instructions to be succinct

• All methods require items to be clearly stated, unambiguous, easily understood

When Constructing Items, Ask:

• Is the item necessary?

• How will it be asked?

• Should it be open-or closed-ended?

• How should the item be worded?

When Constructing Response Formats, Ask:

• What response alternatives should be used?

• In what order should alternatives be stated?

• How will “don’t know”, “no answer” and “neutral” responses be handled?

Item Sequencing and Layout

• Are the instructions clear and easy to read?• Are instructions provided to explain the purpose of

the questions, with directions on how to complete the scales, if necessary?

• Does the questionnaire begin with simple and interesting questions?

• Are sensitive and classificatory items provided last?• The questionnaire has a good flow to it.• The questionnaire is well-laid out and aesthetically

appealing.

Invitation to Participate

• Explicit invitation to participate• An acknowledgement that participation is

voluntary• A statement of the purpose of the project• A statement of the time demands involved in

participation• Contact information for the researcher (or

supervisor).