integrating teaching and research: using student derived data dr. emily summers dr. jon lasser dr....

INTEGRATING TEACHING AND RESEARCH: USING STUDENT DERIVED DATADr. Emily Summers

Dr. Jon Lasser

Dr. Gail Ryser

Mr. William Barry

What is student derived data?• Student Data from Your Class(es)• Student Data from Others’ Class(es)• Student Data from University-Associated Sources

Why use student derived data in your research?

• To Benefit Students or Instruction• Improve Instruction• Understand Students’ Learning• Increase Responsiveness to Student or Instructional Needs• Allow Your Teaching to Inform Your Research and Your Research to

Inform Your Teaching in a Hermeneutically Refining Exchange

• (Not to Increase Publications; Should Have an Authentic Purpose Situated in Teaching, Learning, and/or Instructional Practices)

Setting up a study using student derived data

• IRB• Quantitative design and analysis including statistical power

• Other Power (all the time)

Institutional review board (IRB) 101An Introduction to the IRBTexas State University-San Marcos

Mission

“…charged with protecting the rights and welfare of human research subjects. The IRB reviews proposed research to ensure that the proposed project follows federal guidelines and accepted ethical principles to meet that goal.”

Secondary Mission

To help faculty and students proceed with their research plans in a timely manner that ensures compliance with human subject protection regulations.

Determining the Need for Review

Must answer two questions:

1. Is it research?

2. Are human subjects involved?

Is it research?

For purposes of IRB review, "research" is defined as "a systematic investigation designed to develop or contribute to generalizable knowledge." (45.CFR 46.102).

Projects at the undergraduate or graduate level such as thesis, honors, or seminar projects may also be considered "research."

Are human subjects involved?

For purposes of IRB review, a "human subject" is defined as "a living individual about whom an investigator conducting research obtains (1) data through interaction or intervention with the individual or (2) identifiable private information." (45.CFR 46.102)

Types of Review• Exempt• Expedited• Full• Continuation

Is it research?No Yes

No need for any review

What kind of review?

Exemption review (very brief) Expedited

Full seated review

Balancing Risk and Benefit

All human subject research presents some risk

Benefits must outweigh risks

Consider benefit to subjects vs. society

Researchers tend to overestimate benefits and underestimate risks

Exempt: Six Federal Criteria

See IRB website for the six exemption categories. Examples:

Studies of normal educational practices

Studies of existing data sets without identifying information

And low risk

How to apply at Texas State

Complete CITI training:

https://www.citiprogram.org/default.asp?language=english

Collaborative Institutional Training Initiative

How to apply at Texas State

TSU IRB home page:

http://www.txstate.edu/research/irb/index.php

TSU IRB application page:

http://www.osp.txstate.edu/irb/

Registration required, using TSU e-mail address

The Review Process• Most reviews at Texas State are expedited.• How do reviewers evaluate applications?• Common errors

• No synopsis submitted• No consent form submitted• Consent form does not meet requirements• Applicant does not address risk• Student fails to gain faculty approval

Becoming a Reviewer• Interested in becoming a reviewer? Contact Jon Lasser (

lasser@txstate.edu) or 245-3413

Quantitative Design

•Posttest only•Pre and posttest •Pre and posttest with control group•Add additional control variables

Quantitative Design• Covariates variables: occur in a model in which the

independent variables of interest are categorical, but you also need to control for an observed, continuous variable–the covariate.

• Moderator variables: relationship between two variables depends on the third variable (also known as an interaction effect)

• Mediator variables: intervening and helps explain why two or more other variables are related, may be causally related to the dependent variable

Quantitative Design• Statistical Power

• Four interrelated components that influence the conclusions you might reach when you run a statistical test: • Alpha level (risk of Type I error)• Power (opposite of Type II error or the probability of detecting a

hypothesized effect if one actually exists)• Sample size (n)• Effect size (practical significance or the degree to which the

phenomenon exists)

• Power Calculators• http://homepage.cs.uiowa.edu/~rlenth/Power/• http://

www.psycho.uni-duesseldorf.de/abteilungen/aap/gpower3/download-and-register

Quantitative Design

•Collect data over time•Cross sectional versus longitudinal

Other non-Statistical Power• Inherent Instructor/Student Power Relationships• Ways to Mediate this Balance

• In-Class Forms• After-Class Forms• Open Conversations• Make the Default Participation is Non-Participation

Tips…• Utilize a Neutral Third Party to Distribute/Collect Consent Forms &

Data• Employ Temporal Delays (in Receiving, Collecting, and/or Analyzing

Data)• Notify Students Early in Semester about Intent to Ask Permission to

Use Class Data as Part of a Research Study, Especially for Journals or Other Subjective Reflection Work

• Waive Written Consent for No Record of Who Participated • Try Anonymous Online Surveys• Use Known Identities to Secure Consent After Grades Post

Cautions…• Be Aware of Additional Peer Coercion to Participate• Practice Keen Awareness of Unequal Risks, as Some Student

Subgroups May be More Identifiable than Others; • Watch Extra Credit Options

• If Offering Extra Credit for Participation Provide Equivalent Alternatives for Extra Credit

• Give Credit for Partial Participation to Ensure Students’ Right to Withdrawal

• Recognize that Some Professor/Students Roles Inherently Have Increased Risk of Coercion• Thesis Students• Research Advisees• On the Cusp Grades• Mentees

Resources• Online and Hybrid data• Testing, Research-Support, and Evaluation Center

(TREC)• Snap Surveys• NVivo 10• Initiative for Interdisciplinary Research Design and Analysis (IIRDA)

• Institutional Research

Online & Hybrid Data• BalckBoard Archives• Tracs Data• Discussion Boards• Be Specific (Not a Catch All)

TREC: Snap Surveys• PC based software that can be used to create and

manage web-based surveys• Allows the developer to incorporate the security, access,

and permissions needed to meet the IRB guidelines• Can only access Snap Surveys if connected to the Texas

State domain; user must have administrative rights to the PC

• User must take free recorded training• http://www.txstate.edu/trec/iirda/researchSupport/snap.html

TREC: NVivo 10• PC based qualitative software• Removes many of the manual tasks associated with

qualitative analysis (e.g., classifying, sorting, arranging of information)

• Designed to support a wide range of research methods including grounded theory, ethnography, discourse analysis

• Allows you to code your data, search your data using queries, and examine your data visually using models and charts

• http://www.txstate.edu/trec/iirda/researchSupport/NVivo.html

TREC: IIRDA• IIRDA's mission is to serve as a resource that meets the

need for state-of-the-art research support in a comprehensive, rigorous and interdisciplinary way.

• Services• Externally funded proposal assistance• Faculty consultation services• Professional data analysis services

• Contains a resources page with annotated references, calculators, online trainings, and quantitative methods websites.

• http://www.txstate.edu/trec/iirda/resource.html

Institutional Research (IR)•Will provide additional student data for IRB approved research projects

•http://www.ir.txstate.edu/

Sample Studies

• Microsoft Word study• Core course study

Audience Questions• “I was curious to hear a little about the extent to which we

could collaborate with students in our classes that are teachers, or conducting field work, to implement a research study and use the data they would be using for class projects or papers, for research as well.”

• Others?