How good is good enough?How good is good enough?RRFSS Module ValidationRRFSS Module Validation

Stephanie TottenStephanie TottenEpidemiologistEpidemiologist

Niagara Region Public Health DepartmentNiagara Region Public Health Department

June 20, 2007June 20, 2007

Working Group MembersWorking Group Members

John Barbaro – Simcoe MuskokaJohn Barbaro – Simcoe Muskoka

Alyshia Landry – Waterloo Alyshia Landry – Waterloo

Rachel Savage – Huron CountyRachel Savage – Huron County

Suzanne Sinclair – KFL&ASuzanne Sinclair – KFL&A

Stephanie Totten – Niagara Stephanie Totten – Niagara

OutlineOutline

Background and Rationale Background and Rationale

PHAC Regional Risk Factor Contribution PHAC Regional Risk Factor Contribution ProposalProposal

Current StatusCurrent Status

Next StepsNext Steps

Why Validation?Why Validation?

16/23 (69.6%) RRFSS Reps and 32/41 16/23 (69.6%) RRFSS Reps and 32/41 (78.1%) Health Unit program and research (78.1%) Health Unit program and research staff rated the data quality of RRFSS at staff rated the data quality of RRFSS at least 4/5least 4/5

Lack of validity or reliability testing was Lack of validity or reliability testing was identified as a data quality issueidentified as a data quality issue

Why Now?Why Now?

RRFSS Strategic Plan Short-term Goal #4:RRFSS Strategic Plan Short-term Goal #4:““To increase the number of validated and To increase the number of validated and reliable modules”reliable modules”

Made a priority in light of the 2007 Made a priority in light of the 2007 Workshop themeWorkshop theme

PHAC Regional Risk Factor Contribution PHAC Regional Risk Factor Contribution request for proposalsrequest for proposals

Proposal OverviewProposal Overview

Four activities for module validation Four activities for module validation identified in the Strategic Plan under the identified in the Strategic Plan under the Quality Improvement and Development Quality Improvement and Development Strategy:Strategy:– Create an inventory of what has been Create an inventory of what has been

validated in RRFSSvalidated in RRFSS– Determine criteria for “reliable” and “valid”Determine criteria for “reliable” and “valid”– Identify modules to be validatedIdentify modules to be validated– Find people to do validity studiesFind people to do validity studies

Objective Activities Expected Outcomes

1. Determination of criteria for validity and reliability in RRFSS

-Review of literature for common definitions and methods of validity and reliability-Key Informant Interviews

Definite criteria on how RRFSS defines valid and reliable questions

2. Development of the RRFSS inventory of module validation status

-Key Informant Interviews Detailed spreadsheet outlining validation status of RRFSS modules

3. Prioritize RRFSS modules for validation Studies

-Key Informant interviews-Inventory and Strategic Plan review

Detailed spreadsheet outlining top priorities for RRFSS module validation studies.

4. Develop validation study proposals for one to two modules based on priorities

-Literature review and development of methods, analysis plan, ethical considerations, work plan with timelines, and draft budget

One to two study proposals outlining methods for validation studies.

Current StatusCurrent Status

Partnership with alPHaPartnership with alPHa

Proposal submitted to PHACProposal submitted to PHAC

Support from Ministry of Health PromotionSupport from Ministry of Health Promotion

Next StepsNext Steps

Await funding decisionAwait funding decision– Form advisory groupForm advisory group– Develop work planDevelop work plan

Begin preliminary work on the validation Begin preliminary work on the validation inventoryinventory

Questions?Questions?

Feedback?Feedback?