iccs 2009 4 th nrc meeting, february 15 th - 18 th 2010, madrid 1 sample participation and sampling...
TRANSCRIPT
ICCS 2009 4th NRC Meeting, February 15th - 18th 2010, Madrid1
Sample Participation and Sampling Weights
ICCS 2009 4th NRC Meeting, February 15th - 18th 2010, Madrid
Overview
Progress reportParticipation in ICCSWeights– What are weights?– Why use weights?– What weights are there in ICCS?– How were ICCS weights calculated?
ICCS 2009 4th NRC Meeting, February 15th - 18th 2010, Madrid
Progress report
38 educational systems participated in ICCSSampling weights were calculated for all of them– For the student study and the teacher study– For six additional grades
Variables for variance estimation were preparedParticipation rates were calculated
ICCS 2009 4th NRC Meeting, February 15th - 18th 2010, Madrid
Participation categories
Three categories for sampling participation were defined in ICCS for students and for teachers– Category 1: acceptable participation rate without
the use of replacement schools– Category 2: acceptable participation rate when
replacement schools are included– Category 3: participation rate not acceptable even
when replacement schools are included
Each country was grouped into a category during sampling adjudicationEach country was informed about the decision
ICCS 2009 4th NRC Meeting, February 15th - 18th 2010, Madrid
Participation categories - students
Category 1: 28 countriesCategory 2: 8 countriesCategory 3: 2 countries
ICCS 2009 4th NRC Meeting, February 15th - 18th 2010, Madrid
Participation categories - teachers
Category 1: 22 countriesCategory 2: 5 countriesCategory 3: 9 countriesTwo countries could not be reported
ICCS 2009 4th NRC Meeting, February 15th - 18th 2010, Madrid
Weights. They are everywhere.
Weights are in your dataWeights are in the tables of the reports– All means and percentages in the reports
will be calculated with weighted data
ICCS 2009 4th NRC Meeting, February 15th - 18th 2010, Madrid
What are weights?
Weights are values that are assigned to every sampling unitThe weight of a sampled unit indicates the population that is represented by this sampled unit
2200
ICCS 2009 4th NRC Meeting, February 15th - 18th 2010, Madrid
Why do we need weights?
Weights allow drawing conclusions about the population based on information from the sampleWeights allow unbiased estimates of population parametersUn-weighted data only allow conclusions about the sampled units
ICCS 2009 4th NRC Meeting, February 15th - 18th 2010, Madrid
Population sample
Example: in the population, 20% of the students are in private schools, 80% are in public schoolsThe NRC decides to over-sample students from private schoolsIn the sample, 50% of the students are from private schools, 50% are from public schools
ICCS 2009 4th NRC Meeting, February 15th - 18th 2010, Madrid
Sample estimate
To estimate the correct proportion of students in the population, we must assign different weights to the students in the sample
ICCS 2009 4th NRC Meeting, February 15th - 18th 2010, Madrid
What weights are there in ICCS?
Student weights– TOTWGTS
Teacher weights– TOTWGTT
School weights– TOTWGTC
ICCS 2009 4th NRC Meeting, February 15th - 18th 2010, Madrid
Why do ICCS weights differ?
ICCS student weights differ, because the following five elements differ: Selection probabilities– 1) ... of the sampled schools– 2) ... of the classes within the schools
Non-participation– 3) ... of sampled schools– 4) ... of sampled classes within the schools– 5) ... of students within the classes
Each element corresponds to a weight factor
ICCS 2009 4th NRC Meeting, February 15th - 18th 2010, Madrid
1) School Base Weight
WGTFAC1The school base weight is the inverse of the selection probability of the school– Low selection probability large weight – High selection probability small weight
5
1
ICCS 2009 4th NRC Meeting, February 15th - 18th 2010, Madrid
1) School Base Weight: PPS
Schools were sampled with probability proportional to size (PPS)Large schools had larger selection probabilitiesTherefore, they now have smaller school base weights
Example: the large school is three times largerThe small school has a three times larger school base weight
w w x 3
ICCS 2009 4th NRC Meeting, February 15th - 18th 2010, Madrid
2) Class Base Weight
WGTFAC2SThe class base weight is the inverse of the selection probability of the classDifferent selection probabilities because of the different number of classes in a school
6
2
ICCS 2009 4th NRC Meeting, February 15th - 18th 2010, Madrid
3) School Weight Adjustment
WGTADJ1Adjusts for schools that did not participateExample: Schools from two different explicit strata – one blue school does not participate
5/4
ICCS 2009 4th NRC Meeting, February 15th - 18th 2010, Madrid
4) Class Weight Adjustment
WGTADJ2SAdjusts for classes that did not participateIn most countries, only one class was selected per schoolAlmost always, this factor equals 1
ICCS 2009 4th NRC Meeting, February 15th - 18th 2010, Madrid
5) Student Weight Adjustment
WGTADJ3SAdjusts for students that did not participateExample: Students from two different classes – one blue student does not participate
5/4
ICCS 2009 4th NRC Meeting, February 15th - 18th 2010, Madrid
The final student weight
We now can compute the TOTAL STUDENT WEIGHT
TOTWGTS = x School Base Weightx Class Base Weightx School Adjustment Factorx Class Adjustment Factorx Student Adjustment Factor
ICCS 2009 4th NRC Meeting, February 15th - 18th 2010, Madrid
Teacher weights
The TOTAL TEACHER WEIGHT is computed similarlyTOTWGTT =
x School Base Weightx Teacher Base Weightx School Adjustment Factorx Teacher Adjustment Factorx Teacher Multiplicity Factor
– adjusts for the fact that some teachers teach in more than one school
ICCS 2009 4th NRC Meeting, February 15th - 18th 2010, Madrid
More about weights:
IDB Training on Thursday– how weights are used for data analysis – some examples of what can go wrong if
weights are not used for data analysis
Please join!
ICCS 2009 4th NRC Meeting, February 15th - 18th 2010, Madrid
Country sessions
If you have any questions about participation rates or weights...Please sign up!