using the idb analyzer to analyze iccs data

ICCS 2009 IDB Seminar – Nov 24-26, 2010 – IEA DPC, Hamburg, Germany

Using the IDB Analyzerto Analyze ICCS data

Table of contentI. Overview

– Starting the Analysis Module– Overview of Analysis Methods

II. Percentages onlyIII. Percentages and MeansIV. CorrelationsV. RegressionVI. BenchmarksVII. Hands-On Training

To start the Analysis Module of the IDB Analyzer, select:Start > Programs > IEA > IDB Analyzer > Analysis Module

Starting the Analysis Module

Overview of the Analysis Module

To start the IDB Analyzer Analysis Module choose:Start > Programs > IEA > IDB Analyzer > Analysis Module


Choose file containing the data for analysis


Percentages and Means– Computes means and standard deviation on continuous

variables for specified subgroups, displays percentages of cases within those subgroups

– Also computes the appropriate standard errors for those percentages, means and standard deviations


Percentages Only– Percentages of variables with their BRR standard errors– Computes the percentages of participants within specified

subgroups– Computes the appropriate standard errors for those

percentages


Regression– Calculates a multiple linear regression between a dependent

variable and a set of independent variables– Computes the regression coefficients and their

corresponding standard errors– Can be used to compare means of subgroups and their

significance


Correlations– Calculates correlation coefficients for selected analysis

variables and their BRR standard errors

11

IEA IDB Analyzer – Analysis Module

Benchmarks– Computes percentages of students within, reaching or

surpassing user provided benchmarks (Proficiency Level) of achievement with JRR standard errors for those percentages


Clear all selections: clears all the settings defined by the user– Selected analysis type– Selected grouping, analysis, dependent, weight and replicate

weight variables– Defined output files


Select study variables of interest

Search for variable by name or label


Add or remove variables


Applies to all analysis types


Applies to:- Percentages and Means- Regression- Correlations


Applies to:- Regression


Selected automatically when file is opened


Choose the desired number of decimals in the output


Define the location of the output files to be produced

Student Questionnaire, Q35, p. 34Variable of Interest: IS2P35

Attendance of religious servicesHow frequently students attend religious services outside home together with other people?

Reproducing Table 4.12 from ICCS2009 International report, the last five columnsAnalysis File: merged in the previous step School and Student Background filesC:\ICCS2009\Work\ICG_ISG_INTC2.savAnalysis Type: “Percentages only” with “Exclude Missing from Analysis” checkedGrouping Variables:IS2P35 – Frequency of religious practices outside home[IDCNTRY] – Country IDs (Pre-selected)

Percentages Only

Percentages Only - Settings

C:\ICCS2009\Work\Table_4-12.*

C:\ICCS2009\Work\ICG_ISG_INTC2.sav

IDCNTRY IS2P35

TOTWGTS

JKZONES1

Percentages onlyExclude Missing from Analysis

Percentages Only

The IDB Analyzer creates SPSS Syntax and starts SPSSIn SPSS Syntax Editor Choose: Run > All

Percentages Only

As a result the IDB Analyzer creates the following in the working directory (C:\ICCS2009\Work\):

SPSS Syntax file – contains the syntax with the commands (*.sps)SPSS Data file – contains statistics from the analysis (*.sav)MS Excel Output file – contains statistics from the analysis (*.xls)

Percentages Only - SPSS Output

Number of students in each group

List of countries

Variable name and Value Labels

Percentages Only - Excel Output


Total student weight – population estimate of the groups defined by the grouping variables(IDCNTRY, IS2P35)

Weighted percentage of students for the groups defined by the grouping variables (IDCNTRY, IS2P35)

Standard error of percentages


Percentages Only - Interpretation

On the average 28% of the students across countries never attend religious services outside home with others. This percentage is highest in Czech republic (70%) and lowest in Indonesia (6.5%).

Student Questionnaire, Q02, p. 2Variable of Interest: SGENDER

Distribution of civic knowledge by gender

What is the distribution of students’ civic knowledge in regard with their gender?

Reproducing Table 3.13 from ICCS2009 International report, the last five columns

Percentages & Means - Settings



TOTWGTS

JKZONES

IDCNTRY SGENDER

PVCIV01-05

1

Percentages and MeansExclude Missing from Analysis

Percentages & Means – SPSS Output

Number of students in the sample

List of countries defined by the first groping variable (IDCNTRY)

Achievement scores(PVCIV01-05)

Percentages & Means – Excel Output

Groups of students defined by the second grouping variable (SGENDER)

Weighted percentages of students for the groups defined by the grouping variables (IDCNTRY, SGENDER)

Standard error of percentages


Total student weight – population estimate of the group defined by the grouping variable(IDCNTRY)

Mean of the analysis variable (PV5CIV01-05)

Standard error of the mean of the analysis variable

Standard deviation of the analysis variable

Standard error of the standard deviation


Percentages & Means – Interpretation

In all countries girls outperform boys in regard with civic knowledge. These differences are lowest in Guatemala and greatest in Thailand.

Correlation (Pearson)

Measures the linear relationship between two (continuous) variablesCorrelation coefficient r informs us about the strength and direction of this relationshipScatter plot can give you a first idea if there is a connection between the variables of interestCorrelation coefficient is a sample estimate of a population parameter too!If we want to know how confident we can be in that value, we need to calculate its standard error that accounts for the complex sampling design

Student Questionnaire, Q14A-F, p. 13Variables of Interest: PARTCOM, PVCIV01-05

Participation in communityIs the student participation in wider community associated with their civic knowledge?

Correlations - Settings

C:\ICCS2009\Work\Correlations.*


TOTWGTS

JKZONES

IDCNTRY

PVCIV01-05

PARTCOM

1

CorrelationsExclude Missing from Analysis

Correlations – SPSS Output

Correlations – Excel Output

List of Countries

Achievement Scores

Sum of Weights

Mean achievement

S.E. of the mean achievement


Standard Deviation of Mean Achievement

S.E. of the standard deviation of the mean achievement

Correlation of the set of PV with itself

Correlations – Excel OutputS.E. of correlation of the set of PV with itself

Mean of the analysis variable

S.E. of the mean of the analysis variable

Analysis variable


Standard deviation of the analysis variable

S.E. of the correlation between the set of PVs and the analysis variable

S.E. of the standard deviation of the analysis variable

Correlation between the set of PVs and the analysis variable

Correlations – Interpretation

Most countries show very weak, negative relationship between the participation in community and civic knowledge. However, Latin American countries and Indonesia show moderately strong negative correlation (Guatemala r=-0.28)

Linear Regression Model


y is the dependent variable – here: estimated mean of all 5 plausible valuesx is the independent variableß0 is the intercept (value of y when x is zero)

ß1 is the slope (change in y for each unit increase in x)

10 ββ + xy

Recoding variablesReproducing Table 7.8 from ICCS2009 International report, columns 2 and 10Most regressions in Table 7.8 has been calculated using dummy recoded variablesInformation of the students‘ immigration background (variable IMMIG) needs to be recoded to dummy variableFor replication of the table, the information from the variable IMMIG needs to be recoded in SPSS and added as a new variable to the data file before running the analysis

Native1st

Generation Immigrant

Non-Native

IMMIG 1 2 3

Reg01IMMIG 0 1 1

Dummy Coding for Regression

IMMIG REGIMMIG

Dummy Coding for Regression

IMMIG values:

0

1

1

System Missing

Menu: TRANSFORM Recode into Different Variables...

SPSS: Dummy Coding for SPSS: Dummy Coding for RegressionRegression


1 02 13 1ELSE SYSMISS

Menu: TRANSFORM Recode into Different Variables...


Save the file under different name:C:\ICCS2009\Work\ICG_ISG_INTC2_REG.sav

SPSS: Dummy Coding for SPSS: Dummy Coding for RegressionRegressionOr just run the following syntax (C:\ICCS2009\Work\Recode_IMMIG.sps):

GET FILE = "C:\ICCS2009\Work\ICG_ISG_INTC2.sav".

RECODE IMMIG (MISSING=SYSMISS) (1=0) (ELSE=1) INTO REGIMMIG.VARIABLE LABELS REGIMMIG "RECODED IMMIGRATION BACKGROUND".VALUE LABELS REGIMMIG "0" "No immigration background" "1" "Immigration Background".EXECUTE.

SAVE OUTFILE = "C:\ICCS2009\Work\ICG_ISG_INTC2_REG.sav".

10 ββ + xy


Predictor variable: REGIMMIG

Mean achievement for native students

Difference between mean achieve-ment of native and mean achieve-ment of non-

native students

IDCNTRY

TOTWGTS

JKZONES

C:\ICCS2009\Work\ICG_ISG_INTC2_REG.sav


REGIMMIG

PVICIV01-05

RegressionWith Achievement Scores Exclude Missing from Analysis

Regression - Settings

Regression – SPSS Output

Number of Cases

Multiple R-Squared

Intercept – the mean achievement of native students

S.E. of Intercept

List of countries


Regression coefficient for the independent variable

S.E. of regression coefficient

t-test statistics


In 27 out of 38 countries there are statistically significant difference in civic knowledge between native students and non-native students.

ABS (t-test) > 1.96

↓The difference is statistically

signifficant

Regression – Interpretation

In Austria the difference in civic knowledge between native students and non-native students is statistically significant, but not in Bulgaria.

Austria – ABS (-8.89) > 1.96 Bulgaria – ABS (-1.27) < 1.96

Regression – Interpretation

Mean achievement for native students

Difference between mean achievement of native and mean

achievement of non-native students

56.6516.2 -= xy

Benchmarks/Proficiency Levels

In ICCS the proficiency levels are distinct areas in the distribution providing descriptions of achievement on the scale in relation to performance on the questions asked

Level 3 of International Proficiency Level: ≥ 563 score pointsLevel 2 of International Proficiency Level: 479 562 score pointsLevel 1 of International Proficiency Level: 396 478 score points

IEA IDB Analyzer computes percentages of students within, reaching or surpassing user provided proficiency levels of achievement with JRR standard errors for those percentages

Percentages of Students at Specific Proficiency Levels

What is the percentage of students reaching each one of the ICCS 2009 proficiency levels in each country?

Reproducing Table 3.12 from ICCS2009 International report

Benchmarks - Settings


C:\ICCS2009\Work\ICG_ISG_INTC2_REG.sav

IDCNTRY

TOTWGTS

JKZONES

563 479 395

BenchmarksExclude Missing from Analysis

1

Benchmarks - SPSS Output

Performance groups

List of countries

Achievement scores variable name

Benchmarks - Excel Output

Proficiency levels cut-points

Benchmarks - Excel Output

S.E. of the percentage

Estimated number of students

Total student weight – contains estimates in the population in each group

Weighted percentage of students below or within specific Proficiency Level

Benchmarks - Interpretation

The proportion of students in approximately half of the countries reach or excede the international average for Proficiency Level 3 (28%). The highest percentage of students reaching or exceding Level 3 is Finland, and the lowest is the Dominican Republic.

Any Questions?

Thank you for your attention!

Hands-On TrainingA. Re-produce the presented example using your

country data- Percentage of students’ agreement to the right of expressing opinions freely (IS2P20A)

and/orB. Re-produce the presented example using your

country data- Mean civic and civic knowledge of students by their gender

and/orC. Practice with own selected variables following these

analysis steps (Percentages and/or Percentages and Means)

Hands-On Training

A. Re-produce the example using your country data- Correlation of students’ participation in the wider community with civic and citizenship achievement (PVCIV01-05 with PARTCOM)

and/orB. Re-produce the example using your country data

- Regression of students’ immigration status on civic and citizenship achievement of (IMMIG [recoded] on PVCIV01-05)

and/orC. Practice with own selected variables following these

analysis steps (Correlation and/or Regression)

using the idb analyzer to analyze iccs data

Documents