basic data analysis using r xiao he 1. agenda 1.data cleaning (e.g., missing values) 2.descriptive...

1

Basic Data Analysis Using RXiao He

2

AGENDA

1. Data cleaning (e.g., missing values)

2. Descriptive statistics

3. t-tests

4. ANOVA

5. Linear regression

Data visualization

3

AGENDA



3. t-tests

4. ANOVA


4

1. DATA CLEANINGNA and NaN:

1. NA (Not Available): missing values.

a). Represented in the form of NA, or in the form of <NA>.

2. NaN (Not a number): when an arithmetic operation returns a non-numeric result: e.g., in R, 0/0 gives you NaN.

NOTE: You may have coded missing values using other notations. For example, some researchers/programs use -99 or -999 to denote missing values. If that is the case, you need to check for missing values using other methods that won’t be covered here.

5

1. DATA CLEANINGNA and NaN:

3. Deal with NA and NaN?

a). Check how many cases (rows) do NOT have NA or NaN:complete.cases()#Returns Booleans (TRUE, FALSE). TRUE means no missing #value (in a given row), and FALSE means there is at #least one missing value.

b). Remove cases with NA or NaN:na.omit()#return a data object with NA and NaN removed. For data#frames, an entire row will be removed if it contains

NA#or NaN.

c). More sophisticated ways of dealing with NA (covered by Addie’s workshop in two weeks)

Ex1.1: (refer to handout)


6

AGENDA



3. t-tests

4. ANOVA


7

2. DESCRIPTIVE STATISTICS1. Help you diagnose potential problems w/ data entry or collection:

a. Were any values entered incorrectly? e.g., survey study using 1 – 5 Likert scale, but when you checked the

range of your data, you found that the maximum value in your dataset was 7.

b. Any strange responses?e.g., Did your participants give you any odd responses?

2. Help you get a sense of how your data are distributed.

a. Extreme values (outliers)

b. Non-normality

c. Skewness

d. Unequal variance

8

2. DESCRIPTIVE STATISTICS Compute individual descriptive statistics

1. Location:

mean(x, trim) median(x)

2. Dispersion

var(x) sd(x) range(x); min(x); max(x) IQR(x)



9

2. DESCRIPTIVE STATISTICS Compute a set of descriptive statistics

1. Use the function summary().

2. function describe() in the package `psych`.



10

AGENDA



3. t-tests

4. ANOVA


11

3. T TESTS

t.test()t.test(x, y = NULL,

alternative=c("two.sided", "less", "greater”),mu = 0, paired = FALSE, var.equal = FALSE, conf.level = .95)

1. One-sample t-test:

12

3. T TESTS



1. One-sample t-test:

Suppose someone hypothesized that the mean undergrad age was 19.75. Let’s test whether the mean age was significantly different from 19.75.

H0: mu0 = 19.75

H1: mu0 ≠ 19.75


13

3. T TESTS



2. Independent t-test:

Let’s test whether the mean height of female students is significantly different from the mean height of male students

H0: muFemale = muMale

H1: muFemale ≠ muMale


14

3. T TESTS



t.test(formula, data,…)

2. Independent t-test:

Let’s test whether the mean height of female students is significantly different from the mean height of male students

H0: muFemale = muMale

H1: muFemale ≠ muMale


formula = Y ~ XY: outcome variable (e.g., height)X: 2 level grouping variable (e.g., Sex)

15

3. T TESTS



t.test(formula, data,…)

3. Paired t-test:

Let’s test whether the mean Writing hand span (Wr.Hnd) and the mean Non-writing hand span (NW.Hnd) differ significantly.

H0: muWr.Hnd = muNW.Hnd

H1: muWr.Hnd ≠ muNW.Hnd


16

AGENDA



3. t-tests

4. ANOVA


17

4. ANOVA

aov()aov(formula, data)

1. One-way ANOVA:

Suppose we are interested in whether the mean pulse rates differ amongst people of different exercise statuses.

H0: muNone = muSome = muFreq

H1: Not all groups are equal.


18

We will import a new dataset and will use it for the next exercise.

hsb2 <- read.table("http://www.ats.ucla.edu/stat/r/faq/hsb2.csv", sep=",", header=TRUE)

19

4. ANOVA

aov()aov(formula, data)

2. Two-way ANOVA:

The formula/model for factorial ANOVA (take 2 way interaction for example) is specified as follows:

Y ~ X1 * X2 which is equivalent to Y ~ X1 + X2 + X1:X2

Suppose we are interested the main effects of race and schtyp (school type) as well as the interaction effect between the two variables on read. The formula/model can be specified as:

read ~ as.factor(race) * as.factor(schtyp)


Why do we do this?

20

NoteIn R, by default, ANOVA results are based on Type 1 (“sequential”) Sum of Squares. Some other programs report Type 3 SS (SPSS) or both Type 1 and Type 3 (SAS). The Type 3 SS for each term is calculated given all other terms are in the model.

21

AGENDA



3. t-tests

4. ANOVA


22

Let’s import a new dataset

expenditure <- read.table("http://dornsife.usc.edu/assets/sites/210/docs/GC3/educationExpenditure.txt", sep=",", header=TRUE)

23Ex6.1: (refer to handout)

5. LINEAR REGRESSIONlm()

lm(formula, data)

education: Per-capita education expenditures, dollars.

income: Per-capita income, dollars.

young: Proportion under 18, per 1000.

urban: Proportion urban, per 1000.

1. Simple linear regression

formula = Y ~ X

Suppose we are interested in testing the regression of per-capita education expenditure on per-capita income, the model should be specified as:

formula = education ~ income

24

5. LINEAR REGRESSION

lm()lm(formula, data)

education: Per-capita education expenditures, dollars.

income: Per-capita income, dollars.

young: Proportion under 18, per 1000.

urban: Proportion urban, per 1000.

2. Multiple regression

formula = Y ~ X1 + X2 + … + Xp

Suppose we are interested in testing the regression of education on income, young, and urban, the model should be specified as:

formula = education ~ income + young + urban


25

Thanks!

basic data analysis using r xiao he 1. agenda 1.data cleaning (e.g., missing values) 2.descriptive...

Documents