Introduction to R Lecture 3: Data Manipulation

Andrew Jaffe

9/27/10

Overview

Practice Solutions Indexing Data Management Data Summaries

Practice

Make a 2 x 2 table of sex and dog

> table(dat$sex, dat$dog) no yes F 264 229 M 254 253

Practice

Create a 'BMI' variable using height and weight

> dat$bmi = dat$weight*703/dat$height^2> head(dat$bmi)[1] 23.44931 31.29991 25.69422 23.89881 23.11172 28.13324

Practice

Create an 'overweight' variable, which gives the value 1 for people with BMI > 30 and 0 otherwise

> dat$overweight = ifelse(dat$bmi > 30, 1, 0)> head(dat$overweight)[1] 0 1 0 0 0 0

Practice

Add those two variables to the datasets and save it as a text file somewhere

write.table(dat, "lec2_practice.txt", quote = F, row.names = F, sep="\t")

Overview

Practice Solutions Indexing Data Management Data Summaries

Indexing

Vectors: vector[index] takes ‘index’ elements from vector and returns them

> x = c(1,3,7,34,435)> x[1][1] 1> x[c(1,4)][1] 1 34> x[2:4][1] 3 7 34

> 2:4[1] 2 3 4

Indexing

Replace elements in a vector – combining indexing, is.na(), and rep()

> x = c(1,3,NA,6,NA,8)> which(is.na(x))[1] 3 5> x[is.na(x)] = 0 # or rep(0)> x[1] 1 3 0 6 0 8

Indexing

Data.frames/matrices: dat[row,col]Can subset/extract a row: dat[row,]Can subset/extract a column: dat[,col]

> x = matrix(c(1,2,3,4,5,6), ncol = 3)> x [,1] [,2] [,3][1,] 1 3 5[2,] 2 4 6

Indexing

> x[1,][1] 1 3 5> x[,1][1] 1 2> x[1,1][1] 1> x[1:2,1:2] [,1] [,2][1,] 1 3[2,] 2 4

> x [,1] [,2] [,3][1,] 1 3 5[2,] 2 4 6

Indexing

> x[1,] = rep(1) > x [,1] [,2] [,3][1,] 1 1 1[2,] 2 4 6> x[,1] = rep(2)> x [,1] [,2] [,3][1,] 2 1 1[2,] 2 4 6

> x [,1] [,2] [,3][1,] 1 3 5[2,] 2 4 6

Overview

Practice Solutions Indexing Data Management Data Summaries

Data Management

An aside: save() and load() save(obj_1,…,obj_n, file = “filename.rda”)

Saves R objects (vectors, matrices, or data.frames) as an .rda file (similar to .dta)

load(“filename.rda”) Loads whatever files were saved in the .rda

Easier than reading/writing tables

Data Management

Your workspace can be saved an .rda fileYou get asked this every time you close Rsave.image(“filename.Rdata”) saves all

objects in your workspace (what ls() returns)Each folder might have its own .Rdata file

Doing this is personal preference - if you have a script and it’s a quick analysis, probably don’t need a saved image

Data Management

“lec3_data.rda” can be downloaded from the website

Similar method to read in the data: load(“lec3_data.rda”)Put in the same directory as your scriptSet your working directory Use the full filename

Data Management

What are the dimensions of the dataset?

Data Management

What are the dimensions of the dataset?

> dim(dog_dat)[1] 482 6

Data Management

How many dogs are in this dataset? Is this dataset unique?

Data Management

How many dogs are in this dataset? Is this dataset unique?

> length(unique(dog_dat$dog_id))[1] 482> length(dog_dat$dog_id))[1] 482

Data Management

What are the column/variable names?

Data Management

What are the column/variable names?> head(dog_dat) dog_id owner_id dog_type dog_wt_mo1 dog_len_mo1 dog_food_mo11 1 394 lab 51.5 13.8 25.82 2 571 lab 48.3 24.6 33.13 3 986 poodle 59.3 22.7 29.24 4 750 lab 46.4 22.3 27.65 5 882 husky 48.0 20.9 28.06 6 762 poodle 47.0 19.1 31.0

> names(dog_dat)[1] "dog_id" "owner_id" "dog_type” "dog_wt_mo1" [5] "dog_len_mo1" "dog_food_mo1"

Data Management

Some explanation of the variablesdog_id: id of dogowner_id: id of ownerdog_type: type of dogdog_wt_mo1: dog weight at month 1 (baseline)dog_len_mo1: dog length at month 1dog_food_mo1: baseline dog food consumption

Data Management

Subsetting data: separate data into two data.frames based on a variable:

> lab = dog_dat[dog_dat$dog_type == "lab",]> head(lab) dog_id owner_id dog_type dog_wt_mo1 dog_len_mo1 dog_food_mo11 1 394 lab 51.5 13.8 25.82 2 571 lab 48.3 24.6 33.14 4 750 lab 46.4 22.3 27.67 7 664 lab 53.0 18.2 25.713 13 713 lab 48.3 23.4 31.815 15 480 lab 46.6 20.8 31.3

Data Management

> lab = dog_dat[dog_dat$dog_type == "lab",]

> head(which(dog_dat$dog_type == "lab"))[1] 1 2 4 7 13 15

Taking those specific rows, and all of the columns of the original data

Data Management

> lab2 = dog_dat[dog_dat$dog_type == ”lab",1:3]> head(lab2,3) dog_id owner_id dog_type1 1 394 lab2 2 571 lab4 4 750 lab

Taking those specific rows, and the first 3 columns of the original data

Data Management

Note (stata users…) that we have two data.frames in our workspace! [ls()]

Data Management

Remember we used ifelse() for binary conversions?

> heavy = ifelse(dog_dat[,4] > mean(dog_dat[,4]), 1, 0)

> head(heavy)[1] 1 0 1 0 0 0 Note that you can use

column indexing instead of $name for data.framesThis is just the mean of

that column:> mean(dog_dat[,4])[1] 49.69606

Data Management

The cut() function can split data into more groups – quintiles, tertiles, etc

cut(dat, breaks)dat is a vector of numerical or integer valuesbreaks is where to make the cuts

Data Management

If ‘breaks’ is one number (n), it splits the data into ‘n’ equal sized groups

> x = 1:5 # 1 2 3 4 5 or seq(1,5)> cut(x, 2)[1] (0.996,3] (0.996,3] (0.996,3] (3,5] (3,5] Levels: (0.996,3] (3,5]> cut(x, 3)[1] (0.996,2.33] (0.996,2.33] (2.33,3.67] (3.67,5] (3.67,5] Levels: (0.996,2.33] (2.33,3.67] (3.67,5]> cut(x,3, labels=F) # returns integers of groups, not factors[1] 1 1 2 3 3

FACTORS!

Data Management

What is a factor? Similar to terms like ‘category’ and ‘enumerated type’

Has ‘levels’ associated with it – could be ordinal if factor(…,ordered = T)

Must only have an as.character() method and be sortable to be converted to a factor using factor()

Data Management

If ‘breaks’ are more than one number, splits the vector by those numbers

> x = 1:10> cut(x, c(0,3,6,10)) [1] (0,3] (0,3] (0,3] (3,6] (3,6] (3,6] (6,10] (6,10] (6,10] (6,10]Levels: (0,3] (3,6] (6,10]> cut(x, c(0,3,6,10), FALSE) [1] 1 1 1 2 2 2 3 3 3 3

Data Management

Something more applicable for cut: the quantile(x,probs) function - default ‘probs’ is seq(0,1,0.25), ie quintiles

seq(start, end, by) – creates a sequence from the starting value, to the ending value by the specified amountseq(0,10) ~ 0:10 # 0, 1, 2, …, 9, 10seq(0,10,0.5) # 0, 0.5, 1.0, …, 9.5, 10.0

Data Management

Now for stuff with our data:> quantile(dog_dat$dog_wt_mo1) 0% 25% 50% 75% 100% 10.600 44.600 49.200 55.275 72.500 > quantile(dog_dat$dog_wt_mo1, seq(0,1,0.5)) 0% 50% 100% 10.6 49.2 72.5 > quantile(dog_dat$dog_wt_mo1, 0.6) 60% 51.5 > quantile(dog_dat$dog_wt_mo1, c(0.4,0.6)) 40% 60% 47.24 51.50

Data Management

> sp = quantile(dog_dat$dog_wt_mo1, 0.75)> big = ifelse(dog_dat$dog_wt_mo1 > sp, 1, 0)> head(big)[1] 0 0 1 0 0 0

> quant = cut(dog_dat$dog_wt_mo1, quantile(dog_dat$dog_wt_mo1))> head(quant)[1] (49.2,55.3] (44.6,49.2] (55.3,72.5] (44.6,49.2] (44.6,49.2] (44.6,49.2]Levels: (10.6,44.6] (44.6,49.2] (49.2,55.3] (55.3,72.5]

Overview

Practice Solutions Indexing Data Management Data Summaries

Data Summaries

This is some of the only “statistics” in the course

R functions can perform statistics well, here are some basics for summaries

Data Summaries

mean(dat, na.rm = F) median(dat, na.rm=F)

> x = c(1,2,4,6,NA)> mean(x)[1] NA> mean(x, na.rm=T)[1] 3.25> median(x,na.rm=T)[1] 3

Data Summaries

> x = c(1,2,4,7,9,11)> mean(x)[1] 5.666667> median(x)[1] 5.5> var(x)[1] 15.86667> sd(x)[1] 3.983298

Data Summaries

Let’s combine some concepts! Take the mean food consumption of all of

the labs

Data Summaries

First, figure out which entries correspond to dogs that are labs

> Index = which(dog_dat$dog_type == "lab")> head(Index)[1] 1 2 4 7 13 15

Data Summaries

Then, take the mean of the data you want

> mean(dog_dat$dog_food_mo1[Index])[1] 30.04

Note that we first created a vector of dog food, then indexed it - there are no commas needed for the indexing (because it’s a vector)

Data Summaries

Combined into 1 line/command:

> mean(dog_dat$dog_food_mo1[dog_dat$dog_type == "lab"])[1] 30.04> mean(dog_dat[dog_dat$dog_type == "lab",6])[1] 30.04> mean(dog_dat[dog_dat$dog_type == "lab","dog_food_mo1"])[1] 30.04

Pick your favorite – they’re all the same! Note that the first option might make the most sense…

Practice

Compute the average dog weight, dog length, and dog food consumption for each dog type at baseline

Reminder: the dog types are lab, poodle, husky, and retriever