1. Introduction to R Lecture 3: Data Manipulation Andrew Jaffe 9/27/10

2. Overview Practice Solutions Indexing Data Management Data Summaries

3. Practice Make a 2 x 2 table of sex and dog > table(dat$sex, dat$dog) no yes F 264 229 M 254 253

4. Practice Create a 'BMI' variable using height and weight > dat$bmi = dat$weight*703/dat$height^2 > head(dat$bmi) [1] 23.44931 31.29991 25.69422 23.89881 23.11172 28.13324

5. Practice Create an 'overweight' variable, which gives the value 1 for people with BMI > 30 and 0 otherwise > dat$overweight = ifelse(dat$bmi > 30, 1, 0) > head(dat$overweight) [1] 0 1 0 0 0 0

6. Practice Add those two variables to the datasets and save it as a text file somewhere write.table(dat, "lec2_practice.txt", quote = F, row.names = F, sep="\t")

7. Overview Practice Solutions Indexing Data Management Data Summaries

8. Indexing Vectors: vector[index] takes ‘index’ elements from vector and returns them > x = c(1,3,7,34,435) > x[1] [1] 1 > x[c(1,4)] [1] 1 34 > x[2:4] [1] 3 7 34 > 2:4 [1] 2 3 4

9. Indexing Replace elements in a vector – combining indexing, is.na(), and rep() > x = c(1,3,NA,6,NA,8) > which(is.na(x)) [1] 3 5 > x[is.na(x)] = 0 # or rep(0) > x [1] 1 3 0 6 0 8

10. Indexing Data.frames/matrices: dat[row,col]  Can subset/extract a row: dat[row,]  Can subset/extract a column: dat[,col] > x = matrix(c(1,2,3,4,5,6), ncol = 3) > x [,1] [,2] [,3] [1,] 1 3 5 [2,] 2 4 6

11. Indexing > x[1,] [1] 1 3 5 > x[,1] [1] 1 2 > x[1,1] [1] 1 > x[1:2,1:2] [,1] [,2] [1,] 1 3 [2,] 2 4 > x [,1] [,2] [,3] [1,] 1 3 5 [2,] 2 4 6

12. Indexing > x[1,] = rep(1) > x [,1] [,2] [,3] [1,] 1 1 1 [2,] 2 4 6 > x[,1] = rep(2) > x [,1] [,2] [,3] [1,] 2 1 1 [2,] 2 4 6 > x [,1] [,2] [,3] [1,] 1 3 5 [2,] 2 4 6

13. Overview Practice Solutions Indexing Data Management Data Summaries

14. Data Management An aside: save() and load() save(obj_1,…,obj_n, file = “filename.rda”)  Saves R objects (vectors, matrices, or data.frames) as an.rda file (similar to.dta) load(“filename.rda”)  Loads whatever files were saved in the.rda Easier than reading/writing tables

15. Data Management Your workspace can be saved an.rda file  You get asked this every time you close R  save.image(“filename.Rdata”) saves all objects in your workspace (what ls() returns)  Each folder might have its own.Rdata file Doing this is personal preference - if you have a script and it’s a quick analysis, probably don’t need a saved image

16. Data Management “lec3_data.rda” can be downloaded from the website Similar method to read in the data: load(“lec3_data.rda”)  Put in the same directory as your script  Set your working directory  Use the full filename

17. Data Management What are the dimensions of the dataset?

18. Data Management What are the dimensions of the dataset? > dim(dog_dat) [1] 482 6

19. Data Management How many dogs are in this dataset? Is this dataset unique?

20. Data Management How many dogs are in this dataset? Is this dataset unique? > length(unique(dog_dat$dog_id)) [1] 482 > length(dog_dat$dog_id)) [1] 482

21. Data Management What are the column/variable names?

22. Data Management What are the column/variable names? > head(dog_dat) dog_id owner_id dog_type dog_wt_mo1 dog_len_mo1 dog_food_mo1 1 1 394 lab 51.5 13.8 25.8 2 2 571 lab 48.3 24.6 33.1 3 3 986 poodle 59.3 22.7 29.2 4 4 750 lab 46.4 22.3 27.6 5 5 882 husky 48.0 20.9 28.0 6 6 762 poodle 47.0 19.1 31.0 > names(dog_dat) [1] "dog_id" "owner_id" "dog_type” "dog_wt_mo1" [5] "dog_len_mo1" "dog_food_mo1"

23. Data Management Some explanation of the variables  dog_id: id of dog  owner_id: id of owner  dog_type: type of dog  dog_wt_mo1: dog weight at month 1 (baseline)  dog_len_mo1: dog length at month 1  dog_food_mo1: baseline dog food consumption

24. Data Management Subsetting data: separate data into two data.frames based on a variable: > lab = dog_dat[dog_dat$dog_type == "lab",] > head(lab) dog_id owner_id dog_type dog_wt_mo1 dog_len_mo1 dog_food_mo1 1 1 394 lab 51.5 13.8 25.8 2 2 571 lab 48.3 24.6 33.1 4 4 750 lab 46.4 22.3 27.6 7 7 664 lab 53.0 18.2 25.7 13 13 713 lab 48.3 23.4 31.8 15 15 480 lab 46.6 20.8 31.3

25. Data Management > lab = dog_dat[dog_dat$dog_type == "lab",] > head(which(dog_dat$dog_type == "lab")) [1] 1 2 4 7 13 15 Taking those specific rows, and all of the columns of the original data

26. Data Management > lab2 = dog_dat[dog_dat$dog_type == ”lab",1:3] > head(lab2,3) dog_id owner_id dog_type 1 1 394 lab 2 2 571 lab 4 4 750 lab Taking those specific rows, and the first 3 columns of the original data

27. Data Management Note (stata users…) that we have two data.frames in our workspace! [ls()]

28. Data Management Remember we used ifelse() for binary conversions? > heavy = ifelse(dog_dat[,4] > mean(dog_dat[,4]), 1, 0) > head(heavy) [1] 1 0 1 0 0 0 Note that you can use column indexing instead of $name for data.frames This is just the mean of that column: > mean(dog_dat[,4]) [1] 49.69606

29. Data Management The cut() function can split data into more groups – quintiles, tertiles, etc cut(dat, breaks)  dat is a vector of numerical or integer values  breaks is where to make the cuts

30. Data Management If ‘breaks’ is one number (n), it splits the data into ‘n’ equal sized groups > x = 1:5 # 1 2 3 4 5 or seq(1,5) > cut(x, 2) [1] (0.996,3] (0.996,3] (0.996,3] (3,5] (3,5] Levels: (0.996,3] (3,5] > cut(x, 3) [1] (0.996,2.33] (0.996,2.33] (2.33,3.67] (3.67,5] (3.67,5] Levels: (0.996,2.33] (2.33,3.67] (3.67,5] > cut(x,3, labels=F) # returns integers of groups, not factors [1] 1 1 2 3 3 FACTORS!

31. Data Management What is a factor? Similar to terms like ‘category’ and ‘enumerated type’ Has ‘levels’ associated with it – could be ordinal if factor(…,ordered = T) Must only have an as.character() method and be sortable to be converted to a factor using factor()

32. Data Management If ‘breaks’ are more than one number, splits the vector by those numbers > x = 1:10 > cut(x, c(0,3,6,10)) [1] (0,3] (0,3] (0,3] (3,6] (3,6] (3,6] (6,10] (6,10] (6,10] (6,10] Levels: (0,3] (3,6] (6,10] > cut(x, c(0,3,6,10), FALSE) [1] 1 1 1 2 2 2 3 3 3 3

33. Data Management Something more applicable for cut: the quantile(x,probs) function - default ‘probs’ is seq(0,1,0.25), ie quintiles seq(start, end, by) – creates a sequence from the starting value, to the ending value by the specified amount  seq(0,10) ~ 0:10 # 0, 1, 2, …, 9, 10  seq(0,10,0.5) # 0, 0.5, 1.0, …, 9.5, 10.0

34. Data Management Now for stuff with our data: > quantile(dog_dat$dog_wt_mo1) 0% 25% 50% 75% 100% 10.600 44.600 49.200 55.275 72.500 > quantile(dog_dat$dog_wt_mo1, seq(0,1,0.5)) 0% 50% 100% 10.6 49.2 72.5 > quantile(dog_dat$dog_wt_mo1, 0.6) 60% 51.5 > quantile(dog_dat$dog_wt_mo1, c(0.4,0.6)) 40% 60% 47.24 51.50

35. Data Management > sp = quantile(dog_dat$dog_wt_mo1, 0.75) > big = ifelse(dog_dat$dog_wt_mo1 > sp, 1, 0) > head(big) [1] 0 0 1 0 0 0 > quant = cut(dog_dat$dog_wt_mo1, quantile(dog_dat$dog_wt_mo1)) > head(quant) [1] (49.2,55.3] (44.6,49.2] (55.3,72.5] (44.6,49.2] (44.6,49.2] (44.6,49.2] Levels: (10.6,44.6] (44.6,49.2] (49.2,55.3] (55.3,72.5]

36. Overview Practice Solutions Indexing Data Management Data Summaries

37. This is some of the only “statistics” in the course R functions can perform statistics well, here are some basics for summaries

38. Data Summaries mean(dat, na.rm = F) median(dat, na.rm=F) > x = c(1,2,4,6,NA) > mean(x) [1] NA > mean(x, na.rm=T) [1] 3.25 > median(x,na.rm=T) [1] 3

39. Data Summaries > x = c(1,2,4,7,9,11) > mean(x) [1] 5.666667 > median(x) [1] 5.5 > var(x) [1] 15.86667 > sd(x) [1] 3.983298

40. Data Summaries Let’s combine some concepts! Take the mean food consumption of all of the labs

41. Data Summaries First, figure out which entries correspond to dogs that are labs > Index = which(dog_dat$dog_type == "lab") > head(Index) [1] 1 2 4 7 13 15

42. Data Summaries Then, take the mean of the data you want > mean(dog_dat$dog_food_mo1[Index]) [1] 30.04 Note that we first created a vector of dog food, then indexed it - there are no commas needed for the indexing (because it’s a vector)

43. Data Summaries Combined into 1 line/command: > mean(dog_dat$dog_food_mo1[dog_dat$dog_type == "lab"]) [1] 30.04 > mean(dog_dat[dog_dat$dog_type == "lab",6]) [1] 30.04 > mean(dog_dat[dog_dat$dog_type == "lab","dog_food_mo1"]) [1] 30.04 Pick your favorite – they’re all the same! Note that the first option might make the most sense…

44. Practice Compute the average dog weight, dog length, and dog food consumption for each dog type at baseline Reminder: the dog types are lab, poodle, husky, and retriever