ETW2001 by ed ock

edwinock2003 — 2023-03-25 07:44:20 UTC

What to learn?

1. The if statement
2. Loops - to run a statement or a set of statements multiple times.
3. Functions
4. The apply family

The IF Statement (Basic Level)

edwinock2003 — 2023-03-25 07:45:28 UTC

The "if" statement is a fundamental control flow statement in programming that allows you to execute certain code blocks conditionally based on the truth value of an expression

THE CODE
if (condition) {
# code to execute if the condition is true
}

"condition" is a logical expression that evaluates to either TRUE or FALSE. If the expression is true, the code block inside the curly braces {} is executed. Otherwise, the code block is skipped.

The IF Statement (IF ELSE)

edwinock2003 — 2023-03-25 07:45:50 UTC

The "if-else" function is a control flow statement in R that allows you to specify two different code blocks to execute depending on whether a condition is true or false. Here's the basic syntax of the "if-else" function in R:

if (condition) {
# code to execute if the condition is true
} else {
# code to execute if the condition is false
}

#This is just a advance ver of the previous IF function to print a statement if conditions are NOT met.

The IF Statement (ELSE If + ELSE)

edwinock2003 — 2023-03-25 07:46:55 UTC

if(), else if(), and else()

if(condition 1) {
expr 1
} else if (condition 2) {
expr 2
} else {
expr 3
}

WHILE LOOP (EXAMPLE 2)

edwinock2003 — 2023-03-25 07:48:53 UTC

Observe the outputs.

CODE :

x <- 5
while(x) {
print(x)
x <- x - 1
}

In the while loop you provided, the output does not have 0 because the condition in the while loop is checking if x is a "truthy" value. In R, any non-zero numeric value is considered "truthy" and will evaluate to TRUE in a logical expression, including in the condition of a while loop.

So, when the value of x is 0, the condition in the while loop evaluates to FALSE, and the loop exits before the print(x) statement is executed. This is why the output does not include 0.

If you wanted to include 0 in the output, you could modify the condition to check if x is greater than or equal to 0:

--> Truthy
In contrast, a "falsy" value is a value that is considered to be false when evaluated in a boolean expression. In most programming languages, the number 0, the empty string "", and null or undefined values are considered "falsy".

For Loops (Explanation)

edwinock2003 — 2023-03-25 07:49:04 UTC

For loop is a control structure that allows you to repeat a block of code a fixed number of times. It is useful when you need to perform the same operation multiple times, such as iterating over a collection of data or performing a mathematical calculation.

For Loop (Examples 1)

edwinock2003 — 2023-03-25 07:49:13 UTC

for loop

A for loop is used for iterating over a sequence

Code 1 :
for(i in 1:6) {
print(i)
}

Code 2:
for(OMG in 1:5) {
print(paste("OMG is", OMG*2))
}

Applying "for loops" in your matrix

edwinock2003 — 2023-03-25 07:49:22 UTC

# Create a 3x3 matrix
my_matrix_data <- c(1:8)
my_matrix <- matrix(data = my_matrix_data,
nrow=2,
ncol=4,
byrow=TRUE)

# Print the original matrix
print(my_matrix)

#Name the dimensions
dimnames(my_matrix) <- list(c("Row 1","Row 2"),c("Col1","Col2","Col3","Col4"))

# Multiply each element of the matrix by 10 and assign to a new matrix
my_matrix_2 <- my_matrix
for (x in 1:nrow(my_matrix_2)) {
for (y in 1:ncol(my_matrix_2)) {
my_matrix_2[x, y] <- my_matrix_2[x, y] * 10
}
}

# Print the updated matrix
print(my_matrix_2)

edwinock2003 — 2023-03-25 07:49:31 UTC

Recipe

function_name <- function(arg1, arg2, ...) {
do something
return(something)
}

edwinock2003 — 2023-03-25 07:50:06 UTC

Example #2.1

f.sum <- function(x, y) {
x + y
}

f.sum(4, 7)

edwinock2003 — 2023-03-25 07:52:32 UTC

apply()

apply() function limits itself to 2D arrays
Usage: apply(X, MARGIN, FUN)

edwinock2003 — 2023-03-25 07:52:41 UTC

lapply()

Applies a given function to every element of a list, vector or data frame and obtain a list as a result.
Usage: lapply(X, FUN)

edwinock2003 — 2023-03-25 07:52:49 UTC

sapply()

Works like lapply(), but it tries to simplify the output to the most elementary data structure that is possible.
Usage: sapply(X, FUN)

edwinock2003 — 2023-03-25 07:54:35 UTC

dplyr

dplyr is a grammar of data manipulation, providing a consistent set of verbs/functions that help you solve the most common data manipulation challenges.

edwinock2003 — 2023-03-25 07:54:43 UTC

dplyr functions:

1. select()
2. filter()
3. arrange()
4. mutate()
5. summarise()
6. group_by()

What is Select?

edwinock2003 — 2023-03-25 07:55:43 UTC

select() picks variables (columns) based on their names.

It "zooms in" & picks variables (columns) listed in the functions based on their names.

Select Examples Part 1

edwinock2003 — 2023-03-25 07:55:50 UTC

#1 Selecting selective/specific columns
#Example, you want to select the "name" and "sleep_total" columns

Code
msleep %>%
select(name, sleep_total) %>%
head(5) # to only show first 5 rows

#2 Selecting columns...except (excluding certain columns)
# To select all the columns except a specific column, use the "-" (subtraction) operator (also known as negative indexing)

Code
msleep %>%
select(-c(name, genus)) %>%
head(5)

edwinock2003 — 2023-03-25 07:56:06 UTC

filter() picks cases (rows) based on their values.

edwinock2003 — 2023-03-25 07:56:48 UTC

arrange() changes the ordering of the rows.

edwinock2003 — 2023-03-25 07:56:55 UTC

Example

df %>%
arrange(Height)

Alternative way to use dplyr function:
arrange(df, Height)

edwinock2003 — 2023-03-25 07:57:22 UTC

mutate() adds new variables (COLUMNS) that are functions of existing variables into the df

edwinock2003 — 2023-03-25 07:57:28 UTC

Example

df %>%
mutate(Height_m = Height/100)

edwinock2003 — 2023-03-25 07:57:51 UTC

summarise() reduces multiple values down to a single summary.

or in other words, collapses the data frame into a single row

edwinock2003 — 2023-03-25 07:58:06 UTC

Example

df %>%
summarise(Avg_Height = mean(Height))

edwinock2003 — 2023-03-25 07:58:28 UTC

dplyr functions combine naturally with group_by() which allows you to perform any operation “by group”.

edwinock2003 — 2023-03-25 07:58:44 UTC

Example

df %>%
group_by(Gender) %>%
summarise(Avg_Height = mean(Height))

edwinock2003 — 2023-03-25 08:39:47 UTC

Sales_data <- read.csv("https://www.dropbox.com/s/ddgf4kjv352c6gm/Sales_data.csv?dl=1", na.strings = c("", "NA"))

Exercise1_Clean <- read.csv("https://www.dropbox.com/s/esz83nclbfxkus3/Exercise1_Clean.csv?dl=1")

interviews <- read.csv("https://www.dropbox.com/s/2ga5h6tgdhenlci/Interviews.csv?dl=1", na = "NULL")

edwinock2003 — 2023-03-25 08:42:56 UTC

The goal of tidyr is to help you create tidy data. Tidy data is data where (THESE 3 RULES MUST BE MET)

Every column is variable.
Every row is an observation.
Every cell is a single value.

Tidy data describes a standard way of storing data that is used wherever possible throughout the tidyverse.

edwinock2003 — 2023-03-25 08:43:34 UTC

separate() turns a single character column into multiple columns by splitting the values of the column wherever a separator character appears

tablename %>%
seperate(colname, into = c("col1" , "col2"), sep = "/")

the "/" is just an example which represents where we wish to use a specific character to separate a column

OR

We can use separate_rows function to separate the values in a cell. However, the difference is since the function can't specify the name of two columns to allocate the values to when seperating, they will DOUBLE the number of existing rows.

Code example
table4 %>%
separate_rows(rate, sep = "/")

this would seperate the two values (ex: 757/51263487) at the "/" since it was stated. However the output would be under the same column (hence the double rows...see below)

Rate
757
51263487

edwinock2003 — 2023-03-25 08:43:40 UTC

Example

teaching_staff <- data.frame(name = c("MoganaDarshini Ganggayah", "FungWan Toh", "MinSoo Chun"),
n = c(4, 2, 1))
teaching_staff

edwinock2003 — 2023-03-25 08:43:47 UTC

separate()

teaching_staff_sep <- teaching_staff %>%
separate(name, into = c("first_name", "last_name"), sep = " ")
teaching_staff_sep

edwinock2003 — 2023-03-25 08:43:59 UTC

unite() is a convenient function to unite multiple columns into one by pasting strings together

edwinock2003 — 2023-03-25 08:44:06 UTC

unite()

teaching_staff_unite <- teaching_staff_sep %>%
unite(name,first_name, last_name, sep = " ")
teaching_staff_unite

edwinock2003 — 2023-03-25 08:51:42 UTC

pivot_longer() "lengthens" data, increasing the number of rows and decreasing the number of columns. The inverse transformation is pivot_wider().

edwinock2003 — 2023-03-25 08:51:57 UTC

Example

f2f_is_better <- data.frame(name = c("Mogana", "Fung Wan", "MinSoo"),
response = c("agree", "neutral", "disagree"))
f2f_is_better

edwinock2003 — 2023-03-25 08:52:10 UTC

pivot_wider()

wider <- f2f_is_better %>%
mutate(response_logical = 1) %>%
pivot_wider(names_from = response,
values_from = response_logical,
values_fill = 0)
wide

edwinock2003 — 2023-03-25 08:52:17 UTC

pivot_longer()

longer <- wider %>%
pivot_longer(agree:disagree,
names_to = "response",
values_to = "response_logical") %>%
filter(response_logical == 1) %>%
select(-response_logical)
longer

edwinock2003 — 2023-03-25 08:54:07 UTC

WEEK4

Variables

edwinock2003 — 2023-03-25 10:00:55 UTC

1) There are 3 data types in R.
#1A) Numeric Values : Both Integers (Whole Numbers) and Decimal Values

#1B) Character Values : text (also known as string)
Character should be in " "
Example "character"

#1C) Logical Values : Boolean Values (TRUE or FALSE)

2) Checking data type
#Function is class ( )
class(3)

3) Assigning Variables
#given your variable is X and you want to assign 42 to X
x <- 42

Creating a Vector

edwinock2003 — 2023-03-25 10:02:59 UTC

#Use C Function

vector_A <- c(1:10)
vector_B <- c(2,4,6)

where : in vector_A means values from 1 to 10

#if you want to put in factors as your variables in the vector, you must use as factor function

as factor (c("HOT", "COLD", "HOT", "COLD"))

Arithmetic with Vectors

edwinock2003 — 2023-03-25 10:03:38 UTC

Assume we want a new vector, vector_C

vector_C <- vector_A*2

Naming values in vectors

edwinock2003 — 2023-03-25 11:07:41 UTC

#Step1 is to create your "name values", you can assign them to another vector using function names()

vector_D <- c("Monday", "Tuesday", "Wednedsay")
names(vector_B) <- vector_D

Comparing Vectors

edwinock2003 — 2023-03-25 11:13:30 UTC

#Results will be in logical values

#Are values in Vector B larger than in Vector D?
vector_B > vector_D

result will be (example)
TRUE FALSE TRUE

*ensure vectors both are of same length for apple to apple comparison*

Vectors

edwinock2003 — 2023-03-25 11:14:47 UTC

Vectors are one-dimension arrays that can hold numeric data, character data, or logical data. In other words, a vector is a simple tool to store data

Matrices

edwinock2003 — 2023-03-25 11:52:18 UTC

In R, a matrix is a collection of elements of the same data type (numeric, character, or logical) arranged into a fixed number of rows and columns. Since you are only working with rows and columns, a matrix is called two-dimensional.

#must be same data type (1 only)

Creating a Matrix

edwinock2003 — 2023-03-25 12:04:28 UTC

#To create a matrix, you use the function matrix()

#Example 1 : Creating a matrix with 3 rows and are ARRANGED +FILLED by Rows

matrix_ex1 <- matrix (1:9 , byrow = TRUE, nrow = 3)

#as seen in the first image, values are added in order (1,2,3) with the prioritization on row.

Creating a Matrix #2

edwinock2003 — 2023-03-25 12:08:25 UTC

#Example 2
#Here is another example of how we can use an existing vector to convert into a matrix.
#nrow would mean 2 rows while ncol is 3 columns. byrow = FALSE would mean u want it to be ARRANGED AND FILLED BY COLUMNS
#notice how 1,2 & 3, 4... order goes by column instead

vector_for_matrix_ex2 <- c(1:6)

matrix_ex2 <- matrix(data= vector_for_matrix_ex2,
nrow = 2,
ncol = 3,
byrow = FALSE)

Renaming Columns or Rows of Matrix

edwinock2003 — 2023-03-25 13:33:40 UTC

Use the functions
colnames(matrix) <- c( )
rownames(matrix) <- c( )

or assign them to vectors

Code :

vector_colnames_ex2 <- c("ColE","ColF","ColG")
colnames(matrix_ex2) <- vector_colnames_ex2

Adding new values (in ROWS) - new ROWS

edwinock2003 — 2023-03-25 14:00:13 UTC

#Use the code rbind
# letter r in the rbind represents rows..think of bind as combining

Code
matrix_ex2 <- rbind(matrix_ex2,c(7,8,9))

Adding new rows, via assignment to vectors

edwinock2003 — 2023-03-25 14:01:22 UTC

#Adding new rows, via assignment to vectors

vector_colnames1_ex2 <- c(10,11,12)
matrix_ex2 <- rbind(matrix_ex2, vector_colnames1_ex2)
rownames(matrix_ex2) <- c("RowA","RowB","RowC","RowD")
matrix_ex2

Adding new Columns via Values

edwinock2003 — 2023-03-25 14:11:12 UTC

#Here's the code to add new columns (via values) at the end of matrix

matrix_ex2 <- cbind(matrix_ex2, c(22,33,44,55))

#If u want to add it at the beginning, just rearrange your code
matrix_ex2 <- cbind(c(69,96,690,6999), matrix_ex2)
#notice how the one u want to appear first... is infront!

Selecting a single value to view

edwinock2003 — 2023-03-25 14:13:22 UTC

matrix name [ROW NO, COL NO]

Selecting a multiple values to view

edwinock2003 — 2023-03-25 14:15:02 UTC

matrix name [c(rowA,rowB),c(colA,colB)]

Removing rows & col

edwinock2003 — 2023-03-25 14:19:34 UTC

#Using the function "-"
#U can remove cols and rows
#Noticed how row C and col F disappeared

What is a Factor

edwinock2003 — 2023-03-25 14:27:54 UTC

The term factor refers to a statistical data type used to store categorical variables. The difference between a categorical variable and a continuous variable is that a categorical variable can belong to a limited number of categories. A continuous variable, on the other hand, can correspond to an infinite number of values.

Functions in Factor

edwinock2003 — 2023-03-25 14:41:22 UTC

#To make a vector as a factor (to identify the variables as categorical variables)

#use the factor function
#convert the vector to a factor
NEW VECTOR NAME <- factor(created vector, ordered = TRUE, levels=c("variable A", "variable B", "variable C"))

What is a Data Frame

edwinock2003 — 2023-03-25 15:51:25 UTC

A data frame has the variables of a data set as columns and the observations as rows. Able to store different types of data (unlike matrices where if one is numerical...others have to be numerical) in table form.

Creating Data Frames (using vectors)

edwinock2003 — 2023-03-25 16:10:24 UTC

Refer to image

Creating Data Frames (w/o vector usage)

edwinock2003 — 2023-03-25 16:11:32 UTC

Refer to image

Selecting elements in data frame

edwinock2003 — 2023-03-25 16:17:47 UTC

#1 - Selecting based on rows/col

matrix_A [row,column]
#it can be in number (ex, row3 col2)
matrix_A [3,2]
#or it can be in words, the name of ur respective row and col
matrix_A ["status", "value"]
#or a shortcut
matrix_A$status

Viewing Data Frames in Table format

edwinock2003 — 2023-03-25 16:18:27 UTC

#The code is
library(tidyverse)
view(mydataframe1)

Types of operators

edwinock2003 — 2023-03-25 16:35:30 UTC

Relational Operators
1) Equality ==
In R, "==" is used as the equality operator to test whether two values or expressions are equal.

2) Inequality !=
In R, "!=" is used as the inequality operator to test whether two values or expressions are not equal.

3) More than/equal to >=
In R, ">=" is used as the greater than or equal to operator to test whether one value or expression is greater than or equal to another.

Logical Operators
1) AND &
In R, "&" is the logical AND operator, which is used to combine two logical expressions and return a single logical value. It returns TRUE only if both expressions are TRUE.

2) OR |
In R, "|" is the logical OR operator, which is used to combine two logical expressions and return a single logical value. It returns TRUE if at least one of the expressions is TRUE.

3) NOT !
The NOT operator, represented by an exclamation mark !, simply negates (makes ineffective) the logical value it’s used on. That is, !TRUE evaluates to FALSE, while !FALSE evaluates to TRUE.

#Example Code for NOT

.# Assigning a logical value to a variable
x <- TRUE

# Negating the logical value using the "!" operator
if (!x) {
print("The expression is false")
} else {
print("The expression is true")
}

What's a list?

edwinock2003 — 2023-03-26 04:11:17 UTC

A list in R allows you to gather a variety of objects under one name (that is, the name of the list) in an ordered way. These objects can be matrices, vectors, data frames, even other lists, etc. It is not even required that these objects are related to each other in any way. You could say that a list is some kind super data type: you can store practically any piece of information in it

Creating a List (Example)

edwinock2003 — 2023-03-26 04:23:24 UTC

#As mentioned, a list in R can contain different data types and data structures:

This list contains the following:

numeric_vector: a numeric vector of length 3
character_vector: a character vector of length 3
logical_vector: a logical vector of length 3
integer_vector: an integer vector of length 3
factor_vector: a factor vector of length 3 with levels "blue", "green", "red"
matrix: a 2 x 3 matrix
data_frame: a data frame with 3 columns and 3 rows
nested_list: a list containing 3 nested lists, each with a different data type
named_vector: a named numeric vector with 3 elements
named_list: a named list with 3 elements, each of a different data type.

different data types and data structures:

#Code
my_list <- list(
numeric_vector = c(1, 2, 3),
character_vector = c("a", "b", "c"),
logical_vector = c(TRUE, FALSE, TRUE),
integer_vector = c(4L, 5L, 6L),
factor_vector = factor(c("red", "green", "blue")),
matrix = matrix(1:6, nrow = 2, ncol = 3),
data_frame = data.frame(x = 1:3, y = c("a", "b", "c"), z = c(TRUE, FALSE, TRUE)),
nested_list = list(list(1, 2), list("a", "b", "c"), list(TRUE, FALSE)),
named_vector = c(a = 1, b = 2, c = 3),
named_list = list(a = 1, b = "two", c = TRUE)
)

Prompts to use with LISTS (Part 1)

edwinock2003 — 2023-03-26 04:32:05 UTC

#1 Accessing Items in List

listname$itemname
or
listname[itemname]

Example : my_list$numeric_vector

#2 Checking length of List
length(listname)

#3 Checking if a particular item is in your list
"data" %in% listname$itemname

Example : "red" %in% my_list$factor_vector

#3 Removing list items

Example :
#The 3 is the 3rd row of item in the list
my_list[3]
#TO remove the this, use "-"
my_list_removed_logical_v <- my_list[-3]
my_list_removed_logical_v

Prompts to use with LISTS (Part 2)

edwinock2003 — 2023-03-26 04:37:06 UTC

#1 Changing data value for an item in the list

Example you want to change one of your data content in the factor_vector of your my_list

Code :
my_list$factor_vector[2] <- "gold"

#2 Adding value into a list item

Example, you want to add one data content in the factor_vector of your my_list

Code:
my_list$factor_vector <- c(my_list$factor_vector, "gold")

WHILE LOOP : Explanation

edwinock2003 — 2023-03-26 05:17:54 UTC

A while loop in R is a type of control flow statement that allows you to repeatedly execute a block of code as long as a certain condition is true. Here's the basic syntax for a while loop

Code :

while(condition) {
#code to execute while condition is true
}

Run a statement repeatedly unless the given condition becomes false.
Hence, the condition part of this recipe should become FALSE at some point during the execution. Else, the while loop will go on indefinitely!

WHILE LOOP (EXAMPLE 1)

edwinock2003 — 2023-03-26 05:26:06 UTC

In this example, i is initialized to 1, and the loop will continue to execute as long as i is less than or equal to 10. Inside the loop, the current value of i is printed out using the print() function, and then i is incremented by 1 using the i <- i + 1 statement. Refer to image for output

CODE
i <- 1
while (i <= 10) {
print(i)
i <- i + 1
}

WHILE LOOP (EXAMPLE 1- CONT)

edwinock2003 — 2023-03-26 05:30:54 UTC

If you write i <- i + 1 before print(i) inside the while loop, then the output will be the same, but the order in which the statements are executed will be different.

Have a look at the code :-
i <- 1
while (i <= 10) {
i <- i + 1
print(i)

---------------------------------------------
Notice that the first value printed is 2 instead of 1, because i is incremented before it is printed.

The reason why the order of the statements matters is because it determines the sequence in which the code is executed. In this case, if you increment i first, you'll print the updated value of i during the next iteration of the loop. If you print i first, you'll print the current value of i before it's updated.

WHILE LOOP (IF, ELSE)

edwinock2003 — 2023-03-26 05:47:16 UTC

In this code, we're using a while loop to iterate through the numbers from 1 to 5. For each number, we're using an if-else statement to check if the number is equal to 3, and then printing out a message indicating whether the number is equal to 3 or not.

The condition in the if statement is x == 3, which checks if the value of x is equal to 3. If the condition is true, we print out a message indicating that x is equal to 3. Otherwise, we print out a message indicating that x is not equal to 3, along with the value of x.

What's print ~paste all about?

edwinock2003 — 2023-03-26 05:51:57 UTC

You can think of the paste() function as a tool that allows you to combine different pieces of information into a single sentence or message.

in R, the paste() function is used to concatenate strings together. It takes any number of arguments, which can be strings, variables, or expressions, and returns a single string that combines all of the arguments.

So in summary, print(paste("x is not equal to 3, x =", x)) means that we're printing a message that combines the string "x is not equal to 3, x =" with the current value of x, using the paste() function to concatenate the strings together.

What's Break?

edwinock2003 — 2023-03-26 06:20:05 UTC

break is a control statement in R that allows you to immediately exit a loop when a certain condition is met.

Example Code

x <- 1
while (x <= 10) {
if (x == 5) {
break
} else {
print(x)
x <- x + 1
}
}

Notice how when x equals 5, the loop will exit and the code will stop executing. If x never equals 5, the loop will run until x is greater than 10.

What's the "i" for?

edwinock2003 — 2023-03-26 06:26:12 UTC

In a for loop, you have a variable that takes on different values for each iteration of the loop. The i in for(i in 1:5) is just a variable name that is used to represent the current value of the sequence.

So in the first iteration, i will be equal to 1. The code inside the loop will be executed using i=1. In the second iteration, i will be equal to 2, and the code inside the loop will be executed using i=2, and so on until the loop has executed the code for each value in the sequence.

Think of it like this: You have a list of things to do, and you want to do each thing on the list in order. The for loop is like a way to go through the list one item at a time. Each time you go through the loop, you take the next item on the list and do it. The variable i is like a pointer to the current item on the list that you're working on

Nested Loops (For loop in for loop)

edwinock2003 — 2023-03-26 06:55:19 UTC

A nested loop is a loop inside another loop. The outer loop controls the number of times the inner loop is executed. The inner loop performs some operation for each iteration of the outer loop.

edwinock2003 — 2023-03-26 06:57:18 UTC

break vs next in R

In R, break and next are used to control the flow of execution in loops.

break is used to terminate a loop prematurely. When break is encountered within a loop, the loop is immediately terminated, and the program continues executing from the next statement after the loop.

For example, consider the following code:

Copy codefor (i in 1:10) { if (i == 5) { break } print(i) }

This loop prints the numbers 1 to 4, and then terminates when i equals 5 due to the break statement.

On the other hand, next is used to skip an iteration of a loop. When next is encountered within a loop, the current iteration is immediately skipped, and the loop proceeds to the next iteration.

For example, consider the following code:

Copy codefor (i in 1:10) { if (i %% 2 == 0) { next } print(i) }

This loop prints only the odd numbers from 1 to 9, by skipping the even numbers with the next statement.

In summary, break terminates a loop prematurely, while next skips to the next iteration of a loop.

edwinock2003 — 2023-03-26 06:59:13 UTC

The na.rm argument stands for "remove NAs" and takes a logical value. When na.rm = TRUE, any NAs in the input data are removed before the computation is performed. When na.rm = FALSE (or is not specified), any NA value in the input data causes the result to be NA.

edwinock2003 — 2023-03-26 13:29:14 UTC

To install a package in R, you can use the install.packages() function followed by the name of the package you want to install, enclosed in quotes

Example
install.packages("tidyverse")
library(tidyverse)

1ST DPLYR FUNCTION --> Select

edwinock2003 — 2023-03-26 13:34:36 UTC

Select Examples Part 2

edwinock2003 — 2023-03-26 13:48:19 UTC

#3 Selecting columns ranging from A to C
#Ex: To select a range of columns by name, use the ":" (colon) operator

Code
msleep %>%
select(name: order) %>%
head(5)

#4 Selecting cols that starts or end with a particular string

#1.5
# To select all columns that start with the character string "sl", use the function starts_with()
msleep %>%
select(starts_with(match="sl")) %>%
head(5)

#1.6
# To select all columns that ends with "wt"
msleep %>%
select(ends_with(match="wt")) %>%
head(5)

edwinock2003 — 2023-03-26 13:51:13 UTC

You can think of the pipe operator as an "and then" formula. The "%>%" operator takes the output of the operation on the left-hand side and passes it as the first argument to the function on the right-hand side, which is similar to saying "do this, and then do that".

For example, in the code below:
df %>%
filter(x > 5) %>%
group_by(y) %>%
summarize(mean_x = mean(x))

we are telling R to take the df data frame, filter it to include only rows where x is greater than 5, then group the resulting data by y, and finally calculate the mean value of x for each group. This is equivalent to saying "filter df to include only rows where x is greater than 5, and then group the resulting data by y, and then calculate the mean value of x for each group".

2ND DPLYR FUNCTION --> FILTER

edwinock2003 — 2023-03-26 13:53:26 UTC

FILTER EXAMPLE PART 1

edwinock2003 — 2023-03-26 13:55:39 UTC

#1 Filtering based on values (range) -- refer to image

Code
# Filter the rows for mammals that sleep a total of more than 16 hours
msleep %>%
filter(sleep_total > 16)

#2 Filtering by exact match

# filter by specific character, eg. omni in the vore column
msleep %>%
filter(vore == "omni") %>%
head(5)

# Filter the df for mammals in the “Perissodactyla” and “Primates” taxonomic order.
# %in% is to select several values within the same column/vector

msleep %>%
filter(order%in% c("Perissodactyla", "Primates"))

OR

msleep %>%
filter(order == "Perissodactyla"| order == "Primates")

3RD DPLYR FUNCTION --> ARRANGE

edwinock2003 — 2023-03-26 14:04:41 UTC

edwinock2003 — 2023-03-26 14:07:32 UTC

edwinock2003 — 2023-03-26 14:07:54 UTC

4TH DPLYR FUNCTION --> MUTATE

edwinock2003 — 2023-03-26 14:08:56 UTC

How to keep both ori df and another df containing the mutated function

edwinock2003 — 2023-03-26 14:14:38 UTC

library(dplyr)

# Create a data frame
df <- data.frame(x = 1:5, y = c("a", "b", "c", "d", "e"))

# Add a new variable using mutate
df_mutated <- df %>%
mutate(z = x * 2)

# Combine the original and mutated data frames
df_final <- bind_cols(df, df_mutated)

# Print the final data frame
print(df_final)

edwinock2003 — 2023-03-26 14:15:32 UTC

It keeps only the newly added variables and DELETES the others

edwinock2003 — 2023-03-26 14:16:56 UTC

#2.8 Using ‘summarise()’ function, compute average, minimum and maximum value for
#sleep_total. These three should be named as avg_sleep, min_sleep and max_sleep.

msleep %>%
summarise(
avg_sleep = mean(sleep_total, na.rm = TRUE),
min_sleep = min(sleep_total, na.rm = TRUE),
max_sleep = max(sleep_total, na.rm = TRUE))

Group By Explanation

edwinock2003 — 2023-03-26 14:20:47 UTC

edwinock2003 — 2023-03-26 14:23:08 UTC

group_by() is a function in R that is used to group data by one or more variables.

When you group data using group_by(), R splits your data into smaller subsets based on the unique values of the grouping variable(s). You can then perform operations on each of these subsets separately using functions like summarize() or mutate().

Example #2

#2.9 From the df, group the data by order then compute maximum value of sleep_total by
#naming it as “max_sleep”. Then, arrange the max_sleep in descending order. Once it is all
#done, show the first 5 rows.

msleep %>%
group_by(order) %>%
summarise(max_sleep = max(sleep_total, na.rm = TRUE)) %>%
arrange(desc(max_sleep)) %>%
head(5)

edwinock2003 — 2023-03-26 15:24:13 UTC

PIVOT LONGER #1

edwinock2003 — 2023-03-26 15:38:14 UTC

table4a %>%
pivot_longer(c('1999','2000'), names_to = "year", values_to = "cases")

Hence, the pivot_longer makes datasets :
1) Longer (by increasing number of rows)
2) Decrease number of columns

Slice Function

edwinock2003 — 2023-03-26 15:44:42 UTC

#Example
#1.3
#Remove Last row since it shows total which disturbs tidying the data

Code
tail(Sales_data)
Sales_data <- Sales_data %>%
slice(1:(n()-1))

PIVOT WIDER #1

edwinock2003 — 2023-03-26 15:51:33 UTC

table 2 %>%
pivot_wider(names_from = type, values_from = count

Both pivot_wider and pivot_longer are COMPLEMENTS
where
pivot longer --> narrow and longer
pivot_wider --> shorter and wider

Remember to convert your data type after seperation...

edwinock2003 — 2023-03-26 16:01:26 UTC

Separating intergers (number values) # part 1

edwinock2003 — 2023-03-26 16:03:55 UTC

Utilizing the seperate_rows & pivot_wider #1

edwinock2003 — 2023-03-26 16:16:34 UTC

Utilizing the seperate_rows & pivot_wider #2

edwinock2003 — 2023-03-26 16:21:14 UTC

UNITE EXAMPLE #1

edwinock2003 — 2023-03-26 16:23:28 UTC

UNITE EXAMPLE #2

edwinock2003 — 2023-03-26 16:23:43 UTC

edwinock2003 — 2023-04-14 08:38:42 UTC

How to Deal with Missing Data?
1. Remove any rows or columns that contain missing data.
Bad: Deleting missing data leads data loss and bias data.

2. Impute the missing values using a statistical method i.e. replacing the missing values with the mean or median value of the variable in which the missing values are found.
Bad: Erroneous in some cases

Finding Missing Data

edwinock2003 — 2023-04-14 08:39:29 UTC

is.na(): To test the missing values (TRUE/FALSE)

colSums(is.na()): To identify total number of NAs in each column of the data frame.

*which(is.na()): To identify the location of NAs

* Not covered in lecture

Example

df <- data.frame(Name = c("Alice", "Bob", NA, NA),
Height = c(156, NA, 172, 160))
df

Example Picture 2

edwinock2003 — 2023-04-14 08:43:26 UTC

Example Picture 3

edwinock2003 — 2023-04-14 08:44:19 UTC

Removing Missing Data

edwinock2003 — 2023-04-14 08:44:36 UTC

Removing Missing Data

na.omit(): It simply rules out any rows that contain any missing value and forgets those rows forever.

*na.exclude(): This argument ignores rows having at least one missing value.

*na.pass(): Take no action.

*na.fail(): It terminates the execution if any of the missing values are found.

edwinock2003 — 2023-04-14 08:47:39 UTC

edwinock2003 — 2023-04-14 08:50:02 UTC

edwinock2003 — 2023-04-14 08:50:47 UTC

Outliers

edwinock2003 — 2023-04-14 08:51:19 UTC

What are outliers?
In statistics, outliers are observations or data points that are significantly different from the other data points in a given dataset. These data points can be either unusually high or low in comparison to the rest of the data.

Outliers can occur due to measurement errors, data entry errors, or genuine rare occurrences. It is important to identify and deal with outliers appropriately as they can distort the results of statistical analyses and affect the validity of any conclusions drawn from the data.

Box Plot

edwinock2003 — 2023-04-14 08:53:06 UTC

How to Deal with Outliers?

edwinock2003 — 2023-04-14 08:53:21 UTC

How to Deal with Outliers?

Removal: One common approach is to remove the outliers from the dataset. However, this approach should be used with caution as it can result in a loss of valuable information and can also lead to biased results.

*Transformation: Another approach is to transform the data by applying a mathematical function to it, such as a logarithmic or square root transformation. This can help to reduce the impact of outliers on the analysis.

*Winsorization: Winsorization is a method that involves replacing the extreme values with less extreme values. For example, the maximum or minimum values can be replaced with the next highest or lowest values in the dataset.

*Robust methods: Robust statistical methods, such as median and trimmed mean, are less sensitive to outliers and can be used as an alternative to the mean and standard deviation.

*Modeling: Another approach is to model the data using a more appropriate distribution that accounts for the presence of outliers, such as the t-distribution.

Mutating Join vs Filtering Join

edwinock2003 — 2023-04-14 08:54:43 UTC

Mutating Join vs Filtering Join

Mutating joins add new variables from one table (x) to matching observations in another table (y).
Filtering joins retain observations in one table (x) based on the presence or absence of matches in y.

Mutating join

edwinock2003 — 2023-04-14 08:54:54 UTC

Mutating join

inner_join()
left_join()
right_join()
full_join()

Filtering join

edwinock2003 — 2023-04-14 08:55:02 UTC

Filtering join

semi_join()
anti_join()

Mutating join

edwinock2003 — 2023-04-14 08:55:44 UTC

Data frames x (unit) and y (dept)

edwinock2003 — 2023-04-14 08:55:58 UTC

inner_join()

edwinock2003 — 2023-04-14 08:56:22 UTC

inner_join()

return all rows from x where there are matching values in y, and all columns from x and y.

left_join()

edwinock2003 — 2023-04-14 08:57:58 UTC

left_join()

return all rows from x, and all columns from x and y. Rows in y with no match in x will have NA values in the new columns

right_join()

edwinock2003 — 2023-04-14 08:58:26 UTC

right_join()

return all rows from y, and all columns from x and y. Rows in x with no match in y will have NA values in the new columns

full_join()

edwinock2003 — 2023-04-14 08:59:53 UTC

return all rows and all columns from both x and y. Where there are not matching values, returns NA for the one missing.

semi_join()

edwinock2003 — 2023-04-14 09:01:07 UTC

semi_join()

return all rows from x where there are matching values in y, keeping just columns from x.

anti_join()

edwinock2003 — 2023-04-14 09:01:39 UTC

anti_join()

return all rows from x where there are not matching values in y, keeping just columns from x