purrr_tutorial_slides.Rmd

---
title: "purrr Tutorial"
author: "Caleb Scheidel"
date: "6/30/2017"
output: 
  ioslides_presentation:
    widescreen: true
    smaller: true
    logo: methods_logo.png
    css: purrr_tutorial.css
---

```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE)

library(purrr)
library(devtools)
library(repurrrsive)
library(listviewer)
library(knitr)
library(tibble)
library(tidyverse)

```

## Tutorial Outline

1. Introduce advantages of `purrr` package

2. Use GoT dataset to run through examples of commonly used `purrr` functions: 
    - `map()`, 
    - `map_at()`
    - `map_if()`
    - `lmap()`
    - `walk()`
    - `reduce()`
    - `keep()`
    - `discard()`
    - `map2()`
    - `pmap()` 

3. Walk through a real example where I learned the hard way that `purrr` makes our work easier and more readable to others  

# Advantages of `purrr` package

## Advantages of `purrr`

- efficient, can do more in less code
- `map()` has several advantages over `lapply()`
    - allows you to bypass the `function` function using `~` in place of `function` and a `.` in place of x
    - several versions of `map()` that allow you to specify the structure of your output
    - can easily nest `map()` functions inside of each other.
    
- Disadvantages: 
    - newer package, so conflicts arise when it is updated
        - i.e. `purrr`/`purrrlyr` conflicts
            - motivation to use `packrat`!
    - had some issues working with dates


#  Use GoT dataset to run through examples

## GoT Characters Dataset

- `got_chars` dataset from the `repurrrsive` package

- contains data on all of the point-of-view characters from the Game of Thrones book series in a list of lists format

- The highest-level list contains 29 characters, with each character having a list of 18 variables containing information pertaining to each character, i.e. `name`, `gender`, and `culture`.

--- 

```{r, include = TRUE}

#Only viewing elements of the first character's list
str(got_chars[1])

```

## Introduction to `map()`: extract elements {.smaller}

- `map()` is a function for applying a function to each element of a list.

- Basic map usage: `map(YOUR_LIST, YOUR_FUNCTION)`

- First, a quick example, taking the square root of a list of three numbers:

```{r}

map(c(9, 16, 26), sqrt)

```

## Using `map()` to extract character names

Using the GoT characters dataset, pulling out the names from specified characters:

```{r}

#name and position shortcuts
map(got_chars[1:2], "name")

#can rewrite using the pipe
got_chars[1:2] %>% 
  map("name")

```

## Type-specific maps

You can specify type-specific variants to map, such as `map_chr()`, `map_lgl()`, `map_int()`, and `map_dbl()`:

```{r}

map_chr(got_chars[9:12], "name")

map_int(got_chars[9:12], "id")

map_lgl(got_chars[9:12], "alive")

```

## Extract multiple values without using `map()`

- How to retrieve multiple elements without using `map()`:

```{r}

got_chars[[3]][c("name", "culture", "born")]

```

## Extract multiple values using `map()`

- Recall map() usage: `map(.x, .f, ...)`

- The function .f will be `[`.  We use `...` to pass the character vector of the names of desired elements 

```{r}

x <- map(got_chars, `[`, c("name", "culture", "gender", "born"))
str(x[16:17])

```

## Extract multiple values using `map()`

- This can be rewritten using the pipe and `extract()` function from `magrittr`:

```{r warning=FALSE, message=FALSE}

library(magrittr)

x <- got_chars %>% 
  map(extract,  c("name", "culture", "gender", "born"))
  
str(x[18:19])

```

## `map_df()` to put output into a dataframe

- To get output into a dataframe, use `map_df()`:

```{r}

got_char <- map_df(got_chars, extract, c("name", "culture", "gender", "id", "born", "alive"))

got_char %>% tail() %>% kable()

```

## Specifying the function in map()

- map() function specification

- Recall map() usage: `map(.x, .f, ...)`

- Three more ways to specify a general function, `.f`:

    1. an existing function
    2. an anonymous function, defined on the fly
    3. a formula

## `map()` function specification

- Each GoT character can have aliases, which are stored as a list for each character.

- We want to present each character's aliases combined together in one element of a list, separated by " | ".  

- There are three ways to go about doing this using `map()`:

    1. an existing function
    2. an anonymous function, defined on the fly
    3. a formula
    
---

First, pull a few to use in this demo:

```{r}

aliases <- set_names(map(got_chars, "aliases"),
                     map_chr(got_chars, "name"))

(aliases <- aliases[c("Daenerys Targaryen", "Jon Snow")])

```

## 1. Existing function

- We can define our own function and use it in `map()`:

```{r}

my_fun <- function(x) paste(x, collapse = " | ")
map(aliases, my_fun)

```

## 2. Anonymous function, conventional

- We can also define the function on the fly in the `map()` function itself:

```{r}

map(aliases, function(x) paste(x, collapse = " | "))

#or rewrite it as
map(aliases, paste, collapse = " | ")

```

## 3. Anonymous function, formula

- purrr provides a concise way to define an anonymous function: as a formula.  
- This should start with the `~` symbol and then look like a typical expression.  
    - Use `.x` to refer to the input, i.e. an individual element of the primary vector or list.

```{r}

map(aliases, ~ paste(.x, collapse = " | "))

```

## Converting `aliases` list to a data frame

- Since we've simplified the aliases to a single string for each character, we can hold them as an atomic character vector instead of as a list.  
    - The `tibble::enframe()` function takes a named vector and promotes the names to a proper variable.

```{r}

aliases <- set_names(map(got_chars, "aliases"), map_chr(got_chars, "name"))

aliases_df <- map_chr(aliases[c(9, 23)], ~ paste(.x, collapse = " | ")) %>% 
  tibble::enframe(value = "aliases")

aliases_df %>% kable()

```

## `map_at()`

- `map_at()` maps a function over the elements corresponding to a character vector of names or a numeric vector of positions
- For example, in the `got_char` dataframe we created earlier, we want to change both `culture` and `gender` variables from `chr` to `fctr`:

```{r}

got_char %>%
  map_at(c("culture", "gender"), as.factor) %>%
  str()

```

## `map_if()`

- `map_if()` maps a function over the elements of .x satisfying a predicate.
- For example, we want to change any `int` type variables to `chr` (in this dataset, that is only `id`)

```{r}

got_char %>%
  map_if(is_integer, as.character) %>%
  str()

```

## `lmap()`, `lmap_at()`, and `lmap_if()`

- Operate exclusively on functions that take and return a list (or data frame)
- First, we need a function that takes and returns a list, for example `disjoin`, which takes all factor variables and creates new columns for each factor level:

```{r}

disjoin <- function(x, sep = "_") {
  name <- names(x)
  x <- as.factor(x[[1]])

  out <- lapply(levels(x), function(level) {
    as.numeric(x == level)
  })

  names(out) <- paste(name, levels(x), sep = sep)
  dplyr::as_data_frame(out)
}

```

- We can then create binary variables for each culture and gender in the `got_char` data -------------->

## `lmap()`, `lmap_at()`, and `lmap_if()`

```{r}

got_char %>%
  select(name, culture, gender) %>% 
  filter(culture == "Ironborn" | culture == "Free Folk") %>% 
  map_at(c("culture", "gender"), as.factor) %>% 
  lmap_if(is.factor, disjoin) %>% 
  as.data.frame() %>% tail() %>% kable()

```


## Parallel mapping using `map2()`

- You can use `map2()` to map a function over two vectors or lists in parallel.  Here is the usage:

- `map2(.x, .y, .f, ...)` 

or

- `map2(INPUT_ONE, INPUT_TWO, FUNCTION_TO_APPLY, OPTIONAL_OTHER_STUFF)`

- It can also be type specific: `map2_chr()`, `map2_dbl()`, etc.

- For our example, we will paste the character's name and the date and location of his or her birth to get a sentence.

- First, obtain the two inputs:

```{r}

nms <- got_chars %>% 
  map_chr("name")

birth <- got_chars %>% 
  map_chr("born")

```

## Parallel mapping using `map2()` | 1. Map over both with an existing function, defined by us:

```{r}

my_fun <- function(x, y) paste(x, "was born", y)
map2_chr(nms, birth, my_fun) %>% 
  head()

```

## Parallel mapping using `map2()` | 2. Or with an anonymous function, conventional form:

```{r}

map2_chr(nms, birth, function(x, y) paste(x, "was born", y)) %>% head()

```

## Parallel mapping using `map2()` | 3. Or with an anonymous function via formula:

- Use `.x` and `.y` to refer to individual elements of the two primary inputs:

```{r}

map2_chr(nms[16:18], birth[16:18], ~ paste(.x, "was born", .y))

```

## Parallel mapping using `pmap()`

Use `pmap()` to map a function over two or more vectors or lists in parallel.

Usage:

- `pmap(.l, .f, ...)`

or

- `pmap(LIST_OF_INPUT_LISTS, FUNCTION_TO_APPLY, OPTIONAL_OTHER_STUFF)`

## Parallel mapping using `pmap()`

Function creates sentence for each character stating name, number of aliases and allegiances:

```{r}

df <- got_chars %>% {
  tibble::tibble(
    name = map_chr(., "name"),
    aliases = map(., "aliases"),
    allegiances = map(., "allegiances")
  )
}

my_fun <- function(name, aliases, allegiances) {
  paste(name, "has", length(aliases), "aliases and", length(allegiances), "allegiances")
}

df %>% 
  pmap_chr(my_fun) %>% 
  tail(n = 5)

```

## `walk()`

- `walk()` calls `.f` exclusively for its side-effect (rather than its return value) and returns the original input
    - typically do this because you want to render output to the screen or save files to disk
        - important thing is the action, not the return value
- `walk2()` and `pwalk()` are generally more useful
    - all invisibly return the `.x`, the first argument.  This makes them suitable for use in the middle of pipelines.
      
```{r}

plots <- mtcars %>% 
  split(.$cyl) %>% 
  map(~ggplot(., aes(mpg, wt)) + geom_point())
paths <- paste0(names(plots), ".pdf")

pwalk(list(paths, plots), ggsave, path = tempdir())

```

## `reduce()`

- `reduce()` reduces a list to a single value by iteratively applying a binary function (i.e. function with two primary inputs)
- `reduce()` combines from the left, `reduce_right()` combines from the right
- useful in combination with joins, to bind a list of dataframes together by a common variable/key
- Might have a list of dataframes, and you want to _reduce_ to a single data frame by joining the elements together:

```{r}

dfs <- list(
  age = tibble(name = "John", age = 30),
  sex = tibble(name = c("John", "Mary"), sex = c("M", "F")),
  trt = tibble(name = "Mary", treatment = "A")
)

dfs %>% reduce(full_join, by = "name")

```

## `keep()`

- similar to `filter()`, keeps an element using a predicate function (function that returns either a single `TRUE` or `FALSE`)

```{r}

got_chars %>% 
  keep("alive") %>% 
  map_df(extract, c("name", "culture", "gender", "alive")) %>% 
  head(n = 5) %>% 
  kable()

```

## `discard()`

- opposite of `keep()`, discards an element using a predicate function

```{r}

got_chars %>% 
  discard("alive") %>% 
  map_df(extract, c("name", "culture", "gender", "alive")) %>% 
  head(n = 5) %>% 
  kable()

```

## Using `purrr` to avoid `dplyr::rowwise()`

- For common tasks where you need to have operations applied to each row of a dataframe, you may think of using `dplyr::rowwise()` to accomplish this:

```{r}

iris %>% 
  rowwise() %>% 
  mutate(Max.Length = max(Sepal.Length, Petal.Length)) %>% 
  ungroup() %>% 
  head(n=3) %>% 
  kable()

```

## Using `purrr` to avoid `dplyr::rowwise()`

- But we can use `purrr:map2()` to accomplish this in a quicker and more readable fashion:

```{r}

iris %>% 
  mutate(
    Max.Length = map2(Sepal.Length, Petal.Length, max)
  ) %>% 
  head(n=3) %>% 
  kable()

```

# `purrr` success story

## Scenario

- Reading in dozens of excel tables of different sizes and formats from different files to combine into one dataframe for Optimization Group.

## Long, tedious non-`purrr` way to do this

- I had over 1000 lines of code to do this at first with a different `read_excel()` call for each table 
- :(
- https://gitlab.com/methodsconsultants/optimization-group/marketing-impact/commit/c70aef043c4989b12b88fbf19f1fea6b9a9edd60

## Quick, easy way to do this involving `purrr`

- With Clayton's guidance, reduced to under 50 lines of code by creating a file with details about each specific table's sheet, index, cell range, and other information, writing a function to read all files in, and using a `pmap_df()` call.
- :)
- https://gitlab.com/methodsconsultants/optimization-group/marketing-impact/tree/co_data_prep/data_prep/CA