janitor功能介绍
janitor可以检查并清理脏数据，适用于R语言用户。主要功能如下：
1、完美格式化数据框的列名；
2、创建并格式化1-3个变量的频率表，可以看作是一个改进的table()函数；
3、提供用于清理和检查数据框的其他工具
制表和报告功能类似于SPSS和excel的常用功能。janitor是一个对标tidyverse的包。具体来讲，它与%>%这一pipeline配合的很好，并针对清理readr和readxl包中的数据进行了优化。
janitor的安装
方法一
install.packages("janitor")
方法二
install.packages("devtools")
devtools::install_github("sfirke/janitor")
janitor的使用
具体使用方法可以点击链接。以下是快速入门例子。
清理脏数据
例如下图中的数据
需要清理的部分主要有：
1、顶部标题；
2、列名；
3、包含excel格式但不包含数据的行和列；
4、单列中两种不同格式的日期（MM/DD/YYYY和数字）
5、“Certification”列中的值分布不一致
以下是读入R后的数据展示：
library(readxl); library(janitor); library(dplyr); library(here)
roster_raw <- read_excel(here("dirty_data.xlsx")) # available at https://github.com/sfirke/janitor
glimpse(roster_raw)
#> Rows: 14
#> Columns: 11
#> $ `Data most recently refreshed on:` <chr> "First Name", "Jason", "Jason", "Alicia", "Ada", "Desus", "Chien-…
#> $ ...2 <chr> "Last Name", "Bourne", "Bourne", "Keys", "Lovelace", "Nice", "Wu"…
#> $ ...3 <chr> "Employee Status", "Teacher", "Teacher", "Teacher", "Teacher", "A…
#> $ `Dec-27 2020` <chr> "Subject", "PE", "Drafting", "Music", NA, "Dean", "Physics", "Che…
#> $ ...5 <chr> "Hire Date", "39690", "43479", "37118", "38572", "42791", "11037"…
#> $ ...6 <chr> "% Allocated", "0.75", "0.25", "1", "1", "1", "0.5", "0.5", NA, "…
#> $ ...7 <chr> "Full time?", "Yes", "Yes", "Yes", "Yes", "Yes", "Yes", "Yes", NA…
#> $ ...8 <chr> "do not edit! --->", NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, …
#> $ ...9 <chr> "Certification", "Physical ed", "Physical ed", "Instr. music", "P…
#> $ ...10 <chr> "Certification", "Theater", "Theater", "Vocal music", "Computers"…
#> $ ...11 <chr> "Active?", "YES", "YES", "YES", "YES", "YES", "YES", "YES", NA, "…
现在，从列名开始清理它。名字清洗有两种方式。make_clean_names()对字符向量进行操作，可在数据导入期间使用:
roster_raw_cleaner <- read_excel(here("dirty_data.xlsx"),
skip = 1,
.name_repair = make_clean_names)
glimpse(roster_raw_cleaner)
#> Rows: 13
#> Columns: 11
#> $ first_name <chr> "Jason", "Jason", "Alicia", "Ada", "Desus", "Chien-Shiung", "Chien-Shiung", NA, "J…
#> $ last_name <chr> "Bourne", "Bourne", "Keys", "Lovelace", "Nice", "Wu", "Wu", NA, "Joyce", "Lamarr",…
#> $ employee_status <chr> "Teacher", "Teacher", "Teacher", "Teacher", "Administration", "Teacher", "Teacher"…
#> $ subject <chr> "PE", "Drafting", "Music", NA, "Dean", "Physics", "Chemistry", NA, "English", "Sci…
#> $ hire_date <dbl> 39690, 43479, 37118, 38572, 42791, 11037, 11037, NA, 36423, 27919, 42221, 34700, 4…
#> $ percent_allocated <dbl> 0.75, 0.25, 1.00, 1.00, 1.00, 0.50, 0.50, NA, 0.50, 0.50, NA, NA, 0.80
#> $ full_time <chr> "Yes", "Yes", "Yes", "Yes", "Yes", "Yes", "Yes", NA, "No", "No", "No", "No", "No"
#> $ do_not_edit <lgl> NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA
#> $ certification <chr> "Physical ed", "Physical ed", "Instr. music", "PENDING", "PENDING", "Science 6-12"…
#> $ certification_2 <chr> "Theater", "Theater", "Vocal music", "Computers", NA, "Physics", "Physics", NA, "E…
#> $ active <chr> "YES", "YES", "YES", "YES", "YES", "YES", "YES", NA, "YES", "YES", "YES", "YES", "…
clean_names()是make_clean_names()的便捷版本，可用于管道data.frame工作流。clean_names()的等效步骤如下:
roster_raw <- roster_raw %>%
row_to_names(row_number = 1) %>%
clean_names()
现在dataframe有了干净的列名。进一步整理：
roster <- roster_raw %>%
remove_empty(c("rows", "cols")) %>%
remove_constant(na.rm = TRUE, quiet = FALSE) %>% # remove the column of all "Yes" values
mutate(hire_date = convert_to_date(hire_date, # handle the mixed-format dates
character_fun = lubridate::mdy),
cert = dplyr::coalesce(certification, certification_2)) %>%
select(-certification, -certification_2) # drop unwanted columns
#> Removing 1 constant columns of 10 columns total (Removed: active).
roster
#> # A tibble: 12 × 8
#> first_name last_name employee_status subject hire_date percent_allocated full_time cert
#> <chr> <chr> <chr> <chr> <date> <chr> <chr> <chr>
#> 1 Jason Bourne Teacher PE 2008-08-30 0.75 Yes Physical ed
#> 2 Jason Bourne Teacher Drafting 2019-01-14 0.25 Yes Physical ed
#> 3 Alicia Keys Teacher Music 2001-08-15 1 Yes Instr. music
#> 4 Ada Lovelace Teacher <NA> 2005-08-08 1 Yes PENDING
#> 5 Desus Nice Administration Dean 2017-02-25 1 Yes PENDING
#> 6 Chien-Shiung Wu Teacher Physics 1930-03-20 0.5 Yes Science 6-12
#> 7 Chien-Shiung Wu Teacher Chemistry 1930-03-20 0.5 Yes Science 6-12
#> 8 James Joyce Teacher English 1999-09-20 0.5 No English 6-12
#> 9 Hedy Lamarr Teacher Science 1976-06-08 0.5 No PENDING
#> 10 Carlos Boozer Coach Basketball 2015-08-05 <NA> No Physical ed
#> 11 Young Boozer Coach <NA> 1995-01-01 <NA> No Political sci.
#> 12 Micheal Larsen Teacher English 2009-09-15 0.8 No Vocal music
检查脏数据
寻找重复项
在数据清理期间，使用get_dupes()来识别和检查重复记录。让我们看看是否有教师被多次列出:
roster %>% get_dupes(contains("name"))
#> # A tibble: 4 × 9
#> first_name last_name dupe_count employee_status subject hire_date percent_allocated full_time cert
#> <chr> <chr> <int> <chr> <chr> <date> <chr> <chr> <chr>
#> 1 Chien-Shiung Wu 2 Teacher Physics 1930-03-20 0.5 Yes Science …
#> 2 Chien-Shiung Wu 2 Teacher Chemistry 1930-03-20 0.5 Yes Science …
#> 3 Jason Bourne 2 Teacher PE 2008-08-30 0.75 Yes Physical…
#> 4 Jason Bourne 2 Teacher Drafting 2019-01-14 0.25 Yes Physical…
是的，有些老师会出现两次。我们应该在计算员工人数之前解决这个问题。
制表工具
一个变量(或两个或三个变量的组合)可以用tabyl()制成表格。生成的data.frame可以用一套adorn_函数进行调整和格式化，以便在报告中快速分析和打印漂亮的结果。对于非表类型，adorn_函数也很有帮助。
tabyl()
与table()类似，但是支持管道，基于数据帧，并且功能齐全。
tabyl有两种用法:
1、在向量上，当对单个变量制表时:tabyl(roster$subject)
2、在data.frame上，指定1、2或3个要制表的变量名:roster %>% tabyl(subject，employee_status)。
这里，data.frame通过%>%管道传入；这允许在分析管道中使用tabyl
一个变量:
roster %>%
tabyl(subject)
#> subject n percent valid_percent
#> Basketball 1 0.08333333 0.1
#> Chemistry 1 0.08333333 0.1
#> Dean 1 0.08333333 0.1
#> Drafting 1 0.08333333 0.1
#> English 2 0.16666667 0.2
#> Music 1 0.08333333 0.1
#> PE 1 0.08333333 0.1
#> Physics 1 0.08333333 0.1
#> Science 1 0.08333333 0.1
#> <NA> 2 0.16666667 NA
两个变量：
roster %>%
filter(hire_date > as.Date("1950-01-01")) %>%
tabyl(employee_status, full_time)
#> employee_status No Yes
#> Administration 0 1
#> Coach 2 0
#> Teacher 3 4
三个变量：
roster %>%
tabyl(full_time, subject, employee_status, show_missing_levels = FALSE)
#> $Administration
#> full_time Dean
#> Yes 1
#>
#> $Coach
#> full_time Basketball NA_
#> No 1 1
#>
#> $Teacher
#> full_time Chemistry Drafting English Music PE Physics Science NA_
#> No 0 0 2 0 0 0 1 0
#> Yes 1 1 0 1 1 1 0 1
装饰tabyls
adorn_函数修饰这些制表调用的结果，以实现快速、基本的报告。以下是一些增强报告汇总表的功能:
roster %>%
tabyl(employee_status, full_time) %>%
adorn_totals("row") %>%
adorn_percentages("row") %>%
adorn_pct_formatting() %>%
adorn_ns() %>%
adorn_title("combined")
#> employee_status/full_time No Yes
#> Administration 0.0% (0) 100.0% (1)
#> Coach 100.0% (2) 0.0% (0)
#> Teacher 33.3% (3) 66.7% (6)
#> Total 41.7% (5) 58.3% (7)
在您的RMarkdown报告中直接将它输入到knitter::kable()中。
这些模块化装饰可以分层，以减少R在快速、信息丰富的计数方面相对于Excel和SPSS的不足。从tabyls简介中了解更多关于tabyl()和adorn _ 函数的信息。
