Skip to Content
Course content

50: readr for Fast Data Import

Click on the "Edit" button in the top corner of the screen to edit your slide content.

I remember a project a few years back where I had to analyze a year's worth of server logs—about 800MB of CSV data. I used the standard read.csv() function, went to grab a coffee, and came back to find my R session had completely frozen. When it finally finished, I realized R had guessed the data types incorrectly for several ID columns, turning long numeric strings into scientific notation. It was a frustrating way to spend a Tuesday, and it's exactly why you need to move beyond the base R import functions when dealing with anything larger than a tiny spreadsheet.

The readr package is the modern answer to this problem. It's part of the tidyverse and is designed specifically for speed and consistency. The most immediate difference you'll notice is the naming convention: we use read_csv() (with an underscore) instead of read.csv() (with a dot). That one character change unlocks a much more efficient parser that doesn't try to convert strings to factors by default and is significantly faster on large datasets.

Trading Base R for Tidy Import

When you use read_csv(), the first thing you'll notice is that it returns a tibble rather than a standard data frame. Tibbles are just "lazy" data frames—they don't change variable names or print 10,000 rows to your console and crash your IDE. They only show you a glimpse of the data, which is a lifesaver when you're working with hundreds of columns.

library(readr)

# Instead of the slow read.csv(), we do this:
server_data <- read_csv("server_logs_2023.csv")

One thing I've always appreciated about readr is the column specification message it prints to the console. It tells you exactly what it thinks each column is (e.g., col_double(), col_character()). In base R, you're often left guessing why a column is a factor when it should be a string until you hit an error mid-analysis.

Taking Control of Column Types

The "guessing" mechanism in readr is great, but it's not psychic. A classic trap I've fallen into is the "ZIP code problem." If a column contains ZIP codes, read_csv() will see numbers and assume it's a numeric column. This is a disaster because it will strip leading zeros from East Coast ZIP codes (e.g., 02108 becomes 2108).

To fix this, you can explicitly define your column types using the col_types argument. You can pass a string of shorthand codes or use the cols() function for more precision. I personally prefer cols() because it's more readable when you come back to the code three months later.

# Forcing the 'zip_code' column to be a character to preserve leading zeros
user_data <- read_csv("users.csv", col_types = cols(
  zip_code = col_character(),
  user_id = col_character(),
  signup_date = col_date(format = "%Y-%m-%d")
))

By being explicit, you remove the ambiguity. You're no longer relying on R to guess based on the first 1,000 rows; you're telling R exactly how the data is structured. This not only prevents bugs but actually speeds up the import process because the parser doesn't have to spend time guessing.




📋 Practical Task

Exercise: Fixing the Broken Product Catalog Import

You have been given a CSV file named product_catalog.csv. The file contains the following columns: product_id (which contains alphanumeric codes like "A100"), price (numeric), and sku_code (which are long numbers that should be treated as text to avoid scientific notation).

If you use read_csv("product_catalog.csv") without arguments, R incorrectly guesses that sku_code is a double, causing the IDs to be corrupted.

Your Task:

  • Load the readr library.
  • Import product_catalog.csv using read_csv().
  • Use the col_types argument and the cols() function to explicitly force product_id and sku_code to be characters, and price to be a double.
  • Print the first 5 rows of the resulting tibble to verify the types are correct.
Rating
0 0

There are no comments for now.

to be the first to leave a comment.