How much prose do I actually need to write?

I've seen a lot of analysts swing between two extremes: they either write zero comments and expect the code to "speak for itself," or they write a novel that buries the actual analysis. Here is the truth: code tells me how you did something, but it rarely tells me why you did it.

If you're using R Markdown or Quarto, your goal isn't to transcribe the code into English. Instead, focus on the transitions. Don't say "Now I am filtering for values over 50" when the code clearly says filter(value > 50). That's noise. Instead, explain the business logic: "We're filtering for values over 50 because that is the WHO threshold for 'Unhealthy' air quality."

Think of it as a narrative. If I can read your text and understand the story of the data without staring at every single line of R code, you've hit the sweet spot.

How do I document the "ugly" data cleaning without ruining the report flow?

We've all been there. You spend three hours wrestling with a CSV that has weird encoding and missing headers, and you end up with a 50-line block of gsub() and mutate() calls. If you put that in the middle of your analysis, you'll lose your reader immediately.

My advice? Use a separate "preprocessing" script for the heavy lifting, or use the echo = FALSE or ref.label options in your chunks to hide the plumbing. If you keep it in the main document, wrap the cleaning in a clearly named function. It turns a wall of noise into a single, readable line.

# Instead of 20 lines of regex in the main flow, do this:
clean_sensor_data <- function(df) {
  df %>%
    mutate(timestamp = as.POSIXct(timestamp, format = "%Y-%m-%d %H:%M:%S")) %>%
    filter(!is.na(pm25)) %>%
    # ... all the other messy cleaning here ...
    mutate(city = trimws(city))
}

# In the report, it looks clean:
processed_data <- raw_data %>% clean_sensor_data()

Where should I actually put my assumptions?

This is where most people fail. They document the result, but not the assumptions that led to it. When you make a decision—like deciding to treat "N/A" as 0 or dropping outliers beyond 3 standard deviations—that needs to be a first-class citizen in your documentation.

I personally like to use a "Methodology" or "Assumptions" section at the top of the document. If the assumption happens mid-analysis, put it in a callout box or a bolded paragraph immediately preceding the code chunk. If you don't, you'll be the one answering an email six months from now trying to remember why you deleted 15% of your dataset.

How do I stop my documentation from becoming outdated as the code changes?

The moment you write your documentation in a separate Word doc or a Wiki page, it starts dying. The only way to keep documentation "alive" is to keep it physically attached to the code. This is why literate programming (like Quarto) is the gold standard for R analysis.

When the code and the explanation live in the same .qmd or .Rmd file, the friction of updating the text is much lower. Also, try to use inline R code to report your findings. Instead of writing "The average PM2.5 was 12.4," use an inline expression. That way, if the data changes, the documentation updates itself automatically when you render the report.

# Instead of: "The mean air quality index was 45."
# Use: "The mean air quality index was `r round(mean(data$aqi), 1)`."

📋 Practical Task

Documenting the Urban Heat Island Analysis

You have been handed a messy R script that analyzes temperature differences between city centers and suburbs. The code works, but it's a "black box"—there are no explanations for why certain filters were applied, and the results are hard-coded into the comments.

Your Task: Transform the provided raw script into a professional analysis document (R Markdown or Quarto style). You must:

• Convert the hard-coded result comments into inline R expressions so the report updates automatically.
• Identify the "cleaning" section and wrap it into a named function to improve readability.
• Add a "Methodology/Assumptions" section explaining why temperatures above 45°C were treated as sensor errors (outliers) and removed.
• Replace "What the code is doing" descriptions (e.g., "Filtering the data") with "Why it is being done" descriptions (e.g., "Removing sensor errors to prevent skewing the mean").

Submit your final .Rmd or .qmd file showing the narrative flow between the code and the analysis.