A few years ago, I was helping a colleague optimize a set of API endpoints. He came to me beaming, showing me a dashboard where the "Average Response Time" was a crisp 200ms. On paper, it looked perfect. But the customer support tickets were screaming about the site being "unusable" for a subset of users. When I actually pulled the raw data and looked at the distribution, I found that while most requests were 50ms, a handful of requests were taking 15 seconds. The mean was lying to him. This is why descriptive statistics aren't just academic exercises; they are the only way to tell if your "average" is actually a useful number or a dangerous illusion.

Quick Wins with the Summary Function

When I first open a dataset, I don't start by calculating individual metrics. That's too slow. Instead, I use summary(). It's the Swiss Army knife of descriptive stats in R. It gives you the minimum, the first quartile, the median, the mean, the third quartile, and the maximum all in one go.

# Let's simulate some API response times in milliseconds
response_times <- c(45, 52, 48, 60, 55, 3000, 42, 50, 58, 12000)
summary(response_times)

If you run that, you'll immediately see the gap between the mean and the median. In this case, the mean will be heavily pulled upward by those two massive spikes (the 3000 and 12000 values), while the median remains rooted in the 50s. If the mean is significantly higher than the median, you've got a right-skewed distribution—basically, a few "heavy hitters" are distorting your view.

Measuring the Spread and the Shake

Knowing where the center is is only half the battle. I care more about the variance—how much the data points are scattered. In R, we use var() for variance and sd() for standard deviation. I almost always prefer standard deviation because it's expressed in the same units as your data. If my response times are in milliseconds, the sd() is also in milliseconds, whereas variance is in "milliseconds squared," which is practically meaningless to a human brain.

# Calculating the spread
std_dev <- sd(response_times)
variance_val <- var(response_times)

print(paste("Standard Deviation:", std_dev))

A high standard deviation tells you that your system is inconsistent. In software engineering, consistency is often more important than raw speed. A user would rather have a steady 200ms response every time than a system that fluctuates between 10ms and 2 seconds.

Dealing with the NA Headache

Here is a quirk of R that has tripped up almost every developer I've mentored: by default, if your data contains a single NA (Not Available), R will return NA for your entire calculation. It's R's way of saying, "I can't give you an accurate average if some data is missing."

In the real world, data is always messy. You'll have dropped packets or null database entries. To get around this, you have to explicitly tell R to ignore the missing values using the na.rm = TRUE argument. I've spent way too many hours debugging "missing" results only to realize I forgot this one argument.

# Data with a missing value
messy_times <- c(45, 52, NA, 60, 55)

# This will return NA
bad_mean <- mean(messy_times) 

# This is how you actually do it
good_mean <- mean(messy_times, na.rm = TRUE)

Beyond the basics, you can use quantile() if you need a specific cutoff, like the 95th or 99th percentile. In the SRE (Site Reliability Engineering) world, we almost never care about the average; we care about the P99—the response time that 99% of users stay under. That's where the real pain points live.

📋 Practical Task

Analyzing Production Latency Spikes

You have been handed a vector of latency measurements (in milliseconds) from a problematic production server. The dataset contains some missing values due to logging failures and some extreme outliers.

server_latency <- c(112, 125, 118, 130, 115, NA, 122, 140, 110, 8500, 121, NA, 119, 127, 15000)

Write a script to perform the following analysis:

• Calculate the mean and median latency, ensuring that missing values are ignored.
• Calculate the standard deviation of the latency.
• Find the 95th percentile (the P95) of the latency.
• Print a short message stating whether the mean is higher than the median, and if so, what that implies about the outliers in the data.