A few years ago, I was helping a junior dev who was building a custom logging engine for a high-frequency trading application. He was using fprintf and fwrite from stdio.h, and he was pulling his hair out because the logs weren't appearing in the file in real-time. He’d crash the program, open the log file, and find the last 4KB of critical debugging data simply... missing. He thought he had a disk corruption issue. In reality, he was fighting the C standard library's user-space buffering. The data was sitting in a buffer in his own process, waiting for the library to decide it was "full enough" to actually tell the OS to save it.

That's the moment I told him to stop using stdio.h and start using system calls. When you use fopen or fprintf, you're using a wrapper. When you use open and write, you're talking directly to the kernel. There is no middleman, no hidden buffer, and no guesswork.

The Bridge Between Your Code and the Kernel

In C, most of the "file" functions you've used so far operate on a FILE * object. That's a high-level abstraction. System calls, however, operate on file descriptors. A file descriptor is nothing more than a non-negative integer that the kernel uses to track which "open file" your process is referring to. If open() returns 3, then 3 is your handle to that resource for the rest of the session.

To use these, you'll need and . The open() call is your entry point. Unlike fopen, which takes a mode string like "r" or "w", open uses bitwise flags. You'll commonly see O_RDONLY (read-only), O_WRONLY (write-only), or O_RDWR (both). If you're creating a file, you add O_CREAT and must provide a third argument for the file permissions (like 0644), otherwise, the kernel will assign random, potentially unusable permissions to your new file.

int fd = open("log.txt", O_WRONLY | O_CREAT | O_TRUNC, 0644);
if (fd == -1) {
    perror("Error opening file");
    return 1;
}

Managing File Descriptors and Byte Streams

Once you have that integer fd, you stop thinking about "lines" or "formatted text" and start thinking about bytes. The kernel doesn't know what a "string" is; it only knows buffers and lengths.

The read() and write() calls are the workhorses here. They both follow the same pattern: they take the file descriptor, a pointer to a memory buffer, and the number of bytes you want to move. I always remind people that write() doesn't append a null terminator, and read() doesn't add one for you. You're handling raw memory. If you read 10 bytes into a buffer, you're responsible for making sure that buffer is large enough and, if you plan to print it as a string, that you manually add '\0' at the end.

char buffer[1024];
ssize_t bytesRead = read(fd, buffer, sizeof(buffer));
// bytesRead tells you exactly how much the kernel actually gave you.

Dealing with Partial Reads and Writes

Here is where most developers trip up: read() and write() are not guaranteed to process the entire amount of data you requested in a single call. This is especially true when dealing with sockets or pipes, but it can happen with files too. If you ask to write 10MB but the kernel's internal buffer fills up or a signal interrupts the call, it might only write 4KB and return 4096.

I've seen countless bugs where developers assume write(fd, buf, 100) always writes 100 bytes. If it returns 50, and you just move on, you've just corrupted your data stream. You have to wrap these calls in a loop, tracking how many bytes have been processed and offsetting your buffer pointer until the job is done. It's tedious, but it's the only way to write robust system-level code.

Finally, you must close(fd). While the OS will generally clean up your descriptors when the process exits, failing to close files in a long-running server will lead to "Too many open files" errors, which is a nightmare to debug in production.

📋 Practical Task

Building a Raw Byte-Stream File Copier

Your task is to create a utility called raw_copy that copies the contents of one file to another using only system calls. To make this a real-world challenge, you must ensure the program handles "partial" reads and writes correctly.

• The program should take two command-line arguments: the source filename and the destination filename.
• You must use open(), read(), write(), and close(). Do not use any functions from stdio.h (no fopen, fread, etc.) for the copying process.
• Use a small buffer (e.g., 512 bytes) to demonstrate that the program can handle files larger than the buffer size.
• Implement a while loop for the write() call to ensure that if the kernel performs a partial write, the remaining bytes are still sent.
• Ensure the destination file is created with 0644 permissions.