Skip to Content
Course content

244: The shutil Module for File Operations

Click on the "Edit" button in the top corner of the screen to edit your slide content.

I've seen this mistake happen a dozen times when people start building automation scripts. You have a folder full of logs or user uploads, and you want to back it up to another location. Your first instinct is to reach for shutil.copy() because the name seems obvious. It looks like this:

import shutil
import os

source_dir = 'logs_2023'
backup_dir = 'backup_logs_2023'

# I just want to copy the folder over
shutil.copy(source_dir, backup_dir)

When you run this, Python is going to throw an IsADirectoryError (or a PermissionError on some systems). It's frustrating because you're telling Python to copy a directory, and it's telling you that it can't. The problem is that shutil.copy() is designed strictly for files. It doesn't know how to recurse into a folder, grab all the files inside, create the corresponding directory structure at the destination, and move them over.

Switching to recursive directory copying

To fix this, you need shutil.copytree(). Unlike the basic copy function, copytree is designed specifically for directories. It walks through the entire source tree and replicates it exactly at the destination.

import shutil

source_dir = 'logs_2023'
backup_dir = 'backup_logs_2023'

# This handles the directory and everything inside it
shutil.copytree(source_dir, backup_dir)

One thing to keep in mind: copytree expects the destination directory to not exist yet. If backup_logs_2023 already exists, Python will raise a FileExistsError. If you're on Python 3.8+, you can get around this by adding the argument dirs_exist_ok=True.

Preserving metadata with copy2

Now, if you are copying individual files, you'll notice shutil.copy() copies the file and the permissions, but it doesn't preserve the original creation and modification timestamps. In a production environment—especially when dealing with logs or legal documents—those timestamps are critical.

I always recommend using shutil.copy2() instead. It does everything copy() does, but it also attempts to preserve all the file metadata. It's a small change in the function name, but it saves you from a massive headache during a forensic audit later on.

Moving files across different drives

You might be tempted to use os.rename() to move files. That works fine as long as you're moving a file within the same partition. But the moment you try to move a file from a local SSD to a network drive or a USB stick, os.rename() will fail with an OSError because it can't perform a rename across different file systems.

shutil.move() is the professional's choice here. It first tries to rename the file, but if that fails because the destination is on a different disk, it transparently copies the file over and then deletes the original. You don't have to write any logic to handle the "cross-device" edge case; shutil handles it for you.

Wiping directories and creating archives

Cleaning up is just as important as copying. If you need to delete a directory and everything inside it, os.rmdir() won't work unless the folder is already empty. To nuking a directory regardless of its contents, use shutil.rmtree(). Be careful with this one—there is no "Recycle Bin" here. Once it's gone, it's gone.

Finally, if you need to bundle a directory into a single file for transport, shutil.make_archive() is your best friend. It wraps the complexity of the zipfile or tarfile modules into a single line of code.

import shutil

# Creates 'project_backup.zip' from the 'my_project' folder
shutil.make_archive('project_backup', 'zip', 'my_project')



📋 Practical Task

Build a Project Snapshot and Cleanup Utility

Create a Python script that performs the following sequence of operations to simulate a build-and-archive workflow:

  • Create a directory named build_output and place two dummy text files inside it.
  • Use shutil.copytree() to create a backup of build_output named build_backup.
  • Use shutil.make_archive() to compress the build_backup folder into a zip file named final_release.
  • Use shutil.rmtree() to delete both the build_output and build_backup directories, leaving only the final_release.zip file behind.

Ensure your script handles the case where the backup directory might already exist from a previous run.

Rating
0 0

There are no comments for now.

to be the first to leave a comment.