How To Automate Repetitive Tasks with Python

Beginner

Introduction

Every developer has experienced the monotony of repetitive tasks: renaming thousands of files, backing up project folders on schedule, generating weekly reports, or scanning for files that need processing. These are the moments when you wish a robot would just handle it while you focus on actual coding. The good news? Python makes this incredibly straightforward, and you already have everything you need in the standard library.

Python was designed with automation in mind. Libraries like os, shutil, pathlib, and smtplib give you powerful tools to interact with the file system, schedule tasks, and send notifications. You don’t need to learn complex shell scripts or invest in expensive automation software. A few lines of Python can save you hours of manual work.

In this guide, we’ll explore practical automation patterns starting with file operations and building toward a real-world automated backup system. By the end, you’ll have a toolkit for automating any repetitive task in your workflow.

Quick Example: Rename Files in Bulk

Before diving deep, let’s see automation in action. Imagine you have 500 image files named like IMG_0001.jpg, IMG_0002.jpg, and you want to prefix them with today’s date. Without automation, this takes hours. With Python, it takes seconds:

# bulk_rename.py
import os
import datetime

directory = "./photos"
prefix = datetime.date.today().strftime("%Y%m%d_")

for filename in os.listdir(directory):
    if filename.endswith(".jpg"):
        old_path = os.path.join(directory, filename)
        new_filename = prefix + filename
        new_path = os.path.join(directory, new_filename)
        os.rename(old_path, new_path)
        print(f"Renamed: {filename} -> {new_filename}")

Output:

Renamed: IMG_0001.jpg -> 20260329_IMG_0001.jpg
Renamed: IMG_0002.jpg -> 20260329_IMG_0002.jpg
Renamed: IMG_0003.jpg -> 20260329_IMG_0003.jpg

That script runs instantly and accomplishes what would take manual clicking for hours. This is the power of automation.

Why Automate with Python?

You might be wondering: why Python instead of shell scripts, scheduled tasks, or other tools? The answer is clarity, portability, and power. Here’s how they compare:

Task Aspect	Manual Process	Shell Script	Python Script
Development Time	Hours per occurrence	30-60 minutes	15-30 minutes
Readability	N/A	Cryptic syntax	Human-readable code
Cross-Platform	N/A	Linux/Mac only	Windows, Mac, Linux
Debugging	N/A	Difficult	Easy with proper logging
Email Integration	Manual setup	Complex	Built-in libraries
Maintainability	N/A	Hard to modify	Easy to extend and modify

Python wins for most automation tasks because it balances simplicity with power. You can read Python code six months later and understand what it does, and you can add new features without rewriting everything.

Working with Files and Directories

Using os and pathlib Modules

Python provides two ways to work with file paths and directories: the older os module and the modern pathlib module. pathlib is more intuitive and handles cross-platform differences automatically, but os is still widely used. Let’s explore both:

# file_operations.py
import os
from pathlib import Path

# Using os module
print("Using os module:")
current_dir = os.getcwd()
print(f"Current directory: {current_dir}")

# List files
for item in os.listdir("."):
    if os.path.isfile(item):
        print(f"File: {item}")

# Using pathlib (modern approach)
print("\nUsing pathlib:")
current_path = Path(".")

for item in current_path.iterdir():
    if item.is_file():
        print(f"File: {item.name}")
        print(f"Size: {item.stat().st_size} bytes")
        print(f"Extension: {item.suffix}")

Output:

Using os module:
Current directory: /home/user/projects

Using pathlib:
File: script.py
Size: 1245 bytes
Extension: .py
File: data.csv
Size: 5678 bytes
Extension: .csv

pathlib.Path is generally preferred because it’s more readable and handles path separators automatically (backslash on Windows, forward slash on Unix). However, both work fine depending on your preference and existing codebase.

Renaming and Organizing Files

One of the most common automation tasks is organizing files by type, date, or naming convention. The shutil module and os.rename() make this simple:

# organize_files.py
import os
import shutil
from pathlib import Path

download_dir = "./downloads"

# Create subdirectories if they don't exist
for category in ["Images", "Documents", "Archives", "Other"]:
    Path(download_dir, category).mkdir(exist_ok=True)

# Organize files by extension
for filename in os.listdir(download_dir):
    if filename.startswith("."):
        continue

    filepath = os.path.join(download_dir, filename)

    if not os.path.isfile(filepath):
        continue

    # Determine category based on extension
    ext = os.path.splitext(filename)[1].lower()

    if ext in [".jpg", ".png", ".gif", ".webp"]:
        category = "Images"
    elif ext in [".pdf", ".doc", ".docx", ".txt"]:
        category = "Documents"
    elif ext in [".zip", ".rar", ".7z"]:
        category = "Archives"
    else:
        category = "Other"

    # Move file to appropriate directory
    dest_path = os.path.join(download_dir, category, filename)
    shutil.move(filepath, dest_path)
    print(f"Moved {filename} to {category}/")

Output:

Moved vacation.jpg to Images/
Moved resume.pdf to Documents/
Moved backup.zip to Archives/
Moved config.txt to Documents/

This script is the foundation of smart file organization. In a real system, you’d add error handling, logging, and checks to avoid overwriting files. The Path.mkdir(exist_ok=True) pattern ensures directories exist without throwing errors if they do.

Watching for File Changes with watchdog

Sometimes you need to react the moment a file appears or changes. The watchdog library monitors file system events in real-time. First, install it:

pip install watchdog

Now create a file watcher that triggers actions when new files appear:

# watch_folder.py
import time
from pathlib import Path
from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler

class FileProcessor(FileSystemEventHandler):
    def on_created(self, event):
        if not event.is_directory:
            filename = Path(event.src_path).name
            print(f"New file detected: {filename}")
            print(f"Full path: {event.src_path}")

    def on_modified(self, event):
        if not event.is_directory:
            filename = Path(event.src_path).name
            print(f"File modified: {filename}")

# Watch the current directory
observer = Observer()
observer.schedule(FileProcessor(), path=".", recursive=False)
observer.start()

print("Watching for file changes. Press Ctrl+C to stop.")
try:
    while True:
        time.sleep(1)
except KeyboardInterrupt:
    observer.stop()
    observer.join()

Output (after creating/modifying files):

Watching for file changes. Press Ctrl+C to stop.
New file detected: report.pdf
Full path: ./report.pdf
File modified: report.pdf

The watchdog library is perfect for implementing “drop a file to process it” workflows, such as converting documents, generating thumbnails, or triggering CI/CD pipelines.

Scheduling Tasks with the schedule Library

Many automation tasks need to run at specific times or intervals: daily backups, hourly data syncs, or weekly reports. The schedule library makes this elegant:

pip install schedule

Here’s how to create a task scheduler:

# task_scheduler.py
import schedule
import time
from datetime import datetime

def backup_database():
    timestamp = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
    print(f"[{timestamp}] Running database backup...")
    # Actual backup logic here

def clean_temp_files():
    timestamp = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
    print(f"[{timestamp}] Cleaning temporary files...")
    # Actual cleanup logic here

def generate_report():
    timestamp = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
    print(f"[{timestamp}] Generating daily report...")
    # Actual report generation here

# Schedule tasks
schedule.every().day.at("02:00").do(backup_database)
schedule.every().hour.do(clean_temp_files)
schedule.every().monday.at("09:00").do(generate_report)

# Keep scheduler running
print("Scheduler started. Tasks will run according to schedule.")
while True:
    schedule.run_pending()
    time.sleep(60)  # Check every minute

Output (sample execution):

Scheduler started. Tasks will run according to schedule.
[2026-03-29 02:00:12] Running database backup...
[2026-03-29 03:00:05] Cleaning temporary files...
[2026-03-29 09:00:00] Generating daily report...

The schedule library is straightforward but doesn’t persist across system restarts. For production systems, consider using cron (Linux/Mac) or Task Scheduler (Windows) to run your Python script, or use a more robust library like APScheduler.

Sending Email Notifications with smtplib

Automating tasks is great, but you need to know when something fails or completes. Python’s built-in smtplib library sends email notifications:

# send_email.py
import smtplib
from email.mime.text import MIMEText
from email.mime.multipart import MIMEMultipart

def send_notification(recipient, subject, body):
    sender_email = "automation@example.com"
    sender_password = "your_app_password_here"

    # Create message
    message = MIMEMultipart()
    message["From"] = sender_email
    message["To"] = recipient
    message["Subject"] = subject
    message.attach(MIMEText(body, "plain"))

    # Send email
    try:
        with smtplib.SMTP_SSL("smtp.gmail.com", 465) as server:
            server.login(sender_email, sender_password)
            server.send_message(message)
            print(f"Email sent to {recipient}")
    except Exception as e:
        print(f"Error sending email: {e}")

# Usage
send_notification(
    "admin@example.com",
    "Backup Complete",
    "Daily backup completed successfully at 2026-03-29 02:15:30."
)

Output:

Email sent to admin@example.com

Important: Never hardcode passwords in scripts. Use environment variables or a configuration file outside version control. For Gmail, generate an “App Password” in your account settings rather than using your actual password.

Working with CSV and Excel Files for Reports

Automated reporting is a huge time-saver. Python handles CSV files natively and can create Excel files with the openpyxl library:

# generate_report.py
import csv
from datetime import datetime
from pathlib import Path

# Sample data (from database or API in real scenario)
sales_data = [
    {"date": "2026-03-29", "product": "Widget A", "sales": 150},
    {"date": "2026-03-29", "product": "Widget B", "sales": 200},
    {"date": "2026-03-29", "product": "Widget C", "sales": 175},
]

# Generate CSV report
report_date = datetime.now().strftime("%Y%m%d")
report_filename = f"sales_report_{report_date}.csv"

with open(report_filename, "w", newline="") as csvfile:
    fieldnames = ["date", "product", "sales"]
    writer = csv.DictWriter(csvfile, fieldnames=fieldnames)

    writer.writeheader()
    writer.writerows(sales_data)

print(f"Report generated: {report_filename}")

Output:

Report generated: sales_report_20260329.csv

File contents:

date,product,sales
2026-03-29,Widget A,150
2026-03-29,Widget B,200
2026-03-29,Widget C,175

For more complex reports with formatting, install openpyxl: pip install openpyxl. This lets you create Excel files with colors, formulas, and multiple sheets.

Running System Commands with subprocess

Sometimes you need to call external programs from Python. The subprocess module handles this safely:

# run_commands.py
import subprocess
import os

# Run a simple command
result = subprocess.run(["python", "--version"], capture_output=True, text=True)
print(f"Python version: {result.stdout.strip()}")

# Run a command and capture output
result = subprocess.run(["ls", "-la"], capture_output=True, text=True)
print("Directory listing:")
print(result.stdout)

# Check if command succeeded
result = subprocess.run(["git", "status"], capture_output=True)
if result.returncode == 0:
    print("Git repository is clean")
else:
    print("Not a git repository or git error")

Output (Linux/Mac):

Python version: Python 3.10.6
Directory listing:
total 48
drwxr-xr-x 5 user user 4096 Mar 29 10:15 .
drwxr-xr-x 8 user user 4096 Mar 29 09:00 ..
-rw-r--r-- 1 user user 1245 Mar 29 10:12 script.py
Git repository is clean

Use capture_output=True to collect program output and text=True to get strings instead of bytes. Always check the return code to verify success.

Real-Life Example: Automated Backup System

Now let’s build a complete, production-ready backup system that watches a directory and creates timestamped ZIP archives. This example combines everything we’ve learned:

# backup_system.py
import os
import shutil
import zipfile
import smtplib
import schedule
import time
from pathlib import Path
from datetime import datetime
from email.mime.text import MIMEText
from email.mime.multipart import MIMEMultipart

class BackupManager:
    def __init__(self, source_dir, backup_dir, email_to):
        self.source_dir = source_dir
        self.backup_dir = backup_dir
        self.email_to = email_to
        Path(backup_dir).mkdir(exist_ok=True)

    def create_backup(self):
        timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
        backup_filename = f"backup_{timestamp}.zip"
        backup_path = os.path.join(self.backup_dir, backup_filename)

        try:
            with zipfile.ZipFile(backup_path, "w", zipfile.ZIP_DEFLATED) as zipf:
                for root, dirs, files in os.walk(self.source_dir):
                    for file in files:
                        file_path = os.path.join(root, file)
                        arcname = os.path.relpath(file_path, self.source_dir)
                        zipf.write(file_path, arcname)

            file_size = os.path.getsize(backup_path) / (1024 * 1024)
            print(f"Backup created: {backup_filename} ({file_size:.2f} MB)")

            self.send_notification(
                f"Backup Success",
                f"Backup created successfully: {backup_filename}\nSize: {file_size:.2f} MB"
            )

            # Cleanup old backups (keep last 7)
            self.cleanup_old_backups()

        except Exception as e:
            print(f"Backup failed: {e}")
            self.send_notification("Backup Failed", f"Error: {str(e)}")

    def cleanup_old_backups(self):
        backups = sorted(Path(self.backup_dir).glob("backup_*.zip"))
        if len(backups) > 7:
            for old_backup in backups[:-7]:
                old_backup.unlink()
                print(f"Deleted old backup: {old_backup.name}")

    def send_notification(self, subject, body):
        sender_email = "backup@example.com"
        sender_password = "your_app_password"

        try:
            message = MIMEMultipart()
            message["From"] = sender_email
            message["To"] = self.email_to
            message["Subject"] = subject
            message.attach(MIMEText(body, "plain"))

            with smtplib.SMTP_SSL("smtp.gmail.com", 465) as server:
                server.login(sender_email, sender_password)
                server.send_message(message)
        except Exception as e:
            print(f"Could not send email: {e}")

# Setup and run
if __name__ == "__main__":
    manager = BackupManager(
        source_dir="./important_files",
        backup_dir="./backups",
        email_to="admin@example.com"
    )

    # Schedule daily backups at 2 AM
    schedule.every().day.at("02:00").do(manager.create_backup)

    print("Backup system started. Waiting for scheduled time...")
    while True:
        schedule.run_pending()
        time.sleep(60)

Output (sample):

Backup system started. Waiting for scheduled time...
Backup created: backup_20260329_020015.zip (45.32 MB)
Deleted old backup: backup_20260322_020012.zip

This system handles the full lifecycle: creating backups, managing disk space, and notifying you of success or failure. In production, you’d run this as a background service using systemd (Linux), launchd (Mac), or Task Scheduler (Windows).

Frequently Asked Questions

How do I run a Python script in the background?

Linux/Mac: Use nohup to ignore hangup signals: nohup python backup_system.py &. Or use screen or tmux for interactive backgrounds. Better: use cron to schedule it properly.

Windows: Use Task Scheduler to run the script with python.exe. Create a task that runs at startup or on a schedule without showing a window.

Should I add error handling to automation scripts?

Absolutely. Always wrap file operations in try-except blocks. Log errors to a file so you can debug later. For critical tasks, send notifications on failure. Here’s a pattern:

try:
    # Your automation code
    do_something()
except Exception as e:
    logger.error(f"Task failed: {e}")
    send_alert_email(f"Error: {e}")

Is it safe to put passwords in automation scripts?

No. Use environment variables, config files outside version control, or credential managers. For email, use app-specific passwords instead of your real password. Never commit secrets to GitHub.

import os
password = os.getenv("EMAIL_PASSWORD")  # Load from environment

How do I write automation that works on Windows, Mac, and Linux?

Use pathlib.Path instead of string path concatenation–it handles separators automatically. Use subprocess carefully since some commands differ. Test on all platforms or use Docker for consistency.

What if the user’s system doesn’t have the libraries I need?

Create a requirements.txt file listing dependencies, then users can install them with pip install -r requirements.txt. For standalone scripts, use PyInstaller to bundle Python and libraries into a single executable.

Conclusion

Python automation transforms tedious manual tasks into reliable, repeatable processes. You’ve learned to work with files and directories using os and pathlib, schedule tasks with the schedule library, send email notifications via smtplib, and build complete systems like automated backups. The key is starting simple–automate your most painful task first, then gradually expand your automation toolkit.

For deeper learning, explore the official documentation: os module, pathlib, shutil, and smtplib are all built-in. For external libraries, check schedule and watchdog on PyPI. The automation possibilities are endless once you see Python as your personal robot assistant.

Related Articles

Python File Handling: Reading, Writing, and Manipulating Files

Python Modules and Packages: Organizing Your Code

Python Error Handling: try, except, and finally