How To Schedule Python Scripts To Run Automatically With cron and schedule

Intermediate

How To Schedule Python Scripts To Run Automatically With cron and schedule

If you’re building Python applications, sooner or later you’ll want them to run on a schedule without you having to sit at your computer and trigger them manually. Whether it’s a daily backup, sending reports, checking for updates, or cleaning up temporary files, automation is a game-changer. In this tutorial, we’ll explore how to schedule Python scripts to run automatically using both Linux cron and the Python schedule library, plus we’ll touch on Windows Task Scheduler for our Windows users.

Quick Example (TLDR)

Here’s the fastest way to schedule a Python script using the schedule library:

# Install: pip install schedule
import schedule
import time

def my_job():
    # This code runs on schedule
    print("Job executed at", time.strftime("%Y-%m-%d %H:%M:%S"))

# Schedule the job to run every day at 10:00 AM
schedule.every().day.at("10:00").do(my_job)

# Keep the scheduler running
while True:
    schedule.run_pending()
    time.sleep(60)  # Check every minute if a job should run

Output:

Job executed at 2026-03-12 10:00:15
Job executed at 2026-03-13 10:00:12

Why Automate Your Python Scripts?

Let’s think about some real-world scenarios where automation saves you time and money:

• Backups: Automatically backup your database every night without remembering
• Reports: Generate and email reports every Monday morning at 9 AM
• Data Processing: Process new files as they arrive, every hour
• Monitoring: Check server health every 5 minutes and alert if something’s wrong
• Cleanup: Delete temporary files older than 30 days, weekly

When you automate these tasks, you free up mental energy to focus on more important things. Plus, computers don’t sleep, so they can work 24/7 without complaining!

Method 1: Linux cron Jobs

If you’re running on Linux or macOS, cron is a powerful and built-in scheduler. Let’s walk through how to set up a cron job.

Step 1: Create Your Python Script

First, let’s create a simple backup script:

# backup_script.py
import shutil
import os
from datetime import datetime

def backup_database():
    # Get current date and time
    timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
    
    # Source and destination paths
    source = "/home/user/myapp/data.db"
    destination = f"/home/user/backups/data_{timestamp}.db"
    
    try:
        # Copy the database file
        shutil.copy2(source, destination)
        print(f"Backup successful: {destination}")
    except Exception as e:
        # Log errors for troubleshooting
        print(f"Backup failed: {e}")

if __name__ == "__main__":
    backup_database()

Output:

Backup successful: /home/user/backups/data_20260312_100000.db

Step 2: Make It Executable

chmod +x /home/user/backup_script.py

Step 3: Edit Your Crontab

Open your cron configuration by running:

crontab -e

Add this line to run the backup every day at 2 AM:

# Run backup script every day at 2:00 AM
0 2 * * * /usr/bin/python3 /home/user/backup_script.py >> /var/log/backup.log 2>&1

The cron syntax is: minute hour day month weekday command

• 0 2 * * * = Every day at 2:00 AM
• /usr/bin/python3 = Full path to Python interpreter
• /home/user/backup_script.py = Your script path
• >> /var/log/backup.log 2>&1 = Log output to a file

Method 2: Python schedule Library (Recommended for Cross-Platform)

If you need more control or want your scheduler to run within your Python application, the schedule library is excellent:

# job_scheduler.py
import schedule
import time
import logging
from datetime import datetime

# Setup logging to track what happens
logging.basicConfig(
    level=logging.INFO,
    format='%(asctime)s - %(message)s'
)

def backup_job():
    # This runs every day
    logging.info("Starting daily backup...")
    # Your backup code here
    logging.info("Backup complete!")

def send_report():
    # This runs every Monday at 9 AM
    logging.info("Sending weekly report...")
    # Your email code here
    logging.info("Report sent!")

def cleanup_temp_files():
    # This runs every hour
    logging.info("Cleaning temporary files...")
    # Your cleanup code here
    logging.info("Cleanup complete!")

# Schedule the jobs
schedule.every().day.at("02:00").do(backup_job)
schedule.every().monday.at("09:00").do(send_report)
schedule.every().hour.do(cleanup_temp_files)

# Keep the scheduler running
if __name__ == "__main__":
    print("Scheduler started. Press Ctrl+C to stop.")
    while True:
        schedule.run_pending()
        time.sleep(60)  # Check every minute

Output (simulated run):

2026-03-12 02:00:00 - Starting daily backup...
2026-03-12 02:00:15 - Backup complete!
2026-03-12 09:00:00 - Sending weekly report...
2026-03-12 09:00:25 - Report sent!
2026-03-12 13:00:00 - Cleaning temporary files...
2026-03-12 13:00:05 - Cleanup complete!

Method 3: Windows Task Scheduler

Windows users can use Task Scheduler to run Python scripts automatically:

1. Press Win + R and type taskschd.msc to open Task Scheduler
2. Click “Create Basic Task”
3. Give it a name: “My Python Backup”
4. Set trigger (when to run): Daily at 2:00 AM
5. Set action: Start a program
   • Program: C:Python312python.exe
   • Arguments: C:UsersYourNameackup_script.py
6. Click Finish

Error Handling in Scheduled Tasks

When your script runs automatically, you won’t be there to see errors. That’s why logging is critical:

# robust_scheduler.py
import schedule
import time
import logging
import traceback
from datetime import datetime

# Configure detailed logging
logging.basicConfig(
    level=logging.INFO,
    format='%(asctime)s - %(levelname)s - %(message)s',
    handlers=[
        logging.FileHandler('/var/log/my_jobs.log'),
        logging.StreamHandler()
    ]
)

def safe_job_wrapper(job_func, job_name):
    # Wrapper function that catches and logs errors
    def wrapper():
        try:
            logging.info(f"Starting: {job_name}")
            job_func()
            logging.info(f"Completed: {job_name}")
        except Exception as e:
            # Log the full error trace for debugging
            logging.error(f"Failed: {job_name}")
            logging.error(traceback.format_exc())
    return wrapper

def my_risky_job():
    # This might fail sometimes
    result = 10 / 1  # Would be 10 / 0 to cause error
    print(f"Calculation result: {result}")

# Schedule with error handling
safe_wrapper = safe_job_wrapper(my_risky_job, "my_risky_job")
schedule.every().hour.do(safe_wrapper)

if __name__ == "__main__":
    while True:
        schedule.run_pending()
        time.sleep(60)

Log Output (when error occurs):

2026-03-12 15:00:00 - INFO - Starting: my_risky_job
2026-03-12 15:00:00 - ERROR - Failed: my_risky_job
2026-03-12 15:00:00 - ERROR - Traceback (most recent call last):
  File "scheduler.py", line 15, in wrapper
    job_func()
  File "scheduler.py", line 28, in my_risky_job
    result = 10 / 0
ZeroDivisionError: division by zero

Real-Life Example: Automated Daily Backup Script

Let’s build a complete backup system that you can use right away:

# daily_backup.py
import schedule
import time
import os
import shutil
import logging
from datetime import datetime, timedelta
import gzip
import json

# Configure logging
logging.basicConfig(
    level=logging.INFO,
    format='%(asctime)s - %(levelname)s - %(message)s',
    filename='/var/log/backup.log'
)

class BackupManager:
    def __init__(self, source_dir, backup_dir, keep_days=7):
        # Initialize backup settings
        self.source_dir = source_dir
        self.backup_dir = backup_dir
        self.keep_days = keep_days
        
        # Create backup directory if it doesn't exist
        os.makedirs(backup_dir, exist_ok=True)
    
    def backup_files(self):
        # Create timestamped backup
        timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
        backup_path = os.path.join(self.backup_dir, f"backup_{timestamp}.tar.gz")
        
        try:
            # Compress the entire directory
            shutil.make_archive(
                backup_path.replace('.tar.gz', ''),
                'gztar',
                self.source_dir
            )
            logging.info(f"Backup created: {backup_path}")
            
            # Clean old backups
            self.cleanup_old_backups()
            
        except Exception as e:
            logging.error(f"Backup failed: {e}")
    
    def cleanup_old_backups(self):
        # Remove backups older than keep_days
        cutoff_date = datetime.now() - timedelta(days=self.keep_days)
        
        for filename in os.listdir(self.backup_dir):
            filepath = os.path.join(self.backup_dir, filename)
            file_time = datetime.fromtimestamp(os.path.getmtime(filepath))
            
            if file_time < cutoff_date:
                try:
                    os.remove(filepath)
                    logging.info(f"Deleted old backup: {filename}")
                except Exception as e:
                    logging.error(f"Failed to delete {filename}: {e}")

# Create backup manager instance
backup = BackupManager("/home/user/myapp", "/home/user/backups")

# Schedule daily backup at 2 AM
schedule.every().day.at("02:00").do(backup.backup_files)

# Keep scheduler running
if __name__ == "__main__":
    logging.info("Backup scheduler started")
    while True:
        schedule.run_pending()
        time.sleep(60)

Output:

2026-03-12 02:00:00 - INFO - Backup created: /home/user/backups/backup_20260312_020000.tar.gz
2026-03-13 02:00:00 - INFO - Backup created: /home/user/backups/backup_20260313_020000.tar.gz
2026-03-15 02:00:00 - INFO - Deleted old backup: backup_20260305_020000.tar.gz

FAQ

Q1: How do I know if my scheduled job ran?

Use logging! Redirect output to a log file as shown in our examples. Check the log file to verify execution.

Q2: Can I schedule a job for every 30 minutes?

Yes! With schedule library: schedule.every(30).minutes.do(my_job). With cron: */30 * * * * command

Q3: What's the difference between cron and the schedule library?

Cron is system-level, always running. Schedule library runs inside your Python process. Cron is better for production servers; schedule is better for local testing and development.

Q4: How do I stop a scheduled task?

For cron: crontab -e and delete the line. For schedule: Stop the Python process (Ctrl+C). For Windows: Open Task Scheduler and disable the task.

Q5: What if my Python script takes longer to run than scheduled?

Use a queuing system like Celery for production workloads. For simple scripts, the schedule library won't run overlapping instances by default.

Conclusion

Scheduling Python scripts is one of the most practical skills you can develop. Whether you use cron for server environments, the schedule library for cross-platform applications, or Windows Task Scheduler for local automation, you now have the tools to build reliable automated systems. Start small with a simple daily task, add logging to track what happens, and expand from there. Your future self will thank you for automating those repetitive tasks!

References

• Python Schedule Library Documentation
• Linux crontab Manual
• Python logging Module
• Python shutil for File Operations

cron Basics for Python Scripts

On Linux and macOS, cron is the system-level scheduler. Run crontab -e to open your user's crontab; each line is a schedule plus a command. The five fields before the command are minute hour day-of-month month day-of-week:

# Run every day at 2:30 AM
30 2 * * * /usr/bin/python3 /home/me/scripts/cleanup.py

# Every 15 minutes
*/15 * * * * /home/me/.venv/bin/python /home/me/scripts/poll.py

# Every Monday at 9 AM
0 9 * * 1 /home/me/.venv/bin/python /home/me/scripts/weekly-report.py

# Twice an hour on weekdays
0,30 * * * 1-5 /home/me/.venv/bin/python /home/me/scripts/business-hours.py

Three rules that save hours of debugging:

• Use absolute paths. Cron runs with a minimal PATH. python3 alone may not resolve. Use /usr/bin/python3 or your venv's full path.
• Redirect stdout / stderr. Append >> /var/log/cleanup.log 2>&1 so you can debug. Cron-by-default emails errors to the system mail spool, which you'll never see.
• Test with a 'run in one minute' schedule first. Set the job to run in 1-2 minutes from now while you're watching the log — much faster than waiting until 2:30 AM.

The schedule Library: Cron in Pure Python

For schedules that should travel with your code (containerized apps, cross-platform scripts), the schedule library gives you a Pythonic API:

# pip install schedule

import schedule
import time

def cleanup_temp_files():
    print("Cleaning up...")

def send_daily_report():
    print("Sending report")

schedule.every(15).minutes.do(cleanup_temp_files)
schedule.every().day.at("09:00").do(send_daily_report)
schedule.every().monday.at("08:00").do(send_daily_report)
schedule.every(2).hours.until("18:00").do(cleanup_temp_files)

while True:
    schedule.run_pending()
    time.sleep(60)

The advantage over cron: the schedule travels with the code, no system-level setup. The disadvantage: your script has to stay running. Pair it with systemd on Linux or supervisord to handle restarts.

APScheduler — Production-Grade Scheduling

For real applications, APScheduler beats both cron and schedule: persistent jobs (survive restart), missed-job handling, multiple triggers per job, async support. Three job stores cover most use cases — memory (default), SQLAlchemy (persistent), Redis (distributed):

# pip install apscheduler

from apscheduler.schedulers.background import BackgroundScheduler
from apscheduler.triggers.cron import CronTrigger

def daily_etl():
    print("Running ETL")

def hourly_sync():
    print("Syncing...")

scheduler = BackgroundScheduler()
scheduler.add_job(
    daily_etl,
    trigger=CronTrigger(hour=2, minute=30),
    id="daily-etl",
    replace_existing=True,
)
scheduler.add_job(hourly_sync, "interval", hours=1)
scheduler.start()

# Keep the main thread alive
import time
while True:
    time.sleep(60)

The replace_existing=True idiom is essential — without it, restarting your app fails with "job already exists" against a persistent job store.

Celery Beat — Distributed Scheduled Tasks

For microservices, Celery Beat schedules tasks across a distributed worker pool. The beat process emits scheduled tasks into the queue; any worker consumes them. Survives any single node restart, scales horizontally:

# celery_app.py
from celery import Celery
from celery.schedules import crontab

app = Celery("tasks", broker="redis://localhost:6379")

app.conf.beat_schedule = {
    "daily-report": {
        "task": "tasks.send_daily_report",
        "schedule": crontab(hour=9, minute=0),
    },
    "every-15-min": {
        "task": "tasks.cleanup",
        "schedule": 900.0,  # 15 minutes in seconds
    },
}

@app.task
def send_daily_report():
    print("Daily report sent")

@app.task
def cleanup():
    print("Cleanup done")

# Run beat alongside the workers:
# celery -A celery_app beat
# celery -A celery_app worker

Common Pitfalls

• Time zone confusion. Cron uses the system's local time. APScheduler defaults to UTC. schedule uses local time. Be explicit about which one you intend, or you'll be off by hours after the next DST change.
• Long-running jobs blocking the scheduler. If a 15-minute job kicks off every 15 minutes, the next instance fires while the first is still running. Use max_instances=1 in APScheduler or queue jobs through Celery.
• Forgetting the venv. Cron's python3 isn't your venv's python. Use the full venv path: /home/me/.venv/bin/python.
• Silent failures. Cron emails errors to /var/mail by default. If you've never seen those emails, you're missing failures. Pipe to a log file and tail it occasionally.
• Race conditions on overlap. Two cron jobs that need exclusive file access will collide if their schedules overlap. Use file locks (flock) or a job queue.

FAQ

Q: Cron, schedule, APScheduler, or Celery Beat?
A: Cron for one-off OS-level scripts. schedule for in-process scheduling in small apps. APScheduler when you need persistence or async. Celery Beat for distributed multi-worker setups.

Q: How do I make sure my script doesn't run twice if it overlaps?
A: File lock at the top: fd = open("/tmp/myscript.lock"); fcntl.flock(fd, fcntl.LOCK_EX | fcntl.LOCK_NB). Exits cleanly if another instance holds the lock.

Q: How do I run a Python script on a schedule on Windows?
A: Use Task Scheduler. Create a basic task, set the trigger (daily, weekly, on event), set the action to C:\Python312\python.exe C:\scripts\myscript.py. Or use APScheduler in-process — works identically on Windows.

Q: How do I see what cron jobs are scheduled?
A: crontab -l for the current user, sudo crontab -l -u username for another user. System-level cron lives in /etc/cron.d/* and /etc/cron.{hourly,daily,weekly,monthly}/*.

Q: How do I retry a failed scheduled job?
A: APScheduler doesn't retry out of the box. Wrap your job in a try/except + sleep, or use Celery's built-in retry: @app.task(autoretry_for=(Exception,), retry_backoff=True).

Wrapping Up

Pick the right tool for the scale. A single Python script that should run at 2 AM? Plain cron. A small app with a dozen recurring jobs? schedule in-process. Real production app with persistent jobs and async support? APScheduler. Microservices fleet? Celery Beat. The complexity of each tool tracks the complexity of the use case — don't reach for Celery Beat when cron will do.

Related Articles

• How To Set Up Python Logging With Rotating File Handlers
• How To Manage Python Environment Variables With dotenv
• How To Use Python subprocess To Run Shell Commands Safely