Last Updated: June 01, 2026
Beginner
Selenium is a useful python library to extract web page data especially for pages with javascript loading. Many of you may have tried to use selenium but may have gotten stuck in the installation process. One key thing you have to remember is that Selenium will run an actual browser in the background (or foreground if you wish) to query a given website. So a key step is to install the driver if you haven’t done so already.
Step 1: Locate the right web driver
Since Selenium will use an actual driver, one of the first decisions you’ll need to make is to determine which driver to use. Generally it won’t matter, but the best browser to use, is the one that works the best for your target website. For example, if your target website works best under Firefox, then use that.
| Browser | Supported OS | Maintained by | Download | Issue Tracker |
|---|---|---|---|---|
| Chromium/Chrome | Windows/macOS/Linux | Downloads | Issues | |
| Firefox | Windows/macOS/Linux | Mozilla | Downloads | Issues |
| Edge | Windows 10 | Microsoft | Downloads | Issues |
| Internet Explorer | Windows | Selenium Project | Downloads | Issues |
| Opera | Windows/macOS/Linux | Opera | Downloads | Issues |
So decide which one, and then go to the download page. For this example we will use FireFox. In the above table, the download link goes to this page: https://github.com/mozilla/geckodriver/releases
You can then click on the latest release:

You can then scroll down to the bottom of the page to see the driver list:

Right click on the .gz file, and then get the URL.

Step 2: Download the web driver
Next go to your linux terminal and create a directory to store this file:

Next go into that directory, and then use wget to download the url by pasting the link you copied above:
wget https://github.com/mozilla/geckodriver/releases/download/v0.29.1/geckodriver-v0.29.1-linux32.tar.gz

Step 3: Extract the download web drivers
Next you should see the .gz file when you list the files:

You can the gzip the file to extract it:
gzip -d geckodriver-v0.29.1-linux32.tar.gz

You can then finally untar the file to decompress:
tar -xvf geckodriver-v0.29.1-linux32.tar

Step 4: Configure PATH
What you will be left with is a file called “geckodriver”. This is the driver file. You will need to have it made available via the export path. The reason is that the selenium looks for the driver file from the PATH operating system environment variable.
I simply went to the parent directory, then updated the PATH environment variable by taking the existing PATH value ($PATH) then appending the gdriver folder:
export PATH=$PATH:gdriver
If you do not do the above, you will get the error:
selenium.common.exceptions.WebDriverException: Message: 'geckodriver' executable needs to be in PATH.
Step 5: Test running the web driver
That’s it! Now if you test the following code, you should be able to run a web query by running a firefox driver in the background:
# main.py
from selenium import webdriver
from selenium.webdriver import FirefoxOptions
opts = FirefoxOptions()
opts.add_argument("--headless")
browser = webdriver.Firefox(options=opts)
# Declare a variable containing the URL is going to be scrapped
URL = 'https://pythonhowtoprogram.com/'
# Web driver going into website
browser.get(URL)
# Printing page title
print(browser.title)
You will notice it does take a few seconds to run for the first time. It’s because that an instance of a browser needs to be loaded which does take a few seconds. Just keep this in mind in case you need to have faster performance for which you may need to use urllib or requests instead.
Python developer and educator with 15+ years building production systems across data engineering, web APIs, and AI tooling. Founder of Python How To Program — 270+ in-depth tutorials covering the modern Python stack.
Next Steps
Now that you know how to install a driver, there are numerous webscraping tutorials we have on offer. You can find them all in our web scraping section: https://pythonhowtoprogram.com/category/web-scraping/
Want More Great Articles? Subscribe to our newsletter and have great articles sent right to your inbox as they come:
How To Use Python Decouple for Environment Variable Config
Intermediate
You write a FastAPI app that connects to a database and a third-party payment gateway. You hardcode the credentials during development and tell yourself you will fix it before deploying. A month later the repo is on GitHub and so is your production database password. This is not a hypothetical — it happens constantly, and it happens because os.environ.get() is clunky enough that developers avoid it until it is too late.
Python’s python-decouple library makes the right approach easier than the wrong one. It reads configuration from .env files or .ini files, applies type casting automatically, handles missing values with sensible defaults, and keeps your code clean. One pip install python-decouple is all it takes, and it works with any Python project — Flask, FastAPI, Django, or a plain script.
This article walks through everything you need to use python-decouple confidently: reading values, type casting, setting defaults, working with booleans, using .env vs .ini files, integrating with Django settings, and building a real-world config loader. By the end you will have a pattern you can drop into any project to keep secrets out of your source code for good.
Python Decouple: Quick Example
Here is a minimal working example that shows the core pattern — reading a string, an integer, and a boolean from a .env file — so you can see exactly what decouple does before we explore each feature in depth.
First, create a file named .env in your project root:
# .env
DATABASE_URL=postgresql://localhost:5432/myapp
PORT=8000
DEBUG=True
SECRET_KEY=my-local-dev-secret-key-change-in-prod
Now read those values in Python:
# quick_decouple.py
from decouple import config
# String (default type)
database_url = config('DATABASE_URL')
# Integer -- decouple casts automatically
port = config('PORT', cast=int)
# Boolean -- handles 'True', 'true', '1', 'yes', etc.
debug = config('DEBUG', cast=bool)
# String with a fallback default
secret_key = config('SECRET_KEY', default='fallback-dev-key')
print(f"DB: {database_url}")
print(f"Port: {port} (type: {type(port).__name__})")
print(f"Debug: {debug} (type: {type(debug).__name__})")
print(f"Key: {secret_key[:10]}...")
Output:
DB: postgresql://localhost:5432/myapp
Port: 8000 (type: int)
Debug: True (type: bool)
Key: my-local-d...
Notice that port comes back as a real Python int and debug as a real bool — not strings. With os.environ you would need to write int(os.environ['PORT']) and handle the conversion yourself every time. Decouple does that work once, at the point of reading, so the rest of your code receives properly typed values.
Read on to see how decouple handles missing values, search paths, .ini files, and real-world project layouts.
What Is Python Decouple and Why Use It?
Python decouple is a library that implements the Twelve-Factor App principle of strict separation between configuration and code. Configuration here means anything that is likely to vary between deployment environments: database URLs, API keys, feature flags, port numbers, and debug settings. The idea is that these values live in the environment (a .env file locally, environment variables in production), not in the source code that gets committed to a repository.
Think of it like a restaurant kitchen. The recipes (your code) are written down and shared. The ingredients (your config values) change depending on what the supplier has that day — and the head chef does not write the supplier’s phone number into every recipe card. They keep it in a separate contact file. Decouple is that contact file system for your Python app.
Decouple vs os.environ
Here is how python-decouple compares to using os.environ directly:
| Feature | os.environ | python-decouple |
|---|---|---|
| Read string value | os.environ['KEY'] — raises KeyError if missing | config('KEY') — raises UndefinedValueError with clear message |
| Default value | os.environ.get('KEY', 'default') | config('KEY', default='default') |
| Integer casting | int(os.environ.get('PORT', '8000')) | config('PORT', default=8000, cast=int) |
| Boolean casting | Manual: 'True' == os.environ.get('DEBUG') | config('DEBUG', cast=bool) handles True/true/1/yes |
| Read from .env file | Requires python-dotenv or manual parsing | Built in — searches parent directories automatically |
| Support .ini files | No | Yes — useful for projects with existing .ini configs |
| Test overrides | Must monkeypatch os.environ | Can pass values directly in code during tests |
The bottom line: os.environ is built-in and requires no extra dependency, but every type conversion is manual boilerplate. Decouple pays for itself the moment you have more than two or three config values that need casting.
Installing python-decouple
Install it with pip in your virtual environment:
# install_decouple.py (run this in your terminal, not as a script)
pip install python-decouple
Output:
Successfully installed python-decouple-3.8
There is one important naming note: the library is called python-decouple on PyPI (what you install), but the import name is decouple (what you use in code). Do not confuse it with decouple on PyPI — that is a different package for Django-specific use. Always install python-decouple.
The .env File: Format and Best Practices
A .env file is a plain text file with one KEY=value pair per line. Decouple searches for it starting in the directory of the script being run, then walks up to parent directories. This means you can place it at the root of your project and it will be found regardless of which subdirectory you run from.
# .env (place this in your project root)
# Database
DATABASE_URL=postgresql://user:password@localhost:5432/myapp_dev
# Server
PORT=8000
HOST=0.0.0.0
# Feature flags
DEBUG=True
ENABLE_CACHING=False
# Third-party APIs
STRIPE_SECRET_KEY=sk_test_abc123
SENDGRID_API_KEY=SG.xyz789
# Email
EMAIL_BACKEND=console
EMAIL_HOST=smtp.example.com
EMAIL_PORT=587
There are a few formatting rules to know. Values do not need quotes — DEBUG=True works fine. If your value contains spaces or special characters, wrap it in single or double quotes: FULL_NAME='Ada Lovelace'. Lines starting with # are comments and are ignored. Empty lines are also ignored.
The most important rule: add .env to your .gitignore immediately. Create a .env.example file with the same keys but dummy values, and commit that instead. New developers clone the repo, copy .env.example to .env, fill in their local values, and they are ready to go.
Type Casting with cast=
Every value in a .env file is stored as a string. Decouple’s cast parameter converts the string to the type you need before returning it, so the rest of your code never sees a string where it expects an integer or boolean.
Integers and Floats
Pass cast=int or cast=float to convert numeric config values. This is far cleaner than wrapping every read in a manual conversion.
# cast_examples.py
from decouple import config
# These values exist in .env:
# PORT=8000
# WORKERS=4
# TIMEOUT=30.5
port = config('PORT', default=8000, cast=int)
workers = config('WORKERS', default=2, cast=int)
timeout = config('TIMEOUT', default=30.0, cast=float)
print(f"Port: {port} -- {type(port).__name__}")
print(f"Workers: {workers} -- {type(workers).__name__}")
print(f"Timeout: {timeout} -- {type(timeout).__name__}")
Output:
Port: 8000 -- int
Workers: 4 -- int
Timeout: 30.5 -- float
If the .env value cannot be cast to the requested type — for example, PORT=eight_thousand — decouple raises a ValueError with a clear message pointing to the offending key. You get the error at startup when reading config, not somewhere deep in your app when the value is used.
Booleans
Boolean config values are tricky with os.environ because every string is truthy. "False" evaluates to True in Python because it is a non-empty string. Decouple’s boolean cast handles this correctly by recognizing a set of canonical true and false values.
# cast_bool.py
from decouple import config
# .env contains:
# DEBUG=True
# ENABLE_CACHING=False
# USE_SSL=yes
# MAINTENANCE_MODE=0
debug = config('DEBUG', cast=bool)
caching = config('ENABLE_CACHING', cast=bool)
ssl = config('USE_SSL', cast=bool)
maintenance = config('MAINTENANCE_MODE', cast=bool)
print(f"DEBUG: {debug}")
print(f"ENABLE_CACHING: {caching}")
print(f"USE_SSL: {ssl}")
print(f"MAINTENANCE_MODE: {maintenance}")
Output:
DEBUG: True
ENABLE_CACHING: False
USE_SSL: True
MAINTENANCE_MODE: False
The recognized truthy values are True, true, 1, yes, on. The recognized falsy values are False, false, 0, no, off. Anything else raises a ValueError. This strict set prevents the bug where DEBUG=False still evaluates to True because you forgot to cast.
Comma-Separated Lists
Decouple does not have a built-in list type, but you can pass any callable as the cast argument — including a lambda that splits a string into a list.
# cast_list.py
from decouple import config, Csv
# .env contains:
# ALLOWED_HOSTS=localhost,127.0.0.1,myapp.com
# CORS_ORIGINS=http://localhost:3000,https://app.example.com
# Option 1: built-in Csv helper (strips whitespace, handles quoting)
allowed_hosts = config('ALLOWED_HOSTS', cast=Csv())
# Option 2: lambda for simple cases
cors_origins = config('CORS_ORIGINS', default='', cast=lambda v: [s.strip() for s in v.split(',')])
print(f"Allowed hosts: {allowed_hosts}")
print(f"CORS origins: {cors_origins}")
Output:
Allowed hosts: ['localhost', '127.0.0.1', 'myapp.com']
CORS origins: ['http://localhost:3000', 'https://app.example.com']
The Csv() helper from decouple is the cleaner option for comma-separated values. It handles edge cases like extra whitespace and quoted values with commas inside them. The lambda approach works fine for simple cases where you control the format.
Defaults and Missing Values
When a key is missing from both the .env file and the actual environment, decouple’s behavior depends on whether you provided a default.
# defaults_demo.py
from decouple import config, UndefinedValueError
# KEY_WITH_DEFAULT is not in .env -- returns the default
log_level = config('LOG_LEVEL', default='INFO')
print(f"Log level: {log_level}")
# KEY_WITH_NONE_DEFAULT is not in .env -- returns None
cache_url = config('CACHE_URL', default=None)
print(f"Cache URL: {cache_url}")
# KEY_REQUIRED is not in .env and has no default -- raises UndefinedValueError
try:
api_key = config('REQUIRED_API_KEY')
except UndefinedValueError as e:
print(f"Missing required config: {e}")
Output:
Log level: INFO
Cache URL: None
Missing required config: REQUIRED_API_KEY not found. Declare it as envvar or define a default value.
This behavior is intentional and useful. Required values — things your app absolutely cannot run without — should have no default. That way decouple raises a clear error at startup rather than letting the app start in a broken state and fail later with a cryptic message. Optional values should have a sensible default so the app can run in a minimal configuration without a full .env file in place.
.ini File Support
In addition to .env files, decouple can read from .ini files using the AutoConfig or explicit RepositoryIni approach. This is useful when your project already has a settings.ini or setup.cfg and you do not want to introduce a second config file.
# settings.ini
[settings]
DATABASE_URL=postgresql://localhost:5432/myapp
PORT=8000
DEBUG=True
# read_ini.py
from decouple import Config, RepositoryIni
# Explicitly read from a .ini file instead of .env
config = Config(RepositoryIni('settings.ini'))
database_url = config('DATABASE_URL')
port = config('PORT', cast=int)
debug = config('DEBUG', cast=bool)
print(f"DB: {database_url}")
print(f"Port: {port}")
print(f"Debug: {debug}")
Output:
DB: postgresql://localhost:5432/myapp
Port: 8000
Debug: True
The default config object (imported directly from decouple) uses AutoConfig, which searches for .env first, then .ini, then falls back to actual environment variables. You only need to use RepositoryIni explicitly when you want to force a specific file rather than letting decouple search.
Django Integration
Django’s settings.py is the most common place developers accidentally commit secrets. Decouple is designed to slot in cleanly as a drop-in replacement for hardcoded settings.
# settings.py (Django)
from decouple import config, Csv
# Core Django settings
SECRET_KEY = config('SECRET_KEY')
DEBUG = config('DEBUG', cast=bool, default=False)
ALLOWED_HOSTS = config('ALLOWED_HOSTS', cast=Csv(), default='localhost')
# Database -- dj-database-url makes this even cleaner
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': config('DB_NAME', default='myapp'),
'USER': config('DB_USER', default='postgres'),
'PASSWORD': config('DB_PASSWORD', default=''),
'HOST': config('DB_HOST', default='localhost'),
'PORT': config('DB_PORT', default=5432, cast=int),
}
}
# Email
EMAIL_BACKEND = config('EMAIL_BACKEND', default='django.core.mail.backends.console.EmailBackend')
EMAIL_HOST = config('EMAIL_HOST', default='localhost')
EMAIL_PORT = config('EMAIL_PORT', default=25, cast=int)
EMAIL_USE_TLS = config('EMAIL_USE_TLS', cast=bool, default=False)
# Stripe
STRIPE_PUBLIC_KEY = config('STRIPE_PUBLIC_KEY', default='')
STRIPE_SECRET_KEY = config('STRIPE_SECRET_KEY', default='')
The pattern is consistent throughout: use config('KEY') for required values that must exist in production, and config('KEY', default=...) for optional values with safe development defaults. The entire settings.py file becomes safe to commit because it contains no actual secrets — just the names of the keys and their defaults.
Real-Life Example: Environment-Aware FastAPI App
Here is a realistic FastAPI application config module that uses decouple to manage all its settings. This pattern — a dedicated config.py module that gathers all config into a dataclass — scales cleanly as the project grows.
# config.py
from dataclasses import dataclass
from decouple import config, Csv, UndefinedValueError
@dataclass
class AppConfig:
# Server
host: str
port: int
debug: bool
workers: int
# Database
database_url: str
# Security
secret_key: str
allowed_origins: list
# External APIs
stripe_secret_key: str
sendgrid_api_key: str
slack_webhook_url: str
# Feature flags
enable_caching: bool
enable_email: bool
def load_config() -> AppConfig:
"""Load and validate all application configuration at startup."""
return AppConfig(
# Server
host=config('HOST', default='0.0.0.0'),
port=config('PORT', default=8000, cast=int),
debug=config('DEBUG', default=False, cast=bool),
workers=config('WORKERS', default=1, cast=int),
# Database -- required in production, no default
database_url=config('DATABASE_URL'),
# Security -- required always
secret_key=config('SECRET_KEY'),
allowed_origins=config('ALLOWED_ORIGINS', cast=Csv(), default='http://localhost:3000'),
# External APIs -- optional with empty defaults (check before use)
stripe_secret_key=config('STRIPE_SECRET_KEY', default=''),
sendgrid_api_key=config('SENDGRID_API_KEY', default=''),
slack_webhook_url=config('SLACK_WEBHOOK_URL', default=''),
# Feature flags
enable_caching=config('ENABLE_CACHING', default=False, cast=bool),
enable_email=config('ENABLE_EMAIL', default=False, cast=bool),
)
# main.py
from fastapi import FastAPI
from config import load_config, AppConfig
cfg: AppConfig = load_config() # Fails fast at startup if required vars missing
app = FastAPI(debug=cfg.debug)
@app.get("/health")
def health():
return {
"status": "ok",
"debug": cfg.debug,
"caching": cfg.enable_caching,
"port": cfg.port,
}
if __name__ == "__main__":
import uvicorn
print(f"Starting on {cfg.host}:{cfg.port} (debug={cfg.debug})")
uvicorn.run(app, host=cfg.host, port=cfg.port, workers=cfg.workers)
Output (with example .env values):
Starting on 0.0.0.0:8000 (debug=False)
The key design choice here is that load_config() is called once at module level, so any missing required variable raises UndefinedValueError the moment the app starts — not on the first request five minutes into a production deploy. The dataclass gives you IDE autocomplete throughout the rest of the codebase and makes it obvious what configuration the app expects without opening the .env file.
Testing with Decouple
Config management and testability often conflict — tests need predictable values, but decouple reads from files. There are two clean approaches: use a test-specific .env file, or monkeypatch os.environ.
# test_config.py
import os
import pytest
from unittest.mock import patch
# Approach 1: patch os.environ before importing config
def test_debug_defaults_to_false():
with patch.dict(os.environ, {'DATABASE_URL': 'sqlite:///test.db', 'SECRET_KEY': 'test-key'}, clear=True):
# Reimport or reload config within the patch context
from decouple import config
# config() reads os.environ after checking .env
debug = config('DEBUG', default=False, cast=bool)
assert debug is False
def test_port_casting():
with patch.dict(os.environ, {'PORT': '9000'}):
from decouple import config
port = config('PORT', cast=int)
assert port == 9000
assert isinstance(port, int)
def test_missing_required_raises():
from decouple import config, UndefinedValueError
with patch.dict(os.environ, {}, clear=True):
# Remove DATABASE_URL from the environment
env_without_db = {k: v for k, v in os.environ.items() if k != 'DATABASE_URL'}
with patch.dict(os.environ, env_without_db, clear=True):
with pytest.raises(UndefinedValueError):
config('DATABASE_URL')
Output (pytest):
...
3 passed in 0.12s
The recommended approach for larger projects is to create a .env.test file and use a fixture that temporarily swaps the decouple search path to point at it. This gives you a full, realistic config setup for tests without polluting your development .env. The patch.dict(os.environ, ...)` approach shown above works well for unit tests of individual config values.
Frequently Asked Questions
Does decouple replace os.environ entirely?
Not entirely -- decouple reads from .env files first, then falls back to actual environment variables set in the shell or by the deployment platform. In production you typically do not deploy a .env file; instead, environment variables are set by the platform (Heroku config vars, Docker environment, Kubernetes secrets). Decouple reads those just fine through the os.environ fallback. The .env file is a development convenience, not a production requirement.
Where should I put my .env file?
Place it in the root of your project -- the same directory as manage.py (Django), main.py (FastAPI/Flask), or your top-level package. Decouple uses AutoConfig which walks up from the running script's directory until it finds a .env or .ini file, so as long as it is somewhere in the directory tree above your code, it will be found. Do not commit it to version control -- add it to .gitignore and commit a .env.example instead.
How do I handle multiple environments (dev, staging, prod)?
The cleanest approach is one .env per environment, kept out of the repo. Your CI/CD pipeline injects the appropriate values as environment variables for staging and production. Locally you maintain a .env with development values. You can also use a tool like direnv to switch .env files automatically when you change directories. Never create .env.production files and commit them -- that defeats the entire purpose.
What is the difference between python-decouple and python-dotenv?
Both libraries load .env files, but they take different approaches. python-dotenv loads values into os.environ as a side effect, making them available to os.environ.get() and any other code that reads the environment. python-decouple does not modify os.environ -- instead it provides a config() function that reads directly from the file or the real environment. Decouple is preferable when you want typed values, sensible defaults, and a clean API. Dotenv is useful when you need the values to land in os.environ for libraries that read from there directly.
Should I use .env files in production?
Generally no. Deploying a .env file to a server creates a file on disk containing secrets, which is a security risk if the server is ever compromised. In production, use your platform's secret management: Heroku config vars, AWS Secrets Manager, Docker secrets, Kubernetes secrets, or environment variables set in your deployment config. Decouple reads all of these through its os.environ fallback, so no code changes are needed between development (using .env) and production (using platform secrets).
How do I manage a large number of config values?
Group related config into separate config objects or dataclasses, as shown in the FastAPI example above. A single config.py module that defines everything in one place makes it easy to see what the app needs at a glance. Avoid calling config() scattered throughout your codebase -- centralizing config reads means you only have one place to look when a value is wrong, and one place to update when a key changes.
Conclusion
Python decouple solves a problem that trips up almost every developer at some point: configuration that leaks into source code. The library gives you config('KEY') with type casting, defaults, and clear errors for missing required values -- all reading from a .env file that stays out of your repository. We covered reading strings, integers, booleans, and lists; comparing decouple to raw os.environ; using .ini files; integrating with Django settings; and building a real FastAPI config module with a dataclass pattern.
The next step is to take the config module pattern from the real-life example and adapt it to your own project. Start by identifying every hardcoded string in your settings that could change between environments -- database URLs, API keys, debug flags, port numbers -- and move them behind config() calls. Your future self, and anyone else who has to deploy your app, will thank you.
Full documentation for python-decouple is at github.com/HBNetwork/python-decouple. The Twelve-Factor App methodology that inspired it is documented at 12factor.net/config. For a complementary approach using type-validated settings objects, see our guide on Pydantic Settings for Configuration Management. If you prefer a more flexible multi-environment solution, dynaconf is worth exploring. To keep your secrets extra safe, pair decouple with the built-in Python secrets module for generating tokens and keys.
Related Articles
Further Reading: For more details, see the Python webbrowser module documentation.
Frequently Asked Questions
What is Selenium WebDriver used for in Python?
Selenium WebDriver is a tool for automating web browser interactions. In Python, it is used for web scraping, automated testing of web applications, form filling, screenshot capture, and any task that requires programmatic control of a web browser.
Which browser drivers work with Selenium in Python?
Selenium supports ChromeDriver (Chrome/Chromium), GeckoDriver (Firefox), EdgeDriver (Microsoft Edge), and SafariDriver (Safari). ChromeDriver and GeckoDriver are the most commonly used for Linux-based automation.
How do I install ChromeDriver on Linux?
Download ChromeDriver from the official site matching your Chrome version, extract it, and place it in your PATH (e.g., /usr/local/bin/). Alternatively, use webdriver-manager package: pip install webdriver-manager to handle driver installation automatically.
Why do I get ‘WebDriver not found’ errors?
This typically occurs when the driver executable is not in your system PATH, the driver version does not match your browser version, or the driver file lacks execute permissions. Use chmod +x chromedriver to set permissions and ensure version compatibility.
Can Selenium run without a visible browser window?
Yes. Use headless mode by adding options.add_argument('--headless') to your browser options. This runs the browser in the background without a GUI, which is faster and ideal for servers and CI/CD pipelines.
Installing the Right Driver Binary
Selenium needs a browser-specific driver binary on the system PATH or pointed to explicitly. The two paths that work on Linux:
Option 1 — Selenium Manager (Selenium 4.6+): The library auto-downloads the right driver. Zero setup beyond installing selenium:
# pip install selenium
from selenium import webdriver
driver = webdriver.Chrome() # auto-downloads chromedriver
driver.get("https://example.com")
print(driver.title)
driver.quit()
Option 2 — webdriver-manager: Explicit installation per session, handy when you need to pin a version:
# pip install webdriver-manager
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from webdriver_manager.chrome import ChromeDriverManager
service = Service(ChromeDriverManager().install())
driver = webdriver.Chrome(service=service)
Headless Mode for Servers
On a server with no display, you need headless mode (and matching Chrome / Chromium installed). The minimal Chrome install on Ubuntu 22.04 and Debian:
# Install Chrome and the libraries it needs
sudo apt-get update
sudo apt-get install -y wget gnupg
wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | sudo apt-key add -
echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" | \
sudo tee /etc/apt/sources.list.d/google-chrome.list
sudo apt-get update
sudo apt-get install -y google-chrome-stable
# Python: enable headless
from selenium.webdriver.chrome.options import Options
opts = Options()
opts.add_argument("--headless=new") # use the new headless mode (Chrome 109+)
opts.add_argument("--no-sandbox") # required when running as root
opts.add_argument("--disable-dev-shm-usage") # avoid /dev/shm size issues
opts.add_argument("--window-size=1920,1080") # avoid layout-dependent failures
driver = webdriver.Chrome(options=opts)
The --disable-dev-shm-usage flag fixes a notorious crash in Docker containers where the shared-memory partition is too small. --no-sandbox is required when Chrome runs as root (Docker default).
Firefox / geckodriver
If Chrome isn’t your target, swap in Firefox. Same pattern, different driver:
sudo apt-get install -y firefox
# Python
from selenium import webdriver
from selenium.webdriver.firefox.options import Options as FFOptions
from selenium.webdriver.firefox.service import Service as FFService
from webdriver_manager.firefox import GeckoDriverManager
opts = FFOptions()
opts.add_argument("--headless")
service = FFService(GeckoDriverManager().install())
driver = webdriver.Firefox(service=service, options=opts)
driver.get("https://example.com")
Docker Setup for Selenium
For CI / production, run Selenium in Docker rather than installing system-wide. The official Selenium images have everything bundled:
# Pull a ready-to-go Chrome stack
docker run -d -p 4444:4444 -p 7900:7900 --shm-size=2g \
selenium/standalone-chrome:latest
# Now connect from any host (no local Chrome needed)
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
opts = Options()
opts.add_argument("--headless=new")
driver = webdriver.Remote(
command_executor="http://localhost:4444/wd/hub",
options=opts,
)
driver.get("https://example.com")
The --shm-size=2g on the container fixes the same shared-memory issue as --disable-dev-shm-usage in the Chrome args. Pick whichever is convenient.
Verifying Your Setup
A 6-line smoke test catches 90% of install failures:
# File: test_selenium.py
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
opts = Options()
opts.add_argument("--headless=new")
opts.add_argument("--no-sandbox")
driver = webdriver.Chrome(options=opts)
driver.get("https://www.python.org")
print("Title:", driver.title)
print("URL:", driver.current_url)
driver.quit()
If this runs and prints “Welcome to Python.org” — you’re done. If it fails, the error message tells you exactly what’s missing (driver, browser binary, sandbox flag, etc.).
Common Pitfalls
- Mixing Chrome and chromedriver versions. chromedriver must match Chrome’s major version. Selenium Manager handles this; webdriver-manager handles it; manual installs break every Chrome update.
- Forgetting –no-sandbox in Docker. Chrome refuses to run as root (which Docker default is) without it. Add it OR run as a non-root user.
- Insufficient /dev/shm. Default 64MB shared memory in Docker isn’t enough. Use
--shm-size=2gor--disable-dev-shm-usage. - Missing browser binary. chromedriver alone isn’t enough — you also need Chrome itself installed. Same for Firefox + geckodriver.
- Old –headless flag. Chrome’s old headless mode is deprecated in favor of
--headless=new(Chrome 109+). The new mode is faster and renders more accurately.
FAQ
Q: Selenium or Playwright?
A: For new projects, Playwright is faster, has better selectors, and auto-handles waits. Selenium is mature and ubiquitous — if you have existing Selenium tests or need browser support beyond Chrome/Firefox/WebKit, stick with it.
Q: Headless or headful?
A: Headless for CI, scrapers, and any unattended workflow. Headful when developing — you can SEE what your code is doing, which speeds debugging by 10x.
Q: How do I run as a specific browser version?
A: Install that specific version of Chrome / Firefox, then point Selenium at it: options.binary_location = "/path/to/chrome". webdriver-manager can also pin to a version.
Q: Why is the test slow on the first run?
A: The driver download. Subsequent runs use the cached binary. CI systems should cache ~/.wdm (webdriver-manager) and ~/.cache/selenium.
Q: How do I bypass Cloudflare / bot protection?
A: Standard Selenium gets blocked by Cloudflare. Use undetected-chromedriver (better) or Playwright with stealth plugins (best). For aggressive bot detection, you may need to rotate user agents and use residential proxies.
Wrapping Up
Selenium on Linux comes down to three pieces: Python’s selenium package, the browser binary (Chrome or Firefox), and the driver binary (chromedriver or geckodriver). Selenium Manager handles the driver auto-download. --headless=new, --no-sandbox, and --disable-dev-shm-usage are the three flags that make Chrome work reliably in Docker. Get that combination right and Selenium runs cleanly in CI, on servers, and in production scrapers.
Related Articles
- Scrape Dynamic Websites with Selenium
- Extract Table Data from Webpages
- How To Schedule Python Scripts
Continue Learning Python
Tutorials you might also find useful:
- How To Use Playwright for Web Scraping in Python
- How To Use Python Litestar for Async Web APIs
- How To Use PyJWT for JSON Web Tokens in Python
- How To Build a Flask Web Application in Python
- How To Build Web Apps with Django in Python
- How To Scrape Dynamic Websites With Selenium and BeautifulSoup in Python 3
Thanks for finally writing about > How To Install
Selenium Driver For Python in Linux diatomity