🔵 Intermediate

Virtual Environments & pip

📖 Lesson 26 ⏱ 35 min 🧪 5 questions 💻 3 exercises

🎯 Learning Objectives

Understand why virtual environments are essential
Create and manage virtual environments with venv
Install, upgrade, and remove packages with pip
Use requirements.txt to reproduce environments
Understand pyproject.toml and modern tooling (pip-tools, uv)
Follow dependency management best practices

The Problem Virtual Environments Solve

By default, pip install puts packages in a single global location shared by every Python project on your machine. This causes two major problems:

Version conflicts — Project A needs requests==2.28, Project B needs requests==2.31. They can't both win globally.
Pollution — Packages installed for one project silently affect all others, making bugs hard to reproduce.

Without virtual environments:
┌─────────────────────────────────┐
│        Global Python            │
│  requests 2.28  ←── Project A   │
│  requests 2.31  ←── Project B   │
│  (CONFLICT!)                    │
└─────────────────────────────────┘

With virtual environments:
┌─────────────┐    ┌─────────────┐
│  Project A  │    │  Project B  │
│  .venv/     │    │  .venv/     │
│  requests   │    │  requests   │
│  2.28  ✓    │    │  2.31  ✓   │
└─────────────┘    └─────────────┘

venv_concept.txt

A virtual environment is an isolated copy of the Python interpreter with its own site-packages directory. Packages installed inside it are completely separate from the global Python and from every other virtual environment.

Creating & Activating a Virtual Environment

# Create a virtual environment in a folder called .venv
python -m venv .venv

# ── Activate (adds .venv/bin to your PATH) ──
# Linux / macOS:
source .venv/bin/activate

# Windows (Command Prompt):
.venv\Scripts\activate.bat

# Windows (PowerShell):
.venv\Scripts\Activate.ps1

# Verify — should show the .venv path
which python          # Linux/macOS
where python          # Windows

# Deactivate — return to global Python
deactivate

terminal

After activation your prompt changes:
(.venv) $ python --version
Python 3.12.2

(.venv) $ which python
/home/user/myproject/.venv/bin/python

# pip now installs into .venv, not globally
(.venv) $ pip install requests

activated_prompt.txt

Name your virtual environment .venv (with a leading dot). This is the most common convention, recognised by VS Code, PyCharm, and most tools. Always add .venv/ to your .gitignore — never commit the environment itself, only the dependency file.

What venv creates

.venv/
├── bin/              (Scripts/ on Windows)
│   ├── python        ← symlink to the Python interpreter
│   ├── pip
│   └── activate      ← the activation script
├── lib/
│   └── python3.12/
│       └── site-packages/   ← installed packages live here
└── pyvenv.cfg        ← records the base Python used

venv_structure.txt

pip Essentials

# Install a package
pip install requests

# Install a specific version
pip install requests==2.31.0

# Install with version constraints
pip install "requests>=2.28,<3.0"

# Upgrade an existing package
pip install --upgrade requests

# Uninstall
pip uninstall requests

# List installed packages
pip list

# Show details about a package
pip show requests

# Search PyPI (deprecated — use https://pypi.org instead)
# pip search requests

pip_commands.sh

Installing from different sources

# From PyPI (default)
pip install flask

# From a requirements file
pip install -r requirements.txt

# From a local directory (editable install — changes reflect immediately)
pip install -e .

# From a Git repository
pip install git+https://github.com/psf/requests.git

# From a local wheel or tarball
pip install ./dist/mypackage-1.0.0-py3-none-any.whl

pip_sources.sh

Editable installs (pip install -e .) are the standard way to work on your own packages during development. Changes to your source files are reflected immediately without reinstalling.

requirements.txt

A requirements.txt file lists your project's dependencies so anyone can reproduce the exact same environment:

# Generate from current environment (pinned versions)
pip freeze > requirements.txt

# Install from the file
pip install -r requirements.txt

requirements_commands.sh

# requirements.txt — pinned for reproducibility
requests==2.31.0
certifi==2024.2.2
charset-normalizer==3.3.2
idna==3.6
urllib3==2.2.0

requirements.txt

Direct vs transitive dependencies

# requirements.in — only YOUR direct dependencies (human-maintained)
requests>=2.28
flask>=3.0
pytest>=8.0

# requirements.txt — generated, fully pinned (all transitive deps)
# pip-compile requirements.in  →  requirements.txt
certifi==2024.2.2
charset-normalizer==3.3.2
click==8.1.7
flask==3.0.2
idna==3.6
iniconfig==2.0.0
...
requests==2.31.0
urllib3==2.2.0

requirements_split.txt

Maintain a requirements.in (or pyproject.toml) with your direct dependencies and version constraints. Use pip freeze or pip-compile to generate the fully-pinned requirements.txt for deployment. Never hand-edit the pinned file.

Modern Packaging: `pyproject.toml`

The modern standard (PEP 517/518/621) uses a single pyproject.toml file to declare project metadata and dependencies:

[project]
name = "my-app"
version = "1.0.0"
description = "A sample application"
requires-python = ">=3.11"

# Direct runtime dependencies
dependencies = [
    "requests>=2.28",
    "flask>=3.0",
]

[project.optional-dependencies]
# Extra deps for development/testing
dev = [
    "pytest>=8.0",
    "pytest-cov",
    "ruff",
    "mypy",
]

[build-system]
requires      = ["setuptools>=68"]
build-backend = "setuptools.backends.legacy:build"

pyproject.toml

# Install runtime deps
pip install .

# Install runtime + dev deps
pip install ".[dev]"

# Editable install (development mode)
pip install -e ".[dev]"

pyproject_install.sh

Modern Tooling

pip-tools — lock file management

# Install pip-tools
pip install pip-tools

# Compile requirements.in → requirements.txt (with all transitive deps pinned)
pip-compile requirements.in

# Sync your environment to match exactly
pip-sync requirements.txt

pip_tools.sh

uv — ultra-fast package manager (recommended)

# Install uv (replaces pip + venv + pip-tools in one fast binary)
curl -LsSf https://astral.sh/uv/install.sh | sh   # Linux/macOS
# or: pip install uv

# Create a project with a virtual environment
uv init my-project
cd my-project

# Add a dependency (updates pyproject.toml + creates uv.lock)
uv add requests flask

# Add dev dependency
uv add --dev pytest ruff mypy

# Install all dependencies
uv sync

# Run a command inside the venv (no activation needed)
uv run python main.py
uv run pytest

# Remove a dependency
uv remove requests

uv_commands.sh

uv is written in Rust and is 10–100× faster than pip for installs. It manages Python versions, virtual environments, and lock files in one tool. It is rapidly becoming the community standard for new projects (2024–2025).

Other notable tools

Tool	Purpose	Best for
`venv` + `pip`	Built-in, zero deps	Simple scripts, learning
`pip-tools`	Lock file generation	Teams needing reproducibility
`uv`	All-in-one, fast	New projects, modern teams
`poetry`	Dependency + publish	Libraries, publishing to PyPI
`conda`	Env + non-Python deps	Data science, C/Fortran deps
`pipx`	Install CLI tools globally	Tools like black, ruff, mypy

Day-to-Day Workflow

# ── Starting a new project ──
mkdir my-project && cd my-project
python -m venv .venv
source .venv/bin/activate          # or .venv\Scripts\activate on Windows
pip install flask requests pytest
pip freeze > requirements.txt
echo ".venv/" >> .gitignore

# ── Cloning an existing project ──
git clone https://github.com/org/project
cd project
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

# ── Adding a new dependency ──
pip install httpx
pip freeze > requirements.txt      # update the lock file
git add requirements.txt && git commit -m "Add httpx"

# ── Keeping dependencies up to date ──
pip list --outdated                # see what's stale
pip install --upgrade requests     # upgrade one
pip freeze > requirements.txt      # re-lock

workflow.sh

Security Considerations

# Check for known vulnerabilities in your dependencies
pip install pip-audit
pip-audit

# Use hash verification for truly reproducible installs
pip install --require-hashes -r requirements.txt
# (requirements.txt must contain hashes — generated by pip-compile --generate-hashes)

# Avoid installing as root / with sudo
# Never: sudo pip install ...
# Always use a virtual environment instead

security.sh

Never sudo pip install into the system Python. It can break OS tools that depend on specific package versions, and it defeats the entire purpose of virtual environments. Always install into a virtual environment.

Best Practices

One virtual environment per project — always, no exceptions.
Add .venv/ to .gitignore — commit dependency files, not the env.
Commit your lock file (requirements.txt or uv.lock) — ensures reproducible installs for teammates and CI.
Separate direct and transitive deps — maintain requirements.in or pyproject.toml by hand; generate the pinned file.
Specify minimum Python version — use requires-python = ">=3.11" in pyproject.toml.
Pin transitive deps in applications — for libraries, only pin direct deps (leave room for compatibility).
Audit regularly — run pip-audit in CI to catch known CVEs.

# .gitignore — always include these
.venv/
__pycache__/
*.pyc
*.pyo
*.egg-info/
dist/
build/
.pytest_cache/
.mypy_cache/
.ruff_cache/

.gitignore

Primary sources: Python Docs — venv · pip documentation · Python Packaging Guide · uv documentation

🤖

Ask your AI tutor! Getting dependency conflicts? Not sure whether to use pip-tools, uv, or poetry for your project? Want to migrate from a requirements.txt to pyproject.toml? These are practical decisions worth thinking through.

💻 Exercises

01 Project Bootstrap Script

Write a Python script bootstrap.py that automates setting up a new project directory. It should:

Accept a project name as a command-line argument (sys.argv[1])
Create the project directory and enter it
Write a minimal .gitignore (include .venv/, __pycache__/, *.pyc)
Write a minimal pyproject.toml with the project name and requires-python = ">=3.11"
Print clear instructions for the next steps (create venv, activate, install)

Show solution

#!/usr/bin/env python3
"""bootstrap.py — scaffold a new Python project."""
import sys
from pathlib import Path

def bootstrap(name):
    project_dir = Path(name)
    if project_dir.exists():
        print(f"Error: directory '{name}' already exists", file=sys.stderr)
        sys.exit(1)

    project_dir.mkdir()

    # .gitignore
    (project_dir / ".gitignore").write_text(
        ".venv/\n__pycache__/\n*.pyc\n*.pyo\n*.egg-info/\ndist/\nbuild/\n"
        ".pytest_cache/\n.mypy_cache/\n.ruff_cache/\n",
        encoding="utf-8"
    )

    # pyproject.toml
    (project_dir / "pyproject.toml").write_text(
        f'[project]\nname = "{name}"\nversion = "0.1.0"\n'
        f'description = ""\nrequires-python = ">=3.11"\ndependencies = []\n\n'
        f'[project.optional-dependencies]\ndev = [\n    "pytest>=8.0",\n]\n',
        encoding="utf-8"
    )

    # Stub main module
    (project_dir / "main.py").write_text(
        'def main():\n    print("Hello from ' + name + '!")\n\n'
        'if __name__ == "__main__":\n    main()\n',
        encoding="utf-8"
    )

    print(f"✓ Created project '{name}/'")
    print("\nNext steps:")
    print(f"  cd {name}")
    print("  python -m venv .venv")
    print("  source .venv/bin/activate  # Windows: .venv\\Scripts\\activate")
    print("  pip install -e '.[dev]'")

if __name__ == "__main__":
    if len(sys.argv) != 2:
        print("Usage: python bootstrap.py ", file=sys.stderr)
        sys.exit(1)
    bootstrap(sys.argv[1])

02 Dependency Inspector

Write a Python script dep_check.py that reads a requirements.txt file and:

Parses each pinned package and version (ignore comments and blank lines)
Checks whether each package is currently installed using importlib.metadata
Reports: ✓ installed (with version), ✗ missing, ⚠ wrong version

Show solution

#!/usr/bin/env python3
"""dep_check.py — verify installed packages match requirements.txt"""
import sys
import re
from pathlib import Path
from importlib.metadata import version, PackageNotFoundError

def parse_requirements(path):
    """Yield (package_name, required_version_or_None) tuples."""
    for line in Path(path).read_text(encoding="utf-8").splitlines():
        line = line.strip()
        if not line or line.startswith("#"):
            continue
        m = re.match(r'^([A-Za-z0-9_\-\.]+)==(.+)$', line)
        if m:
            yield m.group(1), m.group(2)
        else:
            # No pinned version
            name = re.match(r'^([A-Za-z0-9_\-\.]+)', line)
            if name:
                yield name.group(1), None

def check_deps(requirements_path="requirements.txt"):
    ok = missing = wrong = 0
    for pkg, required in parse_requirements(requirements_path):
        try:
            installed = version(pkg)
            if required is None or installed == required:
                print(f"  ✓ {pkg} {installed}")
                ok += 1
            else:
                print(f"  ⚠ {pkg}: need {required}, have {installed}")
                wrong += 1
        except PackageNotFoundError:
            print(f"  ✗ {pkg} not installed")
            missing += 1

    print(f"\n{ok} ok, {wrong} wrong version, {missing} missing")
    return missing + wrong

if __name__ == "__main__":
    path = sys.argv[1] if len(sys.argv) > 1 else "requirements.txt"
    sys.exit(check_deps(path))

03 Environment Report

Write a function env_report() that prints a summary of the current Python environment, including:

Python version and executable path
Whether a virtual environment is active (check sys.prefix != sys.base_prefix)
Virtual environment path (if active)
Number of installed packages and their names + versions
Top 5 largest packages by size (hint: importlib.metadata.packages_distributions() and importlib.metadata.PathDistribution)

Show solution

import sys
import importlib.metadata as meta

def env_report():
    print("=" * 50)
    print("Python Environment Report")
    print("=" * 50)

    # Python version and path
    print(f"\nPython version : {sys.version.split()[0]}")
    print(f"Executable     : {sys.executable}")

    # Virtual environment
    in_venv = sys.prefix != sys.base_prefix
    print(f"Virtual env    : {'Yes — ' + sys.prefix if in_venv else 'No (global)'}")

    # Installed packages
    dists = list(meta.distributions())
    print(f"\nInstalled pkgs : {len(dists)}")

    # All packages
    packages = sorted(
        [(d.metadata['Name'], d.metadata['Version']) for d in dists],
        key=lambda x: x[0].lower()
    )
    print("\nAll packages:")
    for name, ver in packages:
        print(f"  {name:<30} {ver}")

    # Top 5 by size
    def dist_size(d):
        try:
            return sum(
                (d.locate_file(f)).stat().st_size
                for f in (d.files or [])
                if (d.locate_file(f)).exists()
            )
        except Exception:
            return 0

    sized = sorted(dists, key=dist_size, reverse=True)[:5]
    print("\nTop 5 by size:")
    for d in sized:
        size_kb = dist_size(d) / 1024
        print(f"  {d.metadata['Name']:<30} {size_kb:>8.1f} KB")

    print("=" * 50)

env_report()

Virtual Environments & pip

🎯 Learning Objectives

The Problem Virtual Environments Solve

Creating & Activating a Virtual Environment

What venv creates

pip Essentials

Installing from different sources

requirements.txt

Direct vs transitive dependencies

Modern Packaging: pyproject.toml

Modern Tooling

pip-tools — lock file management

uv — ultra-fast package manager (recommended)

Other notable tools

Day-to-Day Workflow

Security Considerations

Best Practices

💻 Exercises

Modern Packaging: `pyproject.toml`