Why Your Python Install Breaks, and How Environments Fix It

You install a package for one project and another project stops working. Or a tutorial says pip install and your system answers with a permissions error, or a warning about an externally managed environment. Both are the same problem: packages are being installed into one shared place, and different projects want different versions. Virtual environments solve it, and they take about a minute to learn.

What is actually happening

Python looks for imported packages in a list of directories. Without an environment, pip install requests puts the package in the interpreter's shared site-packages directory, where every project on the machine sees it. There is exactly one version of each package, so the last install wins and the project that needed the older one breaks.

On Linux and macOS there is a second problem: the system Python belongs to the operating system, which has its own packages installed there. Installing into it with sudo can break system tools. Recent distributions block this deliberately, which is what the "externally managed environment" message is telling you.

A virtual environment is a directory holding its own copy of the interpreter's configuration and its own site-packages. Activate it and python and pip point at that directory instead of the shared one. Nothing is magic and nothing is hidden: it is a folder you can delete.

Day to day

# create, once per project
python -m venv .venv

# activate, every new terminal
source .venv/bin/activate      # macOS and Linux
.venv\Scripts\activate         # Windows

# now pip installs into the project
pip install requests

deactivate                      # when you are done

Two habits make this painless. Put .venv inside the project folder, so the environment travels with the code and it is obvious which project it belongs to. And add .venv/ to .gitignore: the environment is a build artefact, not source.

To check which interpreter you are on, which python (or where python on Windows) should point inside the project. Most editors detect a .venv folder automatically and will use it for running and for autocompletion, which saves the classic confusion where the terminal works and the editor reports an unresolved import.

Recording what you installed

An environment that only exists on your laptop is not reproducible. Record the dependencies in the project.

The traditional route is a requirements file. Keep two ideas separate: what you asked for, and what got installed. A file listing your direct dependencies, each pinned to a version you tested, is what you maintain by hand. pip freeze produces the full resolved list including transitive packages, which is what you want for an exact rebuild of a deployment.

pip install -r requirements.txt     # install from the list
pip freeze > requirements.lock.txt  # record exactly what is installed

The modern route is pyproject.toml, where dependencies live alongside the project metadata, with a lock file produced by whichever tool you use. Either is fine. What matters is that a colleague, or you in eight months, can recreate the environment without guessing.

Whatever you choose, pin versions for anything you deploy. An unpinned requirements file is a promise that every future release of every dependency will keep working.

The tools built on top

venv ships with Python and covers most needs. A few alternatives are worth knowing about. Tools that manage projects and environments together create the environment for you and keep a lock file in step, which removes the activate step from daily work. Tools that install command-line applications in their own isolated environments are the right answer for something like a formatter you want available everywhere, rather than installing it globally. And if you need several Python versions on one machine, a version manager handles that; a virtual environment does not, since it uses the interpreter it was created from.

Errors that mean you skipped a step

• ModuleNotFoundError for something you just installed. You installed it in a different environment from the one running the code. Check the interpreter path in both places.
• Permission denied, or a suggestion to use sudo. You are installing into the system Python. Create an environment instead; installing with sudo is how system tools break.
• It works in the terminal and not in the editor. The editor is using a different interpreter. Select the project's .venv explicitly.
• A dependency conflict on install. Two packages want incompatible versions of a third. Pinning the top-level packages to versions that were released around the same time usually resolves it.
• The environment stopped working after a system update. The environment references the interpreter it was built from. When that moves, delete the folder and recreate it; it takes seconds, which is the point of keeping it disposable.

The whole routine

New project: make a folder, create .venv, activate it, install what you need, write the dependencies down, ignore the folder in git. Returning to a project: activate, and install from the requirements file if anything is missing. Something behaving strangely: delete the environment and rebuild it, which is a thirty-second fix rather than an afternoon of debugging a shared install.