Skip to content

Python Virtual Environments: A Complete Guide to venv, virtualenv, and Conda

Updated on

Python developers face a common challenge: dependency conflicts between projects. When multiple applications require different versions of the same library, system-wide installations create incompatibilities that break code. A data science project might need pandas 1.3.5 while a web application requires pandas 2.0.0, making it impossible to run both on the same Python installation.

This problem intensifies in team environments where developers work with different package versions or when deploying applications to production servers. Without proper isolation, installing a new dependency for one project can break another, leading to the classic "it works on my machine" scenario that wastes hours of debugging time.

Python virtual environments solve this by creating isolated spaces where each project maintains its own dependencies independently. This guide covers everything from basic venv usage to advanced environment management with conda, helping you choose the right tool and implement best practices for professional Python development.

📚

Why Virtual Environments Matter

Virtual environments address three critical problems in Python development:

Dependency isolation: Each project gets its own package installations without affecting the system Python or other projects. Installing Django 4.2 in one environment does not impact a legacy project running Django 2.2 in another environment.

Version management: Different projects can use different Python versions. A machine learning project might use Python 3.11 for performance while a legacy codebase requires Python 3.8 for compatibility with older libraries.

Reproducibility: Virtual environments enable exact recreation of development setups across machines. By exporting dependency lists, team members can replicate the same environment, eliminating version mismatch bugs.

Without virtual environments, developers risk breaking their system Python installation, which operating systems often depend on for core functionality. Installing incompatible packages globally can render system tools unusable, requiring complex fixes or complete OS reinstallation in severe cases.

Python venv Module: The Built-in Solution

Python 3.3 introduced venv as a built-in module for creating lightweight virtual environments. It requires no additional installation and provides sufficient functionality for most projects.

Creating a Virtual Environment with venv

Navigate to your project directory and create a virtual environment:

python -m venv myenv

This command creates a myenv directory containing:

  • A copy of the Python interpreter
  • The standard library
  • pip package manager
  • Scripts for environment activation

For specific Python versions when multiple installations exist:

python3.11 -m venv myenv

Activating Virtual Environments

Activation differs by operating system:

Linux and macOS:

source myenv/bin/activate

Windows Command Prompt:

myenv\Scripts\activate.bat

Windows PowerShell:

myenv\Scripts\Activate.ps1

After activation, the terminal prompt displays the environment name in parentheses:

(myenv) user@machine:~/project$

All pip installations now target the virtual environment instead of the system Python.

Installing Packages in venv

With the environment activated, install packages normally:

pip install pandas numpy matplotlib

These packages install only in the virtual environment. To verify:

pip list

This shows packages specific to the active environment, not system-wide installations.

Deactivating Virtual Environments

Return to the system Python:

deactivate

The environment name disappears from the prompt, and pip commands now affect the system Python again.

virtualenv: The Enhanced Alternative

virtualenv preceded venv and offers additional features despite requiring separate installation. It supports older Python versions (2.7+) and provides faster environment creation through optimization.

Installing virtualenv

pip install virtualenv

Creating and Using virtualenv Environments

virtualenv myenv

Activation works identically to venv. virtualenv adds useful options:

Specify Python version:

virtualenv -p python3.10 myenv

Create without pip (install separately later):

virtualenv --no-pip myenv

Copy system site-packages (access globally installed packages):

virtualenv --system-site-packages myenv

venv vs virtualenv

Featurevenvvirtualenv
InstallationBuilt-in (Python 3.3+)Requires separate installation
Python 2 supportNoYes
Creation speedStandardFaster through caching
Advanced optionsLimitedExtensive
MaintenancePython core teamCommunity project
Use caseModern Python 3 projectsLegacy systems, advanced needs

For new Python 3 projects, venv suffices. Use virtualenv when working with Python 2, requiring faster environment creation, or needing advanced configuration options.

Conda Environments: The Scientific Computing Standard

Conda manages both packages and environments, particularly popular in data science for handling complex dependencies like numerical libraries with C/C++ components.

Installing Conda

Download either:

  • Anaconda: Full distribution with 1,500+ pre-installed packages (3GB+)
  • Miniconda: Minimal installer with conda, Python, and essential packages (400MB)

Miniconda suits most developers as it installs packages on demand.

Creating Conda Environments

conda create -n myenv python=3.11

Create with specific packages:

conda create -n datascience python=3.11 pandas numpy scikit-learn jupyter

Activating and Deactivating Conda Environments

conda activate myenv
conda deactivate

Managing Conda Packages

Install from conda repositories:

conda install tensorflow

Conda handles complex dependencies automatically, compiling packages with optimized libraries for better performance than pip equivalents in many cases.

List installed packages:

conda list

Exporting and Cloning Conda Environments

Export environment specification:

conda env export > environment.yml

Recreate environment on another machine:

conda env create -f environment.yml

Clone existing environment:

conda create --name newenv --clone myenv

Conda vs pip

Conda environments can use both conda and pip, but mixing them requires caution:

conda install pandas
pip install custom-package

Install conda packages first, then pip packages to avoid dependency resolution conflicts.

Comprehensive Comparison: Modern Python Environment Tools

Featurevenvvirtualenvcondapipenvpoetry
InstallationBuilt-inpip installSeparate installerpip installpip install
Python versionSystem versionAny installed versionAny version (downloads)System versionSystem version
Package sourcePyPIPyPIConda repos + PyPIPyPIPyPI
Dependency resolutionBasic (pip)Basic (pip)AdvancedAdvancedAdvanced
Lock filesNoNoenvironment.ymlPipfile.lockpoetry.lock
Binary packagesLimitedLimitedExtensiveLimitedLimited
Scientific computingManual setupManual setupOptimizedManual setupManual setup
SpeedFastFastestSlowerModerateModerate
Learning curveLowLowModerateModerateModerate
Best forSimple projectsLegacy PythonData scienceModern appsPublishing packages

Choose venv for straightforward Python 3 projects without complex requirements.

Choose conda for data science, machine learning, or projects with binary dependencies requiring compiled libraries.

Choose poetry when publishing packages to PyPI or requiring strict dependency management.

Choose pipenv for web applications needing reproducible environments with automatic virtual environment management.

Managing Dependencies with requirements.txt

requirements.txt files specify exact package versions for environment reproduction.

Creating requirements.txt

Freeze current environment packages:

pip freeze > requirements.txt

This generates:

pandas==2.1.4
numpy==1.26.2
matplotlib==3.8.2

Installing from requirements.txt

Create a fresh environment:

python -m venv newenv
source newenv/bin/activate  # or appropriate activation for your OS
pip install -r requirements.txt

This installs the exact versions specified, ensuring consistency across environments.

Best Practices for requirements.txt

Pin versions for production:

Django==4.2.7
psycopg2-binary==2.9.9

Use version ranges for development flexibility:

Django>=4.2,<5.0
pytest>=7.0

Separate development and production dependencies:

requirements.txt (production):

Django==4.2.7
gunicorn==21.2.0

requirements-dev.txt (development):

-r requirements.txt
pytest==7.4.3
black==23.12.0

Install development requirements:

pip install -r requirements-dev.txt

Add comments for clarity:

# Web framework
Django==4.2.7

# Database adapter
psycopg2-binary==2.9.9

# Development tools
pytest==7.4.3  # Testing framework

Modern Python Packaging with pyproject.toml

PEP 518 introduced pyproject.toml as a standardized configuration file replacing setup.py and other configuration files.

Basic pyproject.toml Structure

[build-system]
requires = ["setuptools>=61.0"]
build-backend = "setuptools.build_meta"
 
[project]
name = "myproject"
version = "1.0.0"
description = "A sample Python project"
readme = "README.md"
requires-python = ">=3.8"
dependencies = [
    "pandas>=2.0.0",
    "numpy>=1.24.0",
]
 
[project.optional-dependencies]
dev = [
    "pytest>=7.0",
    "black>=23.0",
]

Using pyproject.toml with pip

Install project dependencies:

pip install -e .

Install with development dependencies:

pip install -e ".[dev]"

Poetry Workflow

Poetry uses pyproject.toml for complete dependency management:

poetry new myproject
cd myproject
poetry add pandas numpy
poetry add --group dev pytest black

Poetry automatically creates and manages virtual environments:

poetry install  # Creates venv and installs dependencies
poetry shell    # Activates environment

Virtual Environments in IDEs

VS Code Configuration

VS Code automatically detects virtual environments in standard locations (.venv, venv, env).

Select Python interpreter:

  1. Press Ctrl+Shift+P (Windows/Linux) or Cmd+Shift+P (macOS)
  2. Type "Python: Select Interpreter"
  3. Choose your virtual environment

Automatic activation: VS Code activates the selected environment when opening integrated terminals.

Configure in .vscode/settings.json:

{
    "python.defaultInterpreterPath": "${workspaceFolder}/.venv/bin/python",
    "python.terminal.activateEnvironment": true
}

PyCharm Configuration

PyCharm provides comprehensive virtual environment integration:

Create virtual environment:

  1. File → Settings → Project → Python Interpreter
  2. Click gear icon → Add
  3. Select "Virtualenv Environment"
  4. Choose new or existing environment

Auto-activation: PyCharm automatically activates the project interpreter for terminals and run configurations.

Jupyter Notebooks and Virtual Environments

Install ipykernel in your virtual environment:

source myenv/bin/activate
pip install ipykernel
python -m ipykernel install --user --name=myenv

This makes the environment available as a kernel in Jupyter:

jupyter notebook

Select "myenv" from the kernel menu.

For data scientists working extensively in Jupyter, RunCell (opens in a new tab) provides an AI-powered Jupyter environment that automatically manages virtual environments and dependencies, streamlining the workflow without manual environment configuration.

Common Virtual Environment Issues

Wrong Python Version

Problem: Virtual environment uses incorrect Python version.

Solution: Specify the Python interpreter explicitly:

python3.11 -m venv myenv

Or with virtualenv:

virtualenv -p /usr/bin/python3.11 myenv

Activation Script Not Found

Problem: "activate: No such file or directory"

Solution: Verify environment creation completed successfully. Recreate if necessary:

rm -rf myenv
python -m venv myenv

PowerShell Execution Policy Error

Problem: Windows PowerShell blocks activation scripts.

Solution: Allow script execution for the current user:

Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser

Environment Not Recognized in PATH

Problem: Activated environment still uses system Python.

Solution: Verify activation by checking Python location:

which python  # Linux/macOS
where python  # Windows

Should point to the virtual environment directory. If not, deactivate and reactivate:

deactivate
source myenv/bin/activate

Package Installation Fails

Problem: pip install errors in virtual environment.

Solution: Upgrade pip, setuptools, and wheel:

pip install --upgrade pip setuptools wheel

Conda Environment Conflicts

Problem: Mixing conda and pip causes dependency issues.

Solution: Install conda packages first, then pip packages:

conda install numpy pandas
pip install custom-library

Or use conda exclusively when possible:

conda install -c conda-forge package-name

Best Practices for Virtual Environments

Naming Conventions

Use descriptive, consistent names:

  • .venv - Standard hidden directory for project-specific environments
  • venv - Common alternative
  • myproject-env - Descriptive for multiple projects

Avoid generic names like "env" or "python" that create confusion.

.gitignore Configuration

Always exclude virtual environments from version control:

# Virtual environments
venv/
.venv/
env/
ENV/
myenv/

# Conda environments
conda-env/

# Virtual environment markers
pyvenv.cfg

Project Structure

Organize projects with clear environment separation:

myproject/
├── .venv/                 # Virtual environment (not in git)
├── src/
│   └── myproject/
│       └── __init__.py
├── tests/
│   └── test_main.py
├── requirements.txt       # Production dependencies
├── requirements-dev.txt   # Development dependencies
├── pyproject.toml         # Project configuration
├── README.md
└── .gitignore

One Environment Per Project

Create a dedicated virtual environment for each project rather than sharing environments. This prevents dependency conflicts and simplifies debugging.

Document Environment Setup

Include setup instructions in README.md:

## Setup
 
1. Create virtual environment:

python -m venv .venv


2. Activate environment:
- Linux/macOS: `source .venv/bin/activate`
- Windows: `.venv\Scripts\activate`

3. Install dependencies:

pip install -r requirements.txt

Regular Dependency Updates

Update packages periodically while testing for compatibility:

pip list --outdated
pip install --upgrade package-name
pip freeze > requirements.txt

Use tools like pip-review for batch updates:

pip install pip-review
pip-review --local --interactive

Docker vs Virtual Environments

Docker containers and virtual environments solve different isolation levels:

Virtual environments isolate Python packages within the same operating system. They provide:

  • Lightweight resource usage
  • Fast creation and activation
  • Sufficient isolation for most development
  • Easy integration with local tools

Docker containers isolate entire application stacks including OS, system libraries, and services. They provide:

  • Complete environment reproducibility
  • Consistent behavior across development and production
  • Isolation of system-level dependencies
  • Support for multi-service applications

When to Use Each

Use virtual environments:

  • Local development and testing
  • Python-only projects without system dependencies
  • Quick prototyping and experimentation
  • Individual developer workflows

Use Docker:

  • Production deployments
  • Applications with system-level dependencies
  • Multi-service architectures (web app + database + cache)
  • Team-wide standardized environments
  • CI/CD pipelines

Combining Both

Many projects use virtual environments for development and Docker for deployment:

Dockerfile:

FROM python:3.11-slim
 
WORKDIR /app
 
COPY requirements.txt .
RUN python -m venv /opt/venv
ENV PATH="/opt/venv/bin:$PATH"
 
RUN pip install -r requirements.txt
 
COPY . .
 
CMD ["python", "app.py"]

This approach leverages virtual environments even within containers for cleaner dependency management.

Managing Multiple Python Versions with pyenv

pyenv simplifies installation and switching between Python versions, complementing virtual environment tools.

Installing pyenv

Linux/macOS:

curl https://pyenv.run | bash

Windows: Use pyenv-win:

Invoke-WebRequest -UseBasicParsing -Uri "https://raw.githubusercontent.com/pyenv-win/pyenv-win/master/pyenv-win/install-pyenv-win.ps1" -OutFile "./install-pyenv-win.ps1"; &"./install-pyenv-win.ps1"

Installing Python Versions

List available versions:

pyenv install --list

Install specific versions:

pyenv install 3.11.7
pyenv install 3.10.13

Setting Python Versions

Global (system-wide):

pyenv global 3.11.7

Local (project directory):

cd myproject
pyenv local 3.10.13

This creates .python-version file specifying the version for this directory.

Combining pyenv with Virtual Environments

pyenv local 3.11.7
python -m venv .venv
source .venv/bin/activate

Or use pyenv-virtualenv plugin:

pyenv virtualenv 3.11.7 myproject-env
pyenv activate myproject-env

This integrates Python version management with virtual environment creation in a unified workflow.

Advanced Virtual Environment Techniques

Environment Variables in Virtual Environments

Store configuration in environment-specific files:

.env:

DATABASE_URL=postgresql://localhost/mydb
SECRET_KEY=development-secret-key
DEBUG=True

Load with python-dotenv:

pip install python-dotenv
from dotenv import load_dotenv
import os
 
load_load_dotenv()
database_url = os.getenv('DATABASE_URL')

Pre and Post Activation Scripts

Customize environment activation behavior by modifying activation scripts.

Linux/macOS - Edit myenv/bin/activate:

Add before final line:

export DATABASE_URL="postgresql://localhost/mydb"
export FLASK_ENV="development"

Windows - Create myenv/Scripts/activate.bat:

Add:

set DATABASE_URL=postgresql://localhost/mydb
set FLASK_ENV=development

Sharing Virtual Environments Across Projects

While generally discouraged, shared environments work for related projects with identical dependencies:

python -m venv ~/shared-envs/data-science
source ~/shared-envs/data-science/bin/activate

Activate this environment in multiple project directories. However, this approach risks breaking all projects if dependencies conflict.

FAQ

Conclusion

Python virtual environments form the foundation of professional Python development, preventing dependency conflicts and enabling reproducible project setups. The built-in venv module handles most use cases efficiently, while conda excels for data science projects with complex binary dependencies. Modern tools like poetry and pipenv add advanced dependency resolution for projects requiring strict version control.

Successful virtual environment management follows clear patterns: one environment per project, comprehensive .gitignore rules, and documented setup procedures. Combined with pyenv for Python version management and IDE integration for streamlined workflows, virtual environments transform chaotic dependency management into systematic, predictable processes.

For data scientists working in Jupyter notebooks, RunCell (opens in a new tab) provides an AI-powered environment that automatically handles virtual environment complexity, allowing focus on analysis rather than configuration. Whether using venv for simple projects or conda for scientific computing, virtual environments remain essential tools that every Python developer must master.

Start your next project with a clean virtual environment and document the setup process. Your future self and teammates will appreciate the reproducibility and isolation that prevent hours of debugging mysterious dependency issues.

📚