Managing Python Dependencies: Navigating pip, pipenv, poetry, conda and more¶

Introduction¶

In the realm of Python development, a crucial aspect is managing project dependencies effectively.

This guide delves into four prominent tools—pip, pipenv, poetry, conda and more—each offering distinct approaches to dependency management. Grasping their strengths, weaknesses, and use cases empowers you to make informed decisions for your projects.

Key Considerations¶

Choosing the Right Tool¶

Scope and Size: Consider project scale and complexity.
Team Collaboration: Assess the extent of teamwork involved.
Environment Isolation: Determine the need for isolated environments.
Dependency Management: Evaluate desired features like dependency locking and version conflict resolution.
Project Requirements: Factor in specific project needs or constraints.

Tools Overview¶

pip pipenv poetry conda uv

Core Package Installer: The foundation for Python package management.
Global Installations: Installs packages system-wide by default.
Virtual Environments: Can be used within virtual environments for isolation.
Basic Dependency Management: Relies on requirements.txt for specifying dependencies.

Virtual Environment Creation and Management: Automatically creates and manages virtual environments.
Dependency Locking: Locks dependency versions for reproducibility.
Integration with Pipfile: Uses Pipfile and Pipfile.lock for dependency management.

Dependency Management and Packaging: Comprehensive dependency management and packaging tool.
Virtual Environment Handling: Manages virtual environments effectively.
Dependency Locking and Version Handling: Offers robust dependency locking and version conflict resolution.
Declarative Syntax: Uses pyproject.toml for configuration.

Cross-Platform Package and Environment Management: Manages packages and environments across multiple languages (Python, R, etc.).
Large Ecosystem of Packages: Accesses a vast repository of packages through Anaconda repositories.
Environment Isolation: Creates isolated environments for project-specific dependencies.
Non-Python Dependencies: Handles non-Python dependencies as well.

Extremely Fast Installation: Faster than traditional pip and pip-tools.
Drop-in Replacement: Provides a familiar interface for common pip commands.
Supports Advanced Features: Handles editable installs, Git dependencies, URL dependencies, etc.
Dependency Management: Offers features like dependency overrides and conflict resolution.
Platform-specific Requirements File Generation: Generates requirements files tailored to specific platforms.
Resolution Strategy Customization: Allows customization of resolution strategies for dependencies.

Comparison Table¶

Feature	pip	pipenv	poetry	conda	uv
Installation	Built-in	`pip install pipenv`	`pip install poetry`	Download and install Anaconda or Miniconda	`pip install uv`
Environments	Implicit	Automatic	Automatic	Automatic	Implicit
Locking	Manual	Automatic	Automatic	Automatic	Manual
Configuration	requirements.txt	Pipfile, Pipfile.lock	pyproject.toml	Environment files	N/A
Official packages	PyPI (Python Package Index)	PyPI	PyPI	Conda Forge (conda-forge channel)	PyPI (Python Package Index)
Multi-python-version Resolution	No	No	Yes	No	Yes

Common Commands¶

Feature	pip	pipenv	poetry	conda	uv
Create a project	`python -m venv <env-name>`	`pipenv --python <python-version>`	`poetry init`	`conda create -n <env-name> python=<python-version>`	`uv init .` (creates virtualenv at: `.venv/`)
Use a virtual environment	`source <env-name>/bin/activate` (linux) or `<env-name>/Scripts/activate` (windows)	`pipenv shell`	`poetry shell`	`conda activate <env-name>`	Same as pip
Install a package	`pip install <package-name>`	`pipenv install <package-name>`	`poetry add <package-name>`	`conda install <package-name>`	`uv add <package-name>`
Remove a package	`pip uninstall <package-name>`	`pipenv uninstall <package-name>`	`poetry remove <package-name>`	`conda uninstall <package-name>`	`uv remove <package-name>`
Install from requirements	`pip install -r requirements.txt`	`pipenv install` (reads from Pipfile)	`poetry install` (reads from pyproject.toml)	`conda install --file requirements.txt`	`uv sync requirements.txt` or `uv pip install -r requirements.txt`
Dev packages vs others	No explicit distinction	`--dev` flag for development dependencies	`[tool.poetry.dev-dependencies]` section in pyproject.toml	No explicit distinction	`--dev` flag for development dependencies
List requirements	`pip freeze > requirements.txt` or use `pipreqs`	`pipenv lock -r > requirements.txt`	`poetry export -f requirements.txt > requirements.txt`	`conda list --export > requirements.txt`	Same as pip (e.g., `uv pip freeze > requirements.txt`)
Transport project	Manually copy files or create a setup.py	Copy Pipfile and Pipfile.lock	Copy pyproject.toml and pyproject.toml.lock	Export environment to .yml file (conda env export > environment.yml)	Not directly supported (requires rebuilding with UV)
Install from lock file	N/A	`pipenv install --ignore-pipfile`	`poetry install --no-dev` (reads from pyproject.toml.lock)	`conda env update --file environment.yml`	N/A (Dependency resolution handled by UV directly)
Delete the venv	Remove directory manually	`pipenv --rm`	`poetry env remove <env-name>`	`conda remove --name <env-name> --all`	N/A
Update a package to its latest version	`pip install <package-name> --upgrade`	`pipenv update <package-name>`	`poetry update <package-name>`	`conda update <package-name>`	`uv update <package-name>`
Update all packages to their latest versions	N/A	`pipenv update`	`poetry update`	`conda update -n <env-name> --all` (be aware of potential conflicts)	N/A

Remember !

As you wrap up your exploration of Python's dependency management tools, remember:

Consistency: Maintain a uniform approach within a project for easier maintenance.
Descriptive Naming: Use clear and informative names for environments and dependencies.
Thorough Testing: Ensure compatibility and functionality after dependency changes.
Regular Updates: Keep tools and dependencies up-to-date for security and performance.

More on UV dependencies resolution

Generate Requirements from pyproject.toml:

uv pip compile pyproject.toml -o requirements.txt

Platform-specific Requirements Generation:

uv pip compile requirements.in -o requirements.txt

Command Line Compatibility: uv's install (uv pip install) and compile (uv pip compile) commands support many familiar command-line arguments, such as -r requirements.txt, -c constraints.txt, -e . (for editable installs), --index-url, and more.
Resolution Strategy Customization: uv allows customization of its resolution strategy. With --resolution=lowest, uv pip installs the lowest compatible versions for all dependencies, while --resolution=lowest-direct focuses on lowest compatible versions for direct dependencies and latest compatible versions for transitive dependencies.

read this docu about how to add dependencies from a requirements.txt file

How to use two conda environment at the same time without breaking anything ?

To simultaneously utilize two Conda environments without causing conflicts, you can leverage Conda's nested activation feature. Here's how you can achieve this:

conda activate old_project_env
conda activate --stack new_project_env

This sequence of commands allows you to activate a parent environment (old_project_env) while retaining access to the command-line options from the old environment. Then, by using conda activate --stack new_project_env, you can activate a new environment (new_project_env) within the context of the parent environment.

This approach ensures that you can work within the specified environment (new_project_env) without interfering with the configurations or dependencies of the parent environment (old_project_env).

For more information on Conda environments and nested activation, you can refer to the official Conda documentation.

Conclusion¶

Congratulations! You've now equipped yourself with insights into managing Python dependencies effectively using pip, pipenv, poetry, conda and more. By understanding their strengths and use cases, you're better equipped to make informed decisions for your projects.

Stay proactive in exploring and adapting these tools to optimize your Python development experience!

Relevant Sources¶

Is it safe to update all conda package at once ?