Handling Externally Managed Environment Error With Pip in Ubuntu

Seeing an "externally managed environment" error while using Pip in Ubuntu 23.04? Here's what you can do about this error.
Warp Terminal

I recently upgraded to Ubuntu 23.04. Things are mostly smooth. However, recently I encountered an issue that was not present in the previous version.

I was trying to install a package using Pip, a command line based Python package manager. It works great usually but it threw an error this time:

error: externally-managed-environment

externally-managed-environment error in Ubuntu
πŸ™‹
Why do you see this error?

Linux distributions want you to avoid installing Python packages system wide. You should install them in Python virtual environments. This can be achieved rather easily using pipx tool instead of pip.

Here's the complete error message if you want to read it:

Γ— This environment is externally managed
╰─> To install Python packages system-wide, try apt install
    python3-xyz, where xyz is the package you are trying to
    install.
    
    If you wish to install a non-Debian-packaged Python package,
    create a virtual environment using python3 -m venv path/to/venv.
    Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
    sure you have python3-full installed.
    
    If you wish to install a non-Debian packaged Python application,
    it may be easiest to use pipx install xyz, which will manage a
    virtual environment for you. Make sure you have pipx installed.
    
    See /usr/share/doc/python3.11/README.venv for more information.

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.

And if you read the details, you'll understand why you see this error.

Reason behind the 'Externally Managed Environment' Error

Ubuntu 23.04, Fedora 38 and probably other recent distribution versions are implementing this enhancement on the use of Python packages.

The change has been done to avoid the "conflict between OS package managers and Python-specific package management tools like pip. These conflicts include both Python-level API incompatibilities and conflicts over file ownership."

More details can be found on this page.

PEP 668 – Marking Python base environments as β€œexternally managed” | peps.python.org
Python Enhancement Proposals (PEPs)

What are your options?

You can do three things when you try to install a Python package and see this error.

  1. Install the native package
  2. Create virtual environments in Python
  3. Use Pipx (recommended)

Before you see those methods, let me share a quick and dirty hack shared by an It's FOSS reader.

βœ‹
You could also just delete the element responsible for this warning. If you are comfortable with the command line, just delete the file /usr/lib/python3.x/EXTERNALLY-MANAGED. You have to replace x with the python version(s), of course. 

Option 1: Go for native package

I understand that Pip gives a comfortable way of installing Python packages. However, some Python applications are also packaged as APT or other native packages. Search for it in your distribution's repositories and install it from there if it is available.

For example, I was trying to install WoeUSB-ng. If I was using Arch Linux, the same package is available from AUR.

Option 2: Use Python virtual environment

If you must use a Python package, you have to isolate it in Python virtual environment.

With the help of virtual environments, you can use different versions of package dependencies and Python. This way, you avoid any conflicts between the packages.

πŸ“‹
This method is suitable for software developers and programmers working on Python projects.

Let's quickly see how you do it. Usually, Python should already have the tool for creating virtual environments.

Use the command below to create a virtual environment for your project. Replace project_name with your project's name, of course.

python3 -m venv .venv/project_name

If you see venv errors related, you may have to install it.

sudo apt install python3-venv

Now, you'll see a directory named .env in your home directory and inside the .env, you'll have the project directory.

Here's the exciting part. Each project directory will have its own copy of Python and Pip in it.

Virtual environment in Python

That's your virtual Python environment. You can use this 'local binary' for installing Python packages using Pip inside this virtual environment like this:

.venv/project_name/bin/pip install package_name
Installing Python packages using Pip in virtual environment

Remember that the installed Python package won't be available system wide.

This was only a brief example of virtual environments in Python. Here's a detailed guide if you want to learn more on it.

Python Virtual Environments: A Primer – Real Python
In this tutorial, you’ll learn how to use a Python virtual environment to manage your Python projects. You’ll also dive deep into the structure of virtual environments built using the venv module, as well as the reasoning behind using virtual environments.

What you saw above involves manual work. Pipx automates it.

It automatically creates a new virtual environment for each app you install. Not only that. It also creates a link to it in .local/bin. This way, the user who installed the package can run it from anywhere in the command line.

I guess that's what most desktop Linux users want here.

Install pipx on Ubuntu using this command:

sudo apt install pipx

It will probably install a huge number of dependencies:

Install pipx

Now add it to the PATH so that you can run from anywhere.

pipx ensurepath
Add pipx to PATH
βœ‹
You must close the terminal and log back in for the changes to occur.

Great! Now you can install Python packages using Pipx instead of Pip:

pipx install package_name

Here's an example.

Installing a package with Pipx
πŸ’‘
To remove a package installed with pipx, use pipx uninstall package_name command.
Install and Use pipx in Ubuntu & Other Linux
Pipx addresses the shortcomings of the popular pip tool. Learn to install and use Pipx in Linux.

Conclusion

Pip is a good tool for getting Python packages on the system. I think it was always meant for Python programmers, not for the end users. Clearly, it cannot be used as a replacement for the native distribution packages and the Python devs have made it clear.

The good thing is that alternatives exist for both programmers and end users.

I hope this tutorial helped you understand and overcome the externally-managed-environment error with Pip in Linux.

Please let me know if you have questions or suggestions.

About the author
Abhishek Prakash

Abhishek Prakash

Created It's FOSS 11 years ago to share my Linux adventures. Have a Master's degree in Engineering and years of IT industry experience. Huge fan of Agatha Christie detective mysteries πŸ•΅οΈβ€β™‚οΈ

Become a Better Linux User

With the FOSS Weekly Newsletter, you learn useful Linux tips, discover applications, explore new distros and stay updated with the latest from Linux world

itsfoss happy penguin

Great! You’ve successfully signed up.

Welcome back! You've successfully signed in.

You've successfully subscribed to It's FOSS.

Success! Check your email for magic link to sign-in.

Success! Your billing info has been updated.

Your billing was not updated.