Why using pyenv?
System Python
By default python comes pre-installed within your operating system.
If you are a Mac or Linux user, you can see the “System Python” that comes installed on your operating system:
zsh> which python
/usr/bin/python
Note: this version of python is available to all users (as reflected by its location).
However, this might not be the version you need:
zsh> /usr/local/bin/python3 --version
Python 3.6.8
Another problem is that by running sudo pip install <your-package>, you will be installing the Python package globally. What about if another needs another version of the package e.g. a slightly older version of the package or if two projects requires two different versions because of breaking changes introduced in the newer version?
Last but not the least, some operating system relies heavily on Python to perform operations. Installing a new version of Python could seriously dampen your ability to use your OS.
Pyenv
The logical place to look for to solve all the problems inherent to System Python is pyenv.
Pyenv is a great tool for managing multiple Python versions that can coexists simultaneously on your OS. You can then easily switch between the installed versions and use virtual environments to manage Python packages associated with each Python versions.
Installation
You need to install the following dependencies:
brew install openssl readline sqlite3 xz zlib
Add them within the PATH (macOS):
echo 'export PATH="/usr/local/opt/openssl@3/bin:$PATH"' >> ~/.zshrc
echo 'export LDFLAGS="-L/usr/local/opt/openssl@3/lib"' >> ~/.zshrc
echo 'export CPPFLAGS="-I/usr/local/opt/openssl@3/include"' >> ~/.zshrc
echo 'export PKG_CONFIG_PATH="/usr/local/opt/openssl@3/lib/pkgconfig"' >> ~/.zshrc
echo 'export PATH="/usr/local/opt/sqlite/bin:$PATH"' >> ~/.zshrcc
echo 'export LDFLAGS="-L/usr/local/opt/sqlite/lib"' >> ~/.zshrc
echo 'export CPPFLAGS="-I/usr/local/opt/sqlite/include"' >> ~/.zshrc
echo 'export PKG_CONFIG_PATH="/usr/local/opt/sqlite/lib/pkgconfig"' >> ~/.zshrc
echo 'export LDFLAGS="-L/usr/local/opt/zlib/lib"' >> ~/.zshrc
echo 'export CPPFLAGS="-I/usr/local/opt/zlib/include"' >> ~/.zshrc
echo 'export PKG_CONFIG_PATH="/usr/local/opt/zlib/lib/pkgconfig"' >> ~/.zshrc
Note: Pyenv comes with a set of useful dependencies:
- pyenv: The actual pyenv application
- pyenv-virtualenv: Plugin for pyenv and virtual environments
- pyenv-update: Plugin for updating pyenv
- pyenv-doctor: Plugin to verify that pyenv and build dependencies are installed
- pyenv-which-ext: Plugin to automatically lookup system commands
Then, install pyenv using the pyenv-installer:
curl https://pyenv.run | bash
Restart the terminal for the PATH changes to be reflected:
exec $SHELL
Finally, check that everything did worked it:
zsh> pyenv -v
pyenv 2.3.19
Uninstall pyenv
On MacOS:
brew remove pyenv
Using pyenv
Install python versions
zsh> pyenv install --list
3.6.2
3.6.7
3.7.2
3.8.2
3.9.12
3.10.4
3.11-dev
3.11.4
All the installed version will be located in your pyenv root directory:
zsh> ls ~/.pyenv/versions/
3.10.6 3.11.4 3.6.15 3.6.8 3.6.9 3.8.16 3.8.17 3.9.9
Note: make sure to regularly pyenv update to have access to all the latest python versions.
Uninstall python versions
You can simply remove the versions from the pyenv root folder:
rm -rf ~/.pyenv/versions/3.10.6
or use the provided command:
pyenv uninstall 3.10.6
Switching between Python versions
You can see the python version you have installed:
zsh> pyenv versions
* system (set by /Users/johndoe/.pyenv/version)
3.6.8
3.6.9
3.6.15
3.8.16
3.8.17
3.9.9
3.10.6
3.11.4
Note: the * indicated which version of python is currently active. By default, it is system python. You can confirm is using the which command:
zsh> which python3
/Users/johndoe/.pyenv/shims/python3
pyenv insert itself into the PATH. From the OS’s perspective, pyenv is the executable getting called when you execute which python3. If you want to see the actual, you need to run the following:
zsh> pyenv which python3
/usr/local/bin/python3
zsh> /usr/local/bin/python3 -V
Python 3.6.8
To shift between different versions, you can simply run:
zsh> pyenv global 3.11.4
zsh> python -V
Python 3.11.4
zsh> which python
python: aliased to python3
zsh> pyenv which python
/Users/johndoe/.pyenv/versions/3.11.4/bin/python
zqh> pyenv versions
system
3.6.8
3.6.9
3.6.15
3.8.16
3.8.17
3.9.9
3.10.6
* 3.11.4 (set by /Users/johndoe/.pyenv/version)
shell vs. local vs. global vs. system
Use-cases
Let’s explore the different commands and their use-cases.
To ensure that this python version is gonna be used by default:
zsh> pyenv global 3.11.4
To set an application-specific python version:
zsh> pyenv local 3.11.4
The above command creates a .python-version file in the current directory. If pyenv is active in this an environment, the file will automatically activate this version.
To set a shell-specific python version:
zsh> pyenv shell 3.11.4
The above command activates the version specific by setting the `PYENV_VERSION“ environment variable. It overwrites any application or global setting you have made. To deactivate the version, you need to use the –unset flag:
zsh> echo $PYENV_VERSION
3.11.4
zsh> pyenv shell --unset
Resolution
The System Python is overwritten by pyenv global (~/.pyenv/version).
The pyenv global is overwritten by pyenv local (.python-version file).
The pyenv local is overwritten by pyenv shell ($PYENV_VERSION).
Thus, to determine which version of python to use, pyenv will first look for $PYENV_VERSION, then .python-version then ~/.pyenv/version before finally settling down on the Python System if none of the above have been resolved.
Example
zsh> mkdir /tmp/test && cd /tmp/test
zsh> pyenv versions
* system (set by /Users/johndoe/.pyenv/version)
3.6.8
3.6.9
3.6.15
3.8.16
3.8.17
3.9.9
3.10.6
3.11.4
zsh> python -V
Python 3.6.8
zsh> pyenv local 3.8.16
zsh> ls -a
. .. .python-version
zsh> .python-version
Python 3.8.16
zsh> python -V
Python 3.8.16
zsh> python shell 3.9.9
zsh> echo $PYENV_VERSION
3.9.9
zsh> python -V
Python 3.9.9
And the other way around you can coax it out, layer by layer:
zsh> pyenv shell --unset
zsh> echo $PYENV_VERSION
zsh> python -V
Python 3.8.16
zsh> rm .python-version
zsh> python -V
Python 3.6.8
zsh> pyenv versions
* system (set by /Users/johndoe/.pyenv/version)
3.6.8
3.6.9
3.6.15
3.8.16
3.8.17
3.9.9
3.10.6
3.11.4
Virtual environments and pyenv
To quote this realpython.com article, virtual environments and pyenv are a match made in heaven. Whether you use virtualenv or venv, pyenv plays nicely with either.
You can create virtual environment using the following template:
pyenv virtualenv <python_version> <environment_name>
You can activate your environment running the following:
pyenv local <environment_name>
You can also do it manually:
zsh> pyenv activate <environment_name>
zsh> pyenv deactivate