Arthur Koziel

Working with virtualenv

Published October 22, 2008

Virtualenv is a tool to build isolated Python environments. It's a great way to quickly test new libraries without cluttering your global site-packages or run multiple projects on the same machine which depend on a particular library but not the same version of the library.

You can, for example, install version X of a library in one environment and version Z of the same library in another environment, without one interfering the other. Each environment provides its own Python executable and site-packages directory (which is not shared between environments).

To install virtualenv, simply easy_install it with the following command:

easy_install virtualenv

Once virtualenv is installed, you can use the virtualenv command to create virtual environments. The following command will create an environment called "foobar":

virtualenv foobar

You can activate the environment by sourcing its activate script, which is located in the environment's bin/ directory:

source foobar/bin/activate

The script prepends the name of the currently activated environment to the $PS1 of your shell and sets the environment's site-packages directory as the default one.

If you install a package in your virtual environment, you'll see that executable scripts are placed in foobar/bin/ and eggs in foobar/lib/python2.X/site-packages/:

easy_install yolk

Yolk is a small command line tool which can, among other things, list the currently installed Python packages on your system:

yolk -l

As you might see, virtualenv will inherit the packages from the system's default site-packages directory. This is especially useful when relying on certain packages being available, so you don't have to go through installing them in every environment.

I try to keep my system's site-packages directory fairly minimal and only put packages there which I use in nearly every project. It currently consists of database interfaces like MySQLdb or psycopg2 as well as setuptools and, of course, virtualenv.

However, you can also change this by passing the --no-site-packages option to virtualenv when creating a virtual environment. This will create a clean environment without inheriting the packages from your global site-packages directory:

virtualenv --no-site-packages foobar

To leave an environment, simply run deactivate:

deactivate

Try executing the yolk command now: it won't work because the package was installed only in your virtual environment. Once you reactivate your environment it will be available again.

Virtualenvwrapper

Virtualenvwrapper is a set of bash functions created by Doug Hellmann to keep virtual environments manageable. It expects your environments to be in one directory and provides useful commands to create, remove and switch between environments from anywhere in your cli.

The installation is easy since it's just a bash script which needs to be sourced, but don't forget to set the WORKON_HOME variable before sourcing the script in your .bash_profile or .bash_rc:

WORKON_HOME=$HOME/.virtualenvs
source /path/to/virtualenvwrapper_bashrc

The mkvirtualenv command will create an environment and the rmvirtualenv command will delete an environment. You can activate or switch to an environment by using the workon command.

Bootstrapping environments

Virtualenv has a few hooks which can be used by a custom bootstrap script. This can be useful to automatically install a few additional packages after creating a virtual environment.

If you're doing a lot of Django projects for example, you might want to create a bootstrap script which creates a virtual environment and installs Django for you—it probably saves you some time and avoids boring repetitive tasks. Or you can distribute the script among the developers of your project, which then always use the script to set up the environment. This would not only make sure that everyone has the same libraries installed but would also allow a quick set up of a development/production environment.

In this small example, the bootstrap script will create a virtual environment and easy_install yolk for you:

import virtualenv, textwrap
output = virtualenv.create_bootstrap_script(textwrap.dedent("""
def after_install(options, home_dir):
    subprocess.call([join(home_dir, 'bin', 'easy_install'), 
                    'yolk'])
"""))
print output

You basically pass a string with the additional Python code to the create_bootstrap_script function of virtualenv. It will then return the modified virtualenv script with your additional code inserted. You can then use the modified script to create virtual environments just like the original one.

To test the above bootstrap script, paste the code into a file, run it and save the output to a new file:

python main.py > venv-yolk.py

The venv-yolk.py file can now be used just like the original virtualenv script. In fact, it is the virtualenv script with the above after_install function added:

python venv-yolk.py foobar