Activate a virtualenv via fabric as deploy user

I’m just using a simple wrapper function virtualenv() that can be called instead of run(). It doesn’t use the cd context manager, so relative paths can be used.

def virtualenv(command):
    """
    Run a command in the virtualenv. This prefixes the command with the source
    command.
    Usage:
        virtualenv('pip install django')
    """
    source = 'source %(project_directory)s/bin/activate && ' % env
    run(source + command)

As an update to bitprophet’s forecast: With Fabric 1.0 you can make use of prefix() and your own context managers.

from __future__ import with_statement
from fabric.api import *
from contextlib import contextmanager as _contextmanager

env.hosts = ['servername']
env.user = 'deploy'
env.keyfile = ['$HOME/.ssh/deploy_rsa']
env.directory = '/path/to/virtualenvs/project'
env.activate = 'source /path/to/virtualenvs/project/bin/activate'

@_contextmanager
def virtualenv():
    with cd(env.directory):
        with prefix(env.activate):
            yield

def deploy():
    with virtualenv():
        run('pip freeze')

virtualenvwrapper can make this a little simpler

1. Using @nh2’s approach (this approach also works when using local, but only for virtualenvwrapper installations where workon is in $PATH, in other words — Windows)

   from contextlib import contextmanager
   from fabric.api import prefix
   
   @contextmanager
   def virtualenv():
       with prefix("workon env1"):
           yield
   
   def deploy():
       with virtualenv():
           run("pip freeze > requirements.txt")
   

2. Or deploy your fab file and run this locally. This setup lets you activate the virtualenv for local or remote commands. This approach is powerful because it works around local‘s inability to run .bashrc using bash -l:

   @contextmanager
   def local_prefix(shell, prefix):
       def local_call(command):
           return local("%(sh)s "%(pre)s && %(cmd)s"" % 
               {"sh": shell, "pre": prefix, "cmd": command})
       yield local_prefix
   
   def write_requirements(shell="/bin/bash -lic", env="env1"):
       with local_prefix(shell, "workon %s" % env) as local:
           local("pip freeze > requirements.txt")
   
   write_requirements()  # locally
   run("fab write_requirements")
   

This is my approach on using virtualenv with local deployments.

Using fabric’s path() context manager you can run pip or python with binaries from virtualenv.

from fabric.api import lcd, local, path

project_dir = '/www/my_project/sms/'
env_bin_dir = project_dir + '../env/bin/'

def deploy():
    with lcd(project_dir):
        local('git pull origin')
        local('git checkout -f')
        with path(env_bin_dir, behavior='prepend'):
            local('pip freeze')
            local('pip install -r requirements/staging.txt')
            local('./manage.py migrate') # Django related

            # Note: previous line is the same as:
            local('python manage.py migrate')

            # Using next line, you can make sure that python 
            # from virtualenv directory is used:
            local('which python')

Here is code for a decorator that will result in the use of Virtual Environment for any run/sudo calls:

# This is the bash code to update the $PATH as activate does
UPDATE_PYTHON_PATH = r'PATH="{}:$PATH"'.format(VIRTUAL_ENV_BIN_DIR)

def with_venv(func, *args, **kwargs):
  "Use Virtual Environment for the command"

  def wrapped(*args, **kwargs):
    with prefix(UPDATE_PYTHON_PATH):
      return func(*args, **kwargs)

  wrapped.__name__ = func.__name__
  wrapped.__doc__ = func.__doc__
  return wrapped

and then to use the decorator, note the order of the decorators is important:

@task
@with_venv
def which_python():
  "Gets which python is being used"
  run("which python")

Thanks to all answers posted and I would like to add one more alternative for this. There is an module, fabric-virtualenv, which can provide the function as the same code:

>>> from fabvenv import virtualenv
>>> with virtualenv('/home/me/venv/'):
...     run('python foo')

fabric-virtualenv makes use of fabric.context_managers.prefix, which might be a good way 🙂

If you want to install the packages to environment or want to run commands according to the packages you have in environment, I have found this hack to solve my problem, instead of writing complex methods of fabric or installing new OS packages:

/path/to/virtualenv/bin/python manage.py migrate/runserver/makemigrations  # for running commands under virtualenv

local("/home/user/env/bin/python manage.py migrate")    # fabric command


/path/to/virtualenv/bin/pip install -r requirements.txt   # installing/upgrading virtualenv

local("/home/user/env/bin/pip install -r requirements.txt")  #  fabric command

This way you might not need to activate the environment, but you can execute commands under the environment.

This approach worked for me, you can apply this too.

from fabric.api import run 
# ... other code...
def install_pip_requirements():
    run("/bin/bash -l -c 'source venv/bin/activate' "
        "&& pip install -r requirements.txt "
        "&& /bin/bash -l -c 'deactivate'")

Assuming venv is your virtual env directory and add this method wherever appropriate.

I use pyenv with plugin pyenv-virtualenvwrapper. I had no success with workon, instead I use this (fabric 2.5):

with c.prefix('source /home/mirek/.virtualenvs/%s/bin/activate' % PROJECT):
    with c.prefix('cd /home/mirek/%s/%s' % (PROJECT, PROJECT)):
        c.run('python manage.py ....')

For git pull agent forwarding is good, ie. ssh -A .. or better in ~/.ssh/config something like this:

Host forpsi
    HostName xx.xx.xx.xx
    IdentityFile /home/mirek/.ssh/id_ed25519_xxx
    ForwardAgent yes

And now on the development machine if you have a private key in the agent (after ssh-add or if you have AddKeysToAgent yes in ~/.ssh/config after you made git push) then git pull should not ask for the passphrase of the key.