By John-Scott


2010-07-20 04:33:15 8 Comments

I am trying to run a Django management command from cron. I am using virtualenv to keep my project sandboxed.

I have seen examples here and elsewhere that show running management commands from within virtualenv's like:

0 3 * * * source /home/user/project/env/bin/activate && /home/user/project/manage.py command arg

However, even though syslog shows an entry when the task should have started, this task never actually runs (the log file for the script is empty). If I run the line manually from the shell, it works as expected.

The only way I can currently get the command to run via cron, is to break the commands up and put them in a dumb bash wrapper script:

#!/bin/sh
source /home/user/project/env/bin/activate
cd /home/user/project/
./manage.py command arg

EDIT:

ars came up with a working combination of commands:

0 3 * * * cd /home/user/project && /home/user/project/env/bin/python /home/user/project/manage.py command arg

At least in my case, invoking the activate script for the virtualenv did nothing. This works, so on with the show.

7 comments

@Dmitriy 2018-08-08 15:26:50

I'd like to add this because I spent some time solving the issue and did not find an answer here for combination of variables usage in cron and virtualenv. So maybe it'll help someone.

PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
DIR_SMTH="cd /smth"
VENV=". venv/bin/activate"
CMD="some_python_bin do_something"
# m h  dom mon dow   command
0 * * * * $DIR_SMTH && $VENV && $CMD -k2 some_target >> /tmp/crontest.log 2>&1

It did not work well when it was configured like

DIR_SMTH="cd /smth && . venv/bin/activate"

Thanks @davidwinterbottom, @reed-sandberg and @mkb for giving the right direction. The accepted answer actually works fine until your python need to run a script which have to run another python binary from venv/bin directory.

@Basil Musa 2018-05-27 21:32:19

Don't look any further:

0 3 * * * /usr/bin/env bash -c 'cd /home/user/project && source /home/user/project/env/bin/activate && ./manage.py command arg' > /dev/null 2>&1

Generic approach:

* * * * * /usr/bin/env bash -c 'YOUR_COMMAND_HERE' > /dev/null 2>&1

The beauty about this is you DO NOT need to change the SHELL variable for crontab from sh to bash

@Ivanhoe 2014-03-28 23:02:35

The only correct way to run python cron jobs when using a virtualenv is to activate the environment and then execute the environment's python to run your code.

One way to do this is use virtualenv's activate_this in your python script, see: http://virtualenv.readthedocs.org/en/latest/userguide.html#using-virtualenv-without-bin-python

Another solution is echoing the complete command including activating the environment and piping it into /bin/bash. Consider this for your /etc/crontab:

***** root echo 'source /env/bin/activate; python /your/script' | /bin/bash

@Aaron Schumacher 2015-01-14 18:18:20

I'm very curious as to whether there's consensus that this is in fact the only correct way.

@Will 2015-04-24 13:37:00

This is probably the only correct way. But there are other ways that works.

@canucklesandwich 2015-05-09 19:23:40

This isn't "the only correct way." I've successfully executed a script in a virtualenv simply by pointing the cronjob to the virtualenv's python binary, such as '/home/user/folder/env/bin/python'. No need to activate the environment whatsoever.

@varela 2015-05-29 12:05:48

If you use custom PYTHONPATH in virtual environment env/bin/python will not work for you. That's why using of env/bin/activate is better

@user3850 2015-11-17 01:11:00

it depends on how you set the PYTHONPATH and if you set it in a way that requires "activating" the venv, you're doing it wrong

@here 2014-04-25 08:00:09

The best solution for me was to both

  • use the python binary in the venv bin/ directory
  • set the python path to include the venv modules directory.

man python mentions modifying the path in shell at $PYTHONPATH or in python with sys.path

Other answers mention ideas for doing this using the shell. From python, adding the following lines to my script allows me to successfully run it directly from cron.

import sys
sys.path.insert(0,'/path/to/venv/lib/python3.3/site-packages');

Here's how it looks in an interactive session --

Python 3.3.2+ (default, Feb 28 2014, 00:52:16) 
[GCC 4.8.1] on linux
Type "help", "copyright", "credits" or "license" for more information.

>>> import sys

>>> sys.path
['', '/usr/lib/python3.3', '/usr/lib/python3.3/plat-x86_64-linux-gnu', '/usr/lib/python3.3/lib-dynload']

>>> import requests
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named 'requests'   

>>> sys.path.insert(0,'/path/to/venv/modules/');

>>> import requests
>>>

@joemaller 2012-12-12 04:42:01

Rather than mucking around with virtualenv-specific shebangs, just prepend PATH onto the crontab.

From an activated virtualenv, run these three commands and python scripts should just work:

$ echo "PATH=$PATH" > myserver.cron
$ crontab -l >> myserver.cron
$ crontab myserver.cron

The crontab's first line should now look like this:

PATH=/home/me/virtualenv/bin:/usr/bin:/bin:  # [etc...]

@Victor Schröder 2017-01-05 12:30:45

Not a good solution. Every python tasks in the crontab would then run with the binary from the virtualenv. Making that binary a pseudo-global python goes against the very purpose of virtualenv.

@ars 2010-07-20 04:40:58

You should be able to do this by using the python in your virtual environment:

/home/my/virtual/bin/python /home/my/project/manage.py command arg

EDIT: If your django project isn't in the PYTHONPATH, then you'll need to switch to the right directory:

cd /home/my/project && /home/my/virtual/bin/python ...

You can also try to log the failure from cron:

cd /home/my/project && /home/my/virtual/bin/python /home/my/project/manage.py > /tmp/cronlog.txt 2>&1

Another thing to try is to make the same change in your manage.py script at the very top:

#!/home/my/virtual/bin/python

@John-Scott 2010-07-20 22:47:50

That also does not work. Forgot to put that in my list of things that do not work. Yes, I can run that command manually in the shell but it does not work from cron.

@ars 2010-07-20 23:22:53

Did you replace ~ with the full path? (You probably did, just making sure ...)

@John-Scott 2010-07-21 02:10:13

Ah, you've come up with a working example! I've tried about every combination and activating the virtualenv appears to have no effect whatsoever. I do set my PYTHONPATH in .bashrc but this apparently is not used by cron? Will update my question to highlight your answer.

@ars 2010-07-21 03:23:35

Yeah, I'd forgotten that cron runs under a very minimal environment. The general recommendation is to write bash scripts to set up whatever environment your job will need. You could try sourcing the bash profile directly in cron, but this can lead to subtle bugs depending on what's in your profile (perhaps if you have a separate and minimal profile for such needs, it would be fine).

@Dick 2011-11-16 03:06:39

A good way to test is to execute /bin/sh, and then try to execute your command from there. At least you'll have the same environment setup as cron.

@webjunkie 2012-04-04 13:54:31

+1 for suggesting to log error :) thanks!

@Q Caron 2015-10-27 15:44:04

cron jobs may fail because your packages are not installed in your virtualenv. Make sure you are using your virtualenv's pip so installed packages can be reached with the first instruction of this solution. Maybe you will have to call the virtualenv's pip by using its absolute path when installing packages (stackoverflow.com/questions/20952797/…)

@Austin 2016-09-26 18:03:06

I also had to explicitly declare my settings since I have different ones for local and productions. --settings=path.to.settings

@DavidWinterbottom 2011-08-11 19:33:35

Running source from a cronfile won't work as cron uses /bin/sh as its default shell, which doesn't support source. You need to set the SHELL environment variable to be /bin/bash:

SHELL=/bin/bash
*/10 * * * * root source /path/to/virtualenv/bin/activate && /path/to/build/manage.py some_command > /dev/null

It's tricky to spot why this fails as /var/log/syslog doesn't log the error details. Best to alias yourself to root so you get emailed with any cron errors. Simply add yourself to /etc/aliases and run sendmail -bi.

More info here: http://codeinthehole.com/archives/43-Running-django-cronjobs-within-a-virtualenv.html

@Reed Sandberg 2014-01-23 22:01:54

Or '.' (dot command), which is supported by /bin/sh . /path/to/virtualenv/bin/activate

@joemurphy 2016-02-25 18:22:07

DavidWinterbottom, if that is your real name, you're my hero. I never knew that about sh vs bash and source files. You've shone a light into my little bash-scripting world dude. Thanks.

@dspacejs 2016-11-27 06:53:00

If you have a postactivate file, you should be doing source /path/to/virtualenv/bin/activate && source /path/to/virtualenv/bin/postactivate

@user12345 2017-11-07 20:22:59

@Martin Becker 2018-05-02 07:49:24

Thanks! For me, this works rather than the accepted answer by Gerald.

@adnanmuttaleb 2019-03-05 20:57:58

what is 'root' for? can anybody explain

Related Questions

Sponsored Content

9 Answered Questions

[SOLVED] Issue with virtualenv - cannot activate

  • 2012-01-19 04:54:37
  • user1157538
  • 78685 View
  • 76 Score
  • 9 Answer
  • Tags:   python virtualenv

9 Answered Questions

[SOLVED] How to leave/exit/deactivate a Python virtualenv

21 Answered Questions

[SOLVED] Use different Python version with virtualenv

8 Answered Questions

[SOLVED] Activate a virtualenv with a Python script

  • 2011-08-04 14:27:27
  • h3.
  • 63208 View
  • 72 Score
  • 8 Answer
  • Tags:   python virtualenv

23 Answered Questions

[SOLVED] How do I list all cron jobs for all users?

  • 2008-09-25 18:01:31
  • yukondude
  • 786076 View
  • 788 Score
  • 23 Answer
  • Tags:   unix cron

21 Answered Questions

[SOLVED] Using Python 3 in virtualenv

9 Answered Questions

[SOLVED] Activate a virtualenv via fabric as deploy user

1 Answered Questions

[SOLVED] Supervising virtualenv django app via supervisor

1 Answered Questions

Sponsored Content