Django vs. PYTHONPATH

In the beginning...

So, you've written your first Django app, and deployed... and suddenly, you're getting errors about your apps not existing.

Some helpful person on #django tells you it's because your PYTHONPATH isn't set right. But it always worked with runserver... what gives?

Some background.

Python keeps a list of directories in which to search for your modules when you "import" them. You can see the default list with

>>> import sys
>>> print sys.path

You can add to this list, temporarily, using the PYTHONPATH environment variable.

Now, when I say Python looks inside these dirs, it means if you set PYTHONPATH to "/foo/bar" and try "import baz", Python's going to look for /foo/bar/baz/ or /foo/bar/baz.py {among others}.

The cause

The problem starts with two things: something Python does implicitly, and Django's project layout.

Python implicitly adds the current directory to its search paths.

Django, up to version 1.3, puts "manage.py" inside the project module.

/myproject/
    __init__.py
    settings.py
    manage.py
    urls.py
    app1/
        __init__.py
        models.py
        view.py
    app2/
        __init__.py
        models.py
        views.py
        urls.py

That means, when you run manage.py, Python is implicitly adding your project directory to the list to be look inside.

To compound the confusion, manage.py does some gymnastics to add the project directories parent to the PYTHONPATH. This is the correct thing to do, but it still confuses people.

Deployment and tears

This breaks when you deploy, because typically you don't run manage.py -- at least, not from within your project directory.

Now is the point people find they've been writing all of their apps and code with this assumption .... and typically have "from myapp import models" instead of "from myproject.myapp import models".

Sadly, a lot of people advise that to fix this, you should add your project directory to the PYTHONPATH. "But it works!", they cry.

Sure, until you start doing things that rely on the fact Python won't import the same module twice.

Like signals.

Ever had signals firing off twice? It's usually caused by your code importing the same module by two different "names". Python was given two different names for the same module, so it considers them two separate modules, and imports them twice.

There is hope!

Fortunately, the smart lot we call "the Django devs" have a solution, and it's arriving in Django 1.4. It's as simple as: a new project layout.

/workspace/
    manage.py
    myproject/
        settings.py
        urls.py
    app1/
        __init__.py
        models.py
        view.py
    app2/
        __init__.py
        models.py
        views.py
        urls.py

Now, when you run manage.py you are in the workspace, not the project module. Also, since Python is now adding just the workspace to the search paths, your project is correctly found. No horrid path hacks to get it to work.

The de-facto practice of DJANGO_SETTINGS_MODULE=projectname.settings make clear sense, now, since there it is, right in your workspace.

When you create new apps, they, too, will be put in the workspace, not the project. This will remove some confusion for people trying to write portable apps, and eliminate the need for long, tedious imports starting with the project name.

comments powered by Disqus