Disembrangling Programming

Django and AJAX image uploads

2010-08-20T00:00:00-07:00

Table of contents:

Demo
Summary
Server side (Django)
Client side
Graceful degradation

Demo

Screencast
Screenshots:
- The upload form, empty and ready for action:
- Browsing for an image:
- Uploading the image (in progress):
- The image is uploaded:
- Deleting the image would show a similar progress block as uploading:

Summary

In this post, we’ll go through how to get AJAX uploads to work with Django, including:

csrf protection with Django’s forms
graceful degradation (see also unobtrusive JavaScript)
uploading files in Django
thumbnail generation with PIL
cross-browser uploading of files through AJAX

Note: I’m planning to add upload progress. If you can’t wait for that post (understandably), there are several ways to go about it:

If your site isn’t using multiple webheads, you can just ask the webhead to get you the size of what’s been uploaded so far. Since Django can read in chunks, it can tell you how much has been processed. See this post for implementation ideas.
Or, regardless of the server setup, you can use the File API (in Firefox and Chrome) - easier, cleaner, no server-side interaction required.
Other multi-webhead approaches: writing progress to a file shared among them, or saving directly to a shared folder and e.g. returning the size uploaded so far.

Server side (Django)

First, we’ll look at how the server handles files sent to it.

Model

I created an app called upload with an ImageAttachment model, like so:

apps/upload/models.py:

from django.conf import settings
from django.contrib.auth.models import User
from django.contrib.contenttypes.models import ContentType
from django.contrib.contenttypes import generic
from django.db import models


class ImageAttachment(models.Model):
    """A tag on an item."""
    file = models.ImageField(upload_to=settings.IMAGE_UPLOAD_PATH)
    thumbnail = models.ImageField(upload_to=settings.THUMBNAIL_UPLOAD_PATH)
    creator = models.ForeignKey(User, related_name='image_attachments')
    content_type = models.ForeignKey(ContentType)
    object_id = models.PositiveIntegerField()

    content_object = generic.GenericForeignKey()

    def __unicode__(self):
        return self.file.name

This represents an image attached to a piece of content (using a generic foreign key). Pretty basic stuff. The form is ridiculously simple:

Form

apps/upload/forms.py:

from django import forms


class ImageUploadForm(forms.Form):
    """Image upload form."""
    image = forms.ImageField()

View (uploading image, saving to disk)

The view is a bit more complicated, so I won’t go into the details. But you can have a look at the entire app and contact me if you have questions. Basically, the view does the file upload as you see in Django’s documentation. The function create_image_attachment deals with the part about saving a file to disk.

Generating the thumbnail with PIL

There is also a task for generating thumbnails, which is offloaded from the web server thread to improve performance. If you don’t need that, you can just call generate_thumbnail directly, it’s defined here.

Client side

Now, the magical JavaScript!

We’re using jQuery on SUMO, so I wrote two jQuery extensions:

jQuery.fn.ajaxSubmitInput(options) – wraps an <input type="file"> in a <form> and creates an <iframe> to which that form posts. To get around Django’s csrf protection, it also copies the csrfmiddlewaretoken hidden input into the form. You can’t clone a file input for security reasons (nor can you change or access its value), so you need to wrap it in a form.
jQuery.fn.wrapDeleteInput(options) – wraps an input<type="submit"> in a <form> and creates an <iframe> to which that form posts.

These two pretty much summarize the process:

when the user changes the value of the file input, post the form
- show some progress while the file is uploading
once the file is done uploading, show a thumbnail of the image
- also create the delete input and wrap it in the form using wrapDeleteInput()
when the user clicks on the delete button, post the action
- show some progress while the file is being deleted

A note about graceful degradation

To degrade gracefully, you want to post the file input to whatever view you’re including it to. And you can just do something like:

def some_view(request):
    # ...
    # NOJS: upload image
    if 'upload_image' in request.POST:
        upload_images(request, obj)
    # ...

Thanks for reading, hope it helps!

Access control in Django, and django-authority

2010-05-22T00:00:00-07:00

The past week I’ve been working with Django and permissions. I’ll talk here about the available options and go into details about the one Kitsune is using.

Here are the options I looked at:

Django’s default permissions
AMO’s access app
django-authority

Django’s default permissions

Read more about this in Django’s documentation.

Django provides a convenient and easy to use permissions system. The documentation covers it all, so I’m just going to summarize the pros and cons:

Pros:

Comes with Django (=> robust, well-written and stable)
Permission functions attached to the user object
Per-group permissions and per-user permissions
Model-level (a.k.a. per content_type) permissions (can add custom model-level permissions as well)
Work with Jinja templates, simply use user.has_perm('perm_name')

Cons:

No per-object permissions

AMO’s access app

AMO’s access app adds some flexibility on assigning permissions with wildcards and also stores the user’s groups in request.groups.

Pros:

Flexible way to assign all permissions in a certain group
Does not store permissions in the models Meta class

Cons:

Replaces django’s permission system (either use one or the other)
No per-object permissions

To use this, you can do something like this in your views:

from access import acl

# ...

def some_view_action():
    acl.action_allowed(request, 'Group', 'PermissionName')

django-authority

django-authority was the most promising of the three. It adds per-object permissions and wraps Django’s default permissions system without forcing an either-or choice. It also has a decent documentation, and although still in draft at the time of this writing, it was enough for me to figure out how to use.

Pros:

Per-object permissions
Wraps Django’s default permissions system, so you can use both
Custom permissions
Add permissions to appname/permissions.py, so they are completely separate from the models (unlike Django’s permissions)

Cons:

Needs a bit of work to hook in with Jinja templates
Admin functionality not yet complete - the admin form generated by the edit_permissions admin action does not work at the time of this writing. The alternative is not too bad however, you can manually add to the Permission model using that model’s admin.

django-authority on Kitsune

We are currently focused on rewriting SUMO’s discussion forums. For Kitsune, we have multiple forums and we wanted the ability to have different forum moderators per forum. We’re also using Jinja. django-authority comes with default Django template support, so if you’re using that, you don’t need to write your own template functions/filters.

For Jinja support, just add this to some app’s helpers.py

import jinja2
from jingo import register
import authority

# ...

@register.function
@jinja2.contextfunction
def has_perm(context, perm, obj):
    """
    Check if the user has a permission on a specific object.

    Returns boolean.
    """
    check = authority.get_check(context['request'].user, perm)
    return check(obj)

Then, in your templates, you can just use:

[% if has_perm('perm_cls.codename_model', object) %]
  {# grats, you've got the perms! #}
[% endif %]

(Couldn’t figure out how to escape braces in jekyll, so you’ll need to replace the square brackets above.)

See this link for more information on the perm_cls and codename variables.

If you’re interested, here’s the Kitsune code.

Hope that helps others who may decide to go this route. Questions and suggestions for improvement are always welcome, of course.

My SUMO local development environment

2010-01-04T00:00:00-08:00

Summary

A while back, James wrote about his development environment. I wanted to summarize my approach, and go into a bit more detail about the specific setup you would need to have SUMO work locally. For more information, see also this wiki page.

My LDE: Ubuntu Linux

My LDE is slightly different: I dual boot Windows 7 and Kubuntu 9.10. As of now I’m still a student, and I work on my own machine. There are several reasons I find dual booting a great solution:

Best performance while developing. Although less and less of an issue, being in the native OS is always be faster than running a VM.
A full powered OS that I can use for other work. I do schoolwork and other contracting work, and sometimes Windows is not an option. Even a VM has its limitations when it comes to certain drivers (one example is audio – which I needed for a graphics project)
Keeps my work and pleasure separate – having dual boot is basically like having two machines. Typically, when I’m in Linux, I do work, otherwise I’m in Windows to do personal things.

The only downside is some additional wait time for rebooting if you need to switch to the other OS. Since I work on Linux only, I rarely need to switch while working. However, everything works fine for me on Linux – sound, video, even webcam and VPN (will blog later about all of these).

LAMP

For web development in general, you’ll need to have the LAMP stack installed. Fortunately, Ubuntu provides an easy way to do this in one line. From a terminal, run:

sudo tasksel

Select LAMP server from the list, and choose OK (TAB + ENTER). Follow the steps to choose a MySQL password and any other configuration there may be. Once that’s done, you should be able to go to http://localhost and see the “It works!” text. Here are some useful paths:

Web root: /var/www/
Hosts file: /etc/hosts
Apache sites: /etc/apache2/sites-available/ and /etc/apache2/sites-enabled/
PHP ini: /etc/php5/apache2/php.ini
MySQL ini: /etc/php5/conf.d/mysql.ini

SUMO setup

What you’ll need:

LAMP stack – see above
A host (optional) – see below
SSL – see this guide
Sphinx – see this guide
Memcached – I just install the package and that’s enough: sudo apt-get install memcached php5-memcache
Virtualbox (optional) – for testing: sudo apt-get install virtualbox
A copy of our database – you may ask James, Laura or myself for this – see how, below
SVN – sudo apt-get install subversion
Checking out and configuring the SUMO codebase – see this guide

To set up a SUMO host, I add the entry for it in `/etc/hosts`. Here is the top of my file:

127.0.0.1   localhost
127.0.1.2   sumo

Then, add the site info to apache. Here’s my complete conf file (with SSL), in /etc/apache2/sites-available/sumo:

<VirtualHost 127.0.1.2:80>
    ServerAdmin webmaster@sumo
    ServerName sumo
    ServerAlias sumo
    DocumentRoot /var/www/trunk/webroot
    <Directory /var/www/trunk/webroot>
        Options Indexes FollowSymLinks MultiViews
        AllowOverride All
        Order allow,deny
        allow from all
    </Directory>
</VirtualHost>

<IfModule mod_ssl.c>
<VirtualHost _default_:443>

    ServerAdmin webmaster@sumo
    ServerName sumo
    ServerAlias sumo
    DocumentRoot /var/www/trunk/webroot
    <Directory /var/www/trunk/webroot>
        Options Indexes FollowSymLinks MultiViews
        AllowOverride All
        Order allow,deny
        allow from all
    </Directory>
    SSLEngine on
    SSLCertificateFile /etc/apache2/ssl/apache.pem
</VirtualHost>
</IfModule>

To enable the site, just do this:

cd /etc/apache2/sites-enabled
sudo ln -s ../sites-available/sumo

Then, restart apache:

sudo /etc/init.d/apache2 restart

Testing

For testing, I also use a VM in VirtualBox, running WinXP with multiple IEs. I chose WinXP to save space – only takes up 3GB. You should allow 4GB just in case you want to install more stuff. Using multiple IEs is not the best, as quirksmode points out, so you may consider running a VM for each IE version you wish to test. However, I haven’t experienced that many problems. On WinXP I’ve also installed Firefox (of course), Chrome, and Opera. On Ubuntu I just use the defaults: Firefox and Konqueror (since I’m running Kubuntu).

Help?

I’ve tried to cover everything, but of course there may be stuff I missed. If you need additional info, you may leave a comment. To obtain a copy of our database, you can find us on IRC or contact us by email.