Django Aware Blog - Everything Django

OneToOne Fields And The Primary Key Issue

By Paul Kenjora | June 23, 2008

Django provides one-many ForeignKey , many-many ManyToManyField, and one-one OneToOneField options for defining table relationships. The OneToOneField specifically has recently been the topic of heated debates and ideas. Unlike the other options this field is by always constrained to be a “Primary Key”. The restriction implies that each table may ONLY have one OneToneField. Why is this wrong?

OneToOneField Primary Key Constraint Is Wrong

The “Primary Key” constraint (you cannot turn it off) is wrong because a schema often needs to define a property that can be applied to multiple objects. This property is the same but can be one-one with different tables, its a mutable property. Here is a great example:

Tip (table describing tipping rules) applies to a Person (1-1).
Tip (same rules) applies to Restaurant (1-1).
I want to describe the tipping rules for a Person or a specific Restaurant (or a combination of the two). Each case is a (1-1). A foreign key would complicate things here and not reflect the true meaning of Tip. Tip is always 1-1 with at least one other table, that table can vary though.

This is a very common approach for adding properties to objects. I understand the desire to have one-one raltionships describe only is-a relations but they can and should be so much more powerful. I’ve documented the problem on Django’s Ticket #7367: Cannot have non-primary-key OneToOneField to parent class. The debate over the pure is-a constraint is strange because the workaround to the “Primary Key” issue is a quick 3 liner that does not violate the is-a relationship.

Fixing The OneToOneField Primary Key Constraint

The solution is to use “Primary Key” and thus preserve is-a behavior where appropriate but add the ability to make the relationship optional. I decided to use “null=True” as an indicator that the relationship is optional, same convention as ForeignKey. Once the field becomes optional then its constraint changes to “Unique”. The code below achieves this:

Around Line 585 in Django File: db/models/fields/related.py

class OneToOneField(RelatedField, IntegerField):
    def __init__(self, to, to_field=None, **kwargs):
        try:
            to_name = to._meta.object_name.lower()
        except AttributeError: # to._meta doesn't exist, so it must be RECURSIVE_RELATIONSHIP_CONSTANT
            assert isinstance(to, basestring), "OneToOneField(%r) is invalid. First parameter to OneToOneField must be either a model, a model name, or the 
string %r" % (to, RECURSIVE_RELATIONSHIP_CONSTANT)
        else:
            to_field = to_field or to._meta.pk.name
        kwargs['verbose_name'] = kwargs.get('verbose_name', '')

if kwargs.has_key('edit_inline_type'):
            import warnings
            warnings.warn("edit_inline_type is deprecated. Use edit_inline instead.")
            kwargs['edit_inline'] = kwargs.pop('edit_inline_type')

kwargs['rel'] = OneToOneRel(to, to_field,
            num_in_admin=kwargs.pop('num_in_admin', 0),
            edit_inline=kwargs.pop('edit_inline', False),
            related_name=kwargs.pop('related_name', None),
            limit_choices_to=kwargs.pop('limit_choices_to', None),
            lookup_overrides=kwargs.pop('lookup_overrides', None),
            raw_id_admin=kwargs.pop('raw_id_admin', False))

# PRIMARY KEY ERROR PATCH
        if kwargs.get('null', False):
          kwargs['unique'] = True
        else:
          kwargs['primary_key'] = True
        # END PRIMARY KEY ERROR PATCH

IntegerField.__init__(self, **kwargs)

self.db_index = True

So its possible to have a clean is-a and more mutable is-a relationship without breaking the system. I’m surprised that Django core developers are constraining the database layer, it seems to violate the coolness of Django + Python. I’ve always liked that the Django framework gives me so much out of the box and at the same time allows me to super customize my projects. For no other reason but to keep Django awesome lets not add artificial constraints where they do not need to exist.

Topics: Database API Revisions, Model Revisions, Tutorial | No Comments »

Arkayne Out Of Beta

By Paul Kenjora | June 7, 2008

The road has been fun and at times tedious but Arkayne is finally out of Beta. For those of you not familiar with the project, its a content linking widget that works on relevance. Think of it as the "Amazon Similar Items" for blogs.

Arkayne started as a side project and has grown to almost 100,000 pages and over 1,000,000 hits with a click through rate (CTR) of 1.77%. The decision to move out of beta came to me while having lunch with a fellow site administrator who was running the Arkayne widget. He mentioned that while running the widget, his Google Analytics actually showed a notable flow of traffic from his front page posts to other stories on the site. That was a my moment of Zen that meant Arkayne delivers and needs to be taken to the next level.

Here are some major hurdles we overcame to get out of Beta…

Server uptime is now guranteed, the Arkayne widget will stay up 99.99% of the time and the other 0.01% fail gracefully (without users noticing it). A huge step towards this was moving to Amazons EC2, we now have almost infinite capability to scale and adjust to traffic loads. The basics are on solid ground now.
Giving bloggers more control over their links. Arkayne now has an account system, you can log in and check personal page clicks as well as submit pages for linking, debug parse errors, and track your own personal CTR.
Expanding our core development team. There are now two dedicated developers working on everything from the core linking engine to the portal system. We’re ramping up the team to ensure uptime and constantly improving link quality. Our bloggers have the controls.
Monetizing Arkayne as a product. The API and widgetare currently in final testing for monetization. The widget will always remain free so we’re adding an opt in system for bloggers to buy/sell/bid links. We’ve double checked with Google and are abiding by their official link policy. Stop by Arkayne and sign up to be the first to use this feature when it launches next month. Revenue sharing is near.
Reaching out for round one Angel/VC funding. The technology is proven, the benefits are there, and the response from the community has been very encouraging. The next step is to take Arkayne to large scale markets quicker and better. The funding will allow us to bring in content at a faster rate from both main stream and niche sites to improve the quality of everyones links. Arkayne is going full startup.

For all our Beta user’s, thank you for believing in Arkayne when it was still in its infancy. For our bloggers, Arkayne will strive to drive as much relevant traffic as possible. For the readers of those blogs, Arkayne will work to make content as relevant and useful as possible.

The Arkayne.com has much more information. All statistics like 100,000 pages and over 1,000,000 hits with a CTR of 1.77% are displayed live in the upper right corner of every page. For more information on the Arkayne sponsored links program please visit the official program description.

Topics: Arkayne, Tech News | No Comments »

Amazon EC2 Basics For The Curious

By Paul Kenjora | June 3, 2008

For those of you wondering what it would be like to host and maintain a Django application (or any application) on Amazon’s Elastic Computing Cloud (EC2) here is a basic list of daily operations. These steps assume you’ve set up your Amazon account. This is the basic set of commands you can use to fully administer an EC2 server. A note to some of you, don’t soil your hacking trousers, the keys and file names below are fake.

Show all instances that are currently running, useful if you want to connect to your server.

ec2-describe-instances
Show all images so you can pick one to run. With EC2 you can choose an image and run it on a variety of machine configurations. Great for scaling. Creating images is as simple as picking an existing one and editing it, EC2 comes with a few out of the box images. Default images are covered in the oficial Amazon EC2 getting started guide.

ec2-describe-images
Run an image. This is how you start a server. Same as hitting the power on button on a physical server.

ec2-run-instances ami-4850b521 -k my-gsg-keypair
Connect to an image, and do anything you want. You can use a key or log in and enter the password on the command line. Just like any other server.

ssh -i my-gsg-keypair root@ec2-75-101-194-32.compute-1.amazonaws.com
Attach a static IP to the new instance. This is great if youre running a website, your instance now has a static IP you can point any DNS at. Advantage here is you retain static API even if you upgrade your instance to a more powerful machine.

ec2-associate-address -i i-b148b5gt 75.101.145.136
If creating a new AMI or backing up existing instance you need to upload your key and certificate so the bundle utility knows how to encrypt your stored backup of an image.

scp -i my-gsg-keypair pk-2WWJUDIZFYTYZN4JEBGSR5Z5E3SABDT7.pem cert-2WWJUDIZFYTYZN4JEBGSR5Z5E3SABDT7.pem root@ec2-75-101-194-32.compute-1.amazonaws.com:/mnt
Once your instance (server) is running you can do things within it as root. It looks exactly like any other server I’ve seen. I even installed Django and some python libraries using yum on a fedora instance. So after you log into a running instance…
- Fetch code from a repository and install it to home for example.
  
  cd /home
  
  svn co some project>/p>
  
  export PYTHONPATH=/home
- Fetch database from your current server and install it on your instance. I picked a start image that already had Apache and MySQL.
  
  mysql
  
  CREATE DATABASE project;
  
  GRANT ALL ON project.* TO user@localhost IDENTIFIED BY ’some_password’;
  
  mysql -D project < project.mysql
- After setting your instance up or maybe just to back it up, you need to save it to an image. EC2 instances loose ALL data if they are shut down or crash. The problem is not as bad as you think, a simple cron job to back up the instance to Amazon Simple Storage Service (S3) is all it takes. This is actually better than a typical server because you’re guaranteed to have a backup. So think of it as something you should be doing anyway. This requires step 6 above…
  
  rm -f /mnt/image
  
  ec2-bundle-vol -d /mnt -k /mnt/pk-2WWJUDIZFYTYZN4JEBGSR5Z5E3SABDT7.pem -c /mnt/cert-2WWJUDIZFYTYZN4JEBGSR5Z5E3SABDT7.pem -u 451222037441 -r i386
  
  ec2-upload-bundle -b project -m /mnt/image.manifest.xml -a 13BNUHIBGPAQ48JQ3NV1 -s grAbTUXLUjHnuff7Hpl/hNd3qTSH0WdklZZ2UB+j
Every time you save an instance to an image you have to register it before launching that instance. So if your server crashes you will have to run the following command before launching it again from a backup.

ec2-register project/image.manifest.xml
When you first set up your account you should probably open up a few ports. Opening a port to allow ssh or ftp etc…

ec2-authorize default -p 22

Well thats about it for EC2. I hope the whole thing does not seem as daunting anymore. My experience with the entire system including billing has been great. I set up my first EC2 instance in 35 minutes after hearing about it at a conference. Since then I’ve only used the commands to bundle and backup my instance. I strongly suggest Django developers who outgrow shared hosts move to this service, coupled with some of the tools for auto launching instances and scaling to handle load this is the best hosting option I’ve ever seen.

The intent of this article is to scratch the surface and boil EC2 down for anyone considering it, with only a handful of commands to completely control and back up a server I think the case is solid. For more advanced projects Amazon offers so many tools it would be best to visit the official Amazon Web Services page.

Topics: Tech News, Tutorial | No Comments »

Installing MySQLdb Without Python Egg Problems

By Paul Kenjora | May 31, 2008

When installing Django sooner or later you will come across a system that needs MySQLdb and sooner or later you will see the following error:

[Sat May 31 19:43:19 2008] [error] [client 99.165.134.196] PythonHandler django.core.handlers.modpython: ExtractionError: Can’t extract file(s) to egg cache\n\nThe following error occurred while trying to extract file(s) to the Python egg\ncache:\n\n [Errno 13] Permission denied: ‘/root/.python-eggs’\n\nThe Python egg cache directory is currently set to:\n\n /root/.python-eggs\n\nPerhaps your account does not have write access to this directory? You can\nchange the cache directory by setting the PYTHON_EGG_CACHE environment\nvariable to point to an accessible directory.\n

The fix is to set the PYTHON_EGG_CACHE environment variable before running the install. The install instructions do not make this clear and the error sounds like the path needs to be set after install. Here are the steps to avoid the above error.

Download the tarball from SourceForge.
$ tar zxvf [filename of MySQLdb tarball]
$ cd [name of resulting directory]
$ export PYTHON_EGG_CACHE=/var/cache/python-eggs
$ mkdir /var/cache/python-eggs
$ chmod 777 /var/cache/python-eggs
$ python setup.py build
$ python setup.py install

Thats it. I’ve only run into this issue on Fedora within Amazon EC2. Other platforms and versions of the MySQLdb module may work.

Topics: Environment Setup, Tutorial | 3 Comments »

Authenticating Using Email vs Username

By Paul Kenjora | May 18, 2008

A few weeks back I posted the full source for user logins and password recovery under Everything A Django Developer Needs To Create Logins. There was a little bit of flack from some very passionate and smart individuals (James Bennet) insisting that this middle ware already exists in "contrib.auth;". On the other hand I received many emails thanking me for the post because it boiled down the login system for novice users. In the spirit of helping new people adopt Django and existing users improve it here is a back end modification to allow email based logins in addition to the traditional default username login.

The file: django/contrib/auth/backends.py

from django.contrib.auth.models import User

class ModelBackend:
    """
    Authenticate against django.contrib.auth.models.User
    """
    # TODO: Model, login attribute name and password attribute name should be
    # configurable.

def authenticate(self, **credentials):
        if 'username' in credentials:
            return self.authenticate_by_username(**credentials)
        else:
            return self.authenticate_by_email(**credentials)

def authenticate_by_username(self, username=None, password=None):
        try:
            user = User.objects.get(username=username)
            if user.check_password(password):
                return user
        except User.DoesNotExist:
            return None

def authenticate_by_email(self, email=None, password=None):
        try:
            user = User.objects.get(email=email)
            if user.check_password(password):
                return user
        except User.DoesNotExist:
            return None

def get_user(self, user_id):
        try:
            return User.objects.get(pk=user_id)
        except User.DoesNotExist:
            return None

I started by creating a whole new backend but it turned out to be tons of copy and paste code. The solution above meets all the requirements for a backend interface and is fully compatible with all existing code. I basically created a decision function that checks if "username" or "email" was passed in as a parameter, then decided on the appropriate action. All extreme python developers please feel free to boil this down to a single line.

Now you can use two methods to authenticate, remember all imports etc… remain the same. Here are both interfaces you can use simultaneously.

user = authenticate(username=”some_username”, password=”some_password”)

user = authenticate(email=”some@emai.com”, password=”some_password”)

Thats about it, I guess the next step would be for the code to figure out if its an email or a username on the back end and pick the best one… there are pros and cons here.

PS: I’m looking for a really simple and clever way of adding OpenID to my projects. Everything I’ve found so far requires me to re-code either my site or the solution to make it sync up with my existing logins. I really just want a snippet of HTML for the templates and a single backend authentication function. I know Python OpenID libraries come in a separate directory, I want to keep them there.

Oh yeah… I’m posting the source code and templates for Everything A Django Developer Needs To Create Logins converted to new forms next week…

Topics: Model Revisions, Tutorial | 10 Comments »

UnicodeDecodeError Exception Fix On Templates

By Paul Kenjora | May 14, 2008

My friend Brian at HelpMeSue.com has found a work around for the dreaded UnicodeDecodeError exception. Here is his fix…

In Brian’s Words…

Using special Spanish characters. Try to load a template with a render_to_response and get a template UnicodeDecodeError exception thrown.

To fix the issue:

Run file on the template:

file static_page.htm

That produces: static_page.htm: ISO-8859 text, with very long lines, with CRLF line terminators
Convert the page to utf-8:

iconv -f ISO-8859-1 -t UTF-8 static_page.htm > new_static.htm

cp new_static_htm static_page.htm

file static_page.htm

That produces: static_page.htm: UTF-8 Unicode text, with very long lines, with CRLF line terminators

And now it all work so Brian’s Spanish template guru is Help Me Sue compatible.