meetup django common_problems(1)
DESCRIPTION
Slides from 5/11/2011 Milwaukee Django User Group MeetupTRANSCRIPT
Who is this fool?!A little about me
Python
DjangoJavaScript
VFX
Print DesignSoftwar
e
Web Design
Flash / Flex
Photography
CSS
Graphic Art
Digital Media
PERL & JavaScript
Django = HotPileOfAwesome( yes=True )Django.build_web_app( fast=True )
Django.make_an_app()>>> True
Django.log_in_user()>>> True
Django.comment_on_my_model()>>> True
Django.message_my_user(user='myuser')>>> True
Django.send_emails( emails=['[email protected]'] )>>> True
Django.do_complex_SQL( please=True )>>> No Problem!
Django.make_a_thumbnail()>>> Thumbnail?
Django.send_text_message()>>> Email Exception: no such thing
Django.search_all_my_stuff()>>> WTF?
Django.get_data_in_under_75_queries()>>> Whoa...
Django.alter_table(model='MyModel')>>> Let's not get crazy
Django.be_restful( now=True )>>> you mean request.POST
I've Got an app for that!
Searching
SearchingFind Stuff - Fast
SearchingFind Stuff - Fast
( without crushing my DB )
Haystack
Djapian
Sphinx
haystacksearch.org
code.google.com/p/djapian/
django-sphinx ( github )
MyModel.objects.filter( text__icontains='word' )
OR MyModel.objects.filter( text__search='word' )
Problems:• single model • slow• mysql• manual DB configs
Django
Haystackclass PostIndex( SearchIndex ): body = CharField(document = True, model_attr = 'body') title = CharField( model_attr = 'title') author = CharField( model_attr = 'author__get_full_name') text = CharField( use_template = True )
def get_queryset( self ): return Post.objects.all()
def prepare_url( self, obj ): return obj.get_absolute_url()
site.register(Post, PostIndex)
HaystackSearchQuerySet() .filter( SQ(field=True) | SQ(field__relation="something") ~SQ(field=False) )
>>> [ <SearchResult>, <SearchResult>, <SearchResult> ]
Xapianclass ArticleIndexer( Indexer ): fields = ['title','body'] tags = [ ('title','title', 3), ('body', 'as_plain_text', 1) ]space.add_index(Article, ArticleIndexer, attach_as='indexer')
Xapianfrom djapian.indexer import CompositeIndexer
flags = xapian.QueryParser.FLAG_PARTIAL| \ xapian.QueryParser.FLAG_WILDCARD
indexers = [ Model_1.indexer, Model_2.indexer ]comp = CompositeIndexer( *indexers )
s = comp.search( `a phrase` ).flags( flags )>>> [ <hit:score=100>,<hit:score=98> ]
$ s[0].instance>>> <ModelInstance:Model>
• Pluggable Architecture• Whole Word Matching• Loads all indexers• Multiple Index Hooks• Stored fields• Django-like Syntax• Templates & Tags• Views, Forms & Fields
• Wildcard Matching• Partial word matching• Doesn't Load All indexers• Interactive shell• Close to the metal
( Control )• Watches Models for
changes• Pre-index Filtering
Haystack XapianIndex files
Class Based IndexCustomize Text For Indexing
Link to Indexed ObjectIndex Fields, Methods & Relations
Stemming, Facetting, Highlighting, Spelling
REST API
REST APIExposing Your Data
REST APIExposing Your Data( In a meaningful way )
def view_func( reqeuest, *args, **kwargs): request.GET request.POST request.FILES
Problems:• PUT & DELETE not translated• Can't restrict access based on HTTP
methods• Serialization is left up to you• Manual auth• Tons of URLs
Django
PISTON
TASTYPIE
bitbucket.org/jespern/django-piston
toastdriven.github.com/django-tastypie/
Pistonclass MyHandler( BaseHandler ): methods_allowed =( 'GET', 'PUT') model = MyModel
class MyOtherHandler( BaseHandler ): methods_allowed =( 'GET', 'PUT') model = MyOtherModel fields = ('title','content',('author',('username',) ) ) exclude = ('id', re.compile(r'^private_')) def read( self, request): return [ x for x in MyOtherModel.objects.select_related() ]
def update( self, request ): ...
Tastypieclass MyResource( ModelResource ): fk_field = fields.ForiegnKey( OtherResource, 'fk_field' ) class Meta: authentication = ApiKeyAuthentication() queryset = MyModel.object.all() resource_name = 'resource' fields = ['title', 'content', ] allowed_methods = [ 'get' ] filtering = { 'somfield': ('exact', 'startswith') } def dehydrate_FOO( self, bundle ): return bundle.data[ 'FOO' ] = 'What I want'
Tastypie - Client Sidenew Request.JSONP({ url:'http://www.yoursite.com/api/resource' ,method:'get' ,data:{ username:'billyblanks' ,api_key:'5eb63bbbe01eeed093cb22bb8f5acdc3' ,title__startswith:"Hello World" } ,onSuccess: function( data ){ console.info( data.meta ); console.log( data.objects ):}).send();
http://www.yoursite.com/api/resource/1http://www.yoursite.com/api/resource/set/1;5http://www.yoursite.com/api/resource/?format=xml
• Validation Via Forms• Deep ORM Ties• Data Streaming• OAuth / contrib Auth• URI Templates• Auto API Docs
• Django-like• Built in fields• Auto Meta Data• Resource URIs• ORM ablities ( client )• API Key Auth• Object Caching
( backends )• De / Re hydrations hooks
PISTON TASTYPIEMultiple Formats
ThrottlingAll HTTP Methods
AuthenticationArbitrary Data Resources
Highly Configurable
READY IN MINUTES
DATABASE
QUERYSET-TRANSFORM
DJANGO-SELECTREVERSE
github.com/jbalogh/django-queryset-transform
code.google.com/p/django-selectreverse
DJANGO-SOUTHsouth.aeracode.org
SOUTH
SOUTHDatabase Migrations
DJANGO$ python manage.py syncdb
DJANGO$ python manage.py syncdb
>>> You have a new Database!
DJANGOclass MyModel( models.Model): relation = models.ForiegnKey( Model2 )
DJANGOclass MyModel( models.Model ): relation = models.ForiegnKey( Model2 )
class MyModel( models.Model ): relation = models.ManyToMany( Model2 )
DJANGO$ python manage.py syncdb
DJANGO$ python manage.py syncdb
>>> Sucks to be you!
WTF?!
DJANGO$ python manage.py syncdb
>>> Sucks to be you!
syncdb doesn't really sync your db. Migrations must be done manually.
• For everyedatabase / set up.
Problem:
SOUTHMigrations are a set of sequential .py filesdb agnosticHandle most relation typesRolls migrations forward Handles Dependancies / Reverse DependanciesCan convert existing apps
• Overrides existing syncdb command
DJANGOclass MyModel( models.Model ): relation = models.ForiegnKey( Model2 )
class MyModel( models.Model ): relation = models.ManyToMany( Model2 )
DJANGOclass MyModel( models.Model ): relation = models.ForiegnKey( Model2 )
class MyModel( models.Model ): relation = models.ManyToMany( Model2 )
SOUTH$ python manage.py schemamigration <yourapp>
>>> Sweet, run migrate
SOUTH$ python manage.py migrate <yourapp>
>>> Done.
SOUTH
QUERYSET-TRANSFORM
github.com/jbalogh/django-queryset-transform
( n + 1 )
QUERYSET-TRANSFORM{% for object in object_list %}
{% for object in object.things.all %} {% if object.relation %} {{ object.relation.field.text }} {% else %} {{ object.other_relation }} {% endif %} {% endfor %}
{% empty %} no soup for you{% endfor %}
QUERYSET-TRANSFORMdef lookup_tags(item_qs):
item_pks = [item.pk for item in item_qs] m2mfield = Item._meta.get_field_by_name('tags')[0] tags_for_item = \ Tag.objects.filter( item__in = item_pks) .extra(select = {'item_id': '%s.%s' % ( m2mfield.m2m_db_table(), m2mfield.m2m_column_name() ) }) tag_dict = {} for tag in tags_for_item: tag_dict.setdefault(tag.item_id, []).append(tag)
for item in item_qs: item.fetched_tags = tag_dict.get(item.pk, [])
QUERYSET-TRANSFORM
qs = Item.objects.filter(
name__contains = 'e'
).transform(lookup_tags)
QUERYSET-TRANSFORM
from django.db import connection
len( connection.queries )
>>> 2
DJANGO-SELECTREVERSEcode.google.com/p/django-selectreverse
DJANGO-SELECTREVERSE
model_instance.other_model_set.all()
Tries prefetching on reverse relations
CONTENT MANAGEMENT
DJANGO-CMS
SATCHMO
www.django-cms.org
www.satchmoproject.com
www.webcubecms.com
WEBCUBE-CMS
WEBCUBE-CMS
Feature Complete
WEBCUBE-CMS
Feature Complete
WEBCUBE-CMSRobust & Flexible
Feature Complete
WEBCUBE-CMSRobust & Flexible( Commercial License )
$12,000
$12,000
+ $300 / mo
WTF?!
DJANGO-CMS
• Heavily Customised Admin
• Plugin Support• Template Switching• Menu Control• Translations• Front-End Editing
( latest )• Moderation• Template Tags• Lots of settings
• Lots Of Settings ( again )
• Another Learning Curve• Plone Paradox• Plugins a little wonky
PRO CON
SATCHMO
SATCHMOE-Commerce-y CMS
Django Admin
DJANGO-GRAPPELLI
DJANGO-FILEBROWSE
code.google.com/p/django-grappelli
code.google.com/p/django-filebrowser
DJANGO-ADMIN TOOLSbitbucket.org/izi/django-admin-tools
GRAPPELLI
GRAPPELLI
GRAPPELLI
FILEBROWSER
FILEBROWSER
FILEBROWSER
FILEBROWSER
ADMIN TOOLS
ADMIN TOOLS
ADMIN TOOLS
Image Management
DJANGO-IMAGEKIT
DJANGO-PHOTOLOGUE
bitbucket.org/jdriscoll/django-imagekit
code.google.com/p/django-photologue
SORL-THUMBNAILthumbnail.sorl.net/
DJANGOclass MyModel( models.Model ): image = models.ImageField(upload_to='/' )
DJANGOclass MyModel( models.Model ): image = models.ImageField( upload_to='/' ) thumb = models.ImageField( upload_to='/' )
DJANGOclass MyModel( models.Model ): image = models.ImageField( upload_to='/' ) thumb = models.ImageField( upload_to='/' )
>>> MyModel.objects.get(pk=1)>>> MyModel.objects.image.url>>> MyModel.objects.thumb.url
ImageField
ImageFieldIt Kinda Sucks
IMAGEKITEvolution Of Photologue
IMAGEKITclass Photo( ImageModel ): name = models.CharField(max_length=100) image = models.ImageField(upload_to='photos') views = models.PositiveIntegerField(default=0)
class IKOptions: #define the ImageKit options spec_module = 'myapp.specs' cache_dir = 'photos' image_field = 'image' save_count_as = 'views'
IMAGEKITclass ResizeThumb( processors.Resize ): width = 100 height = 75 crop = True
class ResizeDisplay( processors.Resize ): width = 600
# now we can define our thumbnail specclass Thumbnail( ImageSpec ): access_as = 'thumbnail_image' pre_cache = True processors = [ ResizeThumb ]
class Display( ImageSpec ): increment_count = True processors = [ ResizeDisplay ]
IMAGEKIT
$ photo = Photo.objects.get( pk =1 )
$ photo.display.url>>> u'/path/to/media/photo_display.jpg'
$ photo.thumbnail.width>>> 100
• Only 1 Model• DB Friendly• Format Adjustment• Color Adjustment• Transposing
• Galleries out of the box• Bulk Upload• Admin Integration• Pre-cache command• Very DB Dependant• 4 Models ( 2 Solid )• Templates / Views / Urls• Exif Support
IMAGEKIT PHOTLOGUECaching
Template FriendlyHighly Configurable
Management Commands
EXIF TAGSISSUE 153
( shameless plug )
• Only 1 Model• DB Friendly• Format Adjustment• Color Adjustment• Transposing
• Galleries out of the box• Bulk Upload• Admin Integration• Pre-cache command• Very DB Dependant• 4 Models ( 2 Solid )• Exif Support
IMAGEKIT PHOTLOGUECaching
Template FriendlyHighly Configurable
Management Commands
Beware of View Counts
ImageField
ImageFieldIt's Kinda Messy
SORL
SORLfrom sorl.thumbnail import ImageField
class MyModel( models.Model ): iamge = ImageField( upload_to="/folder" )
SORL{% load thumbnail %}
{% thumbnail obj.image "1000x600" crop='top' as im %} <img src="{{ im.url }}" />
{% endthumbnail %}
MESSAGING
User - to - UserMessaging
DJANGO-POSTMAN
DJANGO-MESSAGES
bitbucket.org/psam/django-postman
code.google.com/p/django-messages
DJANGO-SMScode.google.com/p/django-sms
DJANGO-SMS
DJANGO-SMS1. User Enters Phone #2. User Selects Carrier
THATS IT
FREE
UTILS
DJANGO-EXTENSIONS
DEBUG-TOOLBAR
HOTSAUCE
github.com/django-extensions/django-extensions
github.com/robhudson/django-debug-toolbar
github.com/ericflo/django-paginationDJANGO-PAGINATION
( self plug )
DJANGO-MAINTENANCEMODEpypi.python.org/pypi/django-maintenancemode
DJANGO-GUARDIANgithub.com/lukaszb/django-guardian
DJANGO-EXTENSIONS
$ python manage.py dumpscript>>> your_app.py
$ python manage.py runscript>>> your_app.pyc
$ python manage.py runserver_plus>>> Debugger baked right in
$ python manage.py shell_plus>>> AutoLoad Models
$ python manage.py show_urls>>> /your/endpoint/:id project.app.views.view_func
21 Commands 4 Fields 2 Models 1 Admin Extension ( FK AutoComplete )
RUNSERVER_PLUS
DEBUG TOOLBAR
DEBUG TOOLBAR
MAINTENANCEMODE
MAINTENANCEMODEMAINTENANCE_MODE = True
MAINTENANCEMODEMAINTENANCE_MODE = True
MAINTENANCEMODEMAINTENANCE_MODE = True
HOTSAUCE
HOTSAUCEI Put That $*!T On Everything
HOTSAUCE
HOTSAUCE
HOTSAUCE
Model A
title
content
QueueItem
title
content
title
content
Model A
ChangeSet
ChangeSet
ChangeSet
ChangeSet
HOTSAUCE