python & django ttt

@KevinVanWilder

Virtualenv

• Isolated Python environment

• Virtualenvwrapper

$ which python /home/kevin/.virtualenvs/inuittest/bin/python $ pip install virtualenvwrapper $ mkproject inuittest ... (inuittest)$ which python /home/kevin/.virtualenvs/inuittest/bin/python (inuittest)$ deactivate $ workon inuittest

Installing libraries

• Always work in a virtualenv

$ pip freeze # ONE GAZILLION LIBRARIES!!!!

(inuittest)$ pip freeze argparse==1.2.1 wsgiref==0.1.2 (inuittest)$ pip install yolk django south (inuittest)$ yolk –l Django - 1.4.2 - active Python - 2.7.3 - active development (/usr/lib/python2.7/lib-dynload) South - 0.7.6 - active argparse - 1.2.1 - active development (/usr/lib/python2.7) pip - 1.2.1 - active setuptools - 0.6c11 - active wsgiref - 0.1.2 - active development (/usr/lib/python2.7) yolk - 0.4.3 - active

@KevinVanWilder

introduction

Hi!

• Architecture

• Development Practices

• The Django Ecosystem

• Infrastructure

• Tools

ARCHITECTURE OVERVIEW WTH is this thing?

Architecture

Model

ORM Storage Fixtures

RDBMS

Signals Forms

Template

Template Loader URL Resolver

Middleware

Request

View

Middleware

Response

Model

Model

ORM Storage Fixtures

RDBMS

Signals Forms

Template

Template Loader URL Resolver

Middleware

Request

View

Middleware

Response

Model

from django.db import models

class Poll(models.Model):

question = models.CharField(max_length=200)

pub_date = models.DateTimeField('date published')

class Choice(models.Model):

poll = models.ForeignKey(Poll)

choice = models.CharField(max_length=200)

votes = models.IntegerField()

Model

BEGIN;

CREATE TABLE "polls_poll" (

"id" serial NOT NULL PRIMARY KEY,

"question" varchar(200) NOT NULL,

"pub_date" timestamp with time zone NOT NULL

);

CREATE TABLE "polls_choice" (

"id" serial NOT NULL PRIMARY KEY,

"poll_id" integer NOT NULL REFERENCES "polls_poll" ("id") DEFERRABLE INITIALLY DEFERRED,

"choice" varchar(200) NOT NULL,

"votes" integer NOT NULL

);

COMMIT;

Request Response Cycle

Model

ORM Storage Fixtures

RDBMS

Signals Forms

Template

Template Loader URL Resolver

Middleware

Request

View

Middleware

Response

Request Response Cycle

1. Determine which URLConf to use

2. Load that URLConf and look for urlpatterns variable

3. Run through each pattern in order and stop at the one that matches

4. Once a match is found, import and call the view with the HttpRequest object

5. If no regex matches or an exception is raised, invoke error-handling

Request Response Cycle: URLConf

from django.conf.urls import patterns, include, url

from django.contrib import admin

admin.autodiscover()

urlpatterns = patterns('',

url(r'^polls/$', 'polls.views.index'),

url(r'^polls/(?P<poll_id>\d+)/$', 'polls.views.detail'),

url(r'^polls/(?P<poll_id>\d+)/results/$', 'polls.views.results'),

url(r'^polls/(?P<poll_id>\d+)/vote/$', 'polls.views.vote'),

url(r'^admin/', include(admin.site.urls)),

)

polls.views.detail(request=<HttpRequest object>, poll_id=‘1')

/polls /polls/1 /polls/1/results /polls/1/vote

Request Response Cycle: View

from django.shortcuts import render_to_response

from polls.models import Poll

def index(request):

latest_poll_list = Poll.objects.all().order_by('-pub_date')[:5]

return render_to_response('polls/index.html', {'latest_poll_list': latest_poll_list})

Request Response Cycle: View

from django.http import Http404

def detail(request, poll_id):

p = get_object_or_404(Poll, pk=poll_id)

return render_to_response('polls/detail.html', {'poll': p})

Request Response Cycle: View from django.shortcuts import get_object_or_404, render_to_response

from django.http import HttpResponseRedirect, HttpResponse

from django.core.urlresolvers import reverse

from django.template import RequestContext

from polls.models import Choice, Poll

def vote(request, poll_id):

p = get_object_or_404(Poll, pk=poll_id)

try:

selected_choice = p.choice_set.get(pk=request.POST['choice'])

except (KeyError, Choice.DoesNotExist):

return render_to_response('polls/detail.html', {

'poll': p,

'error_message': "You didn't select a choice.",

}, context_instance=RequestContext(request))

else:

selected_choice.votes += 1

selected_choice.save()

return HttpResponseRedirect(reverse('polls.views.results', args=(p.id,)))

Templating

Model

ORM Storage Fixtures

RDBMS

Signals Forms

Template

Template Loader URL Resolver

Middleware

Request

View

Middleware

Response

Template

{% if latest_poll_list %}

<ul>

{% for poll in latest_poll_list %}

<li><a href="/polls/{{ poll.id }}/">{{ poll.question }}</a></li>

{% endfor %}

</ul>

{% else %}

<p>No polls are available.</p>

{% endif %}

DJANGO ECOSYSTEM The django developer in its natural habitat

Django Debug Toolbar

• https://github.com/dcramer/django-debug-toolbar

• Must have for all your projects

South

• Data schema migration

– Automatic migration creation

– Database independent

– Migration conflict detection

– Under source control

$ ./manage schemamigration –-initial

--- update models

$ ./manage schemamigration --auto

Celery

• Asynchronous task queue

• Message Passing

• Green threads

• RabbitMQ, Redis, MongoDB, CouchDB, …

• Search for Django

• Solr, ElasticSearch, Whoosh, Xapian

Sentry

• getsentry.com

• Exception & error aggregator

• Open Source and SaaS options

Tastypie

• tastypieapi.org

• Webservice API framework

• REST

• HATEOAS Compliant

• Alternative to django-piston

Pinax

• pinaxproject.com

• Standard project layout

• Starter projects

• Default templates for prototyping

• Reusable apps collection

– Get ready...

Pinax Reusable Apps

• Agon (points & positions) • Agon-ratings • Agora (forum) • Aiteo (q&a, cfr. Stackoverflow) • Anafero (referrals) • Atom-format • Biblion (blog) • Brabeion (badges) • Dialogos (flaggable comments) • announcements • Boxes (db driven regions) • Contacts-import • Email-confirmation • Flag (inappropriate/spam) • Friends (contact/invitation) • Mailer (queuing) • Notification • Oauth-access • Observer

• Partitions (querysets) • Reminders • Rubberstamp (permissions) • Timezones • Wakawaka (wiki) • Idios (profile app) • Kaleo (user to user join invites) • Mailout (mail campaigns) • Marturion (testimonials) • Metron (analytics & metrics) • Minesis (media manager) • Nashvegas (db migration) • Phileo (likes) • Pinax (rapid development) • Pinax-shop • Pinax-theme-bootstrap • Pinax-theme-classic • Symposion (apps for conferences)

django-extensions

• Shell_plus – Autoloading of the apps database models

• Additional database fields – AutoSlugField

– CreationDateTimeField

– UUIDField

– EncryptedCharField

• DB Model graphing

• …

INFRASTRUCTURE

Application Server (AS)

Web Server Gateway Interface (WSGI)

• PEP 333 [P. J. Eby, dec 2010]

• Common ground for portable web applications

– Routing, environment variables, load balancing, forwarding, post-prosessing

• 2 parts:

– Gateway (= server)

– Framework (= application)

AS: gunicorn

• “green unicorn” (cfr. green threading)

• Pre-fork worker model

• 4-12 workers = 1000’s req/sec

• Each worker runs an instance of your django project => memory

AS: uWSGI

• Protocol “uwsgi” (lowercase)

• Included in Nginx & Cherokee

• Faster than gunicorn

TOOLS

Tools

• IPython – Tab completion in the interpreter!

• import ipdb; ipdb.set_trace() – Just like pdb, but with added magic

• Fabric (fabfile.org) – $ fab production deploy

– Like Capistrano but without the voodoo

• Supervisord (supervisord.org) – Process monitoring & controlling

QUESTIONS?

CACHING Deja vu for websites

Cache Systems

CACHE_BACKEND = .... – Dummy caching

• ‘dummy:///’ • For development, dummy!

– Local-Memory Caching • ‘locmem:///’ • In memory caching without memcache

– Database Caching • ‘db://my_cache_table’ • When DB I/O is not a problem.

– Filesystem Caching • ‘file:///var/tmp/django_cache’ # directory • Cache values stored in seperate files in pickled format

– Memcache • ‘memcached:///127.0.0.1:112211/’

Approaches to Caching

• Per site caching (generic approach) – Only cache everything without GET or POST parameters

– Only cache anonymous requests

• Per view caching (granular approach) – Cache response of a view

– Set cache timeouts per view

• Template Fragment caching (micro manage approach) – Caching both static and dynamic fragments

• Low level caching (micro micro manage) – Indivudually store parameters in the cache

MIDDLEWARE_CLASSES = ( 'django.middleware.cache.UpdateCacheMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.cache.FetchFromCacheMiddleware',

)

CACHE_MIDDLEWARE_ANONYMOUS_ONLY = True # optional

Approaches to Caching

• Per site caching (generic approach) – Only cache everything without GET or POST parameters

– Only cache anonymous requests

• Per view caching (granular approach) – Cache response of a view

– Set cache timeouts per view

• Template Fragment caching (micro manage approach) – Caching both static and dynamic fragments

• Low level caching (micro micro manage) – Indivudually store parameters in the cache

@cache_page(60 * 15)

def my_view(request):

# ...

Approaches to Caching

• Per site caching (generic approach) – Only cache everything without GET or POST parameters

– Only cache anonymous requests

• Per view caching (granular approach) – Cache response of a view

– Set cache timeouts per view

• Template Fragment caching (micro manage approach) – Caching both static and dynamic fragments

• Low level caching (micro micro manage) – Indivudually store parameters in the cache

Approaches to Caching

• Per site caching (generic approach) – Only cache everything without GET or POST parameters

– Only cache anonymous requests

• Per view caching (granular approach)

– Cache response of a view

– Set cache timeouts per view

• Template Fragment caching (micro manage approach) – Caching both static and dynamic fragments

• Low level caching (micro micro manage) – Indivudually store parameters in the cache

{% load cache %}

{% cache 500 sidebar request.user.username %}

.. sidebar for logged in user ..

{% endcache %}