context processors, caching, and lazy loading. oh my!
March 23, 2008 - 7:50 PM
Both the Dugg and Flickr sections in the sidebar are added using a django context processor. To get the data from the different API's I used publicly available python libraries for both. Then I wrapped the calls to each in a simple class with only one method each getDiggs() and getPhotos().
Context Processor for Flickr API:
def Photos(request):
return {
'photos' : Flickr(API_KEY,USER_ID).getPhotos(),
}
In my settings.py file I added them to the list of context processors.
TEMPLATE_CONTEXT_PROCESSORS = (
...
'matterkkila.context_processors.Digg',
'matterkkila.context_processors.Photos',
...
)
The problem with doing things this way is that for any (and every) page that was loaded requests would be made to both the Flickr and Digg API's, which would add anywhere between 1-3 seconds on the response time for each page. Because it was a global context processor even the RSS and ATOM feed pages were loading the info, which is sad because they don't even display that information. What a waste.
So to help things a little I added cache support for the objects that got returned. The cache back end is configurable inside django and for ease of use I set it to use just a simple file based case.
photos = cache.get(key)
if photos is not None:
return photos
photos = FlickrAPI.getPhotos()
cache.set(key, photos)
This helped a little but I was still a little concerned with that fact that even the cached object was being loaded for the feeds and the sitemap. Enter lazy loading.
In order to get lazy loading to work I modified the wrapper classes for both API's. I overrode the __getattr__() method and added a custom property, for flickr, called 'photos'.
class Flickr(object):
def __getattr__(self, name):
if name == 'photos':
return self.getPhotos()
return getattr(self, name)
def getPhotos(self):
...
Then you modify the context processors to not call the getPhotos() method and instead just instantiate a instance of the wrapper class.
def Photos(request):
return {
'photos' : Flickr(API_KEY, USER_ID),
}
So in the templates you can just call photos.photos and that will in turn call the getPhotos() method for you. So getPhotos() is only called when actually needed by the template.
{% for photo in photos.photos %}
...display photo...
{% endfor %}
This ended up reducing the load times for the feed and sitemap pages to less than 1 second.
