or How I learned to stop worrying and love the JSON
Presenter: Daniel Lindsley (http://www.toastdriven.com/) (@daniellindsley)
PyCon 2012 presentation page: https://us.pycon.org/2012/schedule/presentation/61/
Slides: http://speakerdeck.com/u/daniellindsley/p/restful-apis-with-tastypie
Video: http://pyvideo.org/video/673/restful-apis-with-tastypie
Video running time: 34:07
- Daniel Lindsley
- Consulting & OSS as Toast Driven
- Primary author of Tastypie
- A REST framework for Django
- Designed for extension
- Supports both Model & non-Model data
- http://tastypieapi.org/
- Make good use of HTTP
- Try to be "of the internet" & use the REST methods/status codes properly
- Graceful degradation
- Try to keep backwards compatibility & give users a gradual upgrade path
- Flexible serialization
- not everyone wants JSON
- Flexible EVERYTHING
- Customizability is a core feature
- Data can round-trip
- Anything you can GET, you should be able to POST/PUT
- Reasonable defaults
- but easy to extend
- URIs everywhere!
- Make HATEOAS a reality
- "Hypermedia As The Engine Of Application State"
- Basically the user shouldn't have to know anything in advance
- All about explore-ability
- Deep linking
- http://en.wikipedia.org/wiki/HATEOAS
- Builds on top of Django & plays nicely with other apps
- Full
GET/POST/PUT/DELETE/PATCH
(PATCH
? See5789
) - Any data source (not just Models)
- Designed to be extended
- Supports a variety of serialization formats:
- JSON
- XML
- YAML
- bplist
- Easy to add more
- HATEOAS by default (you'll see soon)
- Lots of hooks for customization
- Well-tested -- about 80% coverage
- Decently documented
pip install django-tastypie
INSTALLED_APPS += ['tastypie']
$ manage.py syncdb
Done.
Set up your app:
# Assuming we're in your project directory...
$ cd <myapp> # Substitute your app_name here
$ mkdir api
$ touch api/__init__.py
$ touch api/resources.py
# Done!
User Resource:
from django.contrib.auth.models import User
from tastypie.resources import ModelResource
class UserResource(ModelResource):
class Meta:
queryset = User.objects.all()
URLconf:
# In your ROOT_URLCONF...
from tastypie.api import Api
from <myapp>.api.resources import UserResource
v1_api = Api()
v1_api.register(UserResource())
urlpatterns = patterns('',
(r'^api/', include(v1_api.urls),
# Then the usual...
)
-------- | ----------------------------------------- |
Curl: | http://localhost:8000/api/v1/ |
Browser: | http://localhost:8000/api/v1/?format=json |
/api/v1/
- A list of all available resources/api/v1/user/
- A list of all users/api/v1/user/2/
- A specific user/api/v1/user/schema/
- A definition of what an individual user consists of/api/v1/user/multiple/1;4;5/
- Get those three users as one request
All serialization formats available (provided lxml, PyYAML, and biplist are installed).
curl -H "Accept: application/xml" http://localhost:8000/api/v1/user/
- http://localhost:8000/api/v1/user/2/?format=yaml
Serialization format negotiated by either Accepts
header or the "?format=json" GET
param
Pagination by default
Everyone has full read-only GET
access
What's not there? (Yet)
- Leaking sensitive information!
email/password/is_staff/is_superuser
- Ability to filter
- Authentication/Authorization
- Caching (disabled by default)
- Throttling (disabled by default)
from django.contrib.auth.models import User
from tastypie.resources import ModelResource
class UserResource(ModelResource):
class Meta:
queryset = User.objects.all()
excludes = ['email', 'password', 'is_staff', 'is_superuser']
from django.contrib.auth.models import User
from tastypie.authentication import BasicAuthentication
from tastypie.resources import ModelResource
class UserResource(ModelResource):
class Meta:
# What was there before...
authentication = BasicAuthentication()
from django.contrib.auth.models import User
from tastypie.authentication import BasicAuthentication
from tastypie.resources import ModelResource, ALL
class UserResource(ModelResource):
class Meta:
# What was there before...
filtering = {
'username': ALL,
'date_joined': ['range', 'gt', 'gte', 'lt', 'lte'],
}
- Using
GET
params, we can now filter out what we want. - Examples:
curl http://localhost:8000/api/v1/user/?username__startswith=a
curl http://localhost:8000/api/v1/user/?date_joined__gte=2011-12-01
from django.contrib.auth.models import User
from tastypie.authentication import BasicAuthentication
from tastypie.authorization import DjangoAuthorization
from tastypie.resources import ModelResource
class UserResource(ModelResource):
class Meta:
# What was there before...
authorization = DjangoAuthorization()
from django.contrib.auth.models import User
from tastypie.authentication import BasicAuthentication
from tastypie.authorization import DjangoAuthorization
from tastypie.cache import SimpleCache
from tastypie.resources import ModelResource
class UserResource(ModelResource):
class Meta:
# What was there before...
cache = SimpleCache()
from django.contrib.auth.models import User
from tastypie.authentication import BasicAuthentication
from tastypie.authorization import DjangoAuthorization
from tastypie.cache import SimpleCache
from tastypie.resources import ModelResource
from tastypie.throttle import CacheDBThrottle
class UserResource(ModelResource):
class Meta:
# What was there before...
throttle = CacheDBThrottle()
- Everything we had before
- Full
GET/POST/PUT/DELETE/PATCH
access - Only registered users can use the API & only perform actions on objects they're allowed to
- Object-level caching (
GET
detail) - Logged throttling that limits users to 150 reqs per hour
- The ability to filter the content
- Why classes?
- Not because I'm OO-crazy.
- It makes extending behavior trivial.
- Why so many classes?
- Composition > Inheritance
- Why so many methods?
- Hooks, hooks, hooks.
- Also makes delegating to composition behaviors easy.
- Tastypie tries to use reasonable defaults:
- You probably want JSON
- You probably want full
POST/PUT/DELETE
by default - You probably want to use the Model's default manager unfiltered
- But Tastypie lets you change all these things
- Plug in custom classes/instances for things like:
- Serialization
- Authentication
- Authorization
- Pagination
- Caching
- Throttling
Resource
has lots of methods, many of which are pretty granular- Override or extend as meets your needs
- As an example, let's customize serialization
- Supports JSON, XML, YAML, bplist by default
- Let's disable everything but JSON and XML, then add a custom type for HTML
- To limit to just JSON and XML:
from django.contrib.auth.models import User
from tastypie.resources import ModelResource
from tastypie.serialization import Serializer
class UserResource(ModelResource):
class Meta:
queryset = User.objects.all()
excludes = ['email', 'password', 'is_staff', 'is_superuser']
serializer = Serializer(formats=['json', 'xml'])
from django.shortcuts import render_to_response
from tastypie.serialization import Serializer
import cgi
from stringio import StringIO
class TemplateSerializer(Serializer):
formats = Serializer.formats + ['html']
def to_html(self, data):
template_name = 'api/api_detail.html'
if 'objects' in data:
template_name = 'api/api_list.html'
return render_to_response(template_name, data)
def from_html(self, content):
form = cgi.FieldStorage(fp=StringIO(content))
data = {}
for key in form:
data[key] = form[key].value
return data
Using it:
from django.contrib.auth.models import User
from tastypie.resources import ModelResource
from myapp.api.serializers import TemplateSerializer
class UserResource(ModelResource):
class Meta:
queryset = User.objects.all()
excludes = ['email', 'password', 'is_staff', 'is_superuser']
serializer = TemplateSerializer(formats=['json', 'xml', 'html'])
- Just like
ModelForm
, you can control all of the exposed fields on aResource/ModelResource
. - Just like Django, you use a declarative syntax.
from django.contrib.auth.models import User
from tastypie import fields
from tastypie.resources import ModelResource
class UserResource(ModelResource):
# Provided they take no args, even callables work!
full_name = fields.CharField('get_full_name', blank=True)
class Meta:
queryset = User.objects.all()
excludes = ['email', 'password', 'is_staff', 'is_superuser']
- You can control how data gets prepared for presentation (
dehydrate
) or accepted from the user (hydrate
). - Happens automatically on fields with
attribute=...
set
def dehydrate_full_name(self, bundle):
return bundle.obj.get_full_name()
def hydrate_full_name(self, bundle):
...
return bundle
- The
SimpleCache
combined withResource.cached_obj_get
caches SINGLE objects only! - Doesn't cache the serialized output
- Doesn't cache the list view
Why?
- More complex behaviors get opinionated fast
- Tastypie would rather be general & give you the tools to build what you need
- Filters and serialization formats make it complex
- Besides...
- https://www.varnish-cache.org/
- Super-fast caching reverse proxy in C
- Already caches by URI/headers
- Way faster than the Django request/response cycle
POST/PUT/DELETE
just pass through- So put Varnish in front of your API (and perhaps the rest of your site) and win in the general case.
- Additionally, use Tastypie's internal caching to further speed up Varnish cache-misses.
- Easy to extend
Resource
to add in more caching - If you get to that point, you're already serving way more load than I ever have.
ModelResource
is just a relatively thin (~300 lines) wrapper on top ofResource
(~1200 lines)- Just the ORM/Model bits.
- So virtually everything in Tastypie is available to non-ORM setups.
- By subclassing from
Resource
and overriding 3 to 9 methods, you can hook up any data source - http://django-tastypie.readthedocs.org/en/latest/non_orm_data_sources.html
(A fair amount of code)
- Takes some work but does a lot for you
- Docs have a more complete example based on Riak
- See also django-tastypie-nonrel.
- Late 2011, tried extracting Tastypie to work anywhere (not just Django). It was called Piecrust.
- http://github.com/toastdriven/piecrust
- Close to functional but failed in terms of complexity and lack of standardization