Why use Django on Google App Engine?
Question:
When researching Google App Engine (GAE), it’s clear that using Django is wildly popular for developing in Python on GAE. I’ve been scouring the web to find information on the costs and benefits of using Django, to find out why it’s so popular. While I’ve been able to find a wide variety of sources on how to run Django on GAE and the various methods of doing so, I haven’t found any comparative analysis on why Django is preferable to using the webapp framework provided by Google.
To be clear, it’s immediately apparent why using Django on GAE is useful for developers with an existing skillset in Django (a majority of Python web developers, no doubt) or existing code in Django (where using GAE is more of a porting exercise). My team, however, is evaluating GAE for use on an all-new project and our existing experience is with TurboGears, not Django.
It’s been quite difficult to determine why Django is beneficial to a development team when the BigTable libraries have replaced Django’s ORM, sessions and authentication are necessarily changed, and Django’s templating (if desirable) is available without using the entire Django stack.
Finally, it’s clear that using Django does have the advantage of providing an “exit strategy” if we later wanted to move away from GAE and need a platform to target for the exodus.
I’d be extremely appreciative for help in pointing out why using Django is better than using webapp on GAE. I’m also completely inexperienced with Django, so elaboration on smaller features and/or conveniences that work on GAE are also valuable to me.
Answers:
I have experience using Django and not GAE. From my experiences with Django it was a very simplistic setup and the deployment process was incredibly easy in terms of web projects. Granted I had to learn Python to really get a good hold on things, but at the end of the day I would use it again on a project. This was almost 2 years ago before it reached 1.0 so I my knowledge is a bit outdated.
If you are worried about changing platforms, then this would be a better choice I suppose.
Django probably isn’t the right choice for you, if you’re sure that GAE is right for you. The strengths of the two technologies don’t align very well – you completely lose a lot of Django’s wonderful orm on GAE, and if you do use it, you write code that isn’t really directly suitable to bigtable and the way GAE works.
The thing about GAE is that it gets the great scalability by forcing you to write code that scales easily from the ground up. You just can’t do a number of things that scale poorly (of course, you can still write poorly scaling code, but you avoid some pitfalls). The tradeoff is that you really end up coding around the framework, if you use something like Django which is designed for a different environment.
If you see yourself ever leaving GAE for any reason, becoming invested in the infrastructure there is a problem for you. Coding for bigtable means that it will be harder to move to a different architecture (though the apache project is working to solve this for you with the HBase component of the Hadoop project). It would still be a lot of work to transition off of GAE.
What’s the driving motivator behind using GAE, besides being a Google product, and a cool buzzword? Is there a reason that scaling using something like mediatemple’s offering is unlikely to work well for you? Are you sure that the ways that GAE scales are right for your application? How does the cost compare to dedicated servers, if you’re expecting to get to that performance realm? Can you solve your problem well using the tools GAE provides, as compared to a more traditional load-balanced server setup?
All this said, unless you absolutely positively need the borderline-ridiculous scaling that GAE offers, I’d personally suggest not letting that particular service structure your choice of framework. I like Django, so I’d say you should use it, but not on GAE.
Edit (June 2010):
As an update to this comment sometime later:
Google has announced sql-like capabilitys for GAE that aren’t free, but will let you easily do things like run SQL-style commands to generate reports on your data.
Additionally, there are upcoming changes to the GAE query language which will allow complex queries in a far easier fashion. Look at the videos from Google I/O 2010.
Furthermore, there is work being done during the Summer of Code 2010 project which should bring no-sql support to django core, and by extension, make working with GAE significantly easier.
GAE is becoming more attractive as a hosting platform.
Edit (August 2011):
And Google just raised the cost to most users of the platform significantly by changing the pricing structure. The lockin problem has gotten better (if your application is big enough you can deploy the apache alternatives), but for most applications, running servers or VPS deployments is cheaper.
Very few people really have bigdata problems. “Oh my startup might scale someday” isn’t a bigdata problem. Build stuff now and get it out the door using the standard tools.
I’ve done lots of projects on GAE. Some in django, some in their normal framework.
For small things, I usually use their normal framework for simplicity and quickness. Like http://stdicon.com, http://yaml-online-parser.appspot.com/, or http://text-twist.appspot.com/.
For large things, I go with django to take advantage of all the nice middleware and plugins. Like http://metaward.com.
Basically my litmus test is Will this take me more than 2 weeks to write and be a REAL software project? If so, go with django for the addons.
It has the added benefit of, if your project is badly suited for BigTable then you quickly port off (like I did Is BigTable slow or am I dumb?)
I cannot answer the question but you may want to look into web2py. It is similar to Django in many respects but its database abstraction layer works on GAE and supports most of the GAE functionality (not all but we try to catch up). In this way if GAE works for you great, if it does not, you can move your code to a different db (SQLite, MySQL, PostgreSQL, Oracle, MSSQL, FireBird, DB2, Informix, Ingres, and – soon – Sybase and MongoDB).
We use django on our appengine instances mostly when we have to serve actual websites to the user. It has a great template engine, url routing and all the request/response/error handling built in. So even while we can’t use the magic orm/admin stuff it has a lot going for it.
For api services we built something very simple on top of webob. It’s far more lightweight because it doesn’t need everything that django offers, and therefore a little quicker in some situations.
If you decide to run you app outside of GAE, you can still use Django. You won’t really have that much luck with the GAE webapp
I am still very new to Google App engine development, but the interfaces Django provides do appear much nicer than the default. The benefits will depend on what you are using to run Django on the app engine. The Google App Engine Helper for Django allows you to use the full power of the Google App Engine with some Django functionality on the side.
Django non-rel attempts to provide as much of Django’s power as possible, but running on the app-engine for possible extra scalability. In particular, it includes Django models (one of Django’s core features), but this is a leaky abstraction due to the differences between relational databases and bigtable. There will most likely be tradeoffs in functionality and efficiency, as well as an increased number of bugs and quirks. Of course, this might be worth it in circumstances like those described in the question, but otherwise would strongly recommend using the helper at the start as then you have the option of moving towards either pure app-engine or Django non-rel later. Also, if you do switch to Django non-rel, your increased knowledge of how app engine works will be useful if the Django abstraction ever breaks – certainly much more useful than knowledge of the quirks/workarounds for Django non-rel if you swap the other way.
I think all this answers are a bit obsolete.
Now you can use Google Cloud SQL
Django is a popular third-party Python web framework. When coupled
with Google Cloud SQL, all of its functionality can be fully supported
by applications running on App Engine. Support for using Google Cloud
SQL with Django is provided by a custom Django database backend which
wraps Django’s MySQL backend.
https://cloud.google.com/python/django/appengine
one more fresh news is, that there is BETA support for PostgreSQL
When researching Google App Engine (GAE), it’s clear that using Django is wildly popular for developing in Python on GAE. I’ve been scouring the web to find information on the costs and benefits of using Django, to find out why it’s so popular. While I’ve been able to find a wide variety of sources on how to run Django on GAE and the various methods of doing so, I haven’t found any comparative analysis on why Django is preferable to using the webapp framework provided by Google.
To be clear, it’s immediately apparent why using Django on GAE is useful for developers with an existing skillset in Django (a majority of Python web developers, no doubt) or existing code in Django (where using GAE is more of a porting exercise). My team, however, is evaluating GAE for use on an all-new project and our existing experience is with TurboGears, not Django.
It’s been quite difficult to determine why Django is beneficial to a development team when the BigTable libraries have replaced Django’s ORM, sessions and authentication are necessarily changed, and Django’s templating (if desirable) is available without using the entire Django stack.
Finally, it’s clear that using Django does have the advantage of providing an “exit strategy” if we later wanted to move away from GAE and need a platform to target for the exodus.
I’d be extremely appreciative for help in pointing out why using Django is better than using webapp on GAE. I’m also completely inexperienced with Django, so elaboration on smaller features and/or conveniences that work on GAE are also valuable to me.
I have experience using Django and not GAE. From my experiences with Django it was a very simplistic setup and the deployment process was incredibly easy in terms of web projects. Granted I had to learn Python to really get a good hold on things, but at the end of the day I would use it again on a project. This was almost 2 years ago before it reached 1.0 so I my knowledge is a bit outdated.
If you are worried about changing platforms, then this would be a better choice I suppose.
Django probably isn’t the right choice for you, if you’re sure that GAE is right for you. The strengths of the two technologies don’t align very well – you completely lose a lot of Django’s wonderful orm on GAE, and if you do use it, you write code that isn’t really directly suitable to bigtable and the way GAE works.
The thing about GAE is that it gets the great scalability by forcing you to write code that scales easily from the ground up. You just can’t do a number of things that scale poorly (of course, you can still write poorly scaling code, but you avoid some pitfalls). The tradeoff is that you really end up coding around the framework, if you use something like Django which is designed for a different environment.
If you see yourself ever leaving GAE for any reason, becoming invested in the infrastructure there is a problem for you. Coding for bigtable means that it will be harder to move to a different architecture (though the apache project is working to solve this for you with the HBase component of the Hadoop project). It would still be a lot of work to transition off of GAE.
What’s the driving motivator behind using GAE, besides being a Google product, and a cool buzzword? Is there a reason that scaling using something like mediatemple’s offering is unlikely to work well for you? Are you sure that the ways that GAE scales are right for your application? How does the cost compare to dedicated servers, if you’re expecting to get to that performance realm? Can you solve your problem well using the tools GAE provides, as compared to a more traditional load-balanced server setup?
All this said, unless you absolutely positively need the borderline-ridiculous scaling that GAE offers, I’d personally suggest not letting that particular service structure your choice of framework. I like Django, so I’d say you should use it, but not on GAE.
Edit (June 2010):
As an update to this comment sometime later:
Google has announced sql-like capabilitys for GAE that aren’t free, but will let you easily do things like run SQL-style commands to generate reports on your data.
Additionally, there are upcoming changes to the GAE query language which will allow complex queries in a far easier fashion. Look at the videos from Google I/O 2010.
Furthermore, there is work being done during the Summer of Code 2010 project which should bring no-sql support to django core, and by extension, make working with GAE significantly easier.
GAE is becoming more attractive as a hosting platform.
Edit (August 2011):
And Google just raised the cost to most users of the platform significantly by changing the pricing structure. The lockin problem has gotten better (if your application is big enough you can deploy the apache alternatives), but for most applications, running servers or VPS deployments is cheaper.
Very few people really have bigdata problems. “Oh my startup might scale someday” isn’t a bigdata problem. Build stuff now and get it out the door using the standard tools.
I’ve done lots of projects on GAE. Some in django, some in their normal framework.
For small things, I usually use their normal framework for simplicity and quickness. Like http://stdicon.com, http://yaml-online-parser.appspot.com/, or http://text-twist.appspot.com/.
For large things, I go with django to take advantage of all the nice middleware and plugins. Like http://metaward.com.
Basically my litmus test is Will this take me more than 2 weeks to write and be a REAL software project? If so, go with django for the addons.
It has the added benefit of, if your project is badly suited for BigTable then you quickly port off (like I did Is BigTable slow or am I dumb?)
I cannot answer the question but you may want to look into web2py. It is similar to Django in many respects but its database abstraction layer works on GAE and supports most of the GAE functionality (not all but we try to catch up). In this way if GAE works for you great, if it does not, you can move your code to a different db (SQLite, MySQL, PostgreSQL, Oracle, MSSQL, FireBird, DB2, Informix, Ingres, and – soon – Sybase and MongoDB).
We use django on our appengine instances mostly when we have to serve actual websites to the user. It has a great template engine, url routing and all the request/response/error handling built in. So even while we can’t use the magic orm/admin stuff it has a lot going for it.
For api services we built something very simple on top of webob. It’s far more lightweight because it doesn’t need everything that django offers, and therefore a little quicker in some situations.
If you decide to run you app outside of GAE, you can still use Django. You won’t really have that much luck with the GAE webapp
I am still very new to Google App engine development, but the interfaces Django provides do appear much nicer than the default. The benefits will depend on what you are using to run Django on the app engine. The Google App Engine Helper for Django allows you to use the full power of the Google App Engine with some Django functionality on the side.
Django non-rel attempts to provide as much of Django’s power as possible, but running on the app-engine for possible extra scalability. In particular, it includes Django models (one of Django’s core features), but this is a leaky abstraction due to the differences between relational databases and bigtable. There will most likely be tradeoffs in functionality and efficiency, as well as an increased number of bugs and quirks. Of course, this might be worth it in circumstances like those described in the question, but otherwise would strongly recommend using the helper at the start as then you have the option of moving towards either pure app-engine or Django non-rel later. Also, if you do switch to Django non-rel, your increased knowledge of how app engine works will be useful if the Django abstraction ever breaks – certainly much more useful than knowledge of the quirks/workarounds for Django non-rel if you swap the other way.
I think all this answers are a bit obsolete.
Now you can use Google Cloud SQL
Django is a popular third-party Python web framework. When coupled
with Google Cloud SQL, all of its functionality can be fully supported
by applications running on App Engine. Support for using Google Cloud
SQL with Django is provided by a custom Django database backend which
wraps Django’s MySQL backend.
https://cloud.google.com/python/django/appengine
one more fresh news is, that there is BETA support for PostgreSQL