How do I use python for web development without relying on a framework?
Question:
I know the various frameworks have their benefits, but I personally want my web development in python to be as straight-forward as possible: less writing to the framework, more writing python.
The only thing I have found so far that lets me do this in the most obvious way possible is web.py but I have slight concerns on its performance.
For those of you using nginx(or another flavour)+mod_wsgi+web.py… how’s performance? Can it be improved further?
For those of you who have used web.py, liked the idea and went on to write something better or found something better… care to point me to the source?
I’d like to hear about all the conspicuous, minimal yet powerful approaches.
Answers:
For what it’s worth, I wrote my website in mod_python without any intervening framework like Django. I didn’t really have any reason to complain. (Well maybe a little, mod_python is kind of quirky in a few ways but not in the common use cases) One thing’s for sure, it will definitely let you write Python 😉
The way to go is wsgi.
WSGI is the Web Server Gateway Interface. It is a specification for web servers and application servers to communicate with web applications (though it can also be used for more than that). It is a Python standard, described in detail in PEP 333.
All current frameworks support wsgi. A lot of webservers support it also (apache included, through mod_wsgi). It is the way to go if you want to write your own framework.
Here is hello world, written to wsgi directly:
def application(environ, start_response):
status = '200 OK'
response_headers = [('Content-type','text/plain')]
start_response(status, response_headers)
return ['Hello world!n']
Put this in a file.py, point your mod_wsgi apache configuration to it, and it will run. Pure python. No imports. Just a python function.
If you are really writing your own framework, you could check werkzeug. It is not a framework, but a simple collection of various utilities for WSGI applications and has become one of the most advanced WSGI utility modules. It includes a powerful debugger, full featured request and response objects, HTTP utilities to handle entity tags, cache control headers, HTTP dates, cookie handling, file uploads, a powerful URL routing system and a bunch of community contributed addon modules. Takes the boring part out of your hands.
You could also check cherrypy. The focus of cherrypy is in being a framework that lets you write python. Cherrypy has its own pretty good webserver, but it is wsgi-compatible so you can run cherrypy applications in apache via mod_wsgi. Here is hello world in cherrypy:
import cherrypy
class HelloWorld(object):
def index(self):
return "Hello World!"
index.exposed = True
cherrypy.quickstart(HelloWorld())
What’s wrong with Django? It doesn’t force you to use it’s ORM and controllers are just plain Python functions instead of Rails-like class methods. Also, url routing is done with regular expressions instead of another framework-invented syntax. If django seems like too much for you anyway, i recommend taking a look at Werkzeug
It’s hilarious how, even prompted with a question asking how to write without a framework, everyone still piles in to promote their favourite framework. The OP whinges about not wanting a “heavyweight framework”, and the replies mention Twisted, of all things?! Come now, really.
Yes, it is perfectly possible to write straight WSGI apps, and grab bits of wanted functionality from standalone modules, instead of fitting your code into one particular framework’s view of the world.
To take this route you’ll generally want to be familiar with the basics of HTTP and CGI (since WSGI inherits an awful lot from that earlier specification). It’s not necessarily an approach to recommend to beginners, but it’s quite doable.
I’d like to hear about all the conspicuous, minimal yet powerful approaches
You won’t hear about them, because no-one has a tribal interest in promoting “do it yourself” as a methodology. Me, I use a particular standalone templating package, a particular standalone form-reading package, a particular data access layer, and a few home-brew utility modules. I’m not writing to one particular philosophy I can proselytise about, they’re all just boring tools that could be swapped out and replaced with something else just as good.
I’m pretty fond of Google AppEngine. I use the ORM and templating system, but otherwise follow a REST-patterned design and just implement Python methods for the corresponding HTTP ones. It makes the raw HTTP interaction central, and optionally gives you other things to use. Plus no more configuring and managing your deployment environment!
I’ve written a few small web applications using mod-python and PSP — mod-python’s equivalent to php.
In one case, I wrote a web page that generates release notes by inspecting our source code repository. I rewrote it into PHP, and was surprised to discover that the PSP version was about 20% faster, as well as being about half as many lines of code.
So, for small problems at least, psp has worked well for me.
+1 to all the answers with WSGI.
Eric Florenzo wrote a great blog post lately you should read: Writing Blazing Fast, Infinitely Scalable, Pure-WSGI Utilities. This will give you a better idea of pure WSGI beyond Hello World. Also pay attention to the comments, especially the first comment by Kevin Dangoor where he recommends at least adding WebOb to your toolset.
I think it depends on the definition of what a framework is and what it should do for you.
As pointed out, a very minimal “framework” would be WSGI as it only defines one small interface for interfacing with a web server. But it’s a powerful approach because of the middleware you can put between your app and the server.
If you want more slightly more, like some URL to function mapping, then you have some choices, some of which have been already mentioned.
If you go further you might come to Pylons or Turbogears or Django, after that maybe Zope but it grows bigger and maybe the pain as well as you always buy into the opinions of that framework.
What I recently use more and more (coming from Zope/Plone) is repoze.bfg. It’s very small, does not come with a ORM bundled (so you can use SQLAlchemy, Storm or simply go to an object database like the ZODB). What it does is basically handling how you come from a URL to a view which is a function. It supports both URL Mapping (a la Routes) or object traversal, which IMHO is very powerful in some circumstances esp. if you have a not so strict mapping. The good thing is that it directly comes with an ACL based security framework which can use if you want to which IMHO is very practical to have. That way you don’t need decorators which seem to be used mostly for such things.
And of course it’s based on WSGI. Also look for the repoze subversion repository for quite a lot of middleware and the Paste stuff is also very useful for WSGI related tasks.
Why do you have concerns about web.py’s performance? As I mentioned here, we use CherryPy (the web server “built into” web.py) behind nginx to serve most of the HTML at Oyster.com — nginx splits the traffic across 2 or 3 web servers each running 4 Python processes, and we can easily handle 100s of requests per second.
Oyster.com is a high-volume website averaging 200,000 dynamically-generated pageviews/day, and peaking to much higher numbers than that. However, we do use a content delivery network (CDN) for our static resources like images and CSS.
We definitely care about performance (most of our pages render in less than 25ms), but web.py isn’t the bottleneck. Our bottlenecks are template rendering (we use Cheetah, which is fast enough but not other-worldly fast) and database queries (we cache heavily and keep the number of database queries per page to 0 or 1) and accessing our 3rd-party hotel pricing providers (these are accessed when you do a search with dates we don’t already have cached).
Remember, premature optimization is the root of all evil — unless you’re serving google.com, web.py will probably work for you.
I know the various frameworks have their benefits, but I personally want my web development in python to be as straight-forward as possible: less writing to the framework, more writing python.
The only thing I have found so far that lets me do this in the most obvious way possible is web.py but I have slight concerns on its performance.
For those of you using nginx(or another flavour)+mod_wsgi+web.py… how’s performance? Can it be improved further?
For those of you who have used web.py, liked the idea and went on to write something better or found something better… care to point me to the source?
I’d like to hear about all the conspicuous, minimal yet powerful approaches.
For what it’s worth, I wrote my website in mod_python without any intervening framework like Django. I didn’t really have any reason to complain. (Well maybe a little, mod_python is kind of quirky in a few ways but not in the common use cases) One thing’s for sure, it will definitely let you write Python 😉
The way to go is wsgi.
WSGI is the Web Server Gateway Interface. It is a specification for web servers and application servers to communicate with web applications (though it can also be used for more than that). It is a Python standard, described in detail in PEP 333.
All current frameworks support wsgi. A lot of webservers support it also (apache included, through mod_wsgi). It is the way to go if you want to write your own framework.
Here is hello world, written to wsgi directly:
def application(environ, start_response):
status = '200 OK'
response_headers = [('Content-type','text/plain')]
start_response(status, response_headers)
return ['Hello world!n']
Put this in a file.py, point your mod_wsgi apache configuration to it, and it will run. Pure python. No imports. Just a python function.
If you are really writing your own framework, you could check werkzeug. It is not a framework, but a simple collection of various utilities for WSGI applications and has become one of the most advanced WSGI utility modules. It includes a powerful debugger, full featured request and response objects, HTTP utilities to handle entity tags, cache control headers, HTTP dates, cookie handling, file uploads, a powerful URL routing system and a bunch of community contributed addon modules. Takes the boring part out of your hands.
You could also check cherrypy. The focus of cherrypy is in being a framework that lets you write python. Cherrypy has its own pretty good webserver, but it is wsgi-compatible so you can run cherrypy applications in apache via mod_wsgi. Here is hello world in cherrypy:
import cherrypy
class HelloWorld(object):
def index(self):
return "Hello World!"
index.exposed = True
cherrypy.quickstart(HelloWorld())
What’s wrong with Django? It doesn’t force you to use it’s ORM and controllers are just plain Python functions instead of Rails-like class methods. Also, url routing is done with regular expressions instead of another framework-invented syntax. If django seems like too much for you anyway, i recommend taking a look at Werkzeug
It’s hilarious how, even prompted with a question asking how to write without a framework, everyone still piles in to promote their favourite framework. The OP whinges about not wanting a “heavyweight framework”, and the replies mention Twisted, of all things?! Come now, really.
Yes, it is perfectly possible to write straight WSGI apps, and grab bits of wanted functionality from standalone modules, instead of fitting your code into one particular framework’s view of the world.
To take this route you’ll generally want to be familiar with the basics of HTTP and CGI (since WSGI inherits an awful lot from that earlier specification). It’s not necessarily an approach to recommend to beginners, but it’s quite doable.
I’d like to hear about all the conspicuous, minimal yet powerful approaches
You won’t hear about them, because no-one has a tribal interest in promoting “do it yourself” as a methodology. Me, I use a particular standalone templating package, a particular standalone form-reading package, a particular data access layer, and a few home-brew utility modules. I’m not writing to one particular philosophy I can proselytise about, they’re all just boring tools that could be swapped out and replaced with something else just as good.
I’m pretty fond of Google AppEngine. I use the ORM and templating system, but otherwise follow a REST-patterned design and just implement Python methods for the corresponding HTTP ones. It makes the raw HTTP interaction central, and optionally gives you other things to use. Plus no more configuring and managing your deployment environment!
I’ve written a few small web applications using mod-python and PSP — mod-python’s equivalent to php.
In one case, I wrote a web page that generates release notes by inspecting our source code repository. I rewrote it into PHP, and was surprised to discover that the PSP version was about 20% faster, as well as being about half as many lines of code.
So, for small problems at least, psp has worked well for me.
+1 to all the answers with WSGI.
Eric Florenzo wrote a great blog post lately you should read: Writing Blazing Fast, Infinitely Scalable, Pure-WSGI Utilities. This will give you a better idea of pure WSGI beyond Hello World. Also pay attention to the comments, especially the first comment by Kevin Dangoor where he recommends at least adding WebOb to your toolset.
I think it depends on the definition of what a framework is and what it should do for you.
As pointed out, a very minimal “framework” would be WSGI as it only defines one small interface for interfacing with a web server. But it’s a powerful approach because of the middleware you can put between your app and the server.
If you want more slightly more, like some URL to function mapping, then you have some choices, some of which have been already mentioned.
If you go further you might come to Pylons or Turbogears or Django, after that maybe Zope but it grows bigger and maybe the pain as well as you always buy into the opinions of that framework.
What I recently use more and more (coming from Zope/Plone) is repoze.bfg. It’s very small, does not come with a ORM bundled (so you can use SQLAlchemy, Storm or simply go to an object database like the ZODB). What it does is basically handling how you come from a URL to a view which is a function. It supports both URL Mapping (a la Routes) or object traversal, which IMHO is very powerful in some circumstances esp. if you have a not so strict mapping. The good thing is that it directly comes with an ACL based security framework which can use if you want to which IMHO is very practical to have. That way you don’t need decorators which seem to be used mostly for such things.
And of course it’s based on WSGI. Also look for the repoze subversion repository for quite a lot of middleware and the Paste stuff is also very useful for WSGI related tasks.
Why do you have concerns about web.py’s performance? As I mentioned here, we use CherryPy (the web server “built into” web.py) behind nginx to serve most of the HTML at Oyster.com — nginx splits the traffic across 2 or 3 web servers each running 4 Python processes, and we can easily handle 100s of requests per second.
Oyster.com is a high-volume website averaging 200,000 dynamically-generated pageviews/day, and peaking to much higher numbers than that. However, we do use a content delivery network (CDN) for our static resources like images and CSS.
We definitely care about performance (most of our pages render in less than 25ms), but web.py isn’t the bottleneck. Our bottlenecks are template rendering (we use Cheetah, which is fast enough but not other-worldly fast) and database queries (we cache heavily and keep the number of database queries per page to 0 or 1) and accessing our 3rd-party hotel pricing providers (these are accessed when you do a search with dates we don’t already have cached).
Remember, premature optimization is the root of all evil — unless you’re serving google.com, web.py will probably work for you.