As I’ve posted before, my wife wanted me to build her a website. Initially, I planned on building it using Plain Old HTML. It was going to be a plain storefront and customers would phone in orders. Then she decided that it would be more convenient if they could upload their images to us rather than email them. CGI would work perfectly for that. Then, we thought that maybe a store catalog and integrated shopping cart would be cool. I started digging into PHP for that. I shied away from TurboGears because I thought hosting would be a problem. After looking around, I decided that hosting was a non-issue, so I built her site in TurboGears.
I settled on Dreamhost for hosting because of price and FastCGI support. FastCGI is one of a handful of methods for deploying Turbogears in a hosted environment. FastCGI has been a source of frustration for me during this process and I don’t expect the frustration to go away any time soon. It just seems really quirky.
I finished my wife’s site yesterday. We did a final walkthrough of the site and I did a few finishing changes. I then began the “deployment to production” process last night. I followed the instructions on the “Installing TurboGears on Dreamhost” wiki.
Thus begins my frustration. Copy my files over. Not a problem. Modify the tg_fastcgi.fcgi script. Not a problem. Make a couple of changes to my TurboGears config file. Not a problem. Drop in a .htaccess file. Not a problem. Test that tg_fastcgi.fcgi runs properly from the command line. Not a problem. Point my browser at my site and get it to kick off the FastCGI process(es). Hmmm. It looked like it was trying to start something. I saw CPU utilization increase, but not on any process I had access to view. Then after what seemed like forever, as if by magic, there were maybe a dozen tg_fastcgi.fcgi processes running. That was liberating. The site was running. And it was pretty snappy, too.
There didn’t appear to be any obvious problems. Except when I needed to change something, then I had to “killall“ the tg_fastcgi.fcgi processes so the change would take effect. FastCGI is apparently more finnicky starting up right after you’ve just killed it. I again saw some unknown process eat a little CPU and then there were entries appearing in my log file that looked like this:
[Wed Jan 18 08:09:17 2006] [error] [client
And then, after a while, it just magically came up.
Again, no obvious problems. Except when I added an item to my shopping cart. When I went to view my cart, there was nothing there. Then, when I clicked “View Cart” again, there was my item. Click again and it’s gone. Click again and it’s there again. Round and round we go. I’ve created a magical disappearing-reappearing shopping cart! Cool! Wait. Not cool. Customers won’t like that. Neither would my wife. I figured that the problem may be caused by the multiple tg_fastcgi.fcgi processes not sharing session data properly. Aarggghh. I switched over from using RAM as session storage to file-based session storage. The problem immediately went away.
Then I started getting 500 errors and entries in my log file that look like this:
server.log: File “/home/(my account)/lib/lib/python2.4/site-packages/CherryPy-2.1.1-py2.4.egg/cherrypy/lib/filter/sessionfilter.py”, line 345, in _lockFile
server.log: raise SessionDeadlockError()
And 500 errors in the browser. And an unusable website. There appears to have been a bug entered against CherryPy which was supposed fix this problem. Maybe I hit a corner case. I don’t know. But it looks like another session-oriented issue. Maybe FastCGI isn’t playing nicely with the session storage files.
So, I have a web application which is difficult to modify quickly because FastCGI doesn’t appear to have a nice “restart” option. (If someone knows of one, I’d appreciate you posting it here. I found a reference to giving a “killall -USR1“, but I really don’t want to try that right now. The server is running OK for the moment). It seemingly randomly spews 500 errors and has session deadlocking issues. There is also sometimes a significant lag during the first request after there have been no requests for a while. The site has been (mostly) fun building. Deployment has been a beast, though.
I’m not blaming Dreamhost or TurboGears or FastCGI or CherryPy or anything else. I’m just venting a bit. It’s good to do that every once in a while. I guess tonight I’ll start trying to find solutions to the relevant problems.