Sunday, January 20, 2008

Boston Python Life

My old habit of missing Boston python meetups for no good reason is over. I'm now in a Wednesday bowling league so I have a good reason.
Or a reason, anyway.

The Wingware hosted Boston-PIG list seems dead. There was some shuffle when the Boston group became the Cambridge group. I think there is a Plone group too which is sometimes the same people. Maybe Doug could clear that up.

Will Guaraldi's Nomadic Telecommuting Herd occasionally meets across the street from me. Maybe I'll run into them sometime (but they don't have a schedule, so that's hard to do).

Monday, January 14, 2008

In The Beginning

This is the story of how I started using python.

Out of college I was a well indoctrinated C++ guy. My resume had silly goals on it like "desires to work at a CMM level 3 or higher organization." It was the mid 90s so I ended up working at a C++ based dot com instead (Musicblvd and later CDNow). I left two years later with my youthful enthusiasm for the "correct way" way stripped. It was now the late 90s and my next job was working at a Perl based dot com in Boston (OpenAir). Things were just so much easier with Perl. And everything was different: OpenAir didn't have a single machine that cost over $5k whereas Musicblvd had more than one that cost over $1Mil (a fully loaded E10k ran the database*). Release cycles were monthly instead of annually. Everything was so plainly better and more productive it wasn't even funny. I was no longer a C++ guy.

So when I quit during the bust in 2001 to start my own company I wanted to use perl. I was afraid to for legal reasons so I started casting around for another "P" language. We did the prototype in PHP. The prototype worked but the code was ugly and slow. Python had some buzz so I downloaded that and translated our small app from PHP to Python. The resulting code was a third smaller, easier to read, and ran faster. If that wasn't enough the C guy in me loved the Python core. Compared to Perl and PHP the Python core was simple, clean, and without much baggage. I was sold.

It is many years later and I'm still happy with Python. I have even added one new feature to python (class decorators) and of course, one bug too.

* and that 256 processor Sun wasn't infallible. When it crashed (which it did) Sun helicoptered in tech support and basically paid us off not to talk about it.

Sunday, January 13, 2008

Three Flavors of mod_*

Ian Bicking talks about the conceptual flavors of web deployment. Go read that because this post is only tangential to his point.

Under the Apache 1.x series I have used mod_perl, mod_php, and mod_python. All three do things slightly differently with regards to loading code. Keep in mind apache 1.x only uses the pre-forking model so each request handling process starts life as an exact copy of the root process. The root process's main job is to spawn child processes. [everything below was true in 2000 and for apache 1.x. Don't make decisions based on my recolections unless you plan on running some ancient servers.]

mod_perl allows you to preload scripts when the root process starts. This means no matter how large your code base you only have to compile and execute it once and then every child process starts life with the same initialized modules already in place. At a perl-based company where I worked the preload script was a single line
    use Everything;
The 'Everything' package loaded, well, everything (is this a standard perl-ism nowadays?). was the most delicate module in a large code base. It imported all the packages in the code base in a way that avoided cycles. As a downside the server couldn't use apache's "graceful" restart because the root server didn't have a way to unload the perl runtime and reload Everything. Besides reloading everything was slooow on the 400MHz CPUs of the day. The work-around was to have a proxy server that could point to one of two local ports that ran mod_perl. To gracefully switch the unused app server was stopped and restarted and the proxy was repointed to the new local port.

mod_php is a bit of yuck by comparison. Or a joy for the reasons Bicking states. The yuck is that mod_php compiles and executes scripts anew on every request. This is great for shared hosts as it acts like CGI but with less overhead because it doesn't have to fork on every request. This sucks for application servers that know they want to run the same code over and over again. It gets worse when the codebase is large and there is a package like that loads the entire codebase of a good sized application on every request [yes, this post is somewhat autobiographical].

mod_python is somewhere between mod_perl and mod_php. The first time a child handles a request it pays the price of loading and executing a script. After that all requests handled by the child are quick because sys.modules still contains all the modules loaded by previous runs. mod_python could support preloading modules in the parent process like mod_perl with a trivial patch [see the mod_python archives circa 2001 for the patch I submitted] but it was decided that the security risk (because the parent process usually starts as root) outweighed the benefit of child nodes having a longer startup time.

In conclusion I would urge you to read the source of the mod_whatever for the language you use. They are small and all work give-or-take the same. It also gives you some idea of how the internals of your favorite language work. My first trip was into mod_perl and I found it readable but with a lot of baggage. When I read mod_php I thought "what a hack! Is this broken on purpose?" Finally I read mod_python and with my perl background I was aghast at how simple and clean the interpreter related code was.

See you at PyCon!