November 27, 2003
Fixing a common/combined Apache log

I was trying another web server statistics package, awstats, based on a testimonial in Alan's blog. Unfortunately, awstats expects a log file to be in the `combined' log format, and blueslugs.com has many entries in the default `common' format (prior to a switch for superior logging). A search showed no obvious small script for this, so I hope that providing massage_log.pl will save someone a few minutes or, at least, a few thousand log entries.

Posted by Stephen at 02:04 PM
November 22, 2003
Dick Venezky Updates

Dina's father has been sick the past year or so, and Dina's been relaying information to friends and family via email. I've archived them all.

Posted by Stephen at 05:06 PM
Caching minidns

I've started using analog to build reports from the blueslugs.com server logs. One of the tools on the site is minidns, which is a small Perl script that runs through the logs, replacing any IPv4 address matches with their DNS-resolved names (if defined). I've improved this slightly, by adding caching inside the application (on the grounds that your site is likely visited by the same communities over time) and, via Storable, to a state file (on the grounds that you probably run the analyzer over your logs rather regularly). [Get cachedns.pl.]

Is it worth it? Read on.

Here are some simple timed runs with the original (minidns.pl) and with cachedns.pl.
$ time perl minidns.pl < ./al.1000 > /dev/null

real    1m56.431s
user    0m0.030s
sys     0m0.030s
al.1000 is the first 1000 lines of an Apache server log. Our name service cache, nscd(1), and the DNS server we're calling (and the perl(1) text) are now reasonably warmed up for subsequent callers.
154 $ time perl minidns.pl < ./al.1000 > /dev/null

real    0m11.210s
user    0m0.030s
sys     0m0.020s
155 $ time perl minidns.pl < ./al.1000 > /dev/null

real    0m12.943s
user    0m0.050s
sys     0m0.010s
So 1000 calls take a little over 10 seconds. Let's run the caching version:
$ time perl cachedns.pl -c dns.cache < ./al.1000 > /dev/null

real    0m8.579s
user    0m0.090s
sys     0m0.020s
So the internal caching is maybe making a little difference. But let's rerun with the now-populated cache file.
$ time perl cachedns.pl -c dns.cache < ./al.1000 > /dev/null

real    0m0.096s
user    0m0.070s
sys     0m0.010s
Since the first part of the log file is processed every night, our cache file means that we're likely only going to perform a DNS lookup for new visitors to the site. (There are many sophisticated DNS resolvers-for-weblogs around, that use C++ or Python or threading or whatever. I just felt that a simple, understandable Perl version, with a boost, was enough for this little site.)
Posted by Stephen at 12:59 PM
November 21, 2003
RSVP, a Flash game

Lifetime TV's website offers a fun solitaire variant, which they call RSVP. Apparently, regular practice will make you an excellent party hostess. ("Now turn to the young lady on your left side, and say something charming...")

Whimsical, well-constructed Flash application: recommended.

Posted by Stephen at 12:12 AM
November 18, 2003
Stash Teas, especially peppermint

Cutbacks at work over the past few years have forced me to take action. As the coffee and teas supplier has moved the offerings from a wide selection of Bigelow teas to a small selection of Bigelow teas to an alternate tea vendor (nameless), the quality of my tea break has dropped so that, now, the tea dust captured by each desiccated bag was only a distant echo of the resonant saga of teas (and tisanes) through the ages.

Something had to be done. Knowing of no obviously heroic possibilities, I turned to ecommerce.

Stash Tea carries a variety of loose and bagged teas. Although I brew from loose teas when deadlines allow, my current workload insists on the highest possible efficiency--a second or three could be the difference between "above average" and "more than satisfactory". My current favourite bagged tea is the peppermint, which is made from dried Oregon-grown mint leaves. (The Darjeeling is nice, but needs multiple bags to brew anything beyond a small pot; the batch of triple ginseng is weaker than I expected.)

Recommended.

Posted by Stephen at 10:16 PM
November 16, 2003
Mark Kurlansky's Cod

Mark Kurlanksy, Cod: a biography of the fish that changed the world, 1997. [Paperback published by Penguin.]

A short book on the role of (mostly) Atlantic cod, which, beyond being yet another species-so-abundant-we-couldn't-help-but-nearly-extinguish-it, was apparently the seventeenth century ocean-crosser's equivalent of interstate service areas (in that great distances could be travelled with comparatively convenient access to food). Kurlansky offers various contemporary recipes for cod-based dishes in between chapters, although I've never seen true salt cod for sale out here in California. (Cloves have been on a multi-century decline from their peak at the "it" herb in the 1700s.)

The discussion never convincingly reconnects to present-day fishermen who are sampling the Grand Banks cod population, although the recollection of the richness of the Banks and the fishing culture of only fifty or sixty years ago is well told. Little discussion of even small-scale aquaculture for cod, or really of the cod ecology, but an excellent survey of the role of an effective natural resource in various periods in the history of North America and Western Europe.

OK.

Posted by Stephen at 08:08 PM