Authority-by-being-employed-by-Google, rather than by-reproducable-data

Many things rub me up the wrong way. This is one of them.

http://developers.slashdot.org/comments.pl?sid=1734346&cid=33051506

In summary - the post asserts some interesting facts (which I believe, having done high volume HTTP stuff myself) but when asked about benchmarks to back up his assertions, he replies:

"Unfortunately, nothing I can publish without permission. I can say that I'm in charge of maintaining the software that terminates all HTTP traffic for Google. Draw your own conclusions."

I really do dislike "I work in this area in Google" as a measure of authority. My conclusion is that the developer in question, as clever as he should be, should likely read some history books on "authority by being high up in the priesthood" and where that takes people.

Grr.

More IPv6 hackery..

I've been spending a little time fleshing out some more of the IPv6 preparation work in Lusca. Since they're rather intrusive patches, I'm doing the work in a separate branch (/playpen/LUSCA_HEAD_ipv6) and will merge back bits and pieces as needed.

I've migrated the client db, external URL rewriters, access logging and the client-facing connection management code over to be IPv6 aware. I still have the request_t state, ACL lookups (which is luckily done - but sitting in a branch!), further DNS work and the protocol-facing stuff (HTTP, FTP.)

There isn't much more work involved in getting LUSCA_HEAD ready enough to serve IPv6-facing clients. That'll let me push out Cacheboy content over IPv6.

But for now, it's back to hacking on commercial, customer code.

FreeBSD/MIPS on AR71xx / Routerstation Pro: NIC alignment/size bug fixed!

I found and squished a small bug in the gigabit NIC driver in FreeBSD/MIPS for the AR71xx chipset. It wasn't all that complicated - TX buffers weren't being thoroughly enough checked for alignment/size constraints before being handed to the DMA engine.

It did however fix a few niggling issues. My tunneling stuff was fixed. IPv6 frames are now correctly handled. And ng_nat doesn't cause a panic in the NIC driver. So there's at least three people who are happy. :)

Cacheboy: Firefox Release 3.6.4

Cacheboy is currently pushing a good 800-1200mbit of firefox 3.6.4 updates. It has about 6% of the total mozilla mirror weight so I predict that there's currently around 16 gigabits of total mozilla updates going out.

Lusca is holding up fine - a single process happily pushed ~ 400mbit on some hosts during the initial peak. I'd obviously like to be able to push a lot more than that but I'm still doing baby steps in the Lusca performance department.

HTTP parser/management changes

I've finally committed changes to Lusca which migrate the HTTP Header management routines away from individual malloc/free calls per HTTP Header entry to (hopefully!) one malloc/free for every N. (N being tunable, of course.)

A lot of incremental preparation work was required in the HTTP parser to tidy things up enough to (hopefully!) minimise any impact on stability and functionality.

There really isn't all that much noticable improvement unless you're working on small, embedded platforms with slow CPUs. It's more of a precursor to further optimisation and reorganisation work. The cumulative effect will be worthwhile.

I've released a tarball with the work - r14674 - which also includes a handful of ATF unit tests in the test-suite/atf/ subdirectory.

I'll likely next work on reorganising some more of the FTP and HTTP protocol specific code which can be migrated out of src/ . I'd like to then spend some more time writing unit tests before (hopefully!) finishing off the IPv6 related DNS changes in preparation for more IPv6 related tomfoolery.

Unit testing using ATF

I've been toying around with the NetBSD unit testing framework called "ATF". Unlike other well-known FOSS unit testing software, ATF seems to have relatively comprehensive support for writing unit tests in C - which is quite important for a piece of C software.

In fact, it seems quite a few popular bits of FOSS software handling unit tests, embedded/API documentation, etc all support C rather poorly. I wonder why.

I've not linked the ATF stubs into the main build tree just yet. The ATF code can be found in the source tree under /test-suite/atf/ . It works enough for me to use "make check" on my iBook (yes, that's a pre-intel, PPC G4 iBook) to check that the basic changes I'm making to the HTTP parser code don't introduce new bugs.

I would love some help writing more unit tests though!

The best news is sometimes no news..

Things have been quiet on the Lusca front lately. I poked some of the users who were reporting earlier issues and asked whether said issues were fixed. All of them responded "yes".

So sometimes no news is good news. But you have to sometimes make sure.