Serving IPv6 from Cacheboy-1.6.PRE2

I've done the very minimum amount of work required to get Cacheboy-1.6.PRE2 to the point where it'll handle IPv6 client requests. I've put it in front of http://www.cacheboy.net/ which now has v4 and v6 records.

There's still plenty of work to do to bring it up to par with the Squid-3 IPv6 support but that will have to wait a while. Specifically, (if anyone feels up to handling it), the dns, ipcache and fqdncache code all needs to be massaged to support IPv4 and IPv6 handling. It shouldn't be that much work.

Cacheboy-1.6 is definitely now in the "freeze and fix bugs as they creep up" stage. I'll continue the memory allocator and HTTP parser code reimplementation in their respective branches and get them ready for merge once I'm happy 1.6 is stable. The rest of the IPv6 support will also have to wait.

Cacheboy IPv6 update

I've made some progress in the IPv6 reorganisation in cacheboy. I've converted the ACL, authentication and ident code over to support v4/v6. I'm now going to convert over the client_db, request_t structure and then the related stuff like logging, x-forwarded-for, etc. I'll then revisit what else is required before I enable v6 sockets on the http client-side. It -should- be pretty minimal - persistent connections/connection pinning (for just assembling the hash key) and some SNMP code to just gloss over IPv6 connections for the time being.

Hm, I was hoping to have this all done by the end of September but I've been a bit busy with paid work. I'll hopefully have this done just after NYCBSDCON. I hope. :)

IPv6 ACL code, sort of!

I'm just doing a spot of testing with my new IPv6 ACL code.

Take a look at this:

(adrian) agnus:~/work/cacheboy/playpen/ipv6_acl/tools% ./squidclient mgr:config@PASSWORD | grep acl
acl all src 0.0.0.0/0.0.0.0
acl all6 src6 ::/::
acl lclnet6 src6 fe80::/fff0::
acl test1 src6 2a01:348:147:5::/ffff:ffff:ffff:ffff::
acl test1 src6 fe80::/fff0::

That there is an IPv6 "src6" ACL (well, three) with somewhat unfriendly netmask display code. I'll tidy that up later. Importantly, the IPv6 code seems to be coming along fine. I'm going to generate up some large random IPv4 and IPv6 ACLs tomorrow to make sure they load in and display out from the splay tree fine, then I'll look at writing some test cases for all of this.

The last bit of code that needs converting before -very basic- client-side IPv6 support can be enabled is to convvert the ACL checklist struct "src_addr" and "my_addr" over to sqaddr_t IPv6 types. This will probably require a whole lot of horrible code changes but luckily I can convert most of them to just be "assign that an IPv4 address thx" and everything should just work as before. Although I need to remind myself to make sure aclMatchIp() checks the _type_ of the ACL its looking up against - doing an IPv4 lookup against an IPv6 splay tree won't really work out.

(Amos / Squid-3 have a single IPv6 "type" for this, and the IPv4 addresses are merged into the IPv6 address space. The ACL types for IP src/dst/myip is then -always- an IPv6 type lookup. I decided to keep seperate IPv4/IPv6 ACL types for now to make testing and development easier. It will double up on the ACL sizes a little - holy crap, I'm doing something less efficient then Squid-3?!? - but thats a small price to pay at the moment for an easier to migrate codebase. Basically, if you compile this up and listen on an IPv6 address, but don't configure an IPv6 ACL, you won't get surprised when IPv6 requests are let through when they shouldn't..)

Cacheboy-1.5: IPv6 DNS servers

I'm just debugging the last couple of issues with the IPv6-aware UDP/TCP DNS code. The Internal DNS resolver still only understands IPv4 code (and, more importantly, the ipcache/fqdncache layer too!) but the code itself will communicate with IPv4/IPv6 DNS servers.

I think I'll stop the development here and concentrate on getting the Cacheboy-1.5 release out the door. I'll then work on IPv6 record resolution in a seperate branch in preparation for Cacheboy-1.6. I may even break out the ipcache/fqdncache code into external libraries so I can reuse/debug/test that code during development.

Upcoming Cacheboy-1.5.PRE3 development release

(Yes, I've been slack in posting about this stuff.)

I'm just about to roll the next Cacheboy-1.5 development pre-release. Cacheboy-1.5 is probably the last "almost but not quite squid-2.HEAD" release. Besides the IPv6 core, Cacheboy-1.5 resembles the Squid code but with a more sensible layout of modules and libraries.

Its main difference is the inclusion of core comm layer changes to support IPv6 in preparation of IPv6 client and server support. This particular pre-release includes some changes to the internal DNS code to decouple it from a few routines in src/ relating to TCP socket connection. Its possible I've busted stuff - just run cacheboy with "debug_options ALL,1 78,2" for a while to see if you're falling back to TCP DNS properly.

I'm about to put Cacheboy-1.5.PRE3 in production for a couple of clients to get some real world feedback.

Standalone HTTP header parser!

I've finally broken out enough of the HTTP header parsing code from src/ into libhttp/ to run the http header parser standalone.

This allows me to write some test cases to make sure I don't break things whilst changing how the HTTP header parser and HTTP header entry code uses (ie, abuses!) the memory allocator. It's also one step closer to being able to reuse bits of the Squid internals in a "simpler" HTTP proxy core.

I'll commit this code reorganisation to Cacheboy trunk after I've released and tested a few developer previews.

So, without further delay:

test1b: test parsing sample headers
| init-ing hdr: 0x7fffffffe6f0 owner: 2
| parsing hdr: (0x7fffffffe6f0)
Host: www.creative.net.au
Content-type: text/html
Foo: bar


| creating entry 0x60ed40: near 'Host: www.creative.net.au'
| created entry 0x60ed40: 'Host: www.creative.net.au'
| 0x7fffffffe6f0 adding entry: 27 at 0
| creating entry 0x60eda0: near 'Content-type: text/html'
| created entry 0x60eda0: 'Content-Type: text/html'
| 0x7fffffffe6f0 adding entry: 18 at 1
| creating entry 0x60ee00: near 'Foo: bar'
| created entry 0x60ee00: 'Foo: bar'
| 0x7fffffffe6f0 adding entry: 68 at 2
  retval from parse: 1
  Parsed Header: Host: www.creative.net.au
  Parsed Header: Content-Type: text/html
  Parsed Header: Foo: bar
| cleaning hdr: 0x7fffffffe6f0 owner: 2
| destroying entry 0x60ed40: 'Host: www.creative.net.au'
| destroying entry 0x60eda0: 'Content-Type: text/html'
| destroying entry 0x60ee00: 'Foo: bar'

IPv6 core merged into cacheboy trunk

I've just completed merging the IPv6 core into the cacheboy trunk. This doesn't mean it handles IPv6 client/server requests yet - there's a lot more to do before that can happen!

I'll next merge in the IPv6 DNS changes from husni's Squid-2.6 IPv6 patch and do up a basic test suite for all of that. Once done, I'll roll the first Cacheboy-1.5 pre-release.