<?xml version='1.0' encoding='UTF-8'?><?xml-stylesheet href="http://www.blogger.com/styles/atom.css" type="text/css"?><feed xmlns='http://www.w3.org/2005/Atom' xmlns:openSearch='http://a9.com/-/spec/opensearchrss/1.0/' xmlns:georss='http://www.georss.org/georss' xmlns:gd='http://schemas.google.com/g/2005' xmlns:thr='http://purl.org/syndication/thread/1.0'><id>tag:blogger.com,1999:blog-4361733293216582854</id><updated>2012-02-13T16:40:50.405-08:00</updated><category term='lighttpd'/><category term='wiki'/><category term='proxy'/><category term='as250'/><category term='cpc'/><category term='usenet'/><category term='news'/><category term='amstrad'/><category term='books'/><category term='apple'/><category term='videolan'/><category term='amiga'/><category term='ubiqiti'/><category term='unit-testing'/><category term='cacheboy'/><category term='nntp'/><category term='ATF'/><category term='nocleanfeed'/><category term='censorship'/><category term='anycast'/><category term='threading'/><category term='sugarlabs'/><category term='AR9100'/><category term='compsci'/><category term='opensource'/><category term='cpc6128'/><category term='nnrp'/><category term='internet'/><category term='windows'/><category term='atheros 802.11n'/><category term='freebsd'/><category term='performance'/><category term='Iusca'/><category term='802.11n'/><category term='armsrace'/><category term='lusca'/><category term='embedded'/><category term='newyork'/><category term='cpc464'/><category term='nbn'/><category term='geoip'/><category term='coss'/><category term='802.11h'/><category term='openbsd'/><category term='tproxy'/><category term='olpc'/><category term='dfs'/><category term='logic'/><category term='cygwin'/><category term='downtime'/><category term='programming'/><category term='BGP'/><category term='holiday'/><category term='ubiquity'/><category term='meatpies'/><category term='cacheboy ipv6'/><category term='ibook'/><category term='windowsupdates'/><category term='heinlein'/><category term='cdn'/><category term='concurrency'/><category term='life'/><category term='squid'/><category term='wishlist'/><category term='AR9132'/><category term='oprofile'/><category term='mips'/><category term='dns'/><category term='Linux'/><category term='wccp'/><category term='quagga'/><category term='modularity'/><category term='openwrt'/><category term='mozilla'/><category term='fail'/><category term='atheros'/><category term='cyberduck'/><category term='caching'/><category term='profiling'/><category term='ipv6'/><category term='university'/><category term='google'/><category term='filtering'/><category term='retrocomputing'/><title type='text'>Adrian Chadd's Ramblings</title><subtitle type='html'></subtitle><link rel='http://schemas.google.com/g/2005#feed' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/posts/default'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default?max-results=100'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/'/><link rel='hub' href='http://pubsubhubbub.appspot.com/'/><link rel='next' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default?start-index=101&amp;max-results=100'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><generator version='7.00' uri='http://www.blogger.com'>Blogger</generator><openSearch:totalResults>199</openSearch:totalResults><openSearch:startIndex>1</openSearch:startIndex><openSearch:itemsPerPage>100</openSearch:itemsPerPage><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-5802620937262074927</id><published>2012-02-13T16:29:00.000-08:00</published><updated>2012-02-13T16:40:50.459-08:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='freebsd'/><category scheme='http://www.blogger.com/atom/ns#' term='Linux'/><title type='text'>.. and the price of packaging up software? Billions.</title><content type='html'>A friend of mine from Perth recently wrote an article outlining the "cost" of Debian Wheezy:&lt;br /&gt;&lt;br /&gt;&lt;a href="http://blog.james.rcpt.to/2012/02/13/debian-wheezy-us19-billion-your-price-free/" target="_blank"&gt;http://blog.james.rcpt.to/&lt;wbr&gt;2012/02/13/debian-wheezy-us19-&lt;wbr&gt;billion-your-price-free/&lt;/a&gt;&lt;span class="HOEnZb"&gt;&lt;span style="color:#888888;"&gt;&lt;br /&gt;&lt;br /&gt;To quote:&lt;br /&gt;&lt;br style="font-style: italic;"&gt;&lt;/span&gt;&lt;/span&gt;&lt;span style="font-style: italic;"&gt;"In my analysis the projected cost of producing &lt;/span&gt;&lt;strong style="font-style: italic;"&gt;Debian Wheezy in February 2012 is US$19,070,177,727 (AU$17.7B, EUR€14.4B, GBP£12.11B)&lt;/strong&gt;&lt;span style="font-style: italic;"&gt;, making each package’s upstream source code wrth an average of &lt;/span&gt;&lt;strong style="font-style: italic;"&gt;US$1,112,547.56 (AU$837K)&lt;/strong&gt;&lt;span style="font-style: italic;"&gt; to produce. Impressively, this is all free (of &lt;/span&gt;&lt;em style="font-style: italic;"&gt;cost&lt;/em&gt;&lt;span style="font-style: italic;"&gt;)."&lt;/span&gt;&lt;br /&gt;&lt;br /&gt;Now this has apparently caused a bit of a stir among Linux and IT news sites. It's a large number, right? It's all free, right?&lt;br /&gt;&lt;br /&gt;However - Debian for the most part is a package repository. Sure, a lot of effort goes into building and maintaining that - and I think _that_ should be assigned a cost - but I think counting all of those package upstreams as part of Debian is hiding the true nature of software development.&lt;br /&gt;&lt;br /&gt;According to Ohloh, the cost of producing FreeBSD, at $72,000 a year per programmer, &lt;a href="https://www.ohloh.net/p/freebsd/estimated_cost"&gt;would be $ &lt;/a&gt;&lt;span id="cocomo_value"&gt;&lt;a href="https://www.ohloh.net/p/freebsd/estimated_cost"&gt;243,777,135,&lt;/a&gt; _before_ all of the packages in the FreeBSD ports repository. FreeBSD has over 23,000 packages too, just like Debian - if those were also counted, it may start to push that figure far up from millions to billions.&lt;br /&gt;&lt;br /&gt;Does it mean Debian Wheezy is equivalent of $20B of effort? Maybe. But then a tiny (comparatively) more effort and you end up with FreeBSD. Or, with different effort - Redhat. Or Gentoo. Or Mandrake. Or NetBSD. Or OpenBSD. Or MacPorts/Fink, which packages this similar software for MacOS X.&lt;br /&gt;&lt;br /&gt;What I guess I'm trying to say is this. You get cool stuff from programmers for free. Including that in your project "cost" just seems silly. &lt;/span&gt;&lt;span id="cocomo_value"&gt;&lt;/span&gt;&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-5802620937262074927?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/5802620937262074927/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2012/02/and-price-of-packaging-up-software.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/5802620937262074927'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/5802620937262074927'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2012/02/and-price-of-packaging-up-software.html' title='.. and the price of packaging up software? Billions.'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-9204442519367519775</id><published>2011-11-26T23:48:00.001-08:00</published><updated>2011-11-26T23:55:15.134-08:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='freebsd'/><category scheme='http://www.blogger.com/atom/ns#' term='802.11n'/><category scheme='http://www.blogger.com/atom/ns#' term='atheros'/><title type='text'>FreeBSD on the TP-Link TL-WR1043nd!</title><content type='html'>I've now included (almost) all of the support needed to run FreeBSD natively on the TP-Link TL-WR1043nd. It's a 3-antenna, 2x2 stream 2.4ghz 802.11n AP which you can get for under AUD $100.&lt;br /&gt;&lt;br /&gt;It supports hostap mode (which is what I bet most of you want to use) and I'm currently using it at home alongside my Ubiquiti Routerstation Pro based hostap (which is what I use to test out all the other pre-11n and 11n NICs that I currently own.)&lt;br /&gt;&lt;br /&gt;I currently get around 50mbit TCP throughput - but I leave full FreeBSD-HEAD debugging on. I'm sure I can push the unit closer to 100mbit. (Compare to the Routerstation Pro + AR9160 hostap - where I routinely get 160mbit of TCP throughput.)&lt;br /&gt;&lt;br /&gt;What works (read: what I've tested):&lt;br /&gt;&lt;ul&gt;&lt;li&gt;Ethernet (at least the WAN port);&lt;/li&gt;&lt;li&gt;Wireless - 802.11bgn - 20/40mhz operation as well as legacy operation (and both, if that's what you need);&lt;/li&gt;&lt;li&gt;Serial console - if you've soldered in one.&lt;/li&gt;&lt;/ul&gt;&lt;p&gt;The firmware image stores the configuration in a 64k flash partition which is read upon boot. You can modify files in /etc and then save these to flash via "cfg_save".&lt;br /&gt;&lt;/p&gt;What isn't supported:&lt;br /&gt;&lt;ul&gt;&lt;li&gt;The onboard switch - so I believe the only port available at the present moment is the WLAN port;&lt;/li&gt;&lt;li&gt;The GPIO lines aren't being configured, so the WLAN, status and USB/QSS buttons don't function.&lt;br /&gt;&lt;/li&gt;&lt;/ul&gt;I haven't tested out Multi-SSID mode yet. The earlier AR9130 revisions have some issues with multi-SSID mode and handling block-ack tracking, so I _think_ I'll need to somehow disable aggregation on the second and later VAP interfaces. Just keep that in mind if you're tinkering.&lt;br /&gt;&lt;br /&gt;Further details about the hardware and how to build the software for yourself can be found &lt;a href="http://code.google.com/p/freebsd-wifi-build/wiki/TpLinkTLWR1043ND"&gt;here&lt;/a&gt; in my FreeBSD wifi development project wiki.&lt;br /&gt;&lt;br /&gt;No, I won't (yet) be putting up firmware images for people to test. Things are changing quite rapidly and there's no easy way to reflash a unit once you've placed FreeBSD on it - you'll need to have added a serial console to the device.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-9204442519367519775?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/9204442519367519775/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2011/11/freebsd-on-tp-link-tl-wr1043nd.html#comment-form' title='7 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/9204442519367519775'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/9204442519367519775'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2011/11/freebsd-on-tp-link-tl-wr1043nd.html' title='FreeBSD on the TP-Link TL-WR1043nd!'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>7</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-2181772512233819967</id><published>2011-11-26T23:42:00.000-08:00</published><updated>2011-11-26T23:47:51.823-08:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='freebsd'/><category scheme='http://www.blogger.com/atom/ns#' term='802.11n'/><category scheme='http://www.blogger.com/atom/ns#' term='atheros'/><title type='text'>FreeBSD 802.11n update: 27 November 2011.</title><content type='html'>I've merged in most of the reset related fixes from my git tree into FreeBSD-HEAD. This means that normal resets (eg stuck beacon, calibration resets, etc) shouldn't drop frames any longer.&lt;br /&gt;&lt;br /&gt;Frames are still dropped during things like channel/operation mode changes and channel scanning (which does do a channel change.) I'll have to look into that at a later stage. If you're using this in station mode you will likely need to disable background scanning or your aggregation sessions may occasionally drop. You'll have random messages logged when frames are dropping during a flush or reset, so just check your system dmesg log for anything from the ath driver.&lt;br /&gt;&lt;br /&gt;I'll be next working on correctly handling failed/filtered frames and then adding some transition stuff to net80211 so the TIM/ATIM bitmaps can be kept correctly up to date. This should fix some of the power saving issues that I'm sure exist.&lt;br /&gt;&lt;br /&gt;Unfortunately transmitting BAR frames is still quite a bit off. There's a lot more tidying up that I'd like to do before I start down the path of handling BAR TX, including trying to figure out how to better handle packet transmission and reception when the NIC is off-channel (eg when doing a background channel scan.)&lt;br /&gt;&lt;br /&gt;I also have a long list of things I'd like to do to the rate control code and all the surrounding code which sets up rates and creates aggregates. The code I ported/wrote is a little too verbose and duplicate-y for me. That likely will occur after the christmas break.&lt;br /&gt;&lt;br /&gt;Enjoy!&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-2181772512233819967?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/2181772512233819967/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2011/11/freebsd-80211n-update-27-november-2011.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/2181772512233819967'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/2181772512233819967'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2011/11/freebsd-80211n-update-27-november-2011.html' title='FreeBSD 802.11n update: 27 November 2011.'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-1360744025106155678</id><published>2011-11-16T18:02:00.000-08:00</published><updated>2011-11-16T18:12:33.810-08:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='freebsd'/><category scheme='http://www.blogger.com/atom/ns#' term='802.11n'/><category scheme='http://www.blogger.com/atom/ns#' term='atheros'/><title type='text'>FreeBSD is now doing (even more) 802.11n..</title><content type='html'>I held off merging in my 802.11n work as much as possible but I decided that I'd like to get it done before the end of the year. Even though 9.0-RELEASE is still around the corner, I decided that it would be better to merge in what I have into -HEAD and then tidy that up then wait for what could be a few more months.&lt;br /&gt;&lt;br /&gt;So, it's in there, bugs and all, supporting both station and &lt;span class="blsp-spelling-error" id="SPELLING_ERROR_0"&gt;hostap&lt;/span&gt; mode. No, &lt;span class="blsp-spelling-error" id="SPELLING_ERROR_1"&gt;wds&lt;/span&gt;, &lt;span class="blsp-spelling-error" id="SPELLING_ERROR_2"&gt;adhoc&lt;/span&gt;, mesh and &lt;span class="blsp-spelling-error" id="SPELLING_ERROR_3"&gt;TDMA&lt;/span&gt; aren't currently supported (I have enough bugs to worry about for the time being, without trying to debug the other operating modes. But I'd like to.)&lt;br /&gt;&lt;br /&gt;What works:&lt;br /&gt;&lt;ul&gt;&lt;li&gt;TX and RX aggregation!&lt;/li&gt;&lt;li&gt;The rest of the 802.11n negotiation stuff, mostly thanks to Bernhard Schmidt who fixed up a lot of the net80211 quirks.&lt;/li&gt;&lt;li&gt;Lots of &lt;span class="blsp-spelling-error" id="SPELLING_ERROR_4"&gt;ANI&lt;/span&gt; changes which hopefully make noisy environments more stable.&lt;br /&gt;&lt;/li&gt;&lt;/ul&gt;&lt;p&gt;What doesn't yet work:&lt;/p&gt;&lt;ul&gt;&lt;li&gt;Interface resets cause frames to be dropped from the RX and TX queues. This messes up aggregation and causes sessions to hang. I'm fixing that up in a git branch at the moment.&lt;/li&gt;&lt;li&gt;BAR TX - I'll implement BAR TX soon - it's just tricky to get right.&lt;/li&gt;&lt;li&gt;Filtered frames - &lt;span class="blsp-spelling-error" id="SPELLING_ERROR_5"&gt;ie&lt;/span&gt;, TX failed frames from the hardware. Instead of the current method of "always try", the hardware supports failing the current and subsequent frames in a set. That way a &lt;span class="blsp-spelling-error" id="SPELLING_ERROR_6"&gt;hostap&lt;/span&gt; seeing a station going into power saving mode can quickly abort all TX frames to said station and then only retransmit them when the station indicates it's again awake. If I don't do this then the hardware will constantly fail a lot of frames, causing BAR frames to be &lt;span class="blsp-spelling-error" id="SPELLING_ERROR_7"&gt;TXed&lt;/span&gt; when they likely shouldn't be.&lt;br /&gt;&lt;/li&gt;&lt;/ul&gt;&lt;p&gt;But it's enough to try. So if you have an AR5416, AR9160, AR9220, AR9280, AR9285, AR9227 or AR9287, give it a whirl. If you have a &lt;span class="blsp-spelling-error" id="SPELLING_ERROR_8"&gt;pre&lt;/span&gt;-11n &lt;span class="blsp-spelling-error" id="SPELLING_ERROR_9"&gt;NIC&lt;/span&gt; then please, give it a go too. I'd like to ensure that the hardware support for earlier &lt;span class="blsp-spelling-error" id="SPELLING_ERROR_10"&gt;chipsets&lt;/span&gt; hasn't broken.&lt;/p&gt;&lt;p&gt;If you'd like to use this in production on a &lt;span class="blsp-spelling-error" id="SPELLING_ERROR_11"&gt;hostap&lt;/span&gt; then please keep in mind that power saving support isn't entirely functional and featured, so stations which go into frequent power saving mode may have some performance issues. I'll tinker with this some more soon.&lt;/p&gt;&lt;p&gt;Finally, &lt;span class="blsp-spelling-error" id="SPELLING_ERROR_12"&gt;thank you&lt;/span&gt; very much to &lt;a href="http://www.hobnob.com/"&gt;Hobnob,&lt;/a&gt; Inc. for sponsoring this work and &lt;a href="http://www.atheros.com/"&gt;&lt;span class="blsp-spelling-error" id="SPELLING_ERROR_13"&gt;Qualcomm&lt;/span&gt; &lt;span class="blsp-spelling-error" id="SPELLING_ERROR_14"&gt;Atheros&lt;/span&gt; &lt;/a&gt;for providing me source code, documentation and assistance in understanding how all of this works.&lt;br /&gt;&lt;/p&gt;&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-1360744025106155678?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/1360744025106155678/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2011/11/freebsd-is-now-doing-even-more-80211n.html#comment-form' title='2 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/1360744025106155678'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/1360744025106155678'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2011/11/freebsd-is-now-doing-even-more-80211n.html' title='FreeBSD is now doing (even more) 802.11n..'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>2</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-3774722263684035043</id><published>2011-10-29T06:53:00.000-07:00</published><updated>2011-10-29T06:55:17.067-07:00</updated><title type='text'>Sitting in a hotel, getting stuck beacons..</title><content type='html'>I'm sitting in a hotel here in Sydney, wondering if Qantas will allow my flight to happen on Monday, when I discover my 11bg AP interface here is going a bit bezerk-o with the stuck beacons.&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;This isn't terribly good - I thought I had fixed all the stuck beacon issues! Apparently not.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;It turns out that in this hotel room, overlooking St Martins Place, I can see a whole lot of random crap. And this includes a very persistent little wireless client:&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;&lt;div&gt;&lt;span class="Apple-style-span"  style="font-size:78%;"&gt;&lt;span class="Apple-style-span" style="font-family: 'courier new'; "&gt;[e8:06:88:8b:23:57] &lt;/span&gt;&lt;span class="Apple-style-span"  style="font-family:'courier new';"&gt;discard probe_req frame, ssid mismatch: "Crime Prevention"&lt;/span&gt;&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span class="Apple-style-span"  style="font-size:78%;"&gt;&lt;span class="Apple-style-span" style="font-family: 'courier new'; "&gt;[e8:06:88:8b:23:57] &lt;/span&gt;&lt;span class="Apple-style-span"  style="font-family:'courier new';"&gt;discard probe_req frame, ssid mismatch: "vodafone8A1F"&lt;/span&gt;&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span class="Apple-style-span"   style="font-family:'courier new';font-size:78%;"&gt;[e8:06:88:8b:23:57] discard probe_req frame, ssid mismatch: "Boingo Hotspot"&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span class="Apple-style-span"   style="font-family:'courier new';font-size:78%;"&gt;[e8:06:88:8b:23:57] discard probe_req frame, ssid mismatch: "Belkin.44FD"&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span class="Apple-style-span"   style="font-family:'courier new';font-size:78%;"&gt;[e8:06:88:8b:23:57] discard probe_req frame, ssid mismatch: "mycloud"&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span class="Apple-style-span"   style="font-family:'courier new';font-size:78%;"&gt;[e8:06:88:8b:23:57] discard probe_req frame, ssid mismatch: "101-108"&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span class="Apple-style-span"   style="font-family:'courier new';font-size:78%;"&gt;[e8:06:88:8b:23:57] discard probe_req frame, ssid mismatch: "Willkommen bei La Maison du Pain"&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span class="Apple-style-span"   style="font-family:'courier new';font-size:78%;"&gt;[e8:06:88:8b:23:57] discard probe_req frame, ssid mismatch: "WLAN-E36697"&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span class="Apple-style-span"   style="font-family:'courier new';font-size:78%;"&gt;[e8:06:88:8b:23:57] discard probe_req frame, ssid mismatch: "FRITZ!BoxWLAN7240 HB"&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span class="Apple-style-span"   style="font-family:'courier new';font-size:78%;"&gt;[e8:06:88:8b:23:57] discard probe_req frame, ssid mismatch: "Telekom_ICE"&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span class="Apple-style-span"   style="font-family:'courier new';font-size:78%;"&gt;[e8:06:88:8b:23:57] discard probe_req frame, ssid mismatch: "minershotspot"&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span class="Apple-style-span"   style="font-family:'courier new';font-size:78%;"&gt;[e8:06:88:8b:23:57] discard probe_req frame, ssid mismatch: "GANAG"&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span class="Apple-style-span"   style="font-family:'courier new';font-size:78%;"&gt;[e8:06:88:8b:23:57] discard probe_req frame, ssid mismatch: "WLAN-001C4A075A8D"&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span class="Apple-style-span"   style="font-family:'courier new';font-size:78%;"&gt;[e8:06:88:8b:23:57] discard probe_req frame, ssid mismatch: "iJumeirah"&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span class="Apple-style-span"   style="font-family:'courier new';font-size:78%;"&gt;[e8:06:88:8b:23:57] discard probe_req frame, ssid mismatch: "bunny"&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span class="Apple-style-span"   style="font-family:'courier new';font-size:78%;"&gt;[e8:06:88:8b:23:57] discard probe_req frame, ssid mismatch: "Vineyard Square Hotel"&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span class="Apple-style-span"   style="font-family:'courier new';font-size:78%;"&gt;[e8:06:88:8b:23:57] discard probe_req frame, ssid mismatch: "Belkin.44FD"&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span class="Apple-style-span"   style="font-family:'courier new';font-size:78%;"&gt;[e8:06:88:8b:23:57] discard probe_req frame, ssid mismatch: "mycloud"&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span class="Apple-style-span"   style="font-family:'courier new';font-size:78%;"&gt;[e8:06:88:8b:23:57] discard probe_req frame, ssid mismatch: "101-108"&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span class="Apple-style-span"   style="font-family:'courier new';font-size:78%;"&gt;[e8:06:88:8b:23:57] discard probe_req frame, ssid mismatch: "Willkommen bei La Maison du Pain"&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span class="Apple-style-span"   style="font-family:'courier new';font-size:78%;"&gt;[e8:06:88:8b:23:57] discard probe_req frame, ssid mismatch: "FRITZ!BoxWLAN7240 HB"&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span class="Apple-style-span"   style="font-family:'courier new';font-size:78%;"&gt;[e8:06:88:8b:23:57] discard probe_req frame, ssid mismatch: "Telekom_ICE"&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span class="Apple-style-span"   style="font-family:'courier new';font-size:78%;"&gt;[e8:06:88:8b:23:57] discard probe_req frame, ssid mismatch: "minershotspot"&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span class="Apple-style-span"   style="font-family:'courier new';font-size:78%;"&gt;[e8:06:88:8b:23:57] discard probe_req frame, ssid mismatch: "GANAG"&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span class="Apple-style-span"   style="font-family:'courier new';font-size:78%;"&gt;[e8:06:88:8b:23:57] discard probe_req frame, ssid mismatch: "WLAN-001C4A075A8D"&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span class="Apple-style-span"   style="font-family:'courier new';font-size:78%;"&gt;[e8:06:88:8b:23:57] discard probe_req frame, ssid mismatch: "iJumeirah"&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span class="Apple-style-span"   style="font-family:'courier new';font-size:78%;"&gt;[e8:06:88:8b:23:57] discard probe_req frame, ssid mismatch: "bunny"&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-3774722263684035043?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/3774722263684035043/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2011/10/sitting-in-hotel-getting-stuck-beacons.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/3774722263684035043'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/3774722263684035043'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2011/10/sitting-in-hotel-getting-stuck-beacons.html' title='Sitting in a hotel, getting stuck beacons..'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-6073973997367918738</id><published>2011-09-25T18:45:00.000-07:00</published><updated>2011-09-25T18:47:02.108-07:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='freebsd'/><category scheme='http://www.blogger.com/atom/ns#' term='openbsd'/><title type='text'>I know this is old..</title><content type='html'>From if_wi.c in FreeBSD:&lt;br /&gt;&lt;br /&gt;&lt;span style="font-family: courier new;"&gt;/*&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family: courier new;"&gt; * Lucent WaveLAN/IEEE 802.11 PCMCIA driver.&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family: courier new;"&gt; *&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family: courier new;"&gt; * Original FreeBSD driver written by Bill Paul &lt;wpaul@ctr.columbia.edu&gt;&lt;/wpaul@ctr.columbia.edu&gt;&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family: courier new;"&gt; * Electrical Engineering Department&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family: courier new;"&gt; * Columbia University, New York City&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family: courier new;"&gt; */&lt;/span&gt;&lt;br /&gt;&lt;br /&gt;From if_wi.c in OpenBSD:&lt;br /&gt;&lt;br /&gt;&lt;span style="font-family: courier new;"&gt;/*&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family: courier new;"&gt; * Lucent WaveLAN/IEEE 802.11 driver for OpenBSD.&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family: courier new;"&gt; *&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family: courier new;"&gt; * Originally written by Bill Paul &lt;wpaul@ctr.columbia.edu&gt;&lt;/wpaul@ctr.columbia.edu&gt;&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family: courier new;"&gt; * Electrical Engineering Department&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family: courier new;"&gt; * Columbia University, New York City&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family: courier new;"&gt; */&lt;/span&gt;&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-6073973997367918738?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/6073973997367918738/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2011/09/i-know-this-is-old.html#comment-form' title='1 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/6073973997367918738'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/6073973997367918738'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2011/09/i-know-this-is-old.html' title='I know this is old..'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>1</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-7175916248450946550</id><published>2011-09-12T20:17:00.001-07:00</published><updated>2011-09-12T20:22:23.083-07:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='freebsd'/><category scheme='http://www.blogger.com/atom/ns#' term='atheros'/><title type='text'>FreeBSD vendor summit - November 2011; Sunnyvale, California</title><content type='html'>Since this doesn't seem to have gotten much notice - there's a FreeBSD vendor summit being held in Sunnyvale, California on the 3rd and 4th of November 2011. This is an opportunity for developers and vendors to share project direction and goals, collaborate on various projects, and likely hit each other with a big "why can't you do X" stick :-)&lt;br /&gt;&lt;br /&gt;NetApp is sponsoring the event.&lt;br /&gt;&lt;br /&gt;I'll be there with someone from Qualcomm Atheros, who will be presenting on Qualcomm/Atheros' open source direction and strategy. There'll likely be some (informal) discussion about how developers and vendors can sign up to their open source developer programme and obtain documentation/reference code for the Atheros wireless and embedded chipsets.&lt;br /&gt;&lt;br /&gt;(This is what I've signed up to and it's been a huge help in getting Atheros 802.11n support up and going in FreeBSD.)&lt;br /&gt;&lt;br /&gt;More details can be found here:&lt;br /&gt;&lt;br /&gt;&lt;a href="http://wiki.freebsd.org/201111VendorSummit"&gt;http://wiki.freebsd.org/201111VendorSummit&lt;/a&gt;&lt;br /&gt;&lt;br /&gt;If you're a vendor using FreeBSD, or you're a vendor thinking about using FreeBSD in a project (wireless or otherwise) this mini-conference is just for you. Personally I'm interested in talking with the nvidia representative about trying to get some CUDA stuff going on FreeBSD.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-7175916248450946550?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/7175916248450946550/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2011/09/freebsd-vendor-summit-november-2011.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/7175916248450946550'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/7175916248450946550'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2011/09/freebsd-vendor-summit-november-2011.html' title='FreeBSD vendor summit - November 2011; Sunnyvale, California'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-3283926884303863610</id><published>2011-09-11T19:17:00.000-07:00</published><updated>2011-09-11T19:59:04.928-07:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='freebsd'/><category scheme='http://www.blogger.com/atom/ns#' term='802.11n'/><category scheme='http://www.blogger.com/atom/ns#' term='atheros'/><title type='text'>Wireless update - 802.11n development</title><content type='html'>A few of us have been working towards 802.11n support in FreeBSD. Bernhard Schmidt and I were hoping to get 802.11n support up to scratch before 9.0-RELEASE but a combination of timing constraints and work constraints has hampered things somewhat.&lt;br /&gt;&lt;br /&gt;But that said, we the net80211 support for 802.11n is in a lot better shape than it was in previous releases. Bernhard has been working out the kinks in the intel driver (if_iwn) and has 802.11n support mostly stable for those NICs. Someone else has been working on if_ral 802.11n support and has had quite a bit of success there. And I've been working on if_ath/ath_hal support.&lt;br /&gt;&lt;br /&gt;(As a side-note, I've ordered some Marvell if_mwl compatible NICs to do some testing with.)&lt;br /&gt;&lt;br /&gt;The 802.11n support in the FreeBSD atheros driver is already mostly usable for testing - 802.11n TX and RX works, but 802.11n TX aggregation doesn't work. So if you'd like to test it out in 9.0-RELEASE, you can do this:&lt;br /&gt;&lt;br /&gt;* build a kernel with "options ATH_ENABLE_11N";&lt;br /&gt;* ifconfig wlan0 -ampdutx&lt;br /&gt;* associate as per normal to an 802.11n network.&lt;br /&gt;&lt;br /&gt;Your download speeds will be good (as RX aggregation works) but as TX aggregation doesn't, you won't be getting full speed.&lt;br /&gt;&lt;br /&gt;Now, as for the current work I'm doing. I've been porting over (and reimplementing in places) the 802.11n TX aggregation support, based on a combination of the atheros reference/carrier codebases and what's filtered through to the Linux ath9k driver.&lt;br /&gt;&lt;br /&gt;In short - so far, so good. There's a lot to do, but basic TX aggregation is working in both station and hostap mode. My current test setup is:&lt;br /&gt;&lt;ul&gt;&lt;li&gt;A routerstation pro with an AR9160 NIC, running locally built firmware (FreeBSD) in hostap mode;&lt;/li&gt;&lt;li&gt;An EEEPC 701 retrofitted with an AR9280 NIC, running FreeBSD, as a station;&lt;/li&gt;&lt;li&gt;My macbook pro (broadcom 802.11n) as a station.&lt;/li&gt;&lt;/ul&gt;So far it's performing well - 200mbit TX/RX UDP in 5ghz HT/40 2x2 stream mode; and 130-140mbit TX/RX TCP.&lt;br /&gt;&lt;br /&gt;I'm currently doing the work in a FreeBSD user svn branch. The details are in the Wiki - &lt;a href="http://wiki.freebsd.org/AdrianChadd/AtherosTxAgg"&gt;http://wiki.freebsd.org/AdrianChadd/AtherosTxAgg&lt;/a&gt; . I currently have exactly one tester (AR5416 STA, AR9280 hostap) who is reporting excellent success. I'd obviously like a few more. :)&lt;br /&gt;&lt;br /&gt;There's still a lot of work to be done before this can be merged into HEAD. I'm sorry to say it won't happen before 9.0-RELEASE as there are some upcoming net80211 changes which will be rather intrusive and a bit risky to throw into the release at this late stage. It may also be difficult to backport as some changes will break kernel ABI.&lt;br /&gt;&lt;br /&gt;But don't let anyone tell you FreeBSD doesn't support 802.11n. 9.0-RELEASE may not have much support, but all the key pieces are there. Once the release has been cut, I'll do some chasing up to get the Intel, Marvell and Ralink 802.11n support updated. (If someone would like to take charge of the broadcom NIC drivers, please drop freebsd-wireless@ a line.)&lt;br /&gt;&lt;br /&gt;The next 6 months will be very interesting in the FreeBSD 802.11 world. :)&lt;br /&gt;&lt;br /&gt;Finally, none of this current work would've been possible without the support of the sponsor paying me to get FreeBSD's net80211 and ath support updated - &lt;a href="http://hobnob.com/"&gt;Hobnob, Inc.&lt;/a&gt; They contacted me a few months ago to help work out kinks in the Atheros NIC they're using in FreeBSD and seem impressed enough by my work to sponsor general net80211/ath improvements which they'll likely roll into future products. So when you're using Atheros 802.11n hardware in FreeBSD and getting that nice 140-150mbit TCP throughput, please give them thanks. :)&lt;br /&gt;&lt;br /&gt;And finally finally, Qualcomm Atheros and the Linux ath9k developers have been instrumental in this work. The documentation, reference code and general interactive discussions have allowed me to get all of this work done in such a short period of time. (Yes,  I did say "Atheros", "Documentation" and "Source Code" there, in the context of open source. Honest.)&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-3283926884303863610?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/3283926884303863610/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2011/09/wireless-update-80211n-development.html#comment-form' title='9 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/3283926884303863610'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/3283926884303863610'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2011/09/wireless-update-80211n-development.html' title='Wireless update - 802.11n development'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>9</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-1971532312918961763</id><published>2011-09-11T07:05:00.000-07:00</published><updated>2011-09-11T07:41:47.456-07:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='freebsd'/><category scheme='http://www.blogger.com/atom/ns#' term='802.11n'/><category scheme='http://www.blogger.com/atom/ns#' term='atheros'/><title type='text'>Wireless update - DFS wrapup</title><content type='html'>Now that the DFS work has been completed, tested and paid for, I guess I can now write about it.&lt;br /&gt;&lt;br /&gt;The DFS work was sponsored by &lt;a href="http://www.kbcnetworks.com/"&gt;KBC Networks&lt;/a&gt;, a wireless product company based in the USA. The eventual aim is to integrate this DFS work into their mesh product. I can't talk any further about what's going on there, except to say that they very graciously allowed the FreeBSD work to be released back to the community.&lt;br /&gt;&lt;br /&gt;So FreeBSD now has:&lt;br /&gt;&lt;br /&gt;&lt;ul&gt;&lt;li&gt;Updated DFS master (hostap) and DFS slave (station) support in net80211;&lt;br /&gt;&lt;/li&gt;&lt;li&gt;Updated regulatory domain entries for the USS ("FCC3");&lt;/li&gt;&lt;li&gt;Changes to the ath(4) driver to fix corner cases in DFS master and DFS slave modes.&lt;/li&gt;&lt;/ul&gt;All of this work is now in FreeBSD-HEAD and will be available in FreeBSD-9.0.&lt;br /&gt;&lt;br /&gt;Future work will hopefully include DFS IBSS and DFS mesh functionality. This work didn't include 802.11n DFS - but that's not too difficult to do (hint: if someone would like the mini-project, please contact me!)&lt;br /&gt;&lt;br /&gt;Along with that was a port of the Atheros radar detection code (from Fusion, their earlier carrier codebase) which is currently proprietary. The Linux wireless developers have been &lt;a href="http://linuxwireless.org/en/developers/DFS"&gt;working towards DFS compliance for a while&lt;/a&gt;, but haven't yet integrated working radar detection into any drivers. So this was a good exercise in uncharted territory to see how difficult it would be.&lt;br /&gt;&lt;br /&gt;It turns out that it wasn't terribly difficult at all. After spending a bit of time chasing down missing bits in the FreeBSD atheros HAL driver, the third party doing said radar code porting managed to get FreeBSD doing radar detection at the same level as the commercial firmware.&lt;br /&gt;&lt;br /&gt;During this particular exercise we discovered a few things about the software radar detection, which I think the community at large should know about.&lt;br /&gt;&lt;br /&gt;The process of getting a device FCC/ETSI certified is just that - the whole device. This includes the NIC, the internal cabling, the enclosure, the antenna(s) and any external cabling. Maybe even whether it's indoors or outdoors, I wasn't involved in that. This likely has repercussions for any open source DFS implementation - although the DFS machinery in a specific FreeBSD release could be certified, the software based radar matching code may need tuning for the specific hardware used. So it's likely that Linux/FreeBSD can't just publish a single "ath_dfs" source module that will work for all Atheros NICs - or even a single NIC. It's likely that things will need tuning based on how a given NIC is used, the environment it's intending to be used in, and a variety of other things I haven't even thought about yet.&lt;br /&gt;&lt;br /&gt;So if/when this does occur, the open source wireless community (FreeBSD, Linux, NetBSD, OpenBSD, DragonflyBSD, whatever..) will have to be responsible and not just simply run an "ath_dfs" radar module, assuming that it works. For complete, correct, certified behaviour, it's going to likely need the community to work with vendors and users in order to create "working, certified" combinations of hardware and software. So, if a vendor (call them U) wishes to release a new "SR-M" NIC, they could work with the open source wireless community to create a certified NIC+board+enclosure+antenna combination and publish a set of radar patterns and settings. They could leverage the proposed existing open source radar pattern matching code and simply add whatever configuration is needed. Users should then be responsible - they should only use the given radar dfs code with the correct setup. If they wish to run a modified setup, they should work with the vendor and upstream community to get things re-certified.&lt;br /&gt;&lt;br /&gt;This way the DFS and radar pattern matching code is open source - what then changes is the radar patterns and configuration, which depends upon the NIC, cabling, antenna and environment.&lt;br /&gt;&lt;br /&gt;For home users/developers, having a set of certified hardware combinations will make things easier for them.&lt;br /&gt;&lt;br /&gt;For commercial vendors, if they adopt open-source FreeBSD or Linux on their platforms then (in an ideal world) they could get their equipment certified once (which as I note, they have to do anyway) and then publish the relevant radar patterns and configuration. The rest of the driver (including the radar pattern matching source and the general DFS support) would already be open source. Users can then use this when running their own kernel/distribution/OS on that platform.&lt;br /&gt;&lt;br /&gt;I know this is all very fluffy and assumes that vendors/users act in a responsible, open manner. But I think this is what needs to occur if the open source community wishes to see stable, supported DFS regulatory compliance in their open source project.&lt;br /&gt;&lt;br /&gt;To wrap this up, if you're a company wishing to leverage open source on their (Atheros) wireless platform and are interested in regulatory compliance, please drop me a line. I'll put you in contact with the relevant people inside Qualcomm Atheros to discuss how to best achieve this.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-1971532312918961763?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/1971532312918961763/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2011/09/wireless-update-dfs-wrapup.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/1971532312918961763'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/1971532312918961763'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2011/09/wireless-update-dfs-wrapup.html' title='Wireless update - DFS wrapup'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-1757089927137358748</id><published>2011-07-30T22:24:00.001-07:00</published><updated>2011-07-30T22:37:20.780-07:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='802.11h'/><category scheme='http://www.blogger.com/atom/ns#' term='freebsd'/><category scheme='http://www.blogger.com/atom/ns#' term='dfs'/><category scheme='http://www.blogger.com/atom/ns#' term='atheros'/><title type='text'>FreeBSD: DFS update</title><content type='html'>I've spent the last few weeks sorting out the DFS support in FreeBSD. I've made some decent progress with it however there's still a long way to go. This has been paid work and I'm glad to see increasing interest in 802.11 support under FreeBSD.&lt;br /&gt;&lt;br /&gt;I've debugged DFS channel switching behaviour in station and access point mode. This now functions correctly in both instances.&lt;br /&gt;&lt;br /&gt;The ath driver now behaves "better" when doing a DFS channel change. The driver would change channels and wait for the first beacon to program the beacon timers - which are then used to signal when there's a lack of beacons (and thus net80211 marks the state as "SCAN" and begins hunting for another AP.) If a channel change occured to a channel that required a CAC (clear access check) and a radar event occured, a second channel switch would occur - and there'd never be any beacons on that channel. The NIC would sit there forever waiting to hear a first beacon that never came.&lt;br /&gt;&lt;br /&gt;I've updated the FCC3 regulatory domain to include frequency bands that require DFS.&lt;br /&gt;&lt;br /&gt;I've also introduced some radar device setup and event parsing code into the ath HAL layer which would allow an interested party to tinker with radar detection/classification code on the AR5212 series NICs. I don't have permission to commit the radar frame decoding code for the 11N NICs - I'm working on this.&lt;br /&gt;&lt;br /&gt;Finally, I've introduced a radar detection layer - called "ath_dfs" - which currently does nothing (hence the only module is "dfs_null".) This provides all the hooks needed to do radar detection, classification and signaling.&lt;br /&gt;&lt;br /&gt;Now, what I haven't done:&lt;br /&gt;&lt;ul&gt;&lt;li&gt;The project also includes working porting the radar classification code and getting it to work. This unfortunately won't be open sourced. I'm going to try and get as much of the radar hardware setup and radar event frame parsing code committed to FreeBSD so an interested party can begin tinkering with this.&lt;/li&gt;&lt;li&gt;I haven't yet ported a lot of the "channel interference" support. It's there, but I haven't really fleshed it out to be useful and then used it.&lt;br /&gt;&lt;/li&gt;&lt;li&gt;There's no DFS support for 11N yet - in particular, 11N HT/40 channels aren't correctly marked as "interference" or "radar"; 11N channels also aren't scanned for during DFS channel selection. That's a later project.&lt;/li&gt;&lt;li&gt;I need to extend things a bit to handle the weather radar stuff when doing DFS - the ETSI requirements include different channel quiet/scan times.&lt;/li&gt;&lt;li&gt;There's no support in net80211 (yet) for the DFS, TPC and Quiet elements. This is going to be required for a fully compliant DFS implementation.&lt;/li&gt;&lt;li&gt;I've not investigated implementing/fixing up the DFS support for IBSS and mesh modes.&lt;br /&gt;&lt;/li&gt;&lt;/ul&gt;However, even though there are shortcomings and some missing support in the DFS support, FreeBSD-9.0 will support DFS STA mode well enough to correctly obey instructions from a DFS-compliant access point.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-1757089927137358748?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/1757089927137358748/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2011/07/freebsd-dfs-update.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/1757089927137358748'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/1757089927137358748'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2011/07/freebsd-dfs-update.html' title='FreeBSD: DFS update'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-3990465410804254091</id><published>2011-07-11T08:36:00.000-07:00</published><updated>2011-07-11T08:38:46.005-07:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='lusca'/><category scheme='http://www.blogger.com/atom/ns#' term='ipv6'/><title type='text'>Unintentional side-effects - IPv4/IPv6 load balancing..</title><content type='html'>Here's a fun one:&lt;br /&gt;&lt;br /&gt;&lt;span style="font-family: courier new;"&gt;1310397101.830    416 X TCP_MISS/200 700 GET http://www.freebsd.org/layout/images/front_get_back.png adrian DIRECT/[2001:4f8:fff6::22] image/png&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family: courier new;"&gt;1310397101.835    413 X TCP_MISS/200 851 GET http://www.freebsd.org/layout/images/front_get_tr.png adrian DIRECT/69.147.83.34 image/png&lt;/span&gt;&lt;br /&gt;&lt;br /&gt;Since the ipcache code already balances outbound connections between multiple hosts, any IPv6 hosts get similarly load balanced.. between IPv4 hosts as well.&lt;br /&gt;&lt;br /&gt;I may end up adding in logic to connect to "only v4" and "only v6" in forward.c...&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-3990465410804254091?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/3990465410804254091/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2011/07/unintentional-side-effects-ipv4ipv6.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/3990465410804254091'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/3990465410804254091'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2011/07/unintentional-side-effects-ipv4ipv6.html' title='Unintentional side-effects - IPv4/IPv6 load balancing..'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-8465785643661563449</id><published>2011-07-11T00:26:00.000-07:00</published><updated>2011-07-11T00:34:46.117-07:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='lusca'/><category scheme='http://www.blogger.com/atom/ns#' term='ipv6'/><title type='text'>Lusca update: IPv6 server work is now working</title><content type='html'>I'm now using the Lusca IPv6 branch for browsing both IPv4 and IPv6 sites.&lt;br /&gt;&lt;br /&gt;There's a bunch of little things to update and test, including testing the neighbor selection, peer and ICP code. It's still not yet doing transparent interception or TPROXY - that'll be among the last thing to work on. Finally, there's a bunch of connection timeout handling code which currently is being ignored - there's no convenient place to attach a timeout handler now since the file descriptor isn't created up front; it's created only once the DNS resolution is complete.&lt;br /&gt;&lt;br /&gt;My main focus at the moment is testing and verifying the peer, neighbor selection and ICP code out.&lt;br /&gt;&lt;br /&gt;But yes, it is actually working!&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-8465785643661563449?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/8465785643661563449/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2011/07/lusca-update-ipv6-server-work-is-now.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/8465785643661563449'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/8465785643661563449'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2011/07/lusca-update-ipv6-server-work-is-now.html' title='Lusca update: IPv6 server work is now working'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-8047497417895261962</id><published>2011-07-07T02:00:00.000-07:00</published><updated>2011-07-07T02:11:28.540-07:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='lusca'/><category scheme='http://www.blogger.com/atom/ns#' term='ipv6'/><category scheme='http://www.blogger.com/atom/ns#' term='squid'/><title type='text'>Lusca development update - IPv6 is almost working</title><content type='html'>Now that I've (hopefully!) completely finished with university, I can get back into using my hard-earned money from Xenion to get more Lusca development done. (And yes, I'll also be doing wireless development too, fear not.)&lt;br /&gt;&lt;br /&gt;The IPv6 branch is a bit messy at the moment, but it's almost able to handle IPv6 server requests.&lt;br /&gt;&lt;br /&gt;The problem? The existing code which handles connecting to remote hosts (ie, src/comm.c) doesn't "know" about IPv6. It assumes all sockets are IPv4 and that all hostnames which are returned are also IPv4.&lt;br /&gt;&lt;br /&gt;There's unfortunately a lot of dirty code in there - commReuseFD() is the main culprit and a good example of this. The 30 second version - since the commConnectStart() API assumes the socket is already created before the connect() occurs, any "connect retry" (for multiple hostnames and multiple attempts at the same end-host) requires the socket to be closed and recreated. But the FD has to stay the same. So commReuseFD() manually creates a new FD, makes it look like the old FD, then calls dup2() to get it into the same FD as the old one.&lt;br /&gt;&lt;br /&gt;The "Correct" Fix is to modify the API to not take an FD, but to return an FD on successful connect() to the remote destination. There's some problems with this though, most notably in the request forwarding layer where the FD is created and comm close handlers are assigned before the connection is attempted. I need to make sure that there's no code which calls comm_close() on the active connection whilst connect() is going on - as said code expects the FD to be valid and assigned by this point.&lt;br /&gt;&lt;br /&gt;The "Dirty" fix is to modify commConnectStart() and commReuseFD() to check the FD address family and destroy/create a "new" socket with the correct address family.&lt;br /&gt;&lt;br /&gt;The "Problem" is that the code allows the outgoing address to be specified, both for transparent interception (source address spoofing) and to be set via an ACL match. Since IPv4 and IPv6 addresses are now possible, the API will have to be modified to handle this case.&lt;br /&gt;&lt;br /&gt;What I'm likely going to do is something inspired by Squid-3. I'll teach the forwarding layer about "try v4 destinations" and "try v6 destinations". The administrator can then configure whether to try v4 or v6 destinations first. Only one outgoing address has to be provided - either "v4" or "v6"; and commConnectStart() will only try connecting to IP addresses that match the family of the outgoing address. That way if a host resolves to a mix of v4 and v6 addresses, they'll be tried in a "v4" group, then a "v6" group (or vice versa.) It's a bit dirty, but it's likely doable in the short-term.&lt;br /&gt;&lt;br /&gt;In the long term, I'd like to fix the API up to be less messy and return an FD, rather than take an existing FD and abuse that. But that can come later.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-8047497417895261962?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/8047497417895261962/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2011/07/lusca-development-update-ipv6-is-almost.html#comment-form' title='1 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/8047497417895261962'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/8047497417895261962'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2011/07/lusca-development-update-ipv6-is-almost.html' title='Lusca development update - IPv6 is almost working'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>1</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-8369658885462114525</id><published>2011-06-17T00:16:00.000-07:00</published><updated>2011-06-17T00:46:06.946-07:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='freebsd'/><category scheme='http://www.blogger.com/atom/ns#' term='802.11n'/><category scheme='http://www.blogger.com/atom/ns#' term='atheros'/><title type='text'>Implementing 11n TX aggregation, why it's not so easy..</title><content type='html'>The only real missing bit from the initial 11n support for ath(4) in FreeBSD is now TX aggregation. This is also quite a tricky thing to do right, because (like everything related to these Atheros NICs) it's all done in software.&lt;br /&gt;&lt;br /&gt;I'm about half way through the first part - implementing per-node, per-TID software queues. Since A-MPDU sessions are implemented using TIDs (although in net80211 they're linked to WME access classes (AC), hm!) traffic for each TID has to be individually managed.&lt;br /&gt;&lt;br /&gt;The basic premise is quite simple. Frames are queued based on the destination node and TID. Non-QoS frames to a unicast destination (ie, a known station) go to the "non-QoS TID" (TID 16 in FreeBSD's net80211 stack.) Multicast frames go to the software-driven multicast/content-after-beacon (cabq) queue. Then a separate task handles de-queuing packets form these queues and queuing them to the hardware.&lt;br /&gt;&lt;br /&gt;The reality however is a little more difficult than that.&lt;br /&gt;&lt;br /&gt;Firstly, frames are queued from multiple contexts. The most obvious is the per-interface queue (ie wlanX). These may run in parallel and target the same hardware, so now I need to add locking to the per-node structure. Previously this wasn't (much) of a problem since the TX code simply directly dispatched to the hardware, and these queues are correctly locked. However now there's software-driven queues (which need new locks) along with some more per-node state. That all needs locking.&lt;br /&gt;&lt;br /&gt;Next, the rate control code wasn't at all locked. I've begun wrapping them in per-node locks. Making sure I get this correct isn't easy.&lt;br /&gt;&lt;br /&gt;Then there's the question of marking nodes and TIDs that have traffic to send. Right now that's done by adding each node to a linked list of "ready nodes". I then check all the TIDs for that node and dequeue what packets I have to. It's ugly code, but it works well enough for testing. I'll finish off implementing what ath9k and the Atheros reference code does (ie, maintaining both a "node" and "TID" linked list, so I can directly walk which TIDs need walking), but there's implications here for getting "QoS" right. (Ie, which order do you dequeue packets? How many do you dequeue from which queue in order for things to be "fair" ?) I'm going to leave all of these questions unanswered for now (as they're not handled in the current code anyway) and hope someone else steps up to it!&lt;br /&gt;&lt;br /&gt;At this point I now have a mostly-clean (for values of "clean") implementation of per-node, per-TID software queues. I'm still in the process of tidying all of this up in preparation to commit to -HEAD before I begin the aggregation code. I'll cover that in a moment.&lt;br /&gt;&lt;br /&gt;Then there's handling the aggregation session stuff.&lt;br /&gt;&lt;br /&gt;Firstly, there's handling the ADDBA exchange (ie, establishing an A-MPDU session.) When this is done, any unicast traffic pending to that node/TID needs to be paused and kept around until the ADDBA request/response occurs. This is known as "pausing" the queue. I've a hack in place to do this and will implement a correct "pause" API soon to clean it up.&lt;br /&gt;&lt;br /&gt;Next there's handling packets already queued to the hardware. This crept up on me without warning.&lt;br /&gt;&lt;br /&gt;Firstly - a little background. The ADDBA exchange establishes the initial sequence number and window size to use as the "window" for subsequent data exchanges. After that, any packets being queued to the destination must fall within this window (typically say 64 sequence numbers long.) The window stays a fixed size, but the window itself slides along when TX packets are successfully completed. Since packets may be lost at any time (whether aggregate or not), the window doesn't move until the sequence numbers at the beginning of the window are successfully received.&lt;br /&gt;&lt;br /&gt;So for example, say my window begins at "0", my window size is "4" and the station sends 4 packets with sequence numbers "0", "1", "2", "3". If I receive all four packets correctly and thus send back a block-ACK indicating this, the new window position will begin at "4". I can then send packets "4", "5", "6" and "7". I can't send "8", even if I have it in my queue.&lt;br /&gt;&lt;br /&gt;But say the other end only received "0", "1" and "3". "2" wasn't received, so I have to retransmit it. I can only advance the window to "2". I can't advance it to "3", since I didn't receive "2". I could advance it to "1", but that'd be pointless as I've already received it. So in this instance, both the station and access point would advance the window position to "2" and the station would then retransmit "2". If it's coded correctly, it could transmit "2", "4", "5". It can't transmit "6", as that's outside the window size of 4 packets. It could just retransmit "2" if it wanted to and not the others. In fact, it could transmit "4", "5" and "2" - the receiver would then be responsible for re-ordering the packets and only passing them up to the network layer once all the packets in the correct order are available.&lt;br /&gt;&lt;br /&gt;Finally, if I don't receive "2" (say I just can't seem to transmit it), the STA need to do a few things. It first has to pause the queue so no further packets are queued to the hardware. At this point, packets may be queued to the hardware, but they'll be for sequence numbers greater than "2" but still within the block-ack window. Once the hardware has finished completing packets (successfully or not), it needs to send a "BAR" frame to the access point to establish a new starting point for the sequence number window. In this instance it could set it to "3", but what if "3" was already in the hardware queue and was successfully transmitted? That would confuse the hell out of the other end. So it looks at the last successfully transmitted sequence number was, then sends the "BAR" with a starting sequence number 1 after that.&lt;br /&gt;&lt;br /&gt;So, if "2" came back as having failed too many times, but "3" and "4" were in the hardware TX queue, the driver will pause the queue, wait for pending frames to complete, then look at what happened. If "3" and "4" were successfully transmitted, it would send a BAR with the sequence beginning at "5". If "3" was successfully transmitted but not "4", it would send a BAR with the sequence beginning at "4". But if "3" failed, and "4" didn't - it would set the BAR at "3" and then try retransmitting it. Finally (and this is another strange corner case I have to handle!) if "3" came back as having failed - and it failed too many times! - but "4" TX'ed ok, then I have to set the initial sequence number at "5" and send the BAR.&lt;br /&gt;&lt;br /&gt;The other end (in this instance, the hostap) would then receive the BAR, set the new expected sequence number to be whatever was sent (say "5" in this instance), flush all pending packets in the reorder queue - even if there are missing packets (in the above example, "3" may be missing but "4" was received ok, so the receive stack would've been waiting for "3" to be received before sending it all up to the network layer!) and send back a block-ACK to the STA confirming the new window position.&lt;br /&gt;&lt;br /&gt;Now (phew!) given all of that, you can see what kind of complicated stuff is needed to properly handle all corner cases when doing software (re)transmitting of packets. I haven't even begun to talk about how to handle sending and re-sending aggregate frames! All of the above needs to be implemented before I can do that.&lt;br /&gt;&lt;br /&gt;So! The above is what I'm now working on. Once that's done, I'll work on handling what's known as "filtered frames" (and that'll be brain-dumped in a future blog post, but it has to do with handling power save stations correctly, or A-MPDU sessions will be torn down prematurely!) and then when THAT is all done and stable, I'll look at handling aggregates.&lt;br /&gt;&lt;br /&gt;And when I handle aggregates correctly, we'll finally have fast 11n TX in FreeBSD. Then I can enable "ATH_ENABLE_11N" by default. :-)&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-8369658885462114525?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/8369658885462114525/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2011/06/implementing-11n-tx-aggregation-why-its.html#comment-form' title='4 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/8369658885462114525'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/8369658885462114525'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2011/06/implementing-11n-tx-aggregation-why-its.html' title='Implementing 11n TX aggregation, why it&apos;s not so easy..'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>4</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-3316111058359895475</id><published>2011-06-10T20:20:00.001-07:00</published><updated>2011-06-10T20:28:39.463-07:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='compsci'/><title type='text'>Uninformed programmers, or "stop assuming virtual memory is awesome."</title><content type='html'>I decided I needed to vent a bit post-exam. So where better to find some clue-void post to answer than slashdot.&lt;br /&gt;&lt;br /&gt;The post, and my reply:&lt;br /&gt;&lt;br /&gt;&lt;a href="http://news.slashdot.org/comments.pl?sid=2229598&amp;amp;cid=36408502"&gt;http://news.slashdot.org/comments.pl?sid=2229598&amp;amp;cid=36408502&lt;/a&gt;&lt;br /&gt;&lt;br /&gt;In summary, the original poster:&lt;br /&gt;&lt;br /&gt;&lt;span style="font-style: italic;"&gt;"In short, don't worry about fine-tuning what's "in memory". Don't change  behavior based on total amount of memory in the system. Operating  systems (OpenBSD aside) ALREADY DO THAT. Just let the memory  manager do its job, and give it enough information (via interactivity  information, memory priority, etc.) to do its job properly. Don't try to  hack around problems at the wrong layers."&lt;/span&gt;&lt;br /&gt;&lt;br /&gt;And my response:&lt;br /&gt;&lt;br /&gt;&lt;span style="font-style: italic;"&gt;"You assume that the OS will make sensible paging decisions. You assume  you can hint to the OS that you're going to make sensible paging  decisions. You hope the application, which is likely big, multithreaded  and such, is doing the sensible thing of not wrapping large accesses to  "memory things" (eg big trees of data, as an example, or image caches,  or whatever takes up more than a small bit of RAM) in mutexes. You  assume that your application is using memory in a sensible fashion, and  not simply using a few bytes here and there in each allocated chunk."&lt;/span&gt;&lt;br /&gt;&lt;br /&gt;I hate to say it, but if you're assuming that virtual memory is the be all and end all of your memory management method, you're setting yourself up for some really bad worse case behaviour. Since you have no control as to where and when the OS decides to punt pages to disk, you can't possibly begin to design methods around it. You can only hope that the OS doesn't page out the data you've got locked, or that the malloc implementation you're using doesn't handle fragmentation poorly.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-3316111058359895475?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/3316111058359895475/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2011/06/uninformed-programmers-or-stop-assuming.html#comment-form' title='1 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/3316111058359895475'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/3316111058359895475'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2011/06/uninformed-programmers-or-stop-assuming.html' title='Uninformed programmers, or &quot;stop assuming virtual memory is awesome.&quot;'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>1</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-8244766086192014188</id><published>2011-05-28T08:46:00.000-07:00</published><updated>2011-05-28T08:50:02.304-07:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='freebsd'/><category scheme='http://www.blogger.com/atom/ns#' term='802.11n'/><category scheme='http://www.blogger.com/atom/ns#' term='atheros'/><title type='text'>AR9287 update</title><content type='html'>I finally gave in and whacked a FreeBSD-HEAD snapshot on my EEEPC so I can test the ath 802.11n code out with the various mini-PCIe NICs I have.&lt;br /&gt;&lt;br /&gt;&lt;span style="font-size:78%;"&gt;&lt;span style="font-family: courier new;"&gt;ath0: &lt;atheros 9287=""&gt; mem 0xfbef0000-0xfbefffff irq 18 at device 0.0 on pci1&lt;/atheros&gt;&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family: courier new;"&gt;ath0: [HT] enabling HT modes&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family: courier new;"&gt;ath0: [HT] 2 RX streams; 2 TX streams&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family: courier new;"&gt;ath0: AR9287 mac 384.2 RF5133 phy 15.15&lt;/span&gt;&lt;br /&gt;&lt;/span&gt;&lt;br /&gt;.. and&lt;br /&gt;&lt;br /&gt;&lt;span style="font-size:78%;"&gt;&lt;span style="font-family: courier new;"&gt;wlan0: flags=8843&lt;up,broadcast,running,simplex,multicast&gt; metric 0 mtu 1500&lt;/up,broadcast,running,simplex,multicast&gt;&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family: courier new;"&gt;        ether b4:82:fe:33:95:53&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family: courier new;"&gt;        inet 10.61.8.27 netmask 0xffffff00 broadcast 10.61.8.255&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family: courier new;"&gt;        media: IEEE 802.11 Wireless Ethernet MCS mode 11ng&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family: courier new;"&gt;        status: associated&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family: courier new;"&gt;        ssid CACHEBOY_RS channel 1 (2412 MHz 11g ht/40+) bssid 00:1b:b1:58:f6:f0&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family: courier new;"&gt;        regdomain ROW country AU indoor ecm authmode WPA2/802.11i privacy ON&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family: courier new;"&gt;        deftxkey UNDEF AES-CCM 2:128-bit AES-CCM 3:128-bit txpower 30 bmiss 7&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family: courier new;"&gt;        scanvalid 450 bgscan bgscanintvl 300 bgscanidle 250 roam:rssi 7&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family: courier new;"&gt;        roam:rate 64 protmode CTS -ampdu ampdulimit 32k ampdudensity 8 -amsdu&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family: courier new;"&gt;        shortgi wme burst roaming MANUAL&lt;/span&gt;&lt;br /&gt;&lt;/span&gt;&lt;br /&gt;(Yes, there's no TX aggregation support for now so I've disabled ampdu.)&lt;br /&gt;&lt;br /&gt;And:&lt;br /&gt;&lt;br /&gt;&lt;span style="font-size:78%;"&gt;&lt;span style="font-family: courier new;"&gt;ADDR               AID CHAN RATE RSSI IDLE  TXSEQ  RXSEQ CAPS FLAG&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family: courier new;"&gt;00:1b:b1:58:f6:f0    1    1 216M 24.5    0  30271  30064 EPS  AQEHTRS RSN HTCAP WPA WME&lt;/span&gt;&lt;br /&gt;&lt;/span&gt;&lt;br /&gt;It's happily sitting at MCS12 from my bedroom (which is what I expect given the noise on the 2.4ghz band where I live.)&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-8244766086192014188?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/8244766086192014188/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2011/05/ar9287-update.html#comment-form' title='1 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/8244766086192014188'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/8244766086192014188'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2011/05/ar9287-update.html' title='AR9287 update'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>1</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-7287224416152483269</id><published>2011-05-26T13:36:00.001-07:00</published><updated>2011-05-26T13:37:31.295-07:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='freebsd'/><category scheme='http://www.blogger.com/atom/ns#' term='802.11n'/><category scheme='http://www.blogger.com/atom/ns#' term='atheros'/><title type='text'>AR9287 support is now in -HEAD</title><content type='html'>I've just committed the last bits of the initial AR9287 support to -HEAD.&lt;br /&gt;&lt;br /&gt;There's a few bits and pieces that need to be added but there's enough code there now to bring up a station interface and exchange traffic.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-7287224416152483269?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/7287224416152483269/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2011/05/ar9287-support-is-now-in-head.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/7287224416152483269'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/7287224416152483269'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2011/05/ar9287-support-is-now-in-head.html' title='AR9287 support is now in -HEAD'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-2038071138464816964</id><published>2011-05-20T06:14:00.000-07:00</published><updated>2011-05-20T06:19:34.787-07:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='books'/><title type='text'>The count of Monte Cristo</title><content type='html'>I spent a couple of weeks reading through the Gutenberg ebook of "The Count of Monte Cristo". All in all I liked it. I could spend some time talking about what I liked, but there's plenty of that already on the internet.&lt;br /&gt;&lt;br /&gt;I did however find the level and detail of the intrigues, manipulation, coercion and general dirty play a little on the creepy side. Don't get me wrong, it made a great book, but it felt almost like Dumas had read "The Prince" and decided to write a book putting it into play; a sort of "what if?" scenario where someone had been betrayed and wanted to get back at the world; then finding a suitable vessel to do this through.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-2038071138464816964?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/2038071138464816964/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2011/05/count-of-monte-cristo.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/2038071138464816964'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/2038071138464816964'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2011/05/count-of-monte-cristo.html' title='The count of Monte Cristo'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-4250126182674068594</id><published>2011-05-15T22:07:00.001-07:00</published><updated>2011-05-15T22:19:19.270-07:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='freebsd'/><category scheme='http://www.blogger.com/atom/ns#' term='802.11n'/><category scheme='http://www.blogger.com/atom/ns#' term='atheros'/><title type='text'>AR9160 11n performance - finally!</title><content type='html'>I've made some good progress with the AR9160 802.11n hostap mode in FreeBSD.&lt;br /&gt;&lt;br /&gt;Firstly, the A-MPDU density setting needed tuning. "0" or "N/A" isn't appropriate for the AR9160 (and likely not appropriate for anything earlier than the AR9300/AR9400 series.) Once set to 8 microseconds, the number of A-MPDU retransmits dropped dramatically.&lt;br /&gt;&lt;br /&gt;I found that there was some missing code to do with disabling RIFS (reduced inter-frame spacing) RX on the AR9160 which I committed to FreeBSD-HEAD. This has eliminated all of the baseband lockups I've been seeing, rare as they were now.&lt;br /&gt;&lt;br /&gt;Then I found there were packets being dropped by if_arge on TX. It turns out the default TX/RX ring buffer size for if_arge was 128 packets - definitely not enough given the uncoupled interrupt/process driven forwarding engine in UNIX these days.&lt;br /&gt;&lt;br /&gt;The background: since NIC drivers aren't doing their work in an interrupt or shared process context, they only do their work when their TX/RX tasks get scheduled. Since they don't have any way to signal each other to "back off" via flow control when the queues are getting filled, it's quite possible to have a very busy device (in this instance wlan0 RX'ing anything more than around 100mbit) queue enough packets to the devices' TX queue faster than the TX task can be scheduled to process packets.&lt;br /&gt;&lt;br /&gt;Bumping if_arge's default TX/RX ring buffer size from 128 to 1024 did the trick - no more packet drops on TX.&lt;br /&gt;&lt;br /&gt;Now I'm getting ~ 105mbit TCP RX in HT/40 5GHz hostap mode (since A-MPDU RX is implemented and working) and ~ 210mbit UDP RX before I begin to drop packets. Anything more than 210mbit starts resulting in the infamous "stuck beacon", but that could be for a variety of reasons. I don't think I'm saturating the PCI bus (read: the CPU scheduler does show about 30% idle clock cycles during a UDP RX test) so there's something else to be blamed.&lt;br /&gt;&lt;br /&gt;Next is figuring out whether there's some scheduling issues with the device TX/RX tasks.&lt;br /&gt;&lt;br /&gt;I find it quite creepy that I can get ~ 100mbit of 802.11n throughput to my macbook pro whilst lying in bed. But that's what 802.11n is for, right? :)&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-4250126182674068594?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/4250126182674068594/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2011/05/ar9160-11n-performance-finally.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/4250126182674068594'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/4250126182674068594'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2011/05/ar9160-11n-performance-finally.html' title='AR9160 11n performance - finally!'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-3626751126254577400</id><published>2011-05-02T08:01:00.000-07:00</published><updated>2011-05-02T08:13:28.199-07:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='freebsd'/><category scheme='http://www.blogger.com/atom/ns#' term='802.11n'/><category scheme='http://www.blogger.com/atom/ns#' term='atheros'/><title type='text'>FreeBSD-HEAD on the PB92 + AR9280</title><content type='html'>&lt;span style="font-size:78%;"&gt;&lt;span style="font-family:courier new;"&gt;&lt;span style="font-family:arial;font-size:100%;"&gt;Here's FreeBSD-HEAD up on the Atheros PB92 reference board (AR7242), complete with an AR9280 mini-PCIe NIC.&lt;br /&gt;&lt;br /&gt;I'm still only getting around 70mbit in HT/40 mode TCP tests and 90mbit in HT/40 UDP mode; but it's only (currently) connected at 100mbit ethernet. I'll tinker some more with gigabit-connected stuff soon. I hope the UDP throughput maxes out above 100mbit.&lt;br /&gt;&lt;br /&gt;It's a far cry from where it should be throughput-wise, but it's getting there.&lt;br /&gt;&lt;br /&gt;Another developer has -HEAD + a small patch working on the Ubiquiti Rocket M5 (AR7240 + AR9280 on-board), which is also great progress.&lt;br /&gt;&lt;/span&gt;&lt;br /&gt;&lt;br /&gt;# dmesg | grep ath&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family:courier new;"&gt;ath0: &lt;atheros 9280=""&gt; at device 0.0 on pci0&lt;/atheros&gt;&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family:courier new;"&gt;ath0: [HT] enabling HT modes&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family:courier new;"&gt;ath0: [HT] 2 RX streams; 2 TX streams&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family:courier new;"&gt;ath0: AR9280 mac 128.2 RF5133 phy 13.0&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family:courier new;"&gt;# uname -a &lt;/span&gt;&lt;br /&gt;&lt;span style="font-family:courier new;"&gt;FreeBSD TEST_AP 9.0-CURRENT FreeBSD 9.0-CURRENT #19 r220911:221321M: Mon May  2 22:46:32 WST 2011     adria&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family:courier new;"&gt;n@pcbsd-3114:/data/freebsd/mips/head/obj/mipseb/mips.mipseb/data/freebsd/mips/head/src/sys/PB92  mips&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family:courier new;"&gt;# ifconfig wlan0 list sta&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family:courier new;"&gt;ADDR               AID CHAN RATE RSSI IDLE  TXSEQ  RXSEQ CAPS FLAG   &lt;/span&gt;&lt;br /&gt;&lt;span style="font-family:courier new;"&gt;00:15:6d:84:05:52    1  157 240M 15.5  120  27094  58288 EP   AQHTR   HTCAP WPA WME&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family:courier new;"&gt;8c:7b:9d:d6:65:ba    2  157 270M 21.5    0  56040  64688 EP   AQHTRS  RSN HTCAP WME&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family:courier new;"&gt;# &lt;/span&gt;&lt;br /&gt;&lt;br /&gt;&lt;/span&gt;&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-3626751126254577400?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/3626751126254577400/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2011/05/freebsd-head-on-pb92-ar9280.html#comment-form' title='6 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/3626751126254577400'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/3626751126254577400'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2011/05/freebsd-head-on-pb92-ar9280.html' title='FreeBSD-HEAD on the PB92 + AR9280'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>6</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-8596132655709263533</id><published>2011-04-28T05:17:00.000-07:00</published><updated>2011-04-28T05:19:54.365-07:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='freebsd'/><category scheme='http://www.blogger.com/atom/ns#' term='802.11n'/><category scheme='http://www.blogger.com/atom/ns#' term='atheros'/><title type='text'>AR913x support in -HEAD, or TP-Link WR-1043nd</title><content type='html'>In summary:&lt;br /&gt;&lt;span style="font-family: courier new;font-size:78%;" &gt;&lt;br /&gt;KDB: debugger backends: ddb&lt;br /&gt;KDB: current backend: ddb&lt;br /&gt;Copyright (c) 1992-2011 The FreeBSD Project.&lt;br /&gt;Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994&lt;br /&gt;        The Regents of the University of California. All rights reserved.&lt;br /&gt;FreeBSD is a registered trademark of The FreeBSD Foundation.&lt;br /&gt;FreeBSD 9.0-CURRENT #6 r220911:221160M: Thu Apr 28 20:09:28 WST 2011&lt;br /&gt;    adrian@pcbsd-3114:/data/freebsd/mips/head/obj/mipseb/mips.mipseb/data/freebsd/mips/head/src/sys/TP-WN1043ND mips&lt;br /&gt;real memory  = 33554432 (32768K bytes)&lt;br /&gt;avail memory = 21123072 (20MB)&lt;br /&gt;nexus0: &lt;mips32 root="" nexus=""&gt;&lt;br /&gt;clock0: &lt;generic mips32="" ticker=""&gt; on nexus0&lt;br /&gt;Timecounter "MIPS32" frequency 200000000 Hz quality 800&lt;br /&gt;Event timer "MIPS32" frequency 200000000 Hz quality 800&lt;br /&gt;apb0 at irq 4 on nexus0&lt;br /&gt;uart0: &amp;lt;16550 or compatible&amp;gt; on apb0&lt;br /&gt;uart0: console (115200,n,8,1)&lt;br /&gt;ehci0: &lt;ar71xx integrated="" usb="" 0="" controller=""&gt; at mem 0x1b000100-0x1bffffff irq 1 on nexus0&lt;br /&gt;usbus0: set host controller mode&lt;br /&gt;usbus0: EHCI version 1.0&lt;br /&gt;usbus0: set host controller mode&lt;br /&gt;usbus0: &lt;ar71xx integrated="" usb="" 0="" controller=""&gt; on ehci0&lt;br /&gt;arge0: &lt;atheros ar71xx="" in="" ethernet="" interface=""&gt; at mem 0x19000000-0x19000fff irq 2 on nexus0&lt;br /&gt;arge0: Overriding MAC from EEPROM&lt;br /&gt;arge0: Ethernet address: 94:0c:6d:fe:4f:20&lt;br /&gt;arge1: &lt;atheros ar71xx="" in="" ethernet="" interface=""&gt; at mem 0x1a000000-0x1a000fff irq 3 on nexus0&lt;br /&gt;device_attach: arge1 attach returned 22&lt;br /&gt;ath0: &lt;atheros 9130=""&gt; at mem 0x180c0000-0x180effff irq 0 on nexus0&lt;br /&gt;ath0: eeprom @ 0x1fff1000&lt;br /&gt;ath0: eeprom data @ 0xbfff1000&lt;br /&gt;ath0: [HT] enabling HT modes&lt;br /&gt;ath0: [HT] 2 RX streams; 2 TX streams&lt;br /&gt;ath0: AR9130 mac 20.1 RF2133 phy 10.2&lt;br /&gt;spi0: &lt;ar71xx spi=""&gt; at mem 0x1f000000-0x1f00000f on nexus0&lt;br /&gt;spibus0: &lt;spibus bus=""&gt; on spi0&lt;br /&gt;mx25l0: &lt;m25pxx flash="" family=""&gt; at cs 0 on spibus0&lt;br /&gt;mx25l0: s25sl064a, sector 65536 bytes, 128 sectors&lt;br /&gt;ar71xx_wdog0: &lt;atheros ar71xx="" watchdog="" timer=""&gt; on nexus0&lt;br /&gt;ar71xx_wdog0: Previous reset was due to watchdog timeout&lt;br /&gt;Timecounters tick every 1.000 msec&lt;br /&gt;usbus0: 480Mbps High Speed USB v2.0&lt;br /&gt;md0.uzip: 855 x 16384 blocks&lt;br /&gt;ugen0.1: &lt;atheros&gt; at usbus0&lt;br /&gt;uhub0: &lt;atheros ehci="" root="" class="" 9="" rev="" 00="" addr="" 1=""&gt; on usbus0&lt;br /&gt;Root mount waiting for: usbus0&lt;br /&gt;uhub0: 1 port with 1 removable, self powered&lt;br /&gt;Root mount waiting for: usbus0&lt;br /&gt;ugen0.2: &lt;sandisk&gt; at usbus0&lt;br /&gt;umass0: &lt;sandisk cruzer="" class="" 0="" rev="" 00="" addr="" 2=""&gt; on usbus0&lt;br /&gt;umass0:  SCSI over Bulk-Only; quirks = 0x0000&lt;br /&gt;Root mount waiting for: usbus0&lt;br /&gt;umass0:0:0:-1: Attached to scbus0&lt;br /&gt;Trying to mount root from ufs:/dev/md0 []...&lt;br /&gt;da0 at umass-sim0 bus 0 scbus0 target 0 lun 0&lt;br /&gt;da0: &lt;sandisk cruzer="" blade="" 02=""&gt; Removable Direct Access SCSI-0 device&lt;br /&gt;da0: 40.000MB/s transfers&lt;br /&gt;da0: 3835MB (7856127 512 byte sectors: 255H 63S/T 489C)&lt;br /&gt;Mounting from ufs:/dev/md0 failed with error 22.&lt;br /&gt;Trying to mount root from ufs:/dev/md0.uzip []...&lt;br /&gt;warning: no time-of-day clock registered, system time will not be set accurately&lt;br /&gt;bridge0: Ethernet address: 06:26:1a:6f:d5:e7&lt;br /&gt;wlan0: Ethernet address: 00:19:e0:66:66:68&lt;/sandisk&gt;&lt;/sandisk&gt;&lt;/sandisk&gt;&lt;/atheros&gt;&lt;/atheros&gt;&lt;/atheros&gt;&lt;/m25pxx&gt;&lt;/spibus&gt;&lt;/ar71xx&gt;&lt;/atheros&gt;&lt;/atheros&gt;&lt;/atheros&gt;&lt;/ar71xx&gt;&lt;/ar71xx&gt;&lt;/generic&gt;&lt;/mips32&gt;&lt;/span&gt;&lt;br /&gt;&lt;br /&gt;And the wireless:&lt;br /&gt;&lt;br /&gt;&lt;span style="font-family: courier new;font-size:78%;" &gt;# ifconfig wlan0&lt;br /&gt;wlan0: flags=8943&lt;up,broadcast,running,promisc,simplex,multicast&gt; metric 0 mtu 1500&lt;br /&gt;        ether 00:19:e0:66:66:68&lt;br /&gt;        inet6 fe80::219:e0ff:fe66:6668%wlan0 prefixlen 64 scopeid 0x6&lt;br /&gt;        nd6 options=21&lt;performnud,auto_linklocal&gt;&lt;br /&gt;        media: IEEE 802.11 Wireless Ethernet autoselect mode 11ng &lt;hostap&gt;&lt;br /&gt;        status: running&lt;br /&gt;        &lt;span style="font-weight: bold;"&gt;ssid CACHEBOY_RS channel 6 (&lt;span style="color: rgb(255, 0, 0);"&gt;2437 MHz 11g ht/40+&lt;/span&gt;)&lt;/span&gt; bssid 00:19:e0:66:66:68&lt;br /&gt;        regdomain ROW country AU ecm authmode WPA1+WPA2/802.11i privacy MIXED&lt;br /&gt;        deftxkey 2 TKIP 2:128-bit txpower 30 scanvalid 60 protmode CTS&lt;br /&gt;        -ampdutx ampdurx ampdulimit 64k -amsdu shortgi wme burst dtimperiod 1&lt;br /&gt;        -dfs&lt;br /&gt;&lt;/hostap&gt;&lt;/performnud,auto_linklocal&gt;&lt;/up,broadcast,running,promisc,simplex,multicast&gt;&lt;/span&gt;&lt;br /&gt;I'll see about sneaking this into -HEAD soon.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-8596132655709263533?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/8596132655709263533/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2011/04/ar913x-support-in-head-or-tp-link-wr.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/8596132655709263533'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/8596132655709263533'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2011/04/ar913x-support-in-head-or-tp-link-wr.html' title='AR913x support in -HEAD, or TP-Link WR-1043nd'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-8916886574777848432</id><published>2011-04-12T19:23:00.000-07:00</published><updated>2011-04-12T19:33:42.744-07:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='freebsd'/><category scheme='http://www.blogger.com/atom/ns#' term='embedded'/><title type='text'>Oh god, so much wireless hardware..</title><content type='html'>Since I've started doing wireless development, people seem to be sending me more and more hardware. (I've also bought some stuff off of ebay too, so I'm partially to blame for this. :-)&lt;br /&gt;&lt;br /&gt;So there's this group of developers over in Europe/Russia who have been busy porting FreeBSD to a variety of wireless devices, but their work hasn't made it into the main FreeBSD tree (or been published at all in any public way.) I've been liaising with Aleksandr Rybalko to push their work into -HEAD in time for FreeBSD-9.0.&lt;br /&gt;&lt;br /&gt;So far this has included:&lt;br /&gt;&lt;br /&gt;&lt;ul&gt;&lt;li&gt;Support for the Ralink RT305x MIPS SoC, with initial peripheral support;&lt;/li&gt;&lt;li&gt;nvram2env, a way of importing the environment from various bootloaders (eg uboot) into the kernel environment;&lt;br /&gt;&lt;/li&gt;&lt;li&gt;geom_map, a GEOM module which describes the flash layout for fixed flash partitioning schemes (again like uboot, but it can be used for anything.)&lt;/li&gt;&lt;/ul&gt;There's a few other things coming up soon, such as support for one of the Broadcom MIPS SoCs, further peripheral support for the RT305x (including the embedded wireless MAC/radio) and other bits and pieces which will prove to be useful.&lt;br /&gt;&lt;br /&gt;It's been a pleasure working with them&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-8916886574777848432?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/8916886574777848432/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2011/04/oh-god-so-much-wireless-hardware.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/8916886574777848432'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/8916886574777848432'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2011/04/oh-god-so-much-wireless-hardware.html' title='Oh god, so much wireless hardware..'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-5683281778242673508</id><published>2011-03-27T07:39:00.001-07:00</published><updated>2011-03-27T07:50:11.302-07:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='freebsd'/><category scheme='http://www.blogger.com/atom/ns#' term='802.11n'/><category scheme='http://www.blogger.com/atom/ns#' term='atheros'/><title type='text'>11n TX and RX experiments..</title><content type='html'>My experiments so far are as follows (all in 5GHz mode - 2GHz is just too busy in my apartment complex):&lt;div&gt;&lt;ul&gt;&lt;li&gt;FreeBSD &amp;lt;-&amp;gt; FreeBSD 11n is stable on the AR9160 and AR9220. TX aggregation is disabled, so it reaches around 35-40mbit in HT/20 and 45mbit in HT/40 mode. I'm getting MCS13 between an AP and STA which are in different rooms in my apartment. MCS15 is achievable if the units are close to each other.&lt;/li&gt;&lt;li&gt;Short-GI in 40MHz works fine.&lt;/li&gt;&lt;li&gt;When RX aggregation is enabled, and IEEE80211_AMPDU_AGE is enabled with net.wlan.ampdu_age set to 5 (it defaults to 500), I get ~ 55mbit RX in HT/20 and ~ 75mbit RX in HT/40 mode. There's problems with out of order packets that are appearing -before- the calculated block-ack window and net80211 is dropping those. I'll investigate why at some point.&lt;/li&gt;&lt;li&gt;FreeBSD works fine as an 11n AP, obviously with A-MPDU TX disabled.&lt;/li&gt;&lt;/ul&gt;&lt;div&gt;There's lots of stuff that needs to be tested at this point, including:&lt;/div&gt;&lt;/div&gt;&lt;div&gt;&lt;ul&gt;&lt;li&gt;2GHz operation, obviously!&lt;/li&gt;&lt;li&gt;11n and legacy co-operation.&lt;/li&gt;&lt;li&gt;Other chipsets - AR5416, AR9280, AR9285.&lt;/li&gt;&lt;li&gt;Behaviour in the presence of noise, busy channel, etc.&lt;/li&gt;&lt;li&gt;Make sure that HT/40 mode is -correctly- protecting frames on both channels and is correctly waiting to make sure both channels are free before transmitting!&lt;/li&gt;&lt;/ul&gt;&lt;div&gt;And stuff that needs to be implemented before we enable it by default:&lt;/div&gt;&lt;/div&gt;&lt;div&gt;&lt;ul&gt;&lt;li&gt;A-MPDU TX aggregation, for STA, AP, Mesh and adhoc modes - this one is really needed and is going to take some time to do.&lt;/li&gt;&lt;li&gt;HT RTS frame protection - I have code for this which I'll commit soon.&lt;/li&gt;&lt;/ul&gt;&lt;/div&gt;&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-5683281778242673508?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/5683281778242673508/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2011/03/11n-tx-and-rx-experiments.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/5683281778242673508'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/5683281778242673508'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2011/03/11n-tx-and-rx-experiments.html' title='11n TX and RX experiments..'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-7349991820291490290</id><published>2011-03-20T02:59:00.000-07:00</published><updated>2011-03-20T03:05:27.389-07:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='freebsd'/><category scheme='http://www.blogger.com/atom/ns#' term='802.11n'/><category scheme='http://www.blogger.com/atom/ns#' term='atheros'/><title type='text'>(More) wireless hackery</title><content type='html'>So instead of doing what I should be doing, I did some more wifi hackery.&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;The short summary version (all 802.11g 2.4ghz testing):&lt;/div&gt;&lt;div&gt;&lt;ul&gt;&lt;li&gt;AR5416 TX is still unhappy and unpredictable. It's also like that with ath9k. AR5416 RX is perfectly fine.&lt;/li&gt;&lt;li&gt;AR9160 TX/RX is fine.&lt;/li&gt;&lt;li&gt;AR9220 (Ubiquiti SR71-12 and Ubiquiti-SR71-15) TX/RX is now fine.&lt;/li&gt;&lt;li&gt;AR9280 closed and open-loop TX is fine, RX is also fine.&lt;/li&gt;&lt;li&gt;AR9285 TX/RX is fine and stable now.&lt;/li&gt;&lt;/ul&gt;&lt;div&gt;I'll now go and test out 5GHz AP/STA modes with the above (where appropriate) and see how they behave. I hope they'll be just as stable now.&lt;/div&gt;&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;Bernard has been making inroads into fleshing out the missing parts of the net80211 802.11n support. Many thanks to him for taking the time to dissect the 802.11n spec, the Linux 802.11 stack and spend time with some APs determining what they're doing.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-7349991820291490290?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/7349991820291490290/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2011/03/more-wireless-hackery_20.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/7349991820291490290'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/7349991820291490290'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2011/03/more-wireless-hackery_20.html' title='(More) wireless hackery'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-705270420825141178</id><published>2011-03-11T05:49:00.001-08:00</published><updated>2011-03-11T05:58:46.586-08:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='freebsd'/><category scheme='http://www.blogger.com/atom/ns#' term='802.11n'/><category scheme='http://www.blogger.com/atom/ns#' term='atheros'/><title type='text'>More wireless hackery</title><content type='html'>&lt;div&gt;I've had a busy week in the land of FreeBSD wireless driver hacking.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;I've been porting over the AR9280 TX calibration changes from ath9k. The AR9280 NIC in my laptop loves it - but the AR9220 based SR71-12 is still unhappy. I'll go poke around and see if I can figure out what the problem is. I have some other AR9280 series NICs to try out.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;I've been using hostap mode on the AR5416 and AR9160 and it still seems quite fine and stable.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;I've ported over a number of changes to the AR9285 codebase from Linux ath9k. These include:&lt;/div&gt;&lt;div&gt;&lt;ul&gt;&lt;li&gt;Fixing the TX power calibration curve code;&lt;/li&gt;&lt;li&gt;Handling a TX power offset at the beginning of the calibration curve;&lt;/li&gt;&lt;li&gt;Updating how the board is initially calibrated on reset;&lt;/li&gt;&lt;li&gt;Adding PA calibration.&lt;/li&gt;&lt;/ul&gt;&lt;div&gt;I'm also trying to remove some of the duplicate code in the HAL that has crept up as chipset support was added.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;I'm waiting to hear back from some other users about what effects it has on their setup. I still have to port over the (completely rewritten) radio board configuration code from ath9k that sets up all kinds of radio related gain, offset and various things. There's also software antenna diversity to port over. I hope that this all fixes the issues users have been reporting with AR9285 NICs just plain not working in a lot of instances - but working fine in others. I also hope it fixes the AR2427 support.&lt;/div&gt;&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;I'm still very unclear what may be up with the AR9280 support. The Ubiquiti high-powered cards tend to have strange EEPROM calibration settings, so it's quite possible FreeBSD just handles one of the options badly. That's going to take some time to figure out.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;There's no point in getting 802.11n support enabled if I can't TX and RX cleanly, so I'm leaving 802.11n alone (for the most part) until the radios and MACs are behaving themselves.&lt;/div&gt;&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-705270420825141178?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/705270420825141178/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2011/03/more-wireless-hackery.html#comment-form' title='3 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/705270420825141178'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/705270420825141178'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2011/03/more-wireless-hackery.html' title='More wireless hackery'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>3</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-7384550923041841797</id><published>2011-03-05T20:05:00.000-08:00</published><updated>2011-03-05T20:17:12.530-08:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='freebsd'/><category scheme='http://www.blogger.com/atom/ns#' term='802.11n'/><category scheme='http://www.blogger.com/atom/ns#' term='atheros'/><title type='text'>AR9280 TX errors - why it's important to care about how your hardware works..</title><content type='html'>One of the issues that I've been having with my AR9280 NIC is that MCS rates aren't being reliable.&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;Having fiddled with the TX power control with the AR9160 (and I'll write about that soon!) I remembered that badly-configured TX power control would result in the transmit side spitting out a (sometimes very) unclean spectral mask, with noise all over the place.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;I did a little digging into how the AR9280 TX-side calibration works. It turns out that there's two methods of TX power control - closed-loop (with a power-detector ADC which compares the signal to a calibrated level) and open-loop (which I'm not yet sure about.) The AR9280 NIC that I have has the "open loop TX power control" bit set in the EEPROM, which indicates it needs open-loop TX power rather than closed-loop. A little digging shows that FreeBSD's HAL doesn't include closed-loop TX power control or any of the newer AR9280 calibration code (in particular the open-loop temperature compensation code.)&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;Long-story short, I've added the open-loop TX calibration and temperature compensation code from ath9k to the FreeBSD HAL (but I haven't yet committed it and won't until I figure out how to make it tidy) and suddenly all MCS rates TX perfectly fine.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;So if you're using an AR9280 NIC, please keep a look out for when I commit this stuff and let me know if it improves things.&lt;/div&gt;&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-7384550923041841797?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/7384550923041841797/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2011/03/ar9280-tx-errors-why-its-important-to.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/7384550923041841797'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/7384550923041841797'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2011/03/ar9280-tx-errors-why-its-important-to.html' title='AR9280 TX errors - why it&apos;s important to care about how your hardware works..'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-8968610910006629430</id><published>2011-02-22T10:30:00.000-08:00</published><updated>2011-02-22T10:40:08.812-08:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='meatpies'/><category scheme='http://www.blogger.com/atom/ns#' term='newyork'/><category scheme='http://www.blogger.com/atom/ns#' term='holiday'/><title type='text'>This is for Emily..</title><content type='html'>&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;My friend Emily asks, "is there anywhere to buy a meat pie in New York?"&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;br /&gt;&lt;div&gt;So I find this place on St Mark's place, east of First Ave.&lt;/div&gt;&lt;br /&gt;&lt;br /&gt;&lt;a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://2.bp.blogspot.com/-5GT4fkCN9ww/TWQCQdhaUGI/AAAAAAAAAFU/q8XpYIPImqk/s1600/CIMG0074.jpg"&gt;&lt;img style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 320px; height: 239px;" src="http://2.bp.blogspot.com/-5GT4fkCN9ww/TWQCQdhaUGI/AAAAAAAAAFU/q8XpYIPImqk/s320/CIMG0074.jpg" border="0" alt="" id="BLOGGER_PHOTO_ID_5576584720417443938" /&gt;&lt;/a&gt;&lt;br /&gt;And apparently it sells meat pies.&lt;br /&gt;&lt;br /&gt;&lt;a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://4.bp.blogspot.com/-nmdtpKBK2Zw/TWQCQpnjEqI/AAAAAAAAAFc/b501n1MB7Ys/s1600/CIMG0073.jpg"&gt;&lt;img style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 239px; height: 320px;" src="http://4.bp.blogspot.com/-nmdtpKBK2Zw/TWQCQpnjEqI/AAAAAAAAAFc/b501n1MB7Ys/s320/CIMG0073.jpg" border="0" alt="" id="BLOGGER_PHOTO_ID_5576584723664409250" /&gt;&lt;/a&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;And lo and behold, they sell meat pies.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://4.bp.blogspot.com/-bRaKpnaDsjw/TWQCQ1_SVAI/AAAAAAAAAFk/VcYEF-b9r8s/s1600/CIMG0075.jpg"&gt;&lt;img style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 320px; height: 239px;" src="http://4.bp.blogspot.com/-bRaKpnaDsjw/TWQCQ1_SVAI/AAAAAAAAAFk/VcYEF-b9r8s/s320/CIMG0075.jpg" border="0" alt="" id="BLOGGER_PHOTO_ID_5576584726985200642" /&gt;&lt;/a&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;The answer is "yes".&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-8968610910006629430?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/8968610910006629430/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2011/02/this-is-for-emily.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/8968610910006629430'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/8968610910006629430'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2011/02/this-is-for-emily.html' title='This is for Emily..'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><media:thumbnail xmlns:media='http://search.yahoo.com/mrss/' url='http://2.bp.blogspot.com/-5GT4fkCN9ww/TWQCQdhaUGI/AAAAAAAAAFU/q8XpYIPImqk/s72-c/CIMG0074.jpg' height='72' width='72'/><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-7146418827055531565</id><published>2011-02-17T09:37:00.000-08:00</published><updated>2011-02-17T09:39:58.494-08:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='university'/><title type='text'>The end is nigh..</title><content type='html'>&lt;a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://1.bp.blogspot.com/-vGQ7WA4j7E0/TV1dLLtBk5I/AAAAAAAAAFM/Z8ZqLrDYu_8/s1600/degree-almost-done.png"&gt;&lt;img style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 320px; height: 129px;" src="http://1.bp.blogspot.com/-vGQ7WA4j7E0/TV1dLLtBk5I/AAAAAAAAAFM/Z8ZqLrDYu_8/s320/degree-almost-done.png" border="0" alt="" id="BLOGGER_PHOTO_ID_5574714360456123282" /&gt;&lt;/a&gt;&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-7146418827055531565?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/7146418827055531565/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2011/02/end-is-nigh.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/7146418827055531565'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/7146418827055531565'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2011/02/end-is-nigh.html' title='The end is nigh..'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><media:thumbnail xmlns:media='http://search.yahoo.com/mrss/' url='http://1.bp.blogspot.com/-vGQ7WA4j7E0/TV1dLLtBk5I/AAAAAAAAAFM/Z8ZqLrDYu_8/s72-c/degree-almost-done.png' height='72' width='72'/><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-5686584668582795426</id><published>2011-02-16T12:37:00.000-08:00</published><updated>2011-02-16T12:45:39.326-08:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='newyork'/><title type='text'>Week 1 - stuff seen randomly about</title><content type='html'>&lt;div&gt;Here's a few random things I saw in the first week here, before I fell stupidly ill.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;The Cupcake Lover:&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://4.bp.blogspot.com/-y4NhhsYdoHI/TVw16lPDk7I/AAAAAAAAAEk/8S7bKsRd4-s/s1600/CIMG0006.jpg"&gt;&lt;img style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 320px; height: 239px;" src="http://4.bp.blogspot.com/-y4NhhsYdoHI/TVw16lPDk7I/AAAAAAAAAEk/8S7bKsRd4-s/s320/CIMG0006.jpg" border="0" alt="" id="BLOGGER_PHOTO_ID_5574389719321777074" /&gt;&lt;/a&gt;&lt;br /&gt;&lt;div&gt;A lot of Converse Shoes:&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;br /&gt;&lt;a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://1.bp.blogspot.com/-0luKSLD46M4/TVw2WYiZ5WI/AAAAAAAAAEs/aNwEPWYb-3U/s1600/CIMG0010.jpg"&gt;&lt;img style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 239px; height: 320px;" src="http://1.bp.blogspot.com/-0luKSLD46M4/TVw2WYiZ5WI/AAAAAAAAAEs/aNwEPWYb-3U/s320/CIMG0010.jpg" border="0" alt="" id="BLOGGER_PHOTO_ID_5574390196949607778" /&gt;&lt;/a&gt;&lt;br /&gt;&lt;div&gt;A nicely washed out photo of Times Square:&lt;br /&gt;&lt;br /&gt;&lt;br /&gt;&lt;a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://2.bp.blogspot.com/-S_psW0_9vWw/TVw2nRg2MHI/AAAAAAAAAE0/C2ZZT4nVj0M/s1600/CIMG0021.jpg"&gt;&lt;img style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 239px; height: 320px;" src="http://2.bp.blogspot.com/-S_psW0_9vWw/TVw2nRg2MHI/AAAAAAAAAE0/C2ZZT4nVj0M/s320/CIMG0021.jpg" border="0" alt="" id="BLOGGER_PHOTO_ID_5574390487121801330" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;&lt;br /&gt;&lt;br /&gt;&lt;a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://1.bp.blogspot.com/-otR6g_NzIYI/TVw3DWIVo_I/AAAAAAAAAE8/Qyv9pd964fk/s1600/CIMG0022.jpg"&gt;&lt;img style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 320px; height: 239px;" src="http://1.bp.blogspot.com/-otR6g_NzIYI/TVw3DWIVo_I/AAAAAAAAAE8/Qyv9pd964fk/s320/CIMG0022.jpg" border="0" alt="" id="BLOGGER_PHOTO_ID_5574390969397519346" /&gt;&lt;/a&gt;&lt;br /&gt;&lt;div&gt;And "Radioactive", an Art display in one of the NY Public Library buildings in Mid-town. By the way, the interior of the library buildings I've seen thus far are gorgeous and make me wish I had even an inkling of photographic technique/equipment on me.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://3.bp.blogspot.com/-ja7WdOBX94Y/TVw3DljO9DI/AAAAAAAAAFE/PbMS04fBzPU/s1600/CIMG0030.jpg"&gt;&lt;img style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 320px; height: 239px;" src="http://3.bp.blogspot.com/-ja7WdOBX94Y/TVw3DljO9DI/AAAAAAAAAFE/PbMS04fBzPU/s320/CIMG0030.jpg" border="0" alt="" id="BLOGGER_PHOTO_ID_5574390973536859186" /&gt;&lt;/a&gt;&lt;br /&gt;&lt;/div&gt;&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-5686584668582795426?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/5686584668582795426/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2011/02/week-1-stuff-seen-randomly-about.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/5686584668582795426'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/5686584668582795426'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2011/02/week-1-stuff-seen-randomly-about.html' title='Week 1 - stuff seen randomly about'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><media:thumbnail xmlns:media='http://search.yahoo.com/mrss/' url='http://4.bp.blogspot.com/-y4NhhsYdoHI/TVw16lPDk7I/AAAAAAAAAEk/8S7bKsRd4-s/s72-c/CIMG0006.jpg' height='72' width='72'/><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-8098559848283075460</id><published>2011-02-08T15:00:00.000-08:00</published><updated>2011-02-16T12:37:32.698-08:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='newyork'/><title type='text'>Space Invaders!</title><content type='html'>&lt;a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://3.bp.blogspot.com/-EBOHXllgKWk/TVw1Pafz-XI/AAAAAAAAAEc/0Oqd87FXiHo/s1600/CIMG0026.jpg"&gt;&lt;img style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 320px; height: 239px;" src="http://3.bp.blogspot.com/-EBOHXllgKWk/TVw1Pafz-XI/AAAAAAAAAEc/0Oqd87FXiHo/s320/CIMG0026.jpg" border="0" alt="" id="BLOGGER_PHOTO_ID_5574388977704892786" /&gt;&lt;/a&gt;&lt;br /&gt;Who remembers this?&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-8098559848283075460?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/8098559848283075460/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2011/02/space-invaders.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/8098559848283075460'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/8098559848283075460'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2011/02/space-invaders.html' title='Space Invaders!'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><media:thumbnail xmlns:media='http://search.yahoo.com/mrss/' url='http://3.bp.blogspot.com/-EBOHXllgKWk/TVw1Pafz-XI/AAAAAAAAAEc/0Oqd87FXiHo/s72-c/CIMG0026.jpg' height='72' width='72'/><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-561867844439597261</id><published>2011-02-04T11:07:00.000-08:00</published><updated>2011-02-16T15:24:50.415-08:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='newyork'/><title type='text'>Sleepless in New York..</title><content type='html'>I had every intent to actually run this as a travel diary for a few weeks, but a few things got in the way.&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;Well, no, I sort of fib - my normal "meh, diaries!" got in the way. So I decided to play a bit of catch-up - then I fell ill with a cold and respiratory infection which has kept me knocked out for a good week or so.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;So, here I go - catchup.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;&lt;b&gt;The Flight Over&lt;/b&gt;&lt;/div&gt;&lt;div&gt;&lt;b&gt;&lt;br /&gt;&lt;/b&gt;&lt;/div&gt;&lt;div&gt;The flight over was mostly uneventful - except for the complete lack of sleep. Since this was a 30 hour transit from Perth to New York, this made for rather interesting experiences near the end of the trip.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;LAX&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;.. was again, a complete and utter mess. I was sent to the wrong terminal; then after walking from terminal 2 to terminal 7 and back,  I find I needed to walk across the road to the Bradley International terminal. I likely should have been much, much more prepared than I was.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;JFK&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;Relatively smooth, for a change.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;Subway to Manhattan&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;I sat next to a Brazilian photographer who told me of his love of asian women. He showed me a few photographs and then proceeded to photograph them secretly without their consent or awareness. I then figured out most of his photos are like that. This creeped me out somewhat.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-561867844439597261?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/561867844439597261/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2011/02/sleepless-in-new-york.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/561867844439597261'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/561867844439597261'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2011/02/sleepless-in-new-york.html' title='Sleepless in New York..'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-946972937102143578</id><published>2011-01-28T20:36:00.000-08:00</published><updated>2011-01-28T20:44:57.508-08:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='freebsd'/><category scheme='http://www.blogger.com/atom/ns#' term='802.11n'/><category scheme='http://www.blogger.com/atom/ns#' term='atheros'/><category scheme='http://www.blogger.com/atom/ns#' term='atheros 802.11n'/><title type='text'>AR9280/AR9285 support now working</title><content type='html'>I managed to find and fix the AR9280 and AR9285 support.&lt;br /&gt;&lt;br /&gt;Things that needed repairing:&lt;br /&gt;&lt;br /&gt;&lt;ul&gt;&lt;li&gt;Updating the AR9280 initvals to the Linux ath9k ones resolved the radio initialisation issues, making everything magically stable!&lt;/li&gt;&lt;li&gt;The v4k EEPROM code (for the AR9285/AR2427) needed some surgery to reflect what was really going on. In particular, the 8 bit radio bias values in the AR9280 (v14) EEPROM are actually lots of 4 bit values; so the bias values being written to the radio were woefully incorrect. This restored AR9285 stability and made the AR2427 function.&lt;/li&gt;&lt;li&gt;The AR9280 RF registers need an extra delay before being written to. I guess since the earlier radios are externally connected via single-bit IO (and thus shift registers are involved), the later radios are too. Without this delay, the AR9220 panics on my MIPS board and I would get occasional unexplained resets when I configured an AR9280 on my eeepc. This fix has resolved both those issues.&lt;/li&gt;&lt;/ul&gt;I have one more fix to integrate for the AR9220 init path so the Ubiquiti SR71-12 and SR71-15 work; then the AR9220 is supported.&lt;br /&gt;&lt;br /&gt;The AR2427 baseband seems to hang after a few hours of use, requiring a complete cold (power off) restart. There's some more initialisation code I need to port from ath9k and there's a lot of AR9280/AR9285 calibration code which I need to port over. I hope porting these two over will fix the stability issues that I was seeing. There's also some noise floor calibration fixes from ath9k which I need to integrate into the FreeBSD-HEAD tree.&lt;br /&gt;&lt;br /&gt;So, there's been quite a bit of progress! I've had reports from users that the AR9280, AR9285 and AR9220 support is now very stable; much more stable than before.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-946972937102143578?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/946972937102143578/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2011/01/ar9280ar9285-support-now-working.html#comment-form' title='1 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/946972937102143578'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/946972937102143578'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2011/01/ar9280ar9285-support-now-working.html' title='AR9280/AR9285 support now working'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>1</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-1332235892712014937</id><published>2011-01-21T09:17:00.000-08:00</published><updated>2011-01-21T09:26:03.510-08:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='freebsd'/><category scheme='http://www.blogger.com/atom/ns#' term='802.11n'/><category scheme='http://www.blogger.com/atom/ns#' term='atheros'/><title type='text'>AR9220 support; why AR9280 is broken</title><content type='html'>I acquired a pair of shiny AR9220 minipci NICs last week - Ubiquiti SR71-12 and SR71-15. They're supposed to be like AR9280 but on PCI, not PCIe. Unfortunately they didn't work under FreeBSD. After some sleuthing (read: guessing), I stumbled across an ath9k commit which reworked some workaround for some AR9220's, which include the Ubiquiti SR71 versions.&lt;br /&gt;&lt;br /&gt;https://patchwork.kernel.org/patch/90926/&lt;br /&gt;&lt;br /&gt;There's also a bus error when reading a random register which is only referenced in a debug statement. That's a whole other bug to try and deal with.&lt;br /&gt;&lt;br /&gt;But now I've verified that with both the AR9280 and the AR9220, the FreeBSD code is definitely tickling the radio wrong. Only one of the two radio chains is even active for the most part, and that chain seems to go deaf quite often, resulting in a baseband hang and chip reset. It also explains why I was getting absolutely shocking 11n performance out of it in my 11n branch.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-1332235892712014937?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/1332235892712014937/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2011/01/ar9220-support-why-ar9280-is-broken.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/1332235892712014937'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/1332235892712014937'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2011/01/ar9220-support-why-ar9280-is-broken.html' title='AR9220 support; why AR9280 is broken'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-4618668803095855205</id><published>2011-01-20T17:52:00.000-08:00</published><updated>2011-01-20T17:56:20.654-08:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='lusca'/><category scheme='http://www.blogger.com/atom/ns#' term='ipv6'/><title type='text'>LUSCA ipv6 - almost there (again!)</title><content type='html'>I've managed to now get the Lusca IPv6 code to perform IPv6 lookups, open a socket to the remote host and then try an IPv6 connection.&lt;br /&gt;&lt;br /&gt;But this doesn't work - the legacy Squid-2 comm code does this:&lt;br /&gt;&lt;ul&gt;&lt;li&gt;Create an AF_INET socket via comm_open();&lt;/li&gt;&lt;li&gt;Pass that socket to commConnectStart() with the relevant hostname or IPv4 address to connect to.&lt;/li&gt;&lt;/ul&gt;If the host is an IPv6 host, the IPv4 socket won't connect to it.&lt;br /&gt;&lt;br /&gt;So now I'll have to change the comm API slightly to make commConnectStart() return a FD on completion, rather than taking a socket to connect to. This will also fix a long-standing hatred of mine - where Squid/Lusca opens up a socket that just sits there until DNS lookups succeed. Ew.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-4618668803095855205?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/4618668803095855205/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2011/01/lusca-ipv6-almost-there-again.html#comment-form' title='1 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/4618668803095855205'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/4618668803095855205'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2011/01/lusca-ipv6-almost-there-again.html' title='LUSCA ipv6 - almost there (again!)'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>1</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-7555860276156636865</id><published>2011-01-16T22:34:00.000-08:00</published><updated>2011-01-16T22:40:20.082-08:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='cacheboy ipv6'/><category scheme='http://www.blogger.com/atom/ns#' term='lusca'/><category scheme='http://www.blogger.com/atom/ns#' term='ipv6'/><title type='text'>Lusca and IPv6 - almost there</title><content type='html'>Today's task - tying together the bits of Lusca's IPv6 work to be able to do an IPv6 AAAA lookup, join it with an IPv4 A lookup, then try to connect to IPv6/IPv4 hosts.&lt;br /&gt;&lt;br /&gt;The result:&lt;br /&gt;&lt;br /&gt;&lt;span style="font-size:78%;"&gt;&lt;span style="font-family: courier new;"&gt;IP Cache Contents:&lt;/span&gt;&lt;br /&gt;&lt;br /&gt;&lt;span style="font-family: courier new;"&gt; Hostname                      Flg lstref    TTL N&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family: courier new;"&gt; www.freebsd.org                          4  21595  2( 1) [2001:4f8:fff6::22]-BAD    69.147.83.34-OK &lt;/span&gt;&lt;br /&gt;&lt;/span&gt;&lt;br /&gt;So far, so good. I don't have it yet connecting to IPv6 destinations, but all the right bits are in play to be able to do so.&lt;br /&gt;&lt;br /&gt;There's plenty of bits in the Squid-3 DNS code that work around potential bugs/gotchas in DNS vs EDNS handling when handling some of the larger IPv6 record replies seen in the real world. I'm going to commit what I have thus far and then look at leveraging what they've done.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-7555860276156636865?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/7555860276156636865/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2011/01/lusca-and-ipv6-almost-there.html#comment-form' title='1 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/7555860276156636865'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/7555860276156636865'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2011/01/lusca-and-ipv6-almost-there.html' title='Lusca and IPv6 - almost there'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>1</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-2913441582503336821</id><published>2011-01-09T23:08:00.000-08:00</published><updated>2011-01-09T23:25:39.956-08:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='university'/><category scheme='http://www.blogger.com/atom/ns#' term='life'/><title type='text'>What am I going to do now?</title><content type='html'>For those who have been following my life elsewhere on the internet, I have struggled for a number of years to finish an undergraduate degree. I've been studying on and off since 2002 and I'm finally two units away from finishing this Bachelor of Arts.&lt;br /&gt;&lt;br /&gt;Which now raises a question - what am I going to do next?&lt;br /&gt;&lt;br /&gt;I'd like to spend more time studying - but after spending 9 years on and off at this undergraduate degree, I can't help but wonder if I'm doing it because I'm scared of not studying. On the flip side, I think now that I finally understand how to "be" a student and achieve excellent results, I should capitalise on this whilst I'm in that mindset.&lt;br /&gt;&lt;br /&gt;I'll see if I get into Linguistics Honours and try to get into something which I can use my psychology and computing skills with as well as straight linguistics.&lt;br /&gt;&lt;br /&gt;I'm enjoying creative writing - but I definitely need a lot more practice with that.&lt;br /&gt;&lt;br /&gt;I'm still enjoying working on CDN and HTTP/proxy stuff in general; I'm getting a lot of enjoyment hacking on the FreeBSD 802.11 stack and making Atheros devices work a little better; I really like working on embedded hardware. The question is whether I can find some more work in that area whilst I'm in Perth.&lt;br /&gt;&lt;br /&gt;Time will tell. I have 6 more months to mull over that whilst focusing primarily on finishing my degree. I'm going to try and finish it on a high note - High Distinctions if possible.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-2913441582503336821?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/2913441582503336821/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2011/01/what-am-i-going-to-do-now.html#comment-form' title='2 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/2913441582503336821'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/2913441582503336821'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2011/01/what-am-i-going-to-do-now.html' title='What am I going to do now?'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>2</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-3162158707575673790</id><published>2010-12-04T07:38:00.001-08:00</published><updated>2010-12-04T07:44:13.211-08:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='freebsd'/><category scheme='http://www.blogger.com/atom/ns#' term='802.11n'/><category scheme='http://www.blogger.com/atom/ns#' term='atheros'/><title type='text'>802.11n update - WPA works</title><content type='html'>I traced down WPA not working to broken CCMP support in FreeBSD-HEAD.&lt;br /&gt;&lt;br /&gt;This has been broken for some time. It was introduced in r204364 which enabled the multicast key search. I'm not sure why it's broken - I'll have to go through the ath9k/ath5k code; maybe even madwifi has some more up to date code in this area.&lt;br /&gt;&lt;br /&gt;In any case, that's one less thing to worry about for now. I'm pretty sure that multi-VAP mode has more issues than this so I'll put that on hold for a while.&lt;br /&gt;&lt;br /&gt;Next - making the 11n TX path a run-time check rather than a compile-time check; then test it with legacy chipsets to ensure things haven't broken.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-3162158707575673790?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/3162158707575673790/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2010/12/80211n-update-wpa-works.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/3162158707575673790'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/3162158707575673790'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2010/12/80211n-update-wpa-works.html' title='802.11n update - WPA works'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-3598799344302943832</id><published>2010-11-30T18:07:00.001-08:00</published><updated>2010-11-30T18:12:20.630-08:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='freebsd'/><category scheme='http://www.blogger.com/atom/ns#' term='802.11n'/><category scheme='http://www.blogger.com/atom/ns#' term='atheros'/><category scheme='http://www.blogger.com/atom/ns#' term='atheros 802.11n'/><title type='text'>Saturday night @ ucc: 11n hackery</title><content type='html'>I'm going to work on the FreeBSD 802.11n support a little more this upcoming Friday night and Saturday at UCC.&lt;br /&gt;&lt;br /&gt;My short-short term TODO list:&lt;br /&gt;&lt;br /&gt;&lt;ol&gt;&lt;li&gt;Figure out why crypto isn't working in 11n mode - I've likely just not fully implemented something there;&lt;/li&gt;&lt;li&gt;Hopefully I'll have the AR9220 based SR-71 cards by then - so if I do, see whether my HAL works on the AR9280 and AR9220 based NICs;&lt;/li&gt;&lt;li&gt;Test the legacy HAL (AR5212 support at least) to make sure I haven't broken that;&lt;/li&gt;&lt;li&gt;Make the TX path run-time configurable rather than a compile #ifdef - ie, if the card says it does 11n, use the 11n-aware TX routines;&lt;/li&gt;&lt;li&gt;Keep breaking apart sys/dev/ath/if_ath.c into smaller parts to make this whole thing more manageable moving forward.&lt;br /&gt;&lt;/li&gt;&lt;/ol&gt;That's all I think I need to finish up before I begin preparation for committing this work to FreeBSD-HEAD.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-3598799344302943832?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/3598799344302943832/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2010/11/saturday-night-ucc-11n-hackery.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/3598799344302943832'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/3598799344302943832'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2010/11/saturday-night-ucc-11n-hackery.html' title='Saturday night @ ucc: 11n hackery'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-2052615375767929230</id><published>2010-11-20T19:03:00.000-08:00</published><updated>2010-11-20T19:10:58.666-08:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='freebsd'/><category scheme='http://www.blogger.com/atom/ns#' term='atheros 802.11n'/><title type='text'>A productive night at the computer club ..</title><content type='html'>I spent most of Saturday at the UWA Computer Club, participating in their "Hardware Hacking Night". Whilst others were hacking on power supplies, coke machines and the like, I sat down to try and figure out how 11n TX works.&lt;br /&gt;&lt;br /&gt;Which I did, around 11:30pm last night. I still have a long way to go. The AR9160 NIC seems quite happy pumping out packets all the way up to MCS15. I couldn't figure out how to make things work correctly using the 11n RTSCTS protection - if I configure that in the TX registers, no packets are ever sent. That's the next thing on my plate to figure out.&lt;br /&gt;&lt;br /&gt;Once I figure that out, I'll worry about the 802.11n "niceities" (short-GI, HT-40 mode to name two) and then see about how to ensure that I'm interoperating (enough!) with legacy non-n devices. There's also hostap and adhoc modes to test.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-2052615375767929230?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/2052615375767929230/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2010/11/productive-night-at-computer-club.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/2052615375767929230'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/2052615375767929230'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2010/11/productive-night-at-computer-club.html' title='A productive night at the computer club ..'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-9057946459240182495</id><published>2010-11-18T17:02:00.000-08:00</published><updated>2010-11-18T17:07:10.028-08:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='freebsd'/><category scheme='http://www.blogger.com/atom/ns#' term='802.11n'/><category scheme='http://www.blogger.com/atom/ns#' term='lusca'/><category scheme='http://www.blogger.com/atom/ns#' term='atheros'/><title type='text'>Now that exams are over ..</title><content type='html'>My university exams are over for another semester. Remind me why I put myself through this? :-)&lt;br /&gt;&lt;br /&gt;So now that they're over, my open source work plate now is rapidly filling up again:&lt;br /&gt;&lt;br /&gt;FreeBSD 802.11n:&lt;br /&gt;&lt;ul&gt;&lt;li&gt;802.11n RX works!&lt;/li&gt;&lt;li&gt;But now I need to grovel through ath9k and figure out how 802.11n TX works.&lt;/li&gt;&lt;li&gt;I'm blissfully ignoring handling sending or receiving A-MPDU and A-MSDU frames for now. That can come (much) later once I have stable 802.11n TX/RX in station and hostap mode (which is enough of a challenge for now, thanks.)&lt;/li&gt;&lt;/ul&gt;Lusca:&lt;br /&gt;&lt;ul&gt;&lt;li&gt;IPv6 client support! I need to finish merging in more stuff to test.&lt;/li&gt;&lt;li&gt;Thread manager - this is an important one.&lt;/li&gt;&lt;li&gt;Find my RPM build patches and include them in the subversion tree.&lt;/li&gt;&lt;/ul&gt;Who needs sleep?&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-9057946459240182495?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/9057946459240182495/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2010/11/now-that-exams-are-over.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/9057946459240182495'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/9057946459240182495'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2010/11/now-that-exams-are-over.html' title='Now that exams are over ..'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-1254339616920265688</id><published>2010-10-12T10:10:00.000-07:00</published><updated>2010-10-12T10:20:49.570-07:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='freebsd'/><category scheme='http://www.blogger.com/atom/ns#' term='802.11n'/><category scheme='http://www.blogger.com/atom/ns#' term='atheros'/><title type='text'>FreeBSD 802.11n: aiee</title><content type='html'>I've been toying with getting 802.11n working on these atheros AR9160's.&lt;br /&gt;&lt;br /&gt;I've got somewhat working 802.11n on the receive side using my public HAL, up to MCS15. But it (currently) isn't implementing AMPDU (datagram aggregation) which is done in software, so it isn't much faster than legacy mode.&lt;br /&gt;&lt;br /&gt;But it -does- work - in 20mhz 11ng mode:&lt;br /&gt;&lt;br /&gt;&lt;span style="font-size:78%;"&gt;&lt;span style="font-family: courier new;"&gt;root@OpenWrt:~# iw  dev wlan0 station dump&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family: courier new;"&gt;Station 00:15:6d:84:05:52 (on wlan0)&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family: courier new;"&gt;        inactive time:  0 ms&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family: courier new;"&gt;        rx bytes:       41144714&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family: courier new;"&gt;        rx packets:     475307&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family: courier new;"&gt;        tx bytes:       1059470517&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family: courier new;"&gt;        tx packets:     692698&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family: courier new;"&gt;        signal:         -50 dBm&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family: courier new;"&gt;        tx bitrate:     130.0 MBit/s MCS 15&lt;/span&gt;&lt;/span&gt;&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-1254339616920265688?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/1254339616920265688/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2010/10/freebsd-80211n-aiee.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/1254339616920265688'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/1254339616920265688'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2010/10/freebsd-80211n-aiee.html' title='FreeBSD 802.11n: aiee'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-5710878339788135026</id><published>2010-09-16T10:51:00.000-07:00</published><updated>2010-09-16T11:00:18.581-07:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='mips'/><category scheme='http://www.blogger.com/atom/ns#' term='freebsd'/><title type='text'>bsdbox - take #1</title><content type='html'>One of the nice things about building embedded Linux "stuff" is busybox. The other is "uClibc", which isn't really the focus of this post.&lt;br /&gt;&lt;br /&gt;I've fleshed out a basic busybox style thing based on the crunched binary stuff which generates sysinstall and the rescue binaries. Thankfully, most of the hard work (read: the hacks needed) are done for you - both in the rescue Makefile and the general FreeBSD build framework.&lt;br /&gt;&lt;br /&gt;So, I give you "bsdbox" - a not-so-small static binary busybox style solution to help build a standalone image.&lt;br /&gt;&lt;br /&gt;&lt;span style="font-size:85%;"&gt;&lt;span style="font-family:courier new;"&gt;# ls -l /sbin/bsdbox&lt;/span&gt;&lt;span style="font-family:courier new;"&gt;&lt;br /&gt;-rwxr-xr-x  1 0  0  2958668 Sep 16 17:44 /sbin/bsdbox&lt;/span&gt;&lt;span style="font-family:courier new;"&gt;&lt;br /&gt;# /sbin/bsdbox      &lt;/span&gt;&lt;span style="font-family:courier new;"&gt;&lt;br /&gt;usage: bsdbox &lt;prog&gt; &lt;args&gt; ..., where &lt;prog&gt; is one of:&lt;/prog&gt;&lt;/args&gt;&lt;/prog&gt;&lt;/span&gt;&lt;span style="font-family:courier new;"&gt;&lt;br /&gt;ls cat dd df cp hostname kill mkdir sleep sh -sh dmesg sysctl init reboot&lt;/span&gt;&lt;span style="font-family:courier new;"&gt;&lt;br /&gt;mount mdmfs mount_mfs mdconfig newfs ifconfig route ping true false hexdump&lt;/span&gt;&lt;span style="font-family:courier new;"&gt;&lt;br /&gt;tail netstat chown chgrp arp hostapd hostapd_cli bsdbox&lt;/span&gt;&lt;/span&gt;&lt;br /&gt;&lt;br /&gt;And with that, I have this TP-Link device happily running FreeBSD as a WPA-enabled wireless access point.&lt;br /&gt;&lt;br /&gt;Combined with the GEOM uzip module, the entire filesystem (which is the above binary and a few shell scripts) - is just slightly above 1 megabyte in size. (geom_lzma will make that even smaller.)&lt;br /&gt;&lt;br /&gt;The bsdbox stuff is in my GIT repository in "bsdbox". It's built as part of the base system and installed in /bsdbox/. The binary can then be cherry picked and built into an image with whatever symlinks are needed. There's more to do - more binaries for a start; but also making it a bit easier to configure bits than a single Makefile - but it's useful right now.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-5710878339788135026?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/5710878339788135026/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2010/09/bsdbox-take-1.html#comment-form' title='3 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/5710878339788135026'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/5710878339788135026'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2010/09/bsdbox-take-1.html' title='bsdbox - take #1'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>3</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-6298291811656474814</id><published>2010-09-13T09:11:00.000-07:00</published><updated>2010-09-13T09:18:35.270-07:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='freebsd'/><category scheme='http://www.blogger.com/atom/ns#' term='atheros'/><title type='text'>Antenna Diversity: Ubiquiti SR-2</title><content type='html'>The Ubiquiti SR-2 is a high-power 11bg card based on the Atheros AR5213 MAC. It has three antenna fittings - one UFL, one MMCX, and a third set of solder pads for a non-existent connector. The first two are treated as separate antennas and are controlled by an external antenna switch. The third looks like it's an alternative to the UFL connector.&lt;br /&gt;&lt;br /&gt;I discovered that these cards occasionally calibrate their NF at some impossibly low level - under -100 dBm. The reason? Antenna diversity is occasionally selecting the MMCX connector as the RX antenna - and this never has an antenna attached. Tsk.&lt;br /&gt;&lt;br /&gt;Antenna diversity also often chooses the MMCX antenna for TX. Grr.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-6298291811656474814?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/6298291811656474814/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2010/09/antenna-diversity-ubiquiti-sr-2.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/6298291811656474814'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/6298291811656474814'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2010/09/antenna-diversity-ubiquiti-sr-2.html' title='Antenna Diversity: Ubiquiti SR-2'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-1764370938476162593</id><published>2010-09-12T09:21:00.000-07:00</published><updated>2010-09-12T09:44:56.010-07:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='freebsd'/><category scheme='http://www.blogger.com/atom/ns#' term='AR9132'/><category scheme='http://www.blogger.com/atom/ns#' term='atheros'/><category scheme='http://www.blogger.com/atom/ns#' term='AR9100'/><title type='text'>FreeBSD/MIPS: AR9132 and AR9100 wireless support</title><content type='html'>I've now made the wireless MAC/PHY work on my AR9132 wireless access point - the TP-LINK TR-WL1043nd.&lt;br /&gt;&lt;br /&gt;The wireless bits are below:&lt;br /&gt;&lt;br /&gt;&lt;span style="font-size:85%;"&gt;&lt;span style="font-family:courier new;"&gt;ath0: &lt;atheros&gt; at mem 0x180c0000-0x180effff irq 0 on nexus0&lt;/atheros&gt;&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family:courier new;"&gt;ath0: [ITHREAD]&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family:courier new;"&gt;eepromdata pointer: 0xbfff1000&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family:courier new;"&gt;eeprom: attaching&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family:courier new;"&gt;eeprom: allocating&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family:courier new;"&gt;eeprom: copying the data!&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family:courier new;"&gt;eeprom: endian check&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family:courier new;"&gt;eeprom: swapsies: 0&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family:courier new;"&gt;&lt;/span&gt;&lt;span style="font-family:courier new;"&gt;ath0: AR9100 mac 20.1 RF2133 phy 10.2&lt;/span&gt;&lt;br /&gt;&lt;/span&gt;&lt;br /&gt;It doesn't currently like client mode - it's something to do with the channel changes and aborting some pending queued stuff isn't working. I'll have to go digging at some point soon.&lt;br /&gt;&lt;br /&gt;But it works in hostap mode:&lt;br /&gt;&lt;br /&gt;&lt;span style="font-size:85%;"&gt;&lt;span style="font-family:courier new;"&gt;# ifconfig wlan0 list sta&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family:courier new;"&gt;ADDR               AID CHAN RATE RSSI IDLE  TXSEQ  RXSEQ CAPS FLAG   &lt;/span&gt;&lt;br /&gt;&lt;span style="font-family:courier new;"&gt;1c:4b:d6:97:ac:26    1    6   1M 32.5   15      2  32160 ES   AQE     WME&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family:courier new;"&gt;&lt;br /&gt;# ifconfig wlan0 list channels&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family:courier new;"&gt;Channel   1 : 2412  MHz 11g ht       Channel   7 : 2442  MHz 11g ht       &lt;/span&gt;&lt;br /&gt;&lt;span style="font-family:courier new;"&gt;Channel   2 : 2417  MHz 11g ht       Channel   8 : 2447  MHz 11g ht       &lt;/span&gt;&lt;br /&gt;&lt;span style="font-family:courier new;"&gt;Channel   3 : 2422  MHz 11g ht       Channel   9 : 2452  MHz 11g ht       &lt;/span&gt;&lt;br /&gt;&lt;span style="font-family:courier new;"&gt;Channel   4 : 2427  MHz 11g ht       Channel  10 : 2457  MHz 11g ht       &lt;/span&gt;&lt;br /&gt;&lt;span style="font-family:courier new;"&gt;Channel   5 : 2432  MHz 11g ht       Channel  11 : 2462  MHz 11g ht       &lt;/span&gt;&lt;br /&gt;&lt;span style="font-family:courier new;"&gt;Channel   6 : 2437  MHz 11g ht       &lt;/span&gt;&lt;br /&gt;&lt;span style="font-family:courier new;"&gt;&lt;br /&gt;# ifconfig wlan0 list sta     &lt;/span&gt;&lt;br /&gt;&lt;span style="font-family:courier new;"&gt;ADDR               AID CHAN RATE RSSI IDLE  TXSEQ  RXSEQ CAPS FLAG   &lt;/span&gt;&lt;br /&gt;&lt;span style="font-family:courier new;"&gt;1c:4b:d6:97:ac:26    1    6  54M 33.5    0   2682  45008 ES   AQE     WME&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family:courier new;"&gt;&lt;br /&gt;# ifconfig wlan0 scan&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family:courier new;"&gt;SSID/MESH ID    BSSID              CHAN RATE   S:N     INT CAPS&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family:courier new;"&gt;WLAN Lucy-Gaby  94:44:52:4b:f5:52    6   54M -65:-96  100 EP   RSN HTCAP WPA WME&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family:courier new;"&gt;CACHEBOY_DA...  00:25:86:d8:7c:da    6   54M -65:-96  100 EP   WPA RSN&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family:courier new;"&gt;davinayarker    00:1c:df:e4:d6:5f    6   54M -83:-96  100 EP   MESHCONF MESHCONF WPS HTCAP WPA RSN WME&lt;/span&gt;&lt;br /&gt;&lt;span style="font-family:courier new;"&gt;surbi           00:22:b0:9a:d0:ab    6   54M -88:-96  100 EPS  WPA ATH WPS&lt;/span&gt;&lt;br /&gt;&lt;/span&gt;&lt;br /&gt;I haven't yet committed the code anywhere - it's a bit of a mess. In particular:&lt;br /&gt;&lt;ul&gt;&lt;li&gt;There's some weird stuff surrounding the EEPROM logic determining when to swap what. The existing code assumes that a Big-Endian system requires swapping the byte order - but the EEPROM contents stored in flash are stored in big endian order already! Tsk.&lt;/li&gt;&lt;li&gt;There's also a bit of a mess in getting the EEPROM contents out from memory-mapped SPI flash into a private buffer. The linux AR9100 commit(s) mention that accessing it direct can be unreliable.&lt;/li&gt;&lt;li&gt;The AR9100 support is sprinkled throughout the AR5416/AR9160 support...&lt;/li&gt;&lt;li&gt;.. and I'm sure there's bits that I've missed in porting it from ath9k ..&lt;/li&gt;&lt;li&gt;.. and I know there's a few bits here and there which are also missing. I'll check that out later.&lt;/li&gt;&lt;/ul&gt;I'll look to tidy this code up and get it pushed into something public in the next week or so.&lt;br /&gt;&lt;br /&gt;&lt;span style="font-weight: bold;"&gt;Update&lt;/span&gt;&lt;br /&gt;&lt;br /&gt;The diff against my GIT repository is available at http://www.creative.net.au/diffs/git_ath_diff_1.diff . The patch includes where the GIT repository can be found.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-1764370938476162593?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/1764370938476162593/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2010/09/freebsdmips-ar9132-and-ar9100-wireless.html#comment-form' title='3 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/1764370938476162593'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/1764370938476162593'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2010/09/freebsdmips-ar9132-and-ar9100-wireless.html' title='FreeBSD/MIPS: AR9132 and AR9100 wireless support'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>3</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-4423265258971912013</id><published>2010-09-07T17:38:00.000-07:00</published><updated>2010-09-07T17:56:40.847-07:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='lusca'/><category scheme='http://www.blogger.com/atom/ns#' term='ipv6'/><title type='text'>LUSCA and IPv6</title><content type='html'>The IPv6 development branch (/playpen/LUSCA_HEAD_ipv6 in subversion) now handles IPv6 HTTP clients. It is still limited to IPv4 servers and other protocols (snmp, icmp, htcp, ftp) are all still IPv4.&lt;br /&gt;&lt;br /&gt;But it's a (long overdue) start.&lt;br /&gt;&lt;br /&gt;I'll look at next fleshing out IPv6-aware SNMP, ICP and HTCP as they're reasonably easy - I just need to change the socket handling to be IPv4/IPv6 agnostic.&lt;br /&gt;&lt;br /&gt;I'll begin merging some of the simple framework changes back to mainline Lusca once I've committed that memory leak fix that I've been discussing.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-4423265258971912013?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/4423265258971912013/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2010/09/lusca-and-ipv6.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/4423265258971912013'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/4423265258971912013'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2010/09/lusca-and-ipv6.html' title='LUSCA and IPv6'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-706470212085903429</id><published>2010-08-28T18:58:00.000-07:00</published><updated>2010-08-28T19:08:44.433-07:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='mips'/><category scheme='http://www.blogger.com/atom/ns#' term='freebsd'/><category scheme='http://www.blogger.com/atom/ns#' term='openwrt'/><category scheme='http://www.blogger.com/atom/ns#' term='atheros'/><title type='text'>FreeBSD/MIPS: AR9132 porting</title><content type='html'>I decided to avoid work a little by porting FreeBSD/MIPS to the AR9132 in the TP-LINK WN-1043nd 802.11n wireless AP I have here. There's existing OpenWRT code for it so I figured the port wouldn't be that difficult.&lt;br /&gt;&lt;br /&gt;It turns out I was right. The AR91xx support took a couple of days to massage into a form which was good enough to commit to FreeBSD-HEAD. I also included some basic AR713x SoC support - not enough to be completely useful, but enough to get a head-start on porting.&lt;br /&gt;&lt;br /&gt;I broke out various CPU specific operations into a set of function pointers which a CPU detection function sets up early during the MIPS init code. The main differences between the SoCs are:&lt;br /&gt;&lt;br /&gt;* Different base frequencies for setting up RAM, peripheral bus (eg UART), etc&lt;br /&gt;* Different register locations for a few things&lt;br /&gt;* The AR713x has a PCIe bus; the AR71xx has a PCI bus; the AR91xx doesn't have a PCI bus&lt;br /&gt;* The AR9132 has different PLL values for the gigabit ethernet MACs&lt;br /&gt;* Each SoC has a different USB peripheral setup path&lt;br /&gt;* Each SoC has different GPIO layouts&lt;br /&gt;* There's slightly different on-board peripheral reset registers&lt;br /&gt;&lt;br /&gt;I've tested the code on the AR71xx and AR9132 and things work fine. I now need to port the AR9100 wireless MAC support from Linux ath9k to complete things. The AR9100 looks a lot like the AR5416/AR9160 (802.11n, 3 radio chains) but it has a few annoying differences:&lt;br /&gt;&lt;br /&gt;* It isn't on a PCI bus, so it has to be manually attached;&lt;br /&gt;* A few registers differ in locations (one of which returns 0xdeadbeef if the AR5416 register location is read!)&lt;br /&gt;* The EEPROM is not in the same location(s) as the AR5416 - it needs to be copied and then shoe-horned into the AR5416 eeprom access code.&lt;br /&gt;&lt;br /&gt;Also, there's an RTL8133RB gigabit switch device on-board. There's also code in Linux/OpenWRT to support this. I'll look at porting this once the AR9100 support is done.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-706470212085903429?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/706470212085903429/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2010/08/freebsdmips-ar9132-porting.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/706470212085903429'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/706470212085903429'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2010/08/freebsdmips-ar9132-porting.html' title='FreeBSD/MIPS: AR9132 porting'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-5023855959350245984</id><published>2010-08-27T06:41:00.000-07:00</published><updated>2010-08-27T06:45:29.385-07:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='nbn'/><category scheme='http://www.blogger.com/atom/ns#' term='internet'/><title type='text'>Satellite instead of NBN? Hello latency..</title><content type='html'>Today's NBN politik comes thanks to the IT Wire:&lt;br /&gt;&lt;br /&gt;http://www.itwire.com/it-policy-news/government-tech-policy/41422-newsat-woos-independents-attacks-nbn&lt;br /&gt;&lt;br /&gt;In particular:&lt;br /&gt;&lt;br /&gt;"Mr Ballantine today argued that for most of Australia’s geography  broadband delivered over satellite would be faster and cheaper. What was  right for the city he argued was not necessarily good for the outback."&lt;br /&gt;&lt;br /&gt;Maybe for download throughput, sure. But latency plays a huge part in satellite (and 3G style solutions) and it will be noticable.&lt;br /&gt;&lt;br /&gt;Satellite IP doesn't scale. Sorry guys.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-5023855959350245984?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/5023855959350245984/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2010/08/satellite-instead-of-nbn-hello-latency.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/5023855959350245984'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/5023855959350245984'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2010/08/satellite-instead-of-nbn-hello-latency.html' title='Satellite instead of NBN? Hello latency..'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-4420579678437071999</id><published>2010-07-28T21:46:00.000-07:00</published><updated>2010-07-28T21:50:25.263-07:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='google'/><title type='text'>Authority-by-being-employed-by-Google, rather than by-reproducable-data</title><content type='html'>Many things rub me up the wrong way. This is one of them.&lt;br /&gt;&lt;br /&gt;http://developers.slashdot.org/comments.pl?sid=1734346&amp;amp;cid=33051506&lt;br /&gt;&lt;br /&gt;In summary - the post asserts some interesting facts (which I believe, having done high volume HTTP stuff myself) but when asked about benchmarks to back up his assertions, he replies:&lt;br /&gt;&lt;br /&gt;"Unfortunately, nothing I can publish without permission. I can say that I'm in charge of maintaining the software that terminates all HTTP traffic for Google. Draw your own conclusions."&lt;br /&gt;&lt;br /&gt;I really do dislike "I work in this area in Google" as a measure of authority. My conclusion is that the developer in question, as clever as he should be, should likely read some history books on "authority by being high up in the priesthood" and where that takes people.&lt;br /&gt;&lt;br /&gt;Grr.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-4420579678437071999?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/4420579678437071999/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2010/07/authority-by-being-employed-by-google.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/4420579678437071999'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/4420579678437071999'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2010/07/authority-by-being-employed-by-google.html' title='Authority-by-being-employed-by-Google, rather than by-reproducable-data'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-8905552602973004053</id><published>2010-07-12T09:12:00.001-07:00</published><updated>2010-07-12T09:16:57.840-07:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='lusca'/><category scheme='http://www.blogger.com/atom/ns#' term='ipv6'/><category scheme='http://www.blogger.com/atom/ns#' term='cacheboy'/><title type='text'>More IPv6 hackery..</title><content type='html'>I've been spending a little time fleshing out some more of the IPv6 preparation work in Lusca. Since they're rather intrusive patches, I'm doing the work in a separate branch (/playpen/LUSCA_HEAD_ipv6) and will merge back bits and pieces as needed.&lt;br /&gt;&lt;br /&gt;I've migrated the client db, external URL rewriters, access logging and the client-facing connection management code over to be IPv6 aware. I still have the request_t state, ACL lookups (which is luckily done - but sitting in a branch!), further DNS work and the protocol-facing stuff (HTTP, FTP.)&lt;br /&gt;&lt;br /&gt;There isn't much more work involved in getting LUSCA_HEAD ready enough to serve IPv6-facing clients. That'll let me push out Cacheboy content over IPv6.&lt;br /&gt;&lt;br /&gt;But for now, it's back to hacking on commercial, customer code.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-8905552602973004053?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/8905552602973004053/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2010/07/more-ipv6-hackery.html#comment-form' title='1 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/8905552602973004053'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/8905552602973004053'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2010/07/more-ipv6-hackery.html' title='More IPv6 hackery..'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>1</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-4508737508170785117</id><published>2010-07-11T00:42:00.000-07:00</published><updated>2010-07-11T00:45:19.147-07:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='mips'/><category scheme='http://www.blogger.com/atom/ns#' term='freebsd'/><category scheme='http://www.blogger.com/atom/ns#' term='ubiqiti'/><title type='text'>FreeBSD/MIPS on AR71xx / Routerstation Pro: NIC alignment/size bug fixed!</title><content type='html'>I found and squished a small bug in the gigabit NIC driver in FreeBSD/MIPS for the AR71xx chipset. It wasn't all that complicated - TX buffers weren't being thoroughly enough checked for alignment/size constraints before being handed to the DMA engine.&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;It did however fix a few niggling issues. My tunneling stuff was fixed. IPv6 frames are now correctly handled. And ng_nat doesn't cause a panic in the NIC driver. So there's at least three people who are happy. :)&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-4508737508170785117?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/4508737508170785117/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2010/07/freebsdmips-on-ar71xx-routerstation-pro.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/4508737508170785117'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/4508737508170785117'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2010/07/freebsdmips-on-ar71xx-routerstation-pro.html' title='FreeBSD/MIPS on AR71xx / Routerstation Pro: NIC alignment/size bug fixed!'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-8148295580951025103</id><published>2010-06-22T21:58:00.000-07:00</published><updated>2010-06-22T22:02:18.786-07:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='lusca'/><category scheme='http://www.blogger.com/atom/ns#' term='cacheboy'/><title type='text'>Cacheboy: Firefox Release 3.6.4</title><content type='html'>Cacheboy is currently pushing a good 800-1200mbit of firefox 3.6.4 updates. It has about 6% of the total mozilla mirror weight so I predict that there's currently around 16 gigabits of total mozilla updates going out.&lt;br /&gt;&lt;br /&gt;Lusca is holding up fine - a single process happily pushed ~ 400mbit on some hosts during the initial peak. I'd obviously like to be able to push a lot more than that but I'm still doing baby steps in the Lusca performance department.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-8148295580951025103?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/8148295580951025103/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2010/06/cacheboy-firefox-release-364.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/8148295580951025103'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/8148295580951025103'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2010/06/cacheboy-firefox-release-364.html' title='Cacheboy: Firefox Release 3.6.4'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-3842593686460795285</id><published>2010-05-19T18:32:00.000-07:00</published><updated>2010-05-19T18:38:55.424-07:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='lusca'/><title type='text'>HTTP parser/management changes</title><content type='html'>I've finally committed changes to Lusca which migrate the HTTP Header management routines away from individual malloc/free calls per HTTP Header entry to (hopefully!) one malloc/free for every N. (N being tunable, of course.)&lt;br /&gt;&lt;br /&gt;A lot of incremental preparation work was required in the HTTP parser to tidy things up enough to (hopefully!) minimise any impact on stability and functionality.&lt;br /&gt;&lt;br /&gt;There really isn't all that much noticable improvement unless you're working on small, embedded platforms with slow CPUs. It's more of a precursor to further optimisation and reorganisation work. The cumulative effect will be worthwhile.&lt;br /&gt;&lt;br /&gt;I've released a tarball with the work - r14674 - which also includes a handful of ATF unit tests in the test-suite/atf/ subdirectory.&lt;br /&gt;&lt;br /&gt;I'll likely next work on reorganising some more of the FTP and HTTP protocol specific code which can be migrated out of src/ . I'd like to then spend some more time writing unit tests before (hopefully!) finishing off the IPv6 related DNS changes in preparation for more IPv6 related tomfoolery.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-3842593686460795285?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/3842593686460795285/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2010/05/http-parsermanagement-changes.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/3842593686460795285'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/3842593686460795285'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2010/05/http-parsermanagement-changes.html' title='HTTP parser/management changes'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-9131225350884717025</id><published>2010-05-13T02:42:00.000-07:00</published><updated>2010-05-13T02:45:56.254-07:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='ATF'/><category scheme='http://www.blogger.com/atom/ns#' term='lusca'/><category scheme='http://www.blogger.com/atom/ns#' term='unit-testing'/><title type='text'>Unit testing using ATF</title><content type='html'>I've been toying around with the NetBSD unit testing framework called "ATF". Unlike other well-known FOSS unit testing software, ATF seems to have relatively comprehensive support for writing unit tests in C - which is quite important for a piece of C software.&lt;br /&gt;&lt;br /&gt;In fact, it seems quite a few popular bits of FOSS software handling unit tests, embedded/API documentation, etc all support C rather poorly. I wonder why.&lt;br /&gt;&lt;br /&gt;I've not linked the ATF stubs into the main build tree just yet. The ATF code can be found in the source tree under /test-suite/atf/ . It works enough for me to use "make check" on my iBook (yes, that's a pre-intel, PPC G4 iBook) to check that the basic changes I'm making to the HTTP parser code don't introduce new bugs.&lt;br /&gt;&lt;br /&gt;I would love some help writing more unit tests though!&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-9131225350884717025?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/9131225350884717025/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2010/05/unit-testing-using-atf.html#comment-form' title='2 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/9131225350884717025'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/9131225350884717025'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2010/05/unit-testing-using-atf.html' title='Unit testing using ATF'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>2</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-8728981983879049789</id><published>2010-05-04T20:38:00.000-07:00</published><updated>2010-05-04T20:40:02.838-07:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='lusca'/><title type='text'>The best news is sometimes no news..</title><content type='html'>Things have been quiet on the Lusca front lately. I poked some of the users who were reporting earlier issues and asked whether said issues were fixed. All of them responded "yes".&lt;br /&gt;&lt;br /&gt;So sometimes no news is good news. But you have to sometimes make sure.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-8728981983879049789?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/8728981983879049789/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2010/05/best-news-is-sometimes-no-news.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/8728981983879049789'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/8728981983879049789'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2010/05/best-news-is-sometimes-no-news.html' title='The best news is sometimes no news..'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-6704956886296526003</id><published>2010-04-21T18:02:00.000-07:00</published><updated>2010-04-21T18:16:49.424-07:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='lusca'/><title type='text'>Modifying the HTTP header parser in Lusca</title><content type='html'>I've been slowly working towards a variety of medium-term goals in Lusca. I've resisted committing various bits of partially finished work partly because they're works in progress but partially because I'm not happy with how the code fits together.&lt;br /&gt;&lt;br /&gt;One of these areas is the HTTP header parser and management routines. Among other things, the main issues I have with the parser and management code is listed below.&lt;br /&gt;&lt;ul&gt;&lt;li&gt;Each header entry is represented by separate strings in memory;&lt;/li&gt;&lt;li&gt;Each header entry has a small, separately allocated object (HttpHeaderEntry), one per header&lt;/li&gt;&lt;li&gt;Parsing the header entries uses various stdio routines to iterate over characters, and these may be implemented slower (to handle unicode/wide/UTF/locale support) than what's needed here (7-bit ASCII);&lt;/li&gt;&lt;li&gt;There's some sanity checks in the header parser - specifically, duplicate content-length - which is likely better once the headers have been parsed.&lt;/li&gt;&lt;/ul&gt;I've been working on the first two items in separate branches. One converts the HttpHeaderEntry items into a single allocated array, which is grown if needed. Another takes the current String API and turns it into fully reference-counted strings. Both of these work fine for me. But shoe-horning it into the current HTTP parser code - which expects individually allocated/created HttpHeaderEntry items which it can destroy on a whim before they're considered a part of the Http Header set - is overly hackish and prone to introduce bugs.&lt;br /&gt;&lt;br /&gt;It's taking me quite a bit of time to slowly change the HTTP parser code to be ready for the new management code. Well, it's taken me about 6 months to slowly modify it in a way that doesn't require rewriting everything and potentially changing expected behaviour and/or introduce subtle bugs.&lt;br /&gt;&lt;br /&gt;The upshoot? Things take time, but the code hopefully will be tidier, cleaner and easier to understand. Oh, and won't include bugs.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-6704956886296526003?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/6704956886296526003/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2010/04/modifying-http-header-parser-in-lusca.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/6704956886296526003'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/6704956886296526003'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2010/04/modifying-http-header-parser-in-lusca.html' title='Modifying the HTTP header parser in Lusca'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-1337204285598849564</id><published>2010-04-03T00:10:00.001-07:00</published><updated>2010-04-03T00:14:22.285-07:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='lusca'/><category scheme='http://www.blogger.com/atom/ns#' term='windows'/><category scheme='http://www.blogger.com/atom/ns#' term='cygwin'/><title type='text'>State of the Cygwin/Windows port!</title><content type='html'>A rather nice chap has been ploughing through the source and making it work under Windows/Cygwin. I've been committing bits and pieces of his work into LUSCA_HEAD as time permits.&lt;br /&gt;&lt;br /&gt;You can find the main port details in &lt;a href="http://code.google.com/p/lusca-cache/issues/detail?id=94"&gt;Issue 94&lt;/a&gt;.&lt;br /&gt;&lt;br /&gt;Thanks!&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-1337204285598849564?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/1337204285598849564/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2010/04/state-of-cygwinwindows-port.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/1337204285598849564'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/1337204285598849564'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2010/04/state-of-cygwinwindows-port.html' title='State of the Cygwin/Windows port!'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-1795824104733809949</id><published>2010-04-02T22:40:00.000-07:00</published><updated>2010-04-02T22:53:08.341-07:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='lusca'/><title type='text'>Hunting down method_t bugs..</title><content type='html'>It all started with &lt;a href="http://code.google.com/p/lusca-cache/issues/detail?id=99"&gt;Issue 99&lt;/a&gt;. There was a random crash in the logging code. It looked like bug in the method handling changes which made it into Squid-2.HEAD a year or two ago. I've been patching issues in the method handling - specifically with NULL and uninitialised method pointers appearing in places - but this time the method_t pointed to junk data.&lt;br /&gt;&lt;br /&gt;A bit of digging found that the pointer value did point to a valid method_t structure instance - but something free'd it. Hm. A little further digging found what was going on:&lt;br /&gt;&lt;ol&gt;&lt;li&gt;A METHOD_OTHER appeared (an RTSP method) which resulted in a new method_t being malloc'ed;&lt;/li&gt;&lt;li&gt;The pointer was copied to the request_t structure;&lt;/li&gt;&lt;li&gt;The request was processed;&lt;/li&gt;&lt;li&gt;The initial method_t pointer was freed, but the request_t method pointer still pointed to it;&lt;/li&gt;&lt;li&gt;The logging code then logged the stuff said request_t method pointer pointed to - but it was already free'd. Sometimes it'd be junk, sometimes it'd be the original contents.&lt;/li&gt;&lt;/ol&gt;The original method code (and the "known" methods) all throw around pointers - and copies of pointers - to statically allocated structures which never go away. Unfortunately this logic wasn't changed when the dynamic "other" methods appeared.&lt;br /&gt;&lt;br /&gt;So I've been quite busy tidying up the method handling code in preparation for the change in how they're handled. LUSCA_HEAD now has some code which logs potential memory leaks when handling the dynamic methods. I'm going to see if I can come up with a way (or two) to log potential risky situations when items are dereferenced after being free'd. But hopefully I can fix the issue without introducing any further bugs.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-1795824104733809949?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/1795824104733809949/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2010/04/hunting-down-methodt-bugs.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/1795824104733809949'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/1795824104733809949'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2010/04/hunting-down-methodt-bugs.html' title='Hunting down method_t bugs..'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-3518100642252461047</id><published>2010-03-28T22:18:00.000-07:00</published><updated>2010-03-28T22:27:36.799-07:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='lusca'/><title type='text'>Today's fun bug: invalid swap metadata</title><content type='html'>One of the Lusca users has issues with swap file contents being invalid. It's linked to the length of the object URL - and fixing this is unfortunately more difficult than first thought.&lt;br /&gt;&lt;br /&gt;In essence - the size of the URL, the size of the metadata and the size of the buffers being used for reading data don't match "right". The TLV encoding code doesn't put an upper limit on the size of the metadata. The TLV decoding code doesn't enforce a maximum buffer size - it tries reading the metadata until it finds the end of said metadata. All of this unfortunately results in stupid behaviour when overly long URLs are stored.&lt;br /&gt;&lt;br /&gt;The current maximum URL is MAX_URL - 4096 bytes. Squid-3 may have redefined this to be longer. The reason I haven't done this is because the URL is included in the swap metadata - and this is only read in in SM_PAGE_SIZE chunks - again, 4096 bytes. So if the URL is say, 4000ish or so bytes long, the total length of the encoded metadata is &gt; 4096 bytes. This is happily written out to the swapfile.&lt;br /&gt;&lt;br /&gt;Reading the data in however is another story. An SM_PAGE_SIZE sized buffer is created and a read is issued. The incomplete metadata is read in. There's unfortunately no check to ensure the passed in buffer actually fully contains all of the metadata - so the code happily trumps in potentially uninitialised memory. The end result is at the very least an invalid object which is eventually deleted; it could be worse. I haven't yet investigated.&lt;br /&gt;&lt;br /&gt;In any case - I'm now going to have to somehow enforce some sensible behaviour. I'd much prefer to make the code handle arbitrary (ie, configurable arbitrary) long URLs and read in the metadata as needed - but that's a bigger issue which will take some further refactoring and redesigning to solve.&lt;br /&gt;&lt;br /&gt;This is all another example of how code works "by magic", rather than "by intent". :)&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-3518100642252461047?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/3518100642252461047/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2010/03/todays-fun-bug-invalid-swap-metadata.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/3518100642252461047'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/3518100642252461047'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2010/03/todays-fun-bug-invalid-swap-metadata.html' title='Today&apos;s fun bug: invalid swap metadata'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-6297520863082810807</id><published>2010-03-25T18:43:00.000-07:00</published><updated>2010-03-25T18:53:51.972-07:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='lusca'/><category scheme='http://www.blogger.com/atom/ns#' term='cygwin'/><title type='text'>Lusca update: important bug fixes, portability work</title><content type='html'>I've finally found and fixed two annoying bugs in Lusca.&lt;br /&gt;&lt;br /&gt;The first is that occasionally the rebuild processed failed to properly rebuild the cache contents and the proxy would start with an empty cache. This ended up being due to the undocumented IPC code in Lusca, inherited from Squid, which would "eat" a few bytes at the beginning of the child process lifetime to establish whether the child was active and ready. It would try to read a hello string ("hi there!\n\0") but it would read 32 bytes instead of the exact string length. If the child process started sending data straight away (as my store rebuild helpers do!) the initial part of their conversation could be eaten.&lt;br /&gt;&lt;br /&gt;The real fix is to modify the helper IPC handshake so it's two way and fixed length rather than the hacky way it's done now. But the temporary workaround seems to work fine - just read the 11 bytes for the hello string, rather than up to 32 bytes.&lt;br /&gt;&lt;br /&gt;The second was a weird situation involving the swap.state files in the UFS store dirs growing to enormous sizes, filling the cache_dir up. This was eventually traced back to the proxy being reconfigured once a minute on some deployments (in this case - pfsense!). The problem was quite simple in the end - a reconfigure would force the swap state logs to be closed and re-opened; but this didn't know whether the swap state logs were pointing at the live ones or the rebuilding ones. In the latter, the rebuild process was reading from swap.state and appending to swap.state.new; a reconfigure would close the swap.state.new file and append to swap.state. This meant that the rebuild process was reading from swap.state and appending to swap.state.new - thus never quite finishing the rebuild process.&lt;br /&gt;&lt;br /&gt;Those have been fixed for *NIX ports and now the rebuild processe seems to be moving forward quite swimmingly!&lt;br /&gt;&lt;br /&gt;I've also had a user pop up recently who is submitting portability fixes for cygwin (and under Windows 7, no less!) I've been committing fixes to the way header files are included so the code mostly compiles under both *NIX and Cygwin. Admittedly most of those portability fixes were my fault - I didn't really bother making my new code "autoconf/automake safe" but, thankfully, revisiting this choice isn't difficult. Thankyou for the continuous assistance with the Cygwin changes!&lt;br /&gt;&lt;br /&gt;Finally, there's still 46 currently active issues in the issue tracker. Some are just placeholders for me to revisit certain bits of code (eg Issue #85 - figuring out how PURGE works!) but I'm still aiming to address, repair and close the majority of those issues - and release a few more stable snapshots! - before I begin including more development stuff into the main tree.&lt;br /&gt;&lt;br /&gt;Not that the snapshots at the moment are unstable - far from it!&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-6297520863082810807?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/6297520863082810807/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2010/03/lusca-update-important-bug-fixes.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/6297520863082810807'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/6297520863082810807'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2010/03/lusca-update-important-bug-fixes.html' title='Lusca update: important bug fixes, portability work'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-1117328368413527653</id><published>2010-03-20T23:20:00.000-07:00</published><updated>2010-03-20T23:27:38.139-07:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='lusca'/><title type='text'>Lusca and HTTP parsing..</title><content type='html'>I've broken out most of the client-side request parsing code into a separate source file. There's now only a single entry and exit point (and one sideways jump for a comm timeout handler for now) from the client-side main code into the request handling code.&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;The main aim here is to eventually isolate and rework the whole process with which a HTTP request is parsed and constructed in memory. The process itself isn't all that CPU intensive these days compared to Squid-2.x and Squid-3.x but it is quite ugly. I won't go into the gory details - you can check it out for yourself if you like. Just cast your eyes over src/client_side_request_parser.c in LUSCA_HEAD.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;I'm going to leave the request parsing code as it is for now. It's ugly but it works and its inefficiencies are dwarfed by the misuses of memory bandwidth/CPU cycles elsewhere.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;I think I've left the codebase slightly easier to understand than before. I think I'm now at the point where I can document a large part of the client-side request and reply handling pipeline. The caching, vary and ETag processing is still very messy and too tightly integrated into the client-side code for my liking but as it also works fine for now I'll be leaving it well alone. There be dragons and all of that.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-1117328368413527653?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/1117328368413527653/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2010/03/lusca-and-http-parsing.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/1117328368413527653'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/1117328368413527653'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2010/03/lusca-and-http-parsing.html' title='Lusca and HTTP parsing..'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-5421613157146664540</id><published>2010-03-15T01:51:00.000-07:00</published><updated>2010-03-15T02:06:15.060-07:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='lusca'/><title type='text'>Lusca Logging, revisited</title><content type='html'>So the obvious thing to do is to not run the logging path at all, and to avoid evaluating the access control list(s) if there's no access control set defined for logging.&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;There are a couple of problems with this!&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;Firstly, the current behaviour is to not pass request/reply information through some of the statistics counters and the clientdb modules if the request doesn't match the logging ACL. If there's no logging ACL then all requests are kicked to the statistics/clientdb counters.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;Secondly, there's also the ACLs used in the access_log directives which allow the administrator to filter which requests/replies are logged to which access log file(s). The current implementation will pass all requests which aren't denied by the top-level logging ACL through to each of the access_log entries, checking those for suitability.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;The question is - can I mostly maintain the behaviour for the use cases. The main two are:&lt;/div&gt;&lt;div&gt;&lt;ol&gt;&lt;li&gt;where logging is completely disabled (via "access_log none") but maintain client counters and the clientdb;&lt;/li&gt;&lt;li&gt;where logging is enabled to one access_log entry, maintaining client counters and the clientdb.&lt;/li&gt;&lt;/ol&gt;&lt;div&gt;Hm, stuff to do, stuff to do..&lt;/div&gt;&lt;/div&gt;&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-5421613157146664540?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/5421613157146664540/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2010/03/lusca-logging-revisited.html#comment-form' title='2 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/5421613157146664540'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/5421613157146664540'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2010/03/lusca-logging-revisited.html' title='Lusca Logging, revisited'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>2</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-4398196450517904380</id><published>2010-03-14T20:46:00.000-07:00</published><updated>2010-03-14T20:55:48.530-07:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='lusca'/><title type='text'>Improving the access logging code</title><content type='html'>The Squid/Lusca access logging logic is .. well, to be honest, it's quite dirty and inefficient. Yes, like most of Squid. But this is a somewhat bite-sized piece of the puzzle which I can tidy up in one corner without too many effects elsewhere.. or is it?&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;For high request rate, small object loads on slower FSB hardware, one of the largest CPU users is actually the client-side access log path. There's two culprits here:&lt;/div&gt;&lt;div&gt;&lt;ol&gt;&lt;li&gt;memcpy() when copying the "hier" struct from the request struct to the access log entry struct (client_side.c:297; revision 14457); and&lt;/li&gt;&lt;li&gt;The ACL list setup in preparation for writing filtered access log entries to multiple files (client_side.c:314; revision 14457).&lt;/li&gt;&lt;/ol&gt;&lt;div&gt;The memcpy() is of a 588 byte HierarchyLogEntry struct - it's this large because of two "SQUIDHOSTNAMELEN" (256 byte) long strings embedded in the struct itself. Annoying, but somewhat fixable with a little code refactoring and use of reference counted strings.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;The ACL list setup is a bit more problematic. It sets up an ACL checklist using the http_access checklist before then checking log_access / acl_access. It then further may use this ACL checklist when evaluating various access_log lines to allow filtering certain access log entries to certain files.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;Now, the latter bites because the really slow paths (setting up and destroying the ACL access stuff) is done even if the default logging configuration is used - one log file - and there's currently no easy way around that. Furthermore, if you disable logging entirely (access_log none) then the initial setup of the access log entry information is done, the ACL checklist is created, and then it's all tossed away without logging. A bit stupid, eh?&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;I'll cover what I'm doing in the codebase in a subsequent post (read; when I'm not running off to class.)&lt;/div&gt;&lt;/div&gt;&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-4398196450517904380?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/4398196450517904380/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2010/03/improving-access-logging-code.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/4398196450517904380'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/4398196450517904380'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2010/03/improving-access-logging-code.html' title='Improving the access logging code'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-8405561081049090960</id><published>2010-03-13T22:58:00.000-08:00</published><updated>2010-03-13T23:05:23.057-08:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='books'/><category scheme='http://www.blogger.com/atom/ns#' term='programming'/><category scheme='http://www.blogger.com/atom/ns#' term='logic'/><title type='text'>Currently reading: "The Science of Programming" by David Gries</title><content type='html'>I've been acquiring older computer science textbooks at various second hand book sale type events over the last few years and occasionally I'll pick up what seems to be like a gem which shouldn't have fallen by the wayside in the name of progress.&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;Today's gem is "The Science of Programming" by David Gries. It's a book first published in the late 1970's which I'm guessing is one of the earlier attempts at a not-quite-academic publication trying to formalise some concepts of program design, provability and deductive reasoning. It does this for a variety of algorithms in a framework which - and this is a useful point - does not involve in any way an object oriented language, functional language, or anything which hides what's mostly going on under the hood from the programmer. Not that I think those are bad things, but I do think that being taught about how things are done closest to how things are run is a good idea. Starting at OO seems to produce programmers who, well, don't seem to have much of a clue about reality and rely on their tools a lot more than they should.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;I'm currently re-reading it with a notebook at hand. Luckily for me, the first few chapters involve propositional/predicate logic and deduction stuff which overlaps nicely with my Semantics course in Linguistics. So it's "almost" related to my university degree. Sort of.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-8405561081049090960?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/8405561081049090960/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2010/03/currently-reading-science-of.html#comment-form' title='1 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/8405561081049090960'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/8405561081049090960'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2010/03/currently-reading-science-of.html' title='Currently reading: &quot;The Science of Programming&quot; by David Gries'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>1</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-8077058012111053182</id><published>2010-03-09T05:39:00.001-08:00</published><updated>2010-03-09T05:50:49.736-08:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='lusca'/><category scheme='http://www.blogger.com/atom/ns#' term='profiling'/><title type='text'>Some more Lusca Profiling..</title><content type='html'>More Lusca profiling of a simple test workload:&lt;div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;&lt;span class="Apple-style-span" style="font-size: small;"&gt;&lt;span class="Apple-style-span"  style="font-family:'courier new';"&gt;samples  %        image name               symbol name&lt;/span&gt;&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span class="Apple-style-span" style="font-size: small;"&gt;&lt;span class="Apple-style-span"  style="font-family:'courier new';"&gt;234266    6.7052  libc-2.3.6.so            _int_malloc&lt;/span&gt;&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span class="Apple-style-span" style="font-size: small;"&gt;&lt;span class="Apple-style-span"  style="font-family:'courier new';"&gt;138031    3.9507  libc-2.3.6.so            vfprintf&lt;/span&gt;&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span class="Apple-style-span" style="font-size: small;"&gt;&lt;span class="Apple-style-span"  style="font-family:'courier new';"&gt;111831    3.2008  libc-2.3.6.so            calloc&lt;/span&gt;&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span class="Apple-style-span" style="font-size: small;"&gt;&lt;span class="Apple-style-span"  style="font-family:'courier new';"&gt;104393    2.9879  libc-2.3.6.so            malloc_consolidate&lt;/span&gt;&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span class="Apple-style-span" style="font-size: small;"&gt;&lt;span class="Apple-style-span"  style="font-family:'courier new';"&gt;98984     2.8331  libc-2.3.6.so            memcpy&lt;/span&gt;&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span class="Apple-style-span" style="font-size: small;"&gt;&lt;span class="Apple-style-span"  style="font-family:'courier new';"&gt;91783     2.6270  libc-2.3.6.so            _int_free&lt;/span&gt;&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span class="Apple-style-span" style="font-size: small;"&gt;&lt;span class="Apple-style-span"  style="font-family:'courier new';"&gt;72578     2.0773  libc-2.3.6.so            memset&lt;/span&gt;&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span class="Apple-style-span" style="font-size: small;"&gt;&lt;span class="Apple-style-span"  style="font-family:'courier new';"&gt;69068     1.9769  libc-2.3.6.so            free&lt;/span&gt;&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span class="Apple-style-span" style="font-size: small;"&gt;&lt;span class="Apple-style-span"  style="font-family:'courier new';"&gt;50473     1.4446  squid                    clientTryParseRequest&lt;/span&gt;&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span class="Apple-style-span" style="font-size: small;"&gt;&lt;span class="Apple-style-span"  style="font-family:'courier new';"&gt;50064     1.4329  squid                    memPoolAlloc&lt;/span&gt;&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span class="Apple-style-span" style="font-size: small;"&gt;&lt;span class="Apple-style-span"  style="font-family:'courier new';"&gt;45211     1.2940  libc-2.3.6.so            re_search_internal&lt;/span&gt;&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span class="Apple-style-span" style="font-size: small;"&gt;&lt;span class="Apple-style-span"  style="font-family:'courier new';"&gt;40469     1.1583  squid                    httpRequestFree&lt;/span&gt;&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span class="Apple-style-span" style="font-size: small;"&gt;&lt;span class="Apple-style-span"  style="font-family:'courier new';"&gt;39227     1.1228  squid                    statHistBin&lt;/span&gt;&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span class="Apple-style-span" style="font-size: small;"&gt;&lt;span class="Apple-style-span"  style="font-family:'courier new';"&gt;38916     1.1139  squid                    comm_select&lt;/span&gt;&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span class="Apple-style-span" style="font-size: small;"&gt;&lt;span class="Apple-style-span"  style="font-family:'courier new';"&gt;36974     1.0583  libc-2.3.6.so            _IO_default_xsputn&lt;/span&gt;&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span class="Apple-style-span" style="font-size: small;"&gt;&lt;span class="Apple-style-span"  style="font-family:'courier new';"&gt;36260     1.0378  squid                    memPoolFree&lt;/span&gt;&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;.. CPU is still taken up in the standard areas: memory allocation, stdio, and memcpy.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;memcpy() I can't tackle right now. malloc() and friends I can tackle, but it may cause strange bugs and I'm not willing to commit anything which upsets stability too much right now. But vfprintf() is fun.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;&lt;div&gt;&lt;span class="Apple-style-span"  style="font-family:'courier new';"&gt;&lt;span class="Apple-style-span" style="font-size: small;"&gt;  51285    22.7881  libc-2.3.6.so            vsprintf&lt;/span&gt;&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span class="Apple-style-span"  style="font-family:'courier new';"&gt;&lt;span class="Apple-style-span" style="font-size: small;"&gt;  168714   74.9667  libc-2.3.6.so            vsnprintf&lt;/span&gt;&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span class="Apple-style-span"  style="font-family:'courier new';"&gt;&lt;span class="Apple-style-span" style="font-size: small;"&gt;156434   100.000  libc-2.3.6.so            vfprintf&lt;/span&gt;&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;.. so most of the CPU is vsnprintf(). &lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;&lt;div&gt;&lt;span class="Apple-style-span"  style="font-family:'courier new';"&gt;&lt;span class="Apple-style-span" style="font-size: small;"&gt;  2549      1.0775  squid                    httpHeaderPutStrf&lt;/span&gt;&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span class="Apple-style-span"  style="font-family:'courier new';"&gt;&lt;span class="Apple-style-span" style="font-size: small;"&gt;  78253    33.0790  squid                    memBufVPrintf&lt;/span&gt;&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span class="Apple-style-span"  style="font-family:'courier new';"&gt;&lt;span class="Apple-style-span" style="font-size: small;"&gt;  153322   64.8121  libc-2.3.6.so            snprintf&lt;/span&gt;&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span class="Apple-style-span"  style="font-family:'courier new';"&gt;&lt;span class="Apple-style-span" style="font-size: small;"&gt;13601    100.000  libc-2.3.6.so            vsnprintf&lt;/span&gt;&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;.. memBufVPrintf():&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;&lt;div&gt;&lt;span class="Apple-style-span" style="font-size: small;"&gt;&lt;span class="Apple-style-span"  style="font-family:'courier new';"&gt;  33231    36.8652  httpHeaderPutStrf&lt;/span&gt;&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span class="Apple-style-span" style="font-size: small;"&gt;&lt;span class="Apple-style-span"  style="font-family:'courier new';"&gt;  55577    61.6549  packerPrintf&lt;/span&gt;&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span class="Apple-style-span" style="font-size: small;"&gt;&lt;span class="Apple-style-span"  style="font-family:'courier new';"&gt;7225     100.000  memBufVPrintf&lt;/span&gt;&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;.. snprintf():&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;&lt;div&gt;&lt;span class="Apple-style-span" style="font-size: small;"&gt;&lt;span class="Apple-style-span"  style="font-family:'courier new';"&gt;  5021      2.8271  squid                    clientAccessCheckDone&lt;/span&gt;&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span class="Apple-style-span" style="font-size: small;"&gt;&lt;span class="Apple-style-span"  style="font-family:'courier new';"&gt;  31897    17.9597  squid                    clientSendHeaders&lt;/span&gt;&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span class="Apple-style-span" style="font-size: small;"&gt;&lt;span class="Apple-style-span"  style="font-family:'courier new';"&gt;  53339    30.0327  squid                    xitoa&lt;/span&gt;&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span class="Apple-style-span" style="font-size: small;"&gt;&lt;span class="Apple-style-span"  style="font-family:'courier new';"&gt;  85548    48.1681  squid                    urlCanonical&lt;/span&gt;&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span class="Apple-style-span" style="font-size: small;"&gt;&lt;span class="Apple-style-span"  style="font-family:'courier new';"&gt;7870     100.000  libc-2.3.6.so            snprintf&lt;/span&gt;&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;.. so it's likely that eliminating the random calls through the printf() code to do "stuff" like assembling the URL and request/reply strings will shave off about 4% of this workload. But the biggest issue right now is the stupidly large amount of CPU being used in the memory allocation routines.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;But the annoying one is memcpy():&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;&lt;div&gt;&lt;span class="Apple-style-span" style="font-size: small;"&gt;&lt;span class="Apple-style-span"  style="font-family:'courier new';"&gt;  10557     7.3665  squid                    httpAccept&lt;/span&gt;&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span class="Apple-style-span" style="font-size: small;"&gt;&lt;span class="Apple-style-span"  style="font-family:'courier new';"&gt;  20779    14.4992  squid                    stringDup&lt;/span&gt;&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span class="Apple-style-span" style="font-size: small;"&gt;&lt;span class="Apple-style-span"  style="font-family:'courier new';"&gt;  37010    25.8250  squid                    httpHeaderEntryPackInto&lt;/span&gt;&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span class="Apple-style-span" style="font-size: small;"&gt;&lt;span class="Apple-style-span"  style="font-family:'courier new';"&gt;  47882    33.4113  squid                    connStateFree&lt;/span&gt;&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span class="Apple-style-span" style="font-size: small;"&gt;&lt;span class="Apple-style-span"  style="font-family:'courier new';"&gt;143177   100.000  libc-2.3.6.so            memcpy&lt;/span&gt;&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;.. there's no memcpy() call in connStateFree. Which means I need to go hunting to figure out what's going on. Grr.&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-8077058012111053182?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/8077058012111053182/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2010/03/some-more-lusca-profiling.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/8077058012111053182'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/8077058012111053182'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2010/03/some-more-lusca-profiling.html' title='Some more Lusca Profiling..'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-5378628125168247009</id><published>2010-03-08T22:57:00.000-08:00</published><updated>2010-03-08T23:11:29.007-08:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='lusca'/><category scheme='http://www.blogger.com/atom/ns#' term='squid'/><title type='text'>Why are some Squid/Lusca ACL types slower than others? And which ones?</title><content type='html'>This post should likely be part of the documentation!&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;One thing which hasn't really been documented is the relative speed of each of the Squid/Lusca ACL types. This is important to know if you're administering a large Squid/Lusca install - it's entirely possible that the performance of your site will be massively impacted with the wrong ACL setup.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;Firstly - the types themselves:&lt;/div&gt;&lt;div&gt;&lt;ol&gt;&lt;li&gt;Splay trees are likely the fastest - src, dst, myip, dstdomain, srcdomain&lt;/li&gt;&lt;li&gt;The wordlist checks are linear but place hits back on the top of the wordlist to try and speed up the most looked up items - portname, method, snmp community, urlgroup, hiercode, &lt;/li&gt;&lt;li&gt;The regular expression checks are also linear and also reshuffle the list based on the most popular items - url regex, path regex,  source/destination domain regex, request/reply mime type&lt;/li&gt;&lt;/ol&gt;&lt;div&gt;Now the exceptions! Some require DNS lookups to match on the IP of the hostname being connected to - eg "dst", "srcdom_regex", "dstdom_regex".&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;A lot of places will simply use URL regular expression ACLs ("url_regex") to filter/forward requests. Unfortunately these scale poorly under high load and are almost always the reason a busy proxy server is pegging at full CPU.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;I'll write up an article explaining how to work around these behaviours if enough people ask me nicely. :)&lt;/div&gt;&lt;/div&gt;&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-5378628125168247009?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/5378628125168247009/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2010/03/why-are-some-squidlusca-acl-types.html#comment-form' title='2 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/5378628125168247009'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/5378628125168247009'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2010/03/why-are-some-squidlusca-acl-types.html' title='Why are some Squid/Lusca ACL types slower than others? And which ones?'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>2</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-8907430218319198556</id><published>2010-02-25T00:38:00.000-08:00</published><updated>2010-02-25T00:40:48.640-08:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='opensource'/><title type='text'>Open Source Economy?</title><content type='html'>I'm reposting this from a Buzz (eww) that I responded to; I'd appreciate feedback and comments.&lt;br /&gt;&lt;br /&gt;The article:&lt;br /&gt;&lt;a href="http://mashable.com/2010/02/24/open-source-threatens-capitalism/?utm_source=feedburner&amp;amp;utm_medium=feed&amp;amp;utm_campaign=Feed%3A+Mashable+%28Mashable%29"&gt;&lt;br /&gt;Lobby Group Says Open Source Threatens Capitalism&lt;/a&gt;&lt;br /&gt;&lt;br /&gt;My response:&lt;br /&gt;&lt;br /&gt;&lt;span class="TSrHSb"&gt;&lt;span class="ze"&gt;That article is .. well, a silly  summary. It's comparing apples to oranges in a way.&lt;br /&gt;&lt;br /&gt;Open/Free Source Software creates a very low entry barrier to a  variety of interesting possibilities. It means companies can leverage  this to create their own solutions without necessarily having to spend a  large amount of money on in-house development or expensively licenced  solutions. In a way, Open/Free Source is forcing a large part of the  commercial market to compete better.&lt;br /&gt;&lt;br /&gt;But the question is how you make that sustainable. Do I think the  current method that Open/Free Source is used in companies is  sustainable? It can be. It can not be. I've worked on open source  projects in both camps.&lt;br /&gt;&lt;br /&gt;Everything looks fine right now with a large wad of Open/Free  projects because the popular ones have a lot of inertia behind them. But  I wonder if there's longer-term flow-on effects in the economy. Plenty  of companies which use and abuse open/free software don't contribute  very much back to the projects. The cost savings they're passing on  sound great in theory, but in practice the "community" is mostly  covering this without their investment.&lt;br /&gt;&lt;br /&gt;So the interesting question is - at what point (if any) does  open/free software use tip the software economy to the point where  developing new solutions is just not cost effective enough to compete  with the established open source base; and what does that mean for  future software development.&lt;/span&gt;&lt;/span&gt;&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-8907430218319198556?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/8907430218319198556/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2010/02/open-source-economy.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/8907430218319198556'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/8907430218319198556'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2010/02/open-source-economy.html' title='Open Source Economy?'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-7712728484764002991</id><published>2010-02-20T00:13:00.000-08:00</published><updated>2010-02-20T08:55:28.438-08:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='lusca'/><title type='text'>Lusca: more reorganisation, new features</title><content type='html'>&lt;p&gt;&lt;br /&gt;More Lusca updates!&lt;br /&gt;&lt;/p&gt;I've further split up the client-side processing. Refresh, IMS handling, ETag handling, location rewriting and various range request support code is now in separate modules. The code isn't all that much easier to follow if you don't have a rough idea what is going on but it's getting there. One of the long-standing issues in the Squid/Lusca codebase is how much of the caching logic is done as part of the client-side handling; I'd like to continue slowly unwinding this so I can start sliding in processing hooks in useful places. I have eventual evil plans here; I'll talk more when they're coming closer to fruition.&lt;br /&gt;&lt;br /&gt;&lt;div&gt;I've also been whittling away a little at the lack of code documentation for the client-side request and reply framework. I'll commit some more comments to client_side.c and the associated routines as I explore things some more.&lt;br /&gt;&lt;br /&gt;I've got a very overdue contract to sort out a TOS bit tagging feature. I'm putting in the basic framework to make this happen. The first step is a new ACL type - dstfwdip - which is the destination IP being handed the request. There's a bunch of other side-changes which need to occur before I can slot in the TOS marking map logic (and I still haven't figured out where -that- will be!) so stay tuned. The aim is to provide a simple way to tag client (and maybe also server) requests with TOS bits based on some property of the request - and this includes the destination IP/network the request is being forwarded to.&lt;/div&gt;&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-7712728484764002991?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/7712728484764002991/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2010/02/lusca-more-reorganisation-new-features.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/7712728484764002991'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/7712728484764002991'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2010/02/lusca-more-reorganisation-new-features.html' title='Lusca: more reorganisation, new features'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-3964857343896011054</id><published>2010-02-14T04:23:00.000-08:00</published><updated>2010-02-14T04:30:16.074-08:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='ibook'/><category scheme='http://www.blogger.com/atom/ns#' term='apple'/><title type='text'>ibook hackery, or "why apple probably did the magnetic connector stuff."</title><content type='html'>I didn't have the foresight to take pictures of the incident beforehand so I'll have to make do with a verbose description of the problem.&lt;br /&gt;&lt;br /&gt;Symptom: the iBook just doesn't seem to handle power very well. "Jiggling" the connector around helps. But if it doesn't get jiggled right - and further jiggling added to the problem - the laptop won't charge fully or .. well, stay on.&lt;br /&gt;&lt;br /&gt;It also can't be good for the power electronics to be given bursts of a couple amps of power every few seconds or so if I'm not paying attention.&lt;br /&gt;&lt;br /&gt;So, after much anger, and Yet Another CD Not Ejecting Properly From The DVD Drive, I decided to strip the iBook down (again!) to take a look.&lt;br /&gt;&lt;br /&gt;It takes quite a bit of disassembly to be able to reach the CD drive to replace it. It doesn't take all that much to get to the power input daughterboard. Then, removal of the daughterboard showed a couple of interesting issues. Firstly, there's some cracked/dry joints rocking hard around the power connector. Secondly, those now dry joints are black with what I'm guessing is a whole lot of DC electrical arc'ing going on.&lt;br /&gt;&lt;br /&gt;A little bit of hot soldering iron action later and the damage was reversed. The power feed seems much more stable now. Thank god! I can stave off replacing the G4 iBook for another day.&lt;br /&gt;&lt;br /&gt;This leads me to the subject of this post. Yes, the magnetic DC power connector Apple introduced into their laptop range is very nifty but it does make sense. Even just the normal insert/remove cycle of these power connectors could cause joints to crack and this sort of damage to occur. But there's a flipside - I've seen at least one magnetic connector blackened with DC arcing after something unknown to me had occured. So there's still apparently a slight chance of DC arc'ing happening - but my pet cat won't be destroying anything by dashing across the room and taking the laptop with it.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-3964857343896011054?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/3964857343896011054/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2010/02/ibook-hackery-or-why-apple-probably-did.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/3964857343896011054'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/3964857343896011054'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2010/02/ibook-hackery-or-why-apple-probably-did.html' title='ibook hackery, or &quot;why apple probably did the magnetic connector stuff.&quot;'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-205970267493543746</id><published>2010-02-13T22:05:00.000-08:00</published><updated>2010-02-13T22:09:54.900-08:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='lusca'/><title type='text'>Lusca: more reorganisation</title><content type='html'>I've been doing some further Lusca re-organisation in preparation for the next round of evil changes.&lt;br /&gt;&lt;br /&gt;The client-side code has been split up a little bit. The breakup has almost exclusively been shifting functions into separate source files to better delineate function - the only refactoring was to extract some common client-side connection creation from the HTTP and HTTPS connection paths.&lt;br /&gt;&lt;br /&gt;My next little bit of fudgery will be to extract out some more of the storage related code into a top-level library so some further unit testing and code reuse becomes easier.&lt;br /&gt;&lt;br /&gt;There's still far, far too much code in the client-side request and reply path. I think I've mentioned it earlier - there's very little code involved in the data pump between the server-&gt;client reply handling code; the majority of the performance issues stem from the initial request and reply processing.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-205970267493543746?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/205970267493543746/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2010/02/lusca-more-reorganisation.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/205970267493543746'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/205970267493543746'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2010/02/lusca-more-reorganisation.html' title='Lusca: more reorganisation'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-1586638906520541278</id><published>2010-02-01T21:18:00.000-08:00</published><updated>2010-02-01T21:40:43.784-08:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='mips'/><category scheme='http://www.blogger.com/atom/ns#' term='freebsd'/><category scheme='http://www.blogger.com/atom/ns#' term='ubiquity'/><title type='text'>FreeBSD-current on the RouterStation Pro</title><content type='html'>I've been working on shoehorning FreeBSD-current onto the RouterStation Pro. It's a rather cheap but nice Atheros MIPS board by Ubiquity. It was tricky, but doable. I now have a cut down kernel and memory root filesystem stored on the on-board flash - enough to run it as a basic access point.&lt;br /&gt;&lt;br /&gt;FreeBSD-current has support for the chipset - it's the AR71XX kernel config file. I've added a few more devices (USB, disk, MSDOS, redboot) to the default kernel. The default cross-build system works just fine.&lt;br /&gt;&lt;br /&gt;I've been using TFTP loaded kernel+mdroot images to test out standalone functionality, along with TFTP + NFS root images to run a mostly-full development environment.&lt;br /&gt;&lt;br /&gt;The board uses RedBoot as a bootloader. There's a tool by Ubiquity which generates firmware images which are understood by the bootloader and will overwrite the non-system area of the on-board flash. This makes dumping an image onto the system highly trivial. The bootloader can also boot the (uncompresed) kernel+mdroot images via TFTP as well as having a compressed version written to flash - so I can easily test the standalone images without constant re-flashing.&lt;br /&gt;&lt;br /&gt;RedBoot makes it easy to TFTP upload a replacement flash image written out by the Ubiquity tool. It takes care of erasing, repartitioning and copying into the onboard flash without overwriting or damaging the RedBoot code, flash partition and configuration areas.&lt;br /&gt;&lt;br /&gt;There was a bug with the PCI probe code until a couple of days ago where PCI cards (ie, stuff like the mini-PCI radio cards!) wouldn't be enumerated unless bootverbose was specified. Gonzo traced it down to a missing DELAY() - Linux was waiting a lot longer between probes than FreeBSD was. PCI devices now probe and attach correctly.&lt;br /&gt;&lt;br /&gt;The geom_redboot module doesn't attach to the on-board flash (device "spi") - it only probes "cfi" flash devices. A quick patch to src/sys/geom/geom_redboot.c to also probe flash/spi has fixed this. I've asked Gonzo/Warner to investigate this and possibly commit a fix. With this in place, FreeBSD can mount a compressed, read-only filesystem from the "rootfs" flash slice rather than needing to pack a kernel+mdroot into the kernel flash slice.&lt;br /&gt;&lt;br /&gt;So far, so good. I've written out a basic image that can be coaxed to be an open (no authentication) access point.&lt;br /&gt;&lt;br /&gt;Interesting links:&lt;br /&gt;&lt;br /&gt;Instructions on re-flashing the unit (but it doesn't mention that the reset button must be held down during power on!): http://www.usualcoding.eu/post/2010/01/15/Flash-OpenWRT-on-the-Ubiquiti-RouterStation-Pro&lt;br /&gt;&lt;br /&gt;My FreeBSD router-station pro wiki: &lt;a href="http://wiki.freebsd.org/AdrianChadd/UbiquityRouterstationPro"&gt;http://wiki.freebsd.org/AdrianChadd/UbiquityRouterstationPro&lt;/a&gt;&lt;br /&gt;&lt;br /&gt;My work-in-progress tarball of "stuff" for building firmware images for the RouterStation Pro: &lt;a href="http://people.freebsd.org/~adrian/rspro/"&gt;http://people.freebsd.org/~adrian/rspro/&lt;/a&gt;&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-1586638906520541278?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/1586638906520541278/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2010/02/freebsd-current-on-routerstation-pro.html#comment-form' title='2 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/1586638906520541278'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/1586638906520541278'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2010/02/freebsd-current-on-routerstation-pro.html' title='FreeBSD-current on the RouterStation Pro'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>2</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-1722946982356880663</id><published>2010-01-25T07:20:00.000-08:00</published><updated>2010-01-25T07:22:57.684-08:00</updated><title type='text'>PS3 hacked! Or, "I'd pay extra for a totally open PS3."</title><content type='html'>&lt;a href="http://games.slashdot.org/story/10/01/25/0654253/PS3-Hacked"&gt;This slashdot article&lt;/a&gt; covers &lt;a href="http://geohotps3.blogspot.com/2010/01/hello-hypervisor-im-geohot.html"&gt;this news article&lt;/a&gt; which describes a hackers success at breaking through the PS3 hypervisor protection layers to get full hardware access. A few of the slashdot posts describe why people may want full PS3 hardware access, and why Sony may be keeping the full platform access away from the user.&lt;br /&gt;&lt;br /&gt;To be perfectly honest, I'd pay extra for a PS3 (and XBox 360 too!) which was unlocked for software that I've written myself, and it wouldn't be used to run pirated games. I'm sure others feel the same way. I'd love to see more hardware manufacturers do this.&lt;br /&gt;&lt;br /&gt;Sigh, I can but only dream. :)&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-1722946982356880663?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/1722946982356880663/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2010/01/ps3-hacked-or-id-pay-extra-for-totally.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/1722946982356880663'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/1722946982356880663'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2010/01/ps3-hacked-or-id-pay-extra-for-totally.html' title='PS3 hacked! Or, &quot;I&apos;d pay extra for a totally open PS3.&quot;'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-7746031503675370598</id><published>2010-01-16T00:07:00.000-08:00</published><updated>2010-01-16T01:02:21.316-08:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='retrocomputing'/><category scheme='http://www.blogger.com/atom/ns#' term='cpc464'/><category scheme='http://www.blogger.com/atom/ns#' term='cpc'/><category scheme='http://www.blogger.com/atom/ns#' term='amstrad'/><category scheme='http://www.blogger.com/atom/ns#' term='cpc6128'/><title type='text'>Hardware hackery: Adding IO to an Amstrad CPC</title><content type='html'>&lt;a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://1.bp.blogspot.com/_37wobiQ3zUs/S1F36LdRbqI/AAAAAAAAADM/95r0Ep-U4I8/s1600-h/16-01-10_1623.jpg"&gt;&lt;img style="margin: 0pt 10px 10px 0pt; float: left; cursor: pointer; width: 400px; height: 300px;" src="http://1.bp.blogspot.com/_37wobiQ3zUs/S1F36LdRbqI/AAAAAAAAADM/95r0Ep-U4I8/s400/16-01-10_1623.jpg" alt="" id="BLOGGER_PHOTO_ID_5427250867349843618" border="0" /&gt;&lt;/a&gt;&lt;br /&gt;One of the benefits to hacking on the old Commodore 64 is that the user port exposes quite a few bidirectional IO pins which can be easily programmed from BASIC. The Amstrad CPC doesn't really have these in the same fashion. Well, ok, it does - if you count the joystick pins as "input" and the printer port pins as "output". But having a fully programmable IO controller is helpful.&lt;br /&gt;&lt;br /&gt;So I breadboarded a basic IO board a couple months ago using an 8255 PIO and some TTL logic to handle address decoding. This worked fine on the Amstrad CPC464 but it didn't work on the CPC6128. A bit of digging into the method used for address decoding showed what I did wrong.&lt;br /&gt;&lt;br /&gt;&lt;div&gt;&lt;a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://2.bp.blogspot.com/_37wobiQ3zUs/S1F4Ar9gFLI/AAAAAAAAADU/HDnM-f3LISQ/s1600-h/16-01-10_1624.jpg"&gt;&lt;img style="margin: 0pt 0pt 10px 10px; float: right; cursor: pointer; width: 400px; height: 300px;" src="http://2.bp.blogspot.com/_37wobiQ3zUs/S1F4Ar9gFLI/AAAAAAAAADU/HDnM-f3LISQ/s400/16-01-10_1624.jpg" alt="" id="BLOGGER_PHOTO_ID_5427250979154171058" border="0" /&gt;&lt;/a&gt;&lt;br /&gt;The Amstrad CPC base peripheral IO list can be found &lt;a href="http://cpcwiki.eu/index.php/Default_I/O_Port_Summary"&gt;here&lt;/a&gt;. The  peripheral IO is decoded by the high 8 bits of the address bus - each peripheral attaches an address line (and /IORQ) to the relevant chip /CS line. The expansion peripheral line is A10, and the "selection" decoding lines are A9 and A8. So, A10 needs to be low, and A8+A9 let you either address multiple peripherals or the registers inside the peripheral itself.&lt;br /&gt;&lt;br /&gt;This was easy - a bit of logic to say /CS = low IFF /IORQ is low and A10 is high.&lt;br /&gt;&lt;br /&gt;This is fine on an unexpanded CPC464 but the CPC6128 floppy disk controller decodes A10 low, A8 low AND A7 low. A7 is the "FDD peripheral expansion" line. So my 8255 PPI board was being accessed the same time that the FDD controller was. This caused all kinds of random lights to blink and the system to get highly confused.&lt;br /&gt;&lt;br /&gt;The logic now is /CS = low IFF /IORQ is low, A10 is high and A7 is high.&lt;br /&gt;&lt;br /&gt;Strictly speaking, I should ensure that A7-A4 are 1110 - ie, A4 is low, the rest are high.&lt;br /&gt;&lt;br /&gt;According to the CPC464 manual, when A10 is low, A4 low is "user", A5 low is "RS232" (for the Amstrad/PACE RS232 adaptor), A6 low is "future", and A7 low is "disk".&lt;/div&gt;&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-7746031503675370598?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/7746031503675370598/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2010/01/hardware-hackery-adding-io-to-amstrad.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/7746031503675370598'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/7746031503675370598'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2010/01/hardware-hackery-adding-io-to-amstrad.html' title='Hardware hackery: Adding IO to an Amstrad CPC'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><media:thumbnail xmlns:media='http://search.yahoo.com/mrss/' url='http://1.bp.blogspot.com/_37wobiQ3zUs/S1F36LdRbqI/AAAAAAAAADM/95r0Ep-U4I8/s72-c/16-01-10_1623.jpg' height='72' width='72'/><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-5768439042411831443</id><published>2010-01-14T19:22:00.001-08:00</published><updated>2010-01-14T19:24:04.974-08:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='heinlein'/><title type='text'>The Origin Of Nerd Phrases, or Robert Heinlein</title><content type='html'>I've read a few Heinlein novels and I have been struck by the sheer amount of quoted material - especially inside Email Signatures! - but by far the highest density of quoted material is in the first few chapters of "Time Enough For Love."&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;Also, the more Heinlein I read, the more cynical I seem to be leaning.&lt;/div&gt;&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-5768439042411831443?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/5768439042411831443/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2010/01/origin-of-nerd-phrases-or-robert.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/5768439042411831443'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/5768439042411831443'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2010/01/origin-of-nerd-phrases-or-robert.html' title='The Origin Of Nerd Phrases, or Robert Heinlein'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-2162602989091075217</id><published>2009-12-21T22:28:00.000-08:00</published><updated>2009-12-21T22:50:30.438-08:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='censorship'/><category scheme='http://www.blogger.com/atom/ns#' term='nocleanfeed'/><category scheme='http://www.blogger.com/atom/ns#' term='proxy'/><title type='text'>The ACMA blacklist, and can it be distributed securely?</title><content type='html'>One sore point is that the ACMA Blacklist for online restricted content is "closed" - that is, we the public have no way at the present time of viewing what is it on it. The pro-filtering advocates quite validly state that opening up the ACMA Blacklist would basically be publishing URLs for naughty people to view - an "illegal content directory", if you will.&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;So what? If they want to find it, they'll find it - whether it is public or not.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;The current downside though is that the blacklist can't be easily used by third-party filter software producers without what I understand to be an elaborate and expensive process.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;So not only is it currently impossible for the public to vet the list to make sure only illegal content makes it on there, but it also means it can't be widely used everywhere unless you're a company with a lot of money to burn.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;It seems like a bit of a silly situation to be in, doesn't it?&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;So, is it feasible to distribute the list in some encrypted way? How easy would it be to find what is on the list itself? This is a good question. The true answer is "no, it isn't feasible." Like everything technological, the true question is how much effort you're willing to go to in order to hide said list.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;The ACMA blacklist is integrated into a few products which are currently available. The problem is hiding the URLs from the user. Software hackers are a clever bunch. If your computer runs the software then it is very possible to determine how to decrypt the URL list and use it. So simply publishing the cleartext ACMA blacklist - encrypted or not - is just never going to be secure. I believe this is how the ACMA blacklist was leaked to Wikileaks earlier in 2009.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;There already exists a perfectly good way to distribute this sort of URL blacklist - eg &lt;a href="http://code.google.com/p/google-safe-browsing/wiki/Protocolv2Spec"&gt;Google SafeSearch&lt;/a&gt;. The ACMA could take the list of URLs, convert them to sets of MD5 strings to match against, and distribute that. They could distribute this openly - so everyone and anyone who wished to filter content based on this list could do so without having to pay the ACMA some stupid amount of money. Finally, it means that web site owners could compare their own URLs against the content of the blacklist to see if any of their pages are on it. It may not be that feasible for very large, dynamic URL sites - but it certainly is more feasible than what can be done today.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;If the ACMA did this then I'd even write up a Squid plugin to filter against said ACMA blacklist. Small companies and schools can then use it for free. That would get the ACMA blacklist more exposure - which benefits the ACMA as much as it benefits anti-censor advocates. More use will translate to a larger cross-section of visited web sites - so people will be more likely to discover if something which shouldn't be blocked suddenly appears on the blacklist.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;But is it truely secure? There's currently no way to take an MD5 string and turn it back into a URL. You could theoretically generate a set of URLs which would hash to that MD5 string but it'd take a damned long time. So, for all practical reasons, it can't be reverse engineered.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;But what &lt;b&gt;can&lt;/b&gt; be done is to log the URLs which match the filter and slowly build up a list of sites that way. Naughty people could then publish the set of URLs which match the blacklist rules. There's no technological method of avoiding that. If people discover a URL has been filtered, they may just share the link online.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;The only real way the government has to counter sharing the cleartext URLs from the blacklist would be to make it illegal and enforce that law very strictly. This means enforcing it when naughty stuff is shared - but it also means that anyone who publishes URLs for content which should &lt;b&gt;not&lt;/b&gt; be on the list may also get punished. That is a whole other debate.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;So in summary - yes, the ACMA could publish the blacklist in a way that is more secure than they currently are. They could publish it - like Google does - to the public, so it can be integrated into arbitrary pieces of software. This may help it be more widely adopted and tested. But they will &lt;b&gt;never&lt;/b&gt; be able to publish the list in a way that makes it impossible to identify and publish cleartext URLs.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;Let me be clear here - there is no technological method for restricting what information people can share between each other, and this includes URLs identified to be on the ACMA blacklist.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-2162602989091075217?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/2162602989091075217/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2009/12/acma-blacklist-and-can-it-be.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/2162602989091075217'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/2162602989091075217'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2009/12/acma-blacklist-and-can-it-be.html' title='The ACMA blacklist, and can it be distributed securely?'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-7448937153121527321</id><published>2009-12-20T20:41:00.000-08:00</published><updated>2009-12-20T21:06:21.500-08:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='censorship'/><category scheme='http://www.blogger.com/atom/ns#' term='nocleanfeed'/><category scheme='http://www.blogger.com/atom/ns#' term='proxy'/><title type='text'>On filtering proxy/anonymizing servers..</title><content type='html'>I'd like to briefly talk about &lt;a href="http://en.wikipedia.org/wiki/Anonymizer"&gt;anonymizing/proxy&lt;/a&gt; servers. These services act as gateways between the user (and their web browser, for example) and the general internet. They typically hide the real user origin from the web site and the ISPs in question so access can not be easily traced. They are also useful diagnostic tools (eg to see whether web sites work from far away networks.) Others use them to circumvent country filters which are blocking access to "free-speech" and social networking web sites (eg China, Iran, etc.)&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;I'm not going to talk about the legitimate and illegitimate uses of these. Many more devices are used and abused for nefarious ways, but we don't see the postal system implement mandatory written filtering; nor do we see (legal!) mandatory monitoring and filtering of the telephone/cellular network.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;One common way of working around URL filters in the workplace, schools and libraries is to use an anonymizer/proxy service on the internet. This is how many schoolchildren log onto facebook and myspace. Their use is dangerous (as you're typically giving the service your facebook/myspace/hotmail/gmail/etc credentials!) but again, there are plenty of legitimate and safe uses for them.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;The problem is constructing filters which block access through these anonymizer/proxy services. Some of them will include the original URL in the request - they're easyish to block. Others will encrypt/obfuscate the URL so a normal filter won't work. There are plenty of tricks which are pulled; describing them will take a long time.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;A growing number of these anonymizer/proxy services are using SSL encryption to totally hide what is going on (ie, blocking not only the URL, but the content itself.) This is just not possible to break without some intrusive additions to the users' computer. Let's not go there.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;So, there really is only a few ways to combat this:&lt;/div&gt;&lt;div&gt;&lt;ol&gt;&lt;li&gt;You create complicated rules for each anonymizer/proxy service which attempts to track and decode the URL, and filter on that; or&lt;/li&gt;&lt;li&gt;You create complicated fingerprints to identify types of traffic which indicate the user of an anonymizer/proxy service, and filter on that; or&lt;/li&gt;&lt;li&gt;You just block any and all proxy anonymizer/proxy sites.&lt;/li&gt;&lt;/ol&gt;&lt;div&gt;The problems!&lt;/div&gt;&lt;div&gt;&lt;ul&gt;&lt;li&gt;1 is difficult and longwinded. A lot of effort would have to be spent to continuously update the set of rules as new proxy services come on board designed to thwart these services.&lt;/li&gt;&lt;li&gt;2 is just as difficult and longwinded - and it becomes possible that these fingerprints will identify legitimate sites as proxy services and filter traffic incorrectly.&lt;/li&gt;&lt;li&gt;3 is what the majority of current content filters do. They don't bother trying to filter what people are doing with anonymizer/proxy services; they just blanket filter all of them.&lt;/li&gt;&lt;/ul&gt;&lt;div&gt;Now, as I've mentioned, plenty of new anonymizer/proxy services pop up every day. I'd hazard a guess and suggest that the majority of them are run by shady, nefarious people who see the value in logging your access credentials to popular webmail/social networking sites and selling them to third parties.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;The real concern - I've seen more than one user log onto their internet banking and work sites using these anonymizer/proxy services because they're so used to using them, they forget not to. Imagine, for a moment, that gambling sites are blocked and users turn to anonymizer/proxy services to gamble online. They use their credit card details. Ruh roh.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;This is another example of the arms race which filtering companies deal with every day.&lt;/div&gt;&lt;div&gt;New anonymizer/proxy services are created every day - many specifically to allow users to bypass country-level filtering. Many of them may be logging and selling your authentication credentials to third parties. Users will simply begin using new anonymizer/proxy services as they creep up to work around any filtering which may be put in place. There is a non-trivial amount of effort required to keep track of all of these sites and noone will ever be 100% effective.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;A large amount of effort will be needed to filter these services and perfectly legitimate uses will be blocked.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;You don't want to push users to begin using anonymizing/proxy services - that is a battle that you won't win.&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-7448937153121527321?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/7448937153121527321/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2009/12/on-filtering-proxyanonymizing-servers.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/7448937153121527321'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/7448937153121527321'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2009/12/on-filtering-proxyanonymizing-servers.html' title='On filtering proxy/anonymizing servers..'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-2797015098583128808</id><published>2009-12-19T23:33:00.000-08:00</published><updated>2009-12-19T23:56:36.157-08:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='filtering'/><category scheme='http://www.blogger.com/atom/ns#' term='BGP'/><category scheme='http://www.blogger.com/atom/ns#' term='nocleanfeed'/><title type='text'>Filtering via BGP, and why this doesn't always quite work out..</title><content type='html'>Another interesting thing to look at is the increasingly popular method of filtering by using a BGP "exception list".  I've heard this anecdotally touted by various parties as "the solution" (but nothing I can quote publicly in any way, sorry) but really, is it?&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;This employs a little bit of routing trickery to redirect sites to be filtered via the proxy and passing the rest through untouched. This hopefully means that the amount of traffic and websites which need to pass through the filter is a lot less than "everything."&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;This is how the British Telecom "cleanfeed" solution worked. They redirected traffic to be filtered via a bunch of Squid proxies to do the actual filtering. This worked out great - until they filtered Wikipedia for a specific image on a specific article. (I'd appreciate links to the above please so I can reference them.) Then everything went pear-shaped:&lt;/div&gt;&lt;div&gt;&lt;ul&gt;&lt;li&gt;From my understanding, all the requests to Wikipedia came from one IP, rather than properly pretending to be the client IP - this noticably upset Wikipedia, who use IP addresses to identify potential spammers; and&lt;/li&gt;&lt;li&gt;The sheer volume of requests to Wikipedia going through the filtering service caused it to slow right down.&lt;/li&gt;&lt;/ul&gt;&lt;/div&gt;&lt;div&gt;So that is problem number 1 - it looks like it will work fine on a set of hardly viewed sites - but it may not work on a very busy site such as Wikipedia. Squid based filtering solutions certainly won't work on the scale of filtering Youtube (at least, not without using the magic Adrian-Squid version which isn't so performance-limited [/advertisement].)&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;The next problem is determining which IP addresses to redirect. Websites may change their IP addresses often - or have many IP addresses! - and so the list of IP addresses needs to be constantly updated. The number of IP addresses which need to be injected into BGP is based on all of the possible IP addresses returned for each site - this varies in the real world from one to hundreds. Again, they may change frequently - requiring constant updating to be correct. This leads to two main potential issues:&lt;/div&gt;&lt;div&gt;&lt;ul&gt;&lt;li&gt;Hardware routing devices (ie, the top of the line ones which large ISPs use to route gigabits of traffic) have limited "slots" for IP addresses/networks. The router typically stops working correctly when those run out. If you're lucky, the number of IP addresses being filtered will fit inside the hardware routing table. If you're unlucky, they won't. The big problem - different equipment from different vendors has different limitations. Upgrading this equipment can costs tens or hundreds of thousands of dollars.&lt;/li&gt;&lt;li&gt;The only traffic being filtered is traffic being redirected to the filter. If the list of IP addresses for a nasty website is not kept 100% up to date, the website will not be properly filtered.&lt;/li&gt;&lt;/ul&gt;&lt;div&gt;The third main problem is filtering websites which employ Content Delivery Networks. This is a combination of the above two problems. So I'm going to pose the question:&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;How do you filter a web page on Google?&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;No, the answer here isn't "Contact Google and ask them to kill the Web page." I'm specifically asking how one filters a particular web page on a very distributed infrastructure. You know; the kind of infrastructure which everyone is deploying these days. This may be something like Google/Yahoo; this may be being hosted on a very large set of end user machines on an illegally run botnet. The problem space is still the same.&lt;/div&gt;&lt;div&gt;&lt;ul&gt;&lt;li&gt;The IP addresses/networks involved in potentially serving that website is dynamic - you don't get an "easy" list of IPs when you resolve the hostname! For example - there's at least hundreds of potential hostnames serving Youtube streaming media content. It just isn't a case of filtering "www.youtube.com".&lt;/li&gt;&lt;li&gt;There are a number of services running on the same infrastructure as Youtube. You may get lucky and only have one website - but you also may get unlucky and end up having to intercept all of the Google services just to filter one particular website.&lt;/li&gt;&lt;/ul&gt;&lt;div&gt;All of a sudden you will end up potentially redirecting a significant portion of your web traffic to your filtering infrastructure. It may be happy filtering a handful of never-visited websites; but then you start feeding it a large part of the internet.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;In summary, BGP based selective filtering doesn't work anywhere near as well as indicated in the ACMA report.&lt;/div&gt;&lt;div&gt;&lt;ul&gt;&lt;li&gt;You can't guarantee that you'll enumerate all IP addresses involved for a specific website;&lt;/li&gt;&lt;li&gt;The ACMA may list something on a large website/CDN which will result in your filtering proxies melting; you may as well have paid the upfront cost in filtering everything in the first place;&lt;/li&gt;&lt;li&gt;The ACMA may list something with so many IP addresses that your network infrastructure either stops working; or the filter itself stops working.&lt;/li&gt;&lt;/ul&gt;&lt;div&gt;Personally - I don't like the idea of the ACMA being able to crash ISPs because they list something which ISPs are just unable to economically filter. Thus, the only logical solution here is to specify a filtering infrastructure to filter -everything-.&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-2797015098583128808?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/2797015098583128808/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2009/12/filtering-via-bgp-and-why-this-doesnt.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/2797015098583128808'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/2797015098583128808'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2009/12/filtering-via-bgp-and-why-this-doesnt.html' title='Filtering via BGP, and why this doesn&apos;t always quite work out..'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-6318145289105971263</id><published>2009-12-19T01:50:00.000-08:00</published><updated>2009-12-19T02:24:50.508-08:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='censorship'/><category scheme='http://www.blogger.com/atom/ns#' term='nocleanfeed'/><category scheme='http://www.blogger.com/atom/ns#' term='proxy'/><title type='text'>How does a proxy interfere with throughput?</title><content type='html'>Another big question with the filtering debate is figuring out how much of an impact on performance an inline proxy filter will have.&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;Well, it's quite easy to estimate how much of an impact an inline server running Windows/UNIX will have on traffic. And this will be important as the filtering mechanisms tested by the government and which will be implemented by more than one of the ISPs.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;Inline proxies are very popular today. They're used in a variety of ISP and corporate environments. Traffic is either forced there via configuration on users' machines, or redirected there transparently by some part of the network (eg a router, or transparent bridge.)&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;The inline proxy will "hijack" the TCP sessions from the client and terminate them locally. The client believes it is talking to the web server but it actually talking to the proxy server.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;Then the inline proxy will issue outbound TCP sessions to the web server as requested by the user - and in some configurations, the web server will think it is talking directly to the client.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;This is all relatively well understood stuff. It's been going on for 10-15 years. I was involved in some of the early implementations of this stuff in Australia back in the mid 1990's. What isn't always well understood is how it impacts performance and throughput. Sometimes this doesn't matter for a variety of reasons - the users may not have a large internet connection in the first place, or the proxy is specifically in place to limit how much bandwidth each user can consume. But these proxies are going to be used for &lt;em&gt;everyone&lt;/em&gt;, some of which will have multi-megabit internet connections. Performance and throughput suddenly matter.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;I'll cover one specific example today - how inline proxies affect data throughput. There are ways that inline proxies affect perceived request times (ie, how long it takes to begin and complete a web request) which will take a lot more space to write about.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;Each request on the inline proxy - client facing and server facing - will have a bit of memory reserved to store data which is being sent and received. The throughput of a connection is, roughly speaking, limited by how big this buffer is. If the buffer is small, then you'll only get fast speeds when speaking to web sites that are next door to you. If the buffer is large, you'll get fast speeds when speaking to web sites that are overseas - but only if they too have large buffers on their servers.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;These buffers take memory, and memory is a fixed commodity in a server. Just to give you an idea - if you have 1GB of RAM assigned for "network buffers", and you're using 64 kilobyte buffers for each session, then you can only hold up (1 gigabyte / 64 kilobyte) sessions - ie, 16,384 sessions. This may sound like a lot of sessions! But how fast can you download with a 64 kilobyte buffer?&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;If you're 1 millisecond away from the webserver (ie, its on the same LAN as you), then that 64 kilobyte buffer will give you (64 / 0.001) kilobytes - or ~ 64 megabytes a second. That's 512 megabits. Quite quick, no?&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;But if you're on DSL, your latency will be at least 10 milliseconds on average. That's 6.4 megabytes a second, or 51.2 megabits. Hm, it's still faster than ADSL2, but suddenly its slower than what bandwidth the NBN is going to give you.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;Say you're streaming from Google. My Perth ISP routes traffic to/from Google in Sydney for a few services. That's 53 milliseconds. With 64 kilobyte buffers, that's (64 / 0.053), or 1207 kilobytes/second. Or, around a megabyte a second. Or, say, 8-10 megabits a second. That isn't even ADSL2 speed (24 megabits), let alone NBN speeds (100 megabits.)&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;So the operative question here is - how do you get such fast speeds when talking to websites in different cities, states or countries to you? The answer is quite simple. Your machine has plenty of RAM for you - so your buffers can be huge. Those streaming websites you're speaking to build servers which are optimised for handling large buffered streams - they'll buy servers which -just- stream flash/video/music, which handle a few thousand clients per server, and have gigabytes of RAM. They're making enough money from the service (I hope!) to just buy more streaming servers where needed - or they'll put the streaming servers all around the world, closer to end-users, so they don't need such big buffers when talking to end users.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;What does this all mean for the performance through a filtering proxy?&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;Well, firstly, the ISP filtering proxy is going to be filtering all requests to a website. So, it'll have to filter all requests (say) to Youtube, or Wikipedia. This means that all streaming content is potentially passing through it. It's going to handle streaming and non-streaming requests for the websites in question.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;So say you've got a filtering proxy with 16GB of RAM, and you've got 64 kilobyte buffers. You have:&lt;/div&gt;&lt;div&gt;&lt;ul&gt;&lt;li&gt;Say, minimum, 262,144 concurrent sessions (16 gigabytes / 64 kilobytes) going through the proxy before you run out of network buffers. You may have more sessions available if there aren't many streaming/downloading connections, but you'll always have that minimum you need to worry about.&lt;/li&gt;&lt;li&gt;Actually, it's half of that - as you have a 64 kilobyte buffer for transmit and a 64 kilobyte buffer for receive. So that's 131,072 concurrent sessions.&lt;/li&gt;&lt;li&gt;If your streaming site is luckily on a LAN, and you're on a LAN to the proxy - you'll get ~ 100mbit.&lt;/li&gt;&lt;li&gt;If you're on ADSL (10 milliseconds) from the proxy - you'll get 6.4 megabytes/second, or 51 megabits/sec from the proxy.&lt;/li&gt;&lt;li&gt;If you're on NBN (1 millisecond, say) from the proxy - you'll get 64 megabytes/second, or 512 megabits from the proxy.&lt;/li&gt;&lt;li&gt;BUT - if the proxy is 50 milliseconds from the web server - then no matter how fast your connection is, you're only going to get maximum (65536 / 0.050) bytes/sec, or 1.2 megabytes/second, or 12 megabits/second.&lt;/li&gt;&lt;li&gt;And woe be if you're talking to a US site. No matter how fast your connection is, the proxy will only achieve speeds of 320 kilobytes/sec, or 2.5 megabits. Not even ADSL1 speed.&lt;/li&gt;&lt;/ul&gt;&lt;div&gt;The only way to increase the throughput your proxy has is to use larger buffers - which means either packing much more RAM into a server, or limiting the number of connections you can handle, or buying more servers. Or, if you're very unlucky, all of the above.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;Now, the technical people in the know will say that modern operating systems have auto-tuning buffering. You only need to use big buffers for distant connections, rather than for all connections. And sure, they're right. This means that the proxy will handle more connections and obtain higher throughput. But the question now is how you design a proxy service which meets certain goals. Sure, you can design for the best case - and things like auto-tuning buffering is certainly good for raising the best-case performance. But the worst case performance doesn't change. If you have lots of international streaming sessions suddenly being filtered (because, say, some very popular US-centric website gets filtered because of one particular video), the performance will suddenly drop to the worst case scenario, and everyone suffers.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;Now, just because I can blow my own trumpet a bit - when I design Squid/Lusca web proxy solutions, I always design for the worst case. And my proxies work better for it. Why? Because I make clear to the customer that the worst case solution is what we should be designing and budgeting for, and new equipment should be purchased based on that. The best case performance is just extra leg room during peak periods. That way clients are never surprised by poorly performing and unstable proxies, and the customer themselves knows exactly when to buy hardware. (They can then choose not to buy new proxies and save money - but then, when you're saving $100k a month on a $6k server, buying that second $6k server to save another $100k a month suddenly makes a lot of sense. Skimping on $6k and risking the wrath of your clients isn't appealing.)&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-6318145289105971263?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/6318145289105971263/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2009/12/how-does-proxy-interfere-with.html#comment-form' title='2 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/6318145289105971263'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/6318145289105971263'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2009/12/how-does-proxy-interfere-with.html' title='How does a proxy interfere with throughput?'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>2</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-685611991093117301</id><published>2009-12-16T19:14:00.000-08:00</published><updated>2009-12-16T19:24:53.837-08:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='censorship'/><category scheme='http://www.blogger.com/atom/ns#' term='nocleanfeed'/><category scheme='http://www.blogger.com/atom/ns#' term='armsrace'/><title type='text'>People who don't understand an arms race aren't doomed to repeat it...`</title><content type='html'>&lt;a href="http://www.thepunch.com.au/articles/hey-geeks-stop-the-whining-and-build-a-better-filter-clean-feed/"&gt;This article&lt;/a&gt; is amusing. Apparently geeks can build Napster to circumvent "stuff" so geeks should be able to build a better RC filter.&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;Here's some history for you.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;"Stuff" initially was "we're already sharing files via DCC on the Internet Relay Chat system (IRC); let's make an indexed, shiny, graphical, automated version of that!" It wasn't to circumvent any kind of censorship or filtering, and it wasn't a great leap of imagination. It was a small, incremental improvement over what existed. The only reason you think it was a big leap for a lone teenager is that Napster popularised file sharing. It made it easy for the average teenager to do.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;Secondly, there are most likely individuals and companies profiting off the construction and use of non-web based distribution of RC materials. Filtering web traffic won't stop this distribution - it will simply stop the web distribution of RC materials. The filtering technology will quickly grow to counter these needs, and then new tools will appear to circumvent the filter. This is a classic arms race, pure and simple.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;The only people who profiteer from an arms race are the arms dealers. In this case, the arms dealers are the companies developing tools to distribute the material, and companies developing tools to filter the material.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;The astute reader should draw a parallel between what I've described and malware/viruses versus anti-virus software. Why is it we can't filter viruses 100%? Because there's money to be made in both writing the nasty software and filtering the nasty software. The end-users end up paying the price.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;This censorship nonsense will suffer the same fate. &lt;/div&gt;&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-685611991093117301?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/685611991093117301/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2009/12/people-who-dont-understand-arms-race.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/685611991093117301'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/685611991093117301'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2009/12/people-who-dont-understand-arms-race.html' title='People who don&apos;t understand an arms race aren&apos;t doomed to repeat it...`'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-6620795416509346111</id><published>2009-12-16T18:38:00.000-08:00</published><updated>2009-12-16T19:05:00.538-08:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='censorship'/><category scheme='http://www.blogger.com/atom/ns#' term='filtering'/><category scheme='http://www.blogger.com/atom/ns#' term='nocleanfeed'/><category scheme='http://www.blogger.com/atom/ns#' term='squid'/><title type='text'>Why would more than 10,000 URLs be a problem?</title><content type='html'>I'm going to preface this (and all other censorship/filtering related posts) with a disclaimer:&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;&lt;b&gt;I believe that mandatory censorship and filtering is wrong, inappropriate and risky.&lt;/b&gt;&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;That said, I'd like others to better understand the various technical issues behind implementing a filter. My hope is that people begin to understand the proper technical issues rather than simply re-stating others' potentially misguided opinions.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;The "10,000 URL" limit is an interesting one. Since the report doesn't mention the specifics behind this view, and I can't find anything about it in my simple web searching, I'm going to make a stab in the dark.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;Many people who implement filters using open source methods such as Squid will typically implement them as a check against a list of URLs. This searching can be implemented via two main methods:&lt;/div&gt;&lt;div&gt;&lt;ol&gt;&lt;li&gt;Building a list of matches (regular expressions, exact-match strings, etc) which is compared against; and&lt;/li&gt;&lt;li&gt;Building a tree/hash/etc to match against in one pass.&lt;/li&gt;&lt;/ol&gt;&lt;div&gt;Squid implements the former for regular expression matching and the latter for dstdomain/IP address matching.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;What this unfortunately means is that full URL matching with regular expressions depends not only on the complexity of the regular expression, but the number of entries. It checks each entry in the list in turn.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;So when Squid (and similar) software is used to filter a large set of URLs, and regular expressions are used to match against, it is quite possible that there will be a limitation on how many URLs can be included before performance degrades.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;So, how would one work around it?&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;It is possible to combine regular expression matches into one larger rule, versus checking against many smaller ones. Technical details - instead of /a/, /b/, /c/; one may use /(a|b|c)/. But unfortunately not all regular expression libraries handle very long regular expressions so for portability reasons this is not always done.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;Squid at least doesn't make it easy to match on the full URL without using regular expressions. Exact-match and glob-style match (eg, http://foo.com/path/to/file/*) will work very nicely. (I also should write that for Squid/Lusca at some point.)&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;A google "&lt;a href="http://code.google.com/p/google-safe-browsing/wiki/Protocolv2Spec"&gt;SafeSearch&lt;/a&gt;" type methodology may be used to avoid the use of regular expressions. This normalises the URL, breaks it up into parts, creates MD5 hashes for each part and compares them in turn to a large database of MD5 hashes. This provides a method of distributing the filtering list without specifically providing the clear-text list of URLs and it turns all of the lookups into simple MD5 comparisons. The downside is the filtering is a lot less powerful than regular expressions.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;To wrap up, I'm specifically not discussing the effectiveness of URL matching and these kinds of rules in building filters. That is a completely different subject - one which will typically end with "it's an arms race; we'll never really win it." The point is that it is possible to filter requests against a list of URLs and regular expressions much, much greater than a low arbitrary limit.&lt;/div&gt;&lt;/div&gt;&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-6620795416509346111?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/6620795416509346111/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2009/12/why-would-more-than-10000-urls-be.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/6620795416509346111'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/6620795416509346111'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2009/12/why-would-more-than-10000-urls-be.html' title='Why would more than 10,000 URLs be a problem?'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-7638000649485781559</id><published>2009-12-16T05:31:00.000-08:00</published><updated>2009-12-16T05:35:01.628-08:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='retrocomputing'/><category scheme='http://www.blogger.com/atom/ns#' term='amiga'/><title type='text'>.. summary from Retro Night, take #2</title><content type='html'>Gah, I deleted the wrong post. Typical me.&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;Three of us got together on Tuesday night to resurrect some Amiga hardware. In summary, we have working machines, one sort of working machine, and a few bad floppy drives. The prize thus far is a working Amiga 2000 with a few megabytes of RAM expansion, a not-yet-working SCSI-and-drive expansion card (ghetto indeed!), a video genlock device to overlay graphics on a PAL/NTSC signal, and a functional Amiga 1200 with extra goodies.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;The aim now is to get an environment working enough to write Amiga floppy images out so we can start playing some more games. I'm hoping the Amiga 1200, when paired with a floppy drive and some form of MS-DOS readable flash device, will fit that bill reasonably nicely.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;More to come next week.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-7638000649485781559?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/7638000649485781559/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2009/12/summary-from-retro-night-take-2.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/7638000649485781559'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/7638000649485781559'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2009/12/summary-from-retro-night-take-2.html' title='.. summary from Retro Night, take #2'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-7726470551992274969</id><published>2009-11-10T19:14:00.000-08:00</published><updated>2010-06-22T22:05:45.775-07:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='freebsd'/><category scheme='http://www.blogger.com/atom/ns#' term='lighttpd'/><category scheme='http://www.blogger.com/atom/ns#' term='cacheboy'/><title type='text'>More issues with Lighttpd</title><content type='html'>So occasionally Lighttpd on FreeBSD-7.x+ZFS gets all upset. I -think- there's something weird going on where I hit mbuf exhaustion somehow when ZFS starts taking a long time to complete IO requests; then all socket IO fails in Lighttpd until it is restarted.&lt;br /&gt;&lt;br /&gt;More investigation is required. Well, more statistics are needed so I can make better judgements. Well, actually, more functional backends are needed so I can take one out of production when something like this occurs, properly debug what is going on and try to fix it.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-7726470551992274969?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/7726470551992274969/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2009/11/more-issues-with-lighttpd.html#comment-form' title='2 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/7726470551992274969'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/7726470551992274969'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2009/11/more-issues-with-lighttpd.html' title='More issues with Lighttpd'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>2</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-449148387520378422</id><published>2009-11-10T18:53:00.000-08:00</published><updated>2010-06-22T22:05:45.800-07:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='lusca'/><category scheme='http://www.blogger.com/atom/ns#' term='quagga'/><category scheme='http://www.blogger.com/atom/ns#' term='cacheboy'/><category scheme='http://www.blogger.com/atom/ns#' term='downtime'/><title type='text'>Cacheboy Update / October/November 2009</title><content type='html'>Howdy,&lt;br /&gt;&lt;br /&gt;Just a few updates this time around!&lt;br /&gt;&lt;ul&gt;&lt;li&gt;Cacheboy was pushing around 800-1200mbit during the Firefox 3.5.4 release cycle. I started to hit issues with the backend server not keeping up with revalidating requests and so I'll have to improve the edge caching logic a little more.&lt;/li&gt;&lt;li&gt;Lusca seems quite happy serving up 300-400mbit from a single node though; which is a big plus.&lt;br /&gt;&lt;/li&gt;&lt;li&gt;I've found some quite horrible memory leaks in Quagga on only one of the edge nodes. I'll have to find some time to login and debug this a little more.&lt;br /&gt;&lt;/li&gt;&lt;li&gt;The second backend server is now offically toast. I need to acquire another 1ru server with 2 SATA slots to magically appear in downtown Manhattan, NY.&lt;br /&gt;&lt;/li&gt;&lt;/ul&gt;&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-449148387520378422?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/449148387520378422/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2009/11/cacheboy-update-octobernovember-2009.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/449148387520378422'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/449148387520378422'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2009/11/cacheboy-update-octobernovember-2009.html' title='Cacheboy Update / October/November 2009'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-7273449207890945845</id><published>2009-10-08T18:38:00.000-07:00</published><updated>2010-06-22T22:05:45.809-07:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='cacheboy'/><category scheme='http://www.blogger.com/atom/ns#' term='downtime'/><title type='text'>Cacheboy downtime - hardware failures</title><content type='html'>Howdy,&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;I've had both backend servers fail today. One is throwing undervolt errors on one PSU line and is having disk issues (most likely related to an undervoltage); the other is just crashed.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;I'm waiting for remote hands to prod the other box into life.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;This is why I'd like some more donated equipment and hosting - I can make things much more fault tolerant. Hint hint.&lt;/div&gt;&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-7273449207890945845?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/7273449207890945845/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2009/10/cacheboy-downtime-hardware-failures.html#comment-form' title='2 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/7273449207890945845'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/7273449207890945845'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2009/10/cacheboy-downtime-hardware-failures.html' title='Cacheboy downtime - hardware failures'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>2</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-6390568478872446676</id><published>2009-09-30T23:39:00.000-07:00</published><updated>2010-03-09T21:32:52.924-08:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='lusca'/><category scheme='http://www.blogger.com/atom/ns#' term='squid'/><category scheme='http://www.blogger.com/atom/ns#' term='proxy'/><title type='text'></title><content type='html'>&lt;span class="Apple-style-span" style="font-family: 'Trebuchet MS', Trebuchet, Verdana, sans-serif; font-size: 13px; "&gt;Just a few Lusca related updates!&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;&lt;ul&gt;&lt;li&gt;All of the Cacheboy CDN nodes are running Lusca-HEAD now and are nice and stable.&lt;/li&gt;&lt;li&gt;I've deployed Lusca at a few customer sites and again, it is nice and stable.&lt;/li&gt;&lt;li&gt;The rebuild logic changes are, for the most part, nice and stable. There seems to be some weirdness with 32 vs 64 bit compilation options which I need to suss out but everything "just works" if you compile Lusca with large file/large cache file support regardless of the platform you're using. I may make that the default option.&lt;/li&gt;&lt;li&gt;I've got a couple of small coding projects to introduce a couple of small new features to Lusca - more on those when they're done!&lt;/li&gt;&lt;li&gt;Finally, I'm going to be migrating some more of the internal code over to use the sqinet_t type in preparation for IPv4/IPv6 agnostic support.&lt;/li&gt;&lt;/ul&gt;Stay Tuned!&lt;/div&gt;&lt;/span&gt;&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-6390568478872446676?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/6390568478872446676/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2009/09/just-few-lusca-related-updates-all-of.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/6390568478872446676'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/6390568478872446676'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2009/09/just-few-lusca-related-updates-all-of.html' title=''/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-3294324651404971566</id><published>2009-09-30T23:31:00.000-07:00</published><updated>2010-06-22T22:05:45.821-07:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='lusca'/><category scheme='http://www.blogger.com/atom/ns#' term='squid'/><category scheme='http://www.blogger.com/atom/ns#' term='proxy'/><title type='text'>Lusca updates - September 2009</title><content type='html'>Just a few Lusca related updates!&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;&lt;ul&gt;&lt;li&gt;All of the Cacheboy CDN nodes are running Lusca-HEAD now and are nice and stable.&lt;/li&gt;&lt;li&gt;I've deployed Lusca at a few customer sites and again, it is nice and stable.&lt;/li&gt;&lt;li&gt;The rebuild logic changes are, for the most part, nice and stable. There seems to be some weirdness with 32 vs 64 bit compilation options which I need to suss out but everything "just works" if you compile Lusca with large file/large cache file support regardless of the platform you're using. I may make that the default option.&lt;/li&gt;&lt;li&gt;I've got a couple of small coding projects to introduce a couple of small new features to Lusca - more on those when they're done!&lt;/li&gt;&lt;li&gt;Finally, I'm going to be migrating some more of the internal code over to use the sqinet_t type in preparation for IPv4/IPv6 agnostic support.&lt;/li&gt;&lt;/ul&gt;Stay Tuned!&lt;/div&gt;&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-3294324651404971566?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/3294324651404971566/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2009/09/lusca-updates-september-2009.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/3294324651404971566'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/3294324651404971566'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2009/09/lusca-updates-september-2009.html' title='Lusca updates - September 2009'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-2234576084184961814</id><published>2009-09-21T22:05:00.000-07:00</published><updated>2010-06-22T22:05:45.834-07:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='wishlist'/><category scheme='http://www.blogger.com/atom/ns#' term='cacheboy'/><title type='text'>My current wishlist</title><content type='html'>I'm going to put this on the website at some point, but I'm currently chasing a few things for Cacheboy:&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;&lt;ul&gt;&lt;li&gt;More US nodes. I'll take anything from 50mbit to 5gbit at this point. I need more US nodes to be able to handle enough aggregate traffic to make optimising the CDN content selection methods worthwhile.&lt;/li&gt;&lt;li&gt;Some donations to cover my upcoming APNIC membership for ASN and IPv4/IPv6 space. This will run to about AUD $3500 this year and then around AUD $2500 a year after that.&lt;/li&gt;&lt;li&gt;Some 1ru/2ru server hardware in the San Francisco area&lt;/li&gt;&lt;li&gt;Another site or two willing to run a relatively low bandwidth "master" mirror site. I have one site in New York but I'd prefer to run a couple of others spread around Europe and the United States.&lt;/li&gt;&lt;/ul&gt;I'm sure more will come to mind as I build things out a little more.&lt;/div&gt;&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-2234576084184961814?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/2234576084184961814/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2009/09/my-current-wishlist.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/2234576084184961814'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/2234576084184961814'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2009/09/my-current-wishlist.html' title='My current wishlist'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-8294057649256579078</id><published>2009-09-21T22:03:00.000-07:00</published><updated>2010-06-22T22:05:45.844-07:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='sugarlabs'/><category scheme='http://www.blogger.com/atom/ns#' term='olpc'/><category scheme='http://www.blogger.com/atom/ns#' term='cacheboy'/><title type='text'>New project - sugar labs!</title><content type='html'>I've just put the finishing touches on the basic sugar labs software repository. I'll hopefully be serving part or all of their software downloads shortly.&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;Sugar is the software behind the OLPC environment. It works on normal intel based PCs as far as I can tell.  More information can be found at http://www.sugarlabs.org/&lt;/div&gt;&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-8294057649256579078?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/8294057649256579078/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2009/09/new-project-sugar-labs.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/8294057649256579078'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/8294057649256579078'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2009/09/new-project-sugar-labs.html' title='New project - sugar labs!'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-2499103806641482535</id><published>2009-08-31T22:50:00.000-07:00</published><updated>2010-06-22T22:06:30.790-07:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='cacheboy'/><title type='text'>Cacheboy presentation at AUSNOG</title><content type='html'>I've just presented on Cacheboy at AUSNOG in Sydney. The feedback so far has been reasonably positive.&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;There's more information available at &lt;a href="http://www.creative.net.au/talks/"&gt;http://www.creative.net.au/talks/&lt;/a&gt;.&lt;/div&gt;&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-2499103806641482535?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/2499103806641482535/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2009/08/cacheboy-presentation-at-ausnog.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/2499103806641482535'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/2499103806641482535'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2009/08/cacheboy-presentation-at-ausnog.html' title='Cacheboy presentation at AUSNOG'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-942542918758251869</id><published>2009-08-17T19:32:00.000-07:00</published><updated>2010-06-22T22:05:45.862-07:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='news'/><category scheme='http://www.blogger.com/atom/ns#' term='cacheboy'/><title type='text'>Cacheboy status update</title><content type='html'>So by and large, the pushing of bits is working quite well. I have a bunch of things to tidy up and a DNS backend to rewrite in C or C++ but that won't stop the bits from being pushed.&lt;br /&gt;&lt;br /&gt;Unfortunately what I'm now lacking is US hosts to send traffic from. I still have more Europe and Asian connectivity than North American - and North America is absolutely where I need connectivity the most. Right now I'm only able to push 350-450 megabits of content from North America - and this puts a big, big limit on how much content I can serve overall.&lt;br /&gt;&lt;br /&gt;Please contact me as soon as possible if you're interested in hosting a node in North America. I ideally need enough nodes to push between a gigabit and ten gigabits of traffic.&lt;br /&gt;&lt;br /&gt;I will be able to start pushing noticable amounts of content out of regional areas once I've sorted out North America. This includes places like Australia, Africa, South America and Eastern Europe. I'd love to be pushing more open source bits out of those locations to keep the transit use low but I just can't do so at the moment.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-942542918758251869?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/942542918758251869/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2009/08/cacheboy-status-update.html#comment-form' title='2 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/942542918758251869'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/942542918758251869'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2009/08/cacheboy-status-update.html' title='Cacheboy status update'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>2</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-6465064986962087447</id><published>2009-08-17T19:30:00.000-07:00</published><updated>2010-06-22T22:05:45.873-07:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='news'/><category scheme='http://www.blogger.com/atom/ns#' term='cacheboy'/><title type='text'>Canada node online and pushing bits!</title><content type='html'>The Canada/TORIX node is online thanks to John Nistor at &lt;a href="http://www.prioritycolo.com/"&gt;prioritycolo&lt;/a&gt; in Toronto, Canada.&lt;br /&gt;&lt;br /&gt;Thanks John!&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-6465064986962087447?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/6465064986962087447/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2009/08/canada-node-online-and-pushing-bits.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/6465064986962087447'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/6465064986962087447'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2009/08/canada-node-online-and-pushing-bits.html' title='Canada node online and pushing bits!'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-650588384586696438</id><published>2009-08-17T18:55:00.000-07:00</published><updated>2010-06-22T22:06:30.792-07:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='cacheboy'/><title type='text'>Cacheboy is on WAIX!</title><content type='html'>Yesterday's traffic from mirror1.au into WAIX:&lt;br /&gt;&lt;table cellpadding="1" cellspacing="1"&gt;&lt;tbody&gt;&lt;tr&gt;&lt;th&gt;ASN&lt;/th&gt;&lt;th&gt;MBytes&lt;/th&gt;&lt;th&gt;Requests&lt;/th&gt;&lt;th&gt;% of overall&lt;/th&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;AS7545&lt;/td&gt;&lt;td&gt;17946.77&lt;/td&gt;&lt;td&gt;7437&lt;/td&gt;&lt;td&gt;29.85&lt;/td&gt;&lt;td&gt;TPG-INTERNET-AP TPG Internet Pty Ltd&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;AS4802&lt;/td&gt;&lt;td&gt;12973.47&lt;/td&gt;&lt;td&gt;4476&lt;/td&gt;&lt;td&gt;21.58&lt;/td&gt;&lt;td&gt;ASN-IINET iiNet Limited&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;AS4739&lt;/td&gt;&lt;td&gt;8497.92&lt;/td&gt;&lt;td&gt;2947&lt;/td&gt;&lt;td&gt;14.13&lt;/td&gt;&lt;td&gt;CIX-ADELAIDE-AS Internode Systems Pty Ltd&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;AS9543&lt;/td&gt;&lt;td&gt;2524.57&lt;/td&gt;&lt;td&gt;1241&lt;/td&gt;&lt;td&gt;4.20&lt;/td&gt;&lt;td&gt;WESTNET-AS-AP Westnet Internet Services&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;AS4854&lt;/td&gt;&lt;td&gt;2097.32&lt;/td&gt;&lt;td&gt;941&lt;/td&gt;&lt;td&gt;3.49&lt;/td&gt;&lt;td&gt;NETSPACE-AS-AP Netspace Online Systems&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;AS17746&lt;/td&gt;&lt;td&gt;1881.17&lt;/td&gt;&lt;td&gt;1050&lt;/td&gt;&lt;td&gt;3.13&lt;/td&gt;&lt;td&gt;ORCONINTERNET-NZ-AP Orcon Internet&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;AS9822&lt;/td&gt;&lt;td&gt;1425.44&lt;/td&gt;&lt;td&gt;456&lt;/td&gt;&lt;td&gt;2.37&lt;/td&gt;&lt;td&gt;AMNET-AU-AP Amnet IT Services Pty Ltd&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;AS17435&lt;/td&gt;&lt;td&gt;1161.01&lt;/td&gt;&lt;td&gt;411&lt;/td&gt;&lt;td&gt;1.93&lt;/td&gt;&lt;td&gt;WXC-AS-NZ WorldxChange Communications LTD&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;AS9443&lt;/td&gt;&lt;td&gt;1140.62&lt;/td&gt;&lt;td&gt;701&lt;/td&gt;&lt;td&gt;1.90&lt;/td&gt;&lt;td&gt;INTERNETPRIMUS-AS-AP Primus Telecommunications&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;AS7657&lt;/td&gt;&lt;td&gt;891.93&lt;/td&gt;&lt;td&gt;1187&lt;/td&gt;&lt;td&gt;1.48&lt;/td&gt;&lt;td&gt;VODAFONE-NZ-NGN-AS Vodafone NZ Ltd.&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;AS7718&lt;/td&gt;&lt;td&gt;740.74&lt;/td&gt;&lt;td&gt;272&lt;/td&gt;&lt;td&gt;1.23&lt;/td&gt;&lt;td&gt;TRANSACT-SDN-AS TransACT IP Service Provider&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;AS7543&lt;/td&gt;&lt;td&gt;732.11&lt;/td&gt;&lt;td&gt;423&lt;/td&gt;&lt;td&gt;1.22&lt;/td&gt;&lt;td&gt;PI-AU Pacific Internet (Australia) Pty Ltd&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;AS24313&lt;/td&gt;&lt;td&gt;527.38&lt;/td&gt;&lt;td&gt;252&lt;/td&gt;&lt;td&gt;0.88&lt;/td&gt;&lt;td&gt;NSW-DET-AS NSW Department of Education and Training&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;AS9790&lt;/td&gt;&lt;td&gt;436.80&lt;/td&gt;&lt;td&gt;389&lt;/td&gt;&lt;td&gt;0.73&lt;/td&gt;&lt;td&gt;CALLPLUS-NZ-AP CallPlus Services Limited&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;AS17412&lt;/td&gt;&lt;td&gt;365.13&lt;/td&gt;&lt;td&gt;228&lt;/td&gt;&lt;td&gt;0.61&lt;/td&gt;&lt;td&gt;WOOSHWIRELESSNZ Woosh Wireless&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;AS17486&lt;/td&gt;&lt;td&gt;349.27&lt;/td&gt;&lt;td&gt;116&lt;/td&gt;&lt;td&gt;0.58&lt;/td&gt;&lt;td&gt;SWIFTEL1-AP People Telecom Pty. Ltd.&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;AS17808&lt;/td&gt;&lt;td&gt;311.65&lt;/td&gt;&lt;td&gt;248&lt;/td&gt;&lt;td&gt;0.52&lt;/td&gt;&lt;td&gt;VODAFONE-NZ-AP AS number for Vodafone NZ IP Networks&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;AS24093&lt;/td&gt;&lt;td&gt;303.40&lt;/td&gt;&lt;td&gt;114&lt;/td&gt;&lt;td&gt;0.50&lt;/td&gt;&lt;td&gt;BIGAIR-AP BIGAIR. Multihoming ASN&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;AS9889&lt;/td&gt;&lt;td&gt;288.85&lt;/td&gt;&lt;td&gt;197&lt;/td&gt;&lt;td&gt;0.48&lt;/td&gt;&lt;td&gt;MAXNET-NZ-AP Auckland&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;AS17705&lt;/td&gt;&lt;td&gt;282.49&lt;/td&gt;&lt;td&gt;84&lt;/td&gt;&lt;td&gt;0.47&lt;/td&gt;&lt;td&gt;INSPIRENET-AS-AP InSPire Net Ltd&lt;/td&gt;&lt;/tr&gt;&lt;/tbody&gt;&lt;/table&gt;&lt;br /&gt;Query content served: 54878.07 mbytes; 23170 requests.&lt;br /&gt;Total content served: 60123.25 mbytes; 28037 requests.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-650588384586696438?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/650588384586696438/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2009/08/cacheboy-is-on-waix.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/650588384586696438'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/650588384586696438'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2009/08/cacheboy-is-on-waix.html' title='Cacheboy is on WAIX!'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-2193032414546437669</id><published>2009-08-17T18:33:00.000-07:00</published><updated>2010-06-22T22:05:45.896-07:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='dns'/><category scheme='http://www.blogger.com/atom/ns#' term='BGP'/><category scheme='http://www.blogger.com/atom/ns#' term='cacheboy'/><title type='text'>BGP aware DNS</title><content type='html'>I've just written up the first "test" hack of BGP aware DNS.&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;The basic logic is simple but evil. I'm simply mapping BGP next-hop to a set of weighted servers. A server is then randomly chosen from this pool.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;I'm not doing this for -all- prefixes and POPs - it is only being used for two specific POPs where there is a lot of peering and almost no transit. There are a few issues regarding split horizon BGP/DNS and request routing which I'd like to fully sort out before I enable it for everything. I don't want a quirk to temporarily redirect -all- requests to -one- server cluster!&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;In any case, the test is working well. I'm serving ~10mbit to WAIX (Western Australia) and ~ 30mbit to TORIX (Toronto, Canada.)&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;All of the DNS based redirection caveats apply - most certainly that not all client requests to the caches will also be over peering. I'll have to craft some method(s) of tracking this.&lt;/div&gt;&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-2193032414546437669?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/2193032414546437669/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2009/08/bgp-aware-dns.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/2193032414546437669'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/2193032414546437669'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2009/08/bgp-aware-dns.html' title='BGP aware DNS'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-2408490445634240369</id><published>2009-08-16T05:43:00.000-07:00</published><updated>2010-03-09T21:32:52.955-08:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='squid'/><title type='text'>Squid-3 isn't a rewrite!</title><content type='html'>G'day,&lt;br /&gt;&lt;br /&gt;There seems to be this strange misconception that Squid-3 is a "rewrite" of Squid in C++. I am not sure where this particular little tidbit gets copy/pasted from but just for the record:&lt;br /&gt;&lt;br /&gt;Squid-3 is the continuation of Squid-2.5, made to compile using the GNU C++ compiler. It is not a rewrite.&lt;br /&gt;&lt;br /&gt;If Squid-3 -were- a rewrite, and the resultant code -was- as much of a crappy-performing, bastardised C/C++ hybrid, then I'd have suggested the C++ coders in question need to relearn C++. Luckily for them, the codebase is a hybrid of C and C++ because it did just start as a C codebase with bits and pieces part-migrated to C++.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-2408490445634240369?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/2408490445634240369/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2009/08/squid-3-isn-rewrite.html#comment-form' title='1 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/2408490445634240369'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/2408490445634240369'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2009/08/squid-3-isn-rewrite.html' title='Squid-3 isn&amp;#39;t a rewrite!'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>1</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-1839983628922508442</id><published>2009-08-09T11:28:00.000-07:00</published><updated>2010-06-22T22:05:45.920-07:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='cacheboy'/><title type='text'>Updates - or why I've not been doing very much</title><content type='html'>G'day! Cacheboy has been running on autopilot for the last couple of months whilst I've been focusing on paid work and growing my little company. So far (mostly) so good there.&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;The main issue scaling traffic has been the range request handling in Squid/Lusca, so I've been working on fixing things up "just enough" to make it work in the firefox update environment. I think I've finally figured it out - and figured out the bugs in the range request handling in Squid too! - so I'll push out some updates to the network next week and throw it some more traffic.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;I really am hoping to ramp traffic up past the gigabit mark once this is done. We'll just have to see!&lt;/div&gt;&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-1839983628922508442?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/1839983628922508442/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2009/08/updates-or-why-i-not-been-doing-very.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/1839983628922508442'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/1839983628922508442'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2009/08/updates-or-why-i-not-been-doing-very.html' title='Updates - or why I&amp;#39;ve not been doing very much'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-879531005881453874</id><published>2009-08-06T23:57:00.000-07:00</published><updated>2010-03-09T21:32:52.979-08:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='lusca'/><category scheme='http://www.blogger.com/atom/ns#' term='ipv6'/><category scheme='http://www.blogger.com/atom/ns#' term='proxy'/><title type='text'>Preparation for next release; IPv6 checklist</title><content type='html'>I've been slowly working on tidying up the codebase before the next snapshot release. I've been avoiding doing further large scale code reorganisation until I'm confident that this codebase is as stable and performs as well as it should.&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;I'll hopefully have the next stable snapshot online tonight. I'll then re-evaluate where things are at right now and come up with a short-list of things to do over the next couple of weeks. It'll almost certainly be the remainder of the IPv6 preparation work - I'd like to prepare the last few bits of infrastructure for IPv6 - and make certain that is all stable before I start converting the client-side and server-side code to actively using the IPv6 routines.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;The current IPv6 shortlist, if I decide to do it:&lt;/div&gt;&lt;div&gt;&lt;ol&gt;&lt;li&gt;client database code - convert to a radix tree instead of a hash on the IP address; make IPv4/IPv6 agnostic.&lt;/li&gt;&lt;li&gt;persistent connection code - up the pconn hash key length to fit the text version of the IPv6 address. I'll worry about migrating the pconn code to a tree later on.&lt;/li&gt;&lt;li&gt;Importing the last remaining bits of the IPv6 related code into the internal DNS code.&lt;/li&gt;&lt;li&gt;Make sure the internal and external DNS choices both function properly when handling IPv6 addresses for forward and reverse lookups.&lt;/li&gt;&lt;li&gt;Import the IP protocol ACL type and IPv6 address ACL types - src6 and dst6.&lt;/li&gt;&lt;li&gt;Modify the ACL framework to use the IPv6 datatype instead of "sockaddr_in" and "inaddr" structs; then enable src6/dst6.&lt;/li&gt;&lt;li&gt;Make certain the source and destination hostname ACLs function correctly for both IPv4 and IPv6.&lt;/li&gt;&lt;li&gt;Test, test, test!&lt;/li&gt;&lt;/ol&gt;The last time I did a "hack" conversion to support IPv6 client side code I found a number of places which expected a newly-allocated struct to be zero'ed, and thus the "in_addr" embedded inside it to be INADDR_ANY. This caused some crashes to occur in production testing. I'm thus going to hold off on pushing through the IPv6 client side changes (which are actually surprisingly simple once the above is done!) until I've enumerated and fixed all of those particular nightmares.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;The IPv6 server-side stuff is a whole different barrel of fun. I'm going to ignore a lot of that for now until I've made certain the client-side code is stable and performing as well as the current IPv4-only code.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;I don't even want to think about the FTP related changes that need to occur. I may leave the FTP support IPv4 only until someone asks (nicely) about it. The FTP code is rife with C string pointer manipulations which need to be rewritten to use the provided string primitives. I'd really like to do -that- before I consider upgrading it to handle IPv6.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;Anyway. Lots to do, not enough spare time to do it all in.&lt;/div&gt;&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-879531005881453874?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/879531005881453874/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2009/08/preparation-for-next-release-ipv6.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/879531005881453874'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/879531005881453874'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2009/08/preparation-for-next-release-ipv6.html' title='Preparation for next release; IPv6 checklist'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-5566419902337327572</id><published>2009-07-28T01:03:00.000-07:00</published><updated>2010-03-09T21:32:53.014-08:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='caching'/><category scheme='http://www.blogger.com/atom/ns#' term='lusca'/><category scheme='http://www.blogger.com/atom/ns#' term='coss'/><category scheme='http://www.blogger.com/atom/ns#' term='squid'/><category scheme='http://www.blogger.com/atom/ns#' term='proxy'/><title type='text'>Updates - rebuild logic, peering and COSS work</title><content type='html'>I've committed the initial modifications to the storage rebuilding code. The changes mostly live in the AUFS and COSS code - the rest of Lusca isn't affected.&lt;br /&gt;&lt;br /&gt;The change pushes the rebuild logic itself into external helpers which simply stream swaplog entries to the main process. Lusca doesn't care how the swaplog entries are generated.&lt;br /&gt;&lt;br /&gt;The external helper method is big boost for AUFS. Each storedir creates a single rebuild helper process which can block on disk IO without blocking anything else. The original code in Squid will do a little disk IO work at a time - which almost always involved blocking the process until said disk IO completed.&lt;br /&gt;&lt;br /&gt;The main motivation of this work was the removal of a lot of really horrible, twisty code and further modularisation of the codebase. The speedups to the rebuild process are a nice side-effect. The next big improvement will be sorting out how the swap logs are written. Fixing that will be key to allowing enormous caches to properly function without log rotation potentially destroying the proxy service.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-5566419902337327572?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/5566419902337327572/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2009/07/updates-rebuild-logic-peering-and-coss.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/5566419902337327572'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/5566419902337327572'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2009/07/updates-rebuild-logic-peering-and-coss.html' title='Updates - rebuild logic, peering and COSS work'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-4870785848736867378</id><published>2009-07-13T20:19:00.000-07:00</published><updated>2010-03-09T21:32:53.037-08:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='caching'/><category scheme='http://www.blogger.com/atom/ns#' term='windowsupdates'/><category scheme='http://www.blogger.com/atom/ns#' term='lusca'/><title type='text'>Caching Windows Updates</title><content type='html'>&lt;span class="Apple-style-span" style="font-family: 'Trebuchet MS', Trebuchet, Verdana, sans-serif; font-size: 13px; "&gt;There are two issues with caching windows updates in squid/lusca:&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;* the requests for data themselves are all range requests, which means the content is never cached in Squid/Lusca;&lt;/div&gt;&lt;div&gt;* the responses contain validation information (eg ETags) but the object is -always- returned regardless of whether the validators match or not.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;This feels a lot like Google Maps who did the same thing with revalidation. Grr.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;I'm not sure why Microsoft (and Google!) did this with their web services. I'll see if I can find someone inside Microsoft who can answer questions about the Windows Update related stuff to see if it is intentional (and document why) or whether it is an oversight which they would be interested in fixing.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;In any case, I'm going to fix it for the handful of commercial supported customers which I have here.&lt;/div&gt;&lt;/span&gt;&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-4870785848736867378?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/4870785848736867378/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2009/07/caching-windows-updates.html#comment-form' title='5 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/4870785848736867378'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/4870785848736867378'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2009/07/caching-windows-updates.html' title='Caching Windows Updates'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>5</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-1108617877363223965</id><published>2009-07-08T09:53:00.000-07:00</published><updated>2010-06-22T22:05:45.935-07:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='cdn'/><category scheme='http://www.blogger.com/atom/ns#' term='videolan'/><category scheme='http://www.blogger.com/atom/ns#' term='cacheboy'/><title type='text'>VLC 1.0 released</title><content type='html'>VLC-1.0 has been released. The CDN is pushing out between 550 and 700mbit of VLC downloads. I'm sure it can do more but as I'm busy working elsewhere, I'm going to be overly conservative and leave the mirror weighting where it is.&lt;br /&gt;&lt;br /&gt;Graphs to follow!&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-1108617877363223965?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/1108617877363223965/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2009/07/vlc-10-released.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/1108617877363223965'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/1108617877363223965'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2009/07/vlc-10-released.html' title='VLC 1.0 released'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-2460901424720608704</id><published>2009-07-08T01:49:00.000-07:00</published><updated>2010-03-09T21:32:53.057-08:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='lusca'/><title type='text'>Storage rebuilding / logging project - proposal</title><content type='html'>I've put forward a basic proposal to the fledgling Lusca community to get funding to fix up the storage logging and rebuilding code.&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;Right now the storage logging (ie, "swap.state" logging) is done using synchronous IO and this starts to lag Lusca if there is a lot of disk file additions/deletions. It also takes a -long- time to rotate the store swap log (which effectively halts the proxy whilst the logs are rotated) and an even longer time to rebuild the cache index at startup.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;I've braindumped the proposal here - http://code.google.com/p/lusca-cache/wiki/ProjectStoreRebuildChanges .&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;Now, the good news is that I've implemented the rebuild helper programs and the results are -fantastic-. UFS cache dirs will still take forever to rebuild if the logfile doesn't exist or is corrupt but the helper programs speed this up by a factor of "LOTS". It also parallelises correctly - if you have 15 disks and you aren't hitting CPU/bus/controller limits, all the cache dirs will rebuild at full speed in parallel.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;Rebuilding from the log files takes seconds rather than minutes.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;Finally, I've sketched out how to solve the COSS startup/rebuild times and the even better news is that fixing the AUFS rebuild code will give me about 90% of what I need to fix COSS.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;The bad news is that integrating this into the Lusca codebase and fixing up the rebuild process to take advantage of this parallelism is going to take 4 to 6 weeks of solid work. I'm looking for help from the community (and other interested parties) who would like to see this work go in. I have plenty of testers but nothing to help -coding- along and I unfortunately have to focus on projects that provide me with some revenue.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;Please contact me if you're able to help with either coding or funding for this.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-2460901424720608704?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/2460901424720608704/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2009/07/storage-rebuilding-logging-project.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/2460901424720608704'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/2460901424720608704'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2009/07/storage-rebuilding-logging-project.html' title='Storage rebuilding / logging project - proposal'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4361733293216582854.post-3269326807735441212</id><published>2009-06-29T01:36:00.000-07:00</published><updated>2010-06-22T22:05:45.952-07:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='cacheboy'/><category scheme='http://www.blogger.com/atom/ns#' term='downtime'/><title type='text'>Current Downtime/issues</title><content type='html'>&lt;div&gt;There's a current issue with content not being served correctly. It stemmed from a ZFS related panic on one of the backend servers (note to self - update to the very latest FreeBSD-7-stable code; these are all fixed!) which then came up with lighttpd but no ZFS mounts. Lighttpd then started returning 404's.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;I'm now watching the backend(s) throw random connection failures and the Lusca caches then cache an error rather than the object.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;I've fixed the backend giving trouble so it won't start up in that failed mode again and I've set the negative caching in the Lusca cache nodes to 30 seconds instead of the default 5 minutes. Hopefully the traffic levels now pick up to where its supposed to be.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;EDIT: The problem is again related to the Firefox range requests and Squid/Lusca's inability to cache range request fragments.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;The backend failure(s) removed the objects from the cache. The problem now is that the objects aren't re-entering the cache because they are all range requests.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;I'm going to wind down the Firefox content serving for now until I get some time to hack up Lusca "enough" to cache the range request objects. I may just do something dodgy with the URL rewriter to force a full object request to occur in the background. Hm, actually..&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4361733293216582854-3269326807735441212?l=adrianchadd.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://adrianchadd.blogspot.com/feeds/3269326807735441212/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://adrianchadd.blogspot.com/2009/06/current-downtimeissues.html#comment-form' title='2 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/3269326807735441212'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4361733293216582854/posts/default/3269326807735441212'/><link rel='alternate' type='text/html' href='http://adrianchadd.blogspot.com/2009/06/current-downtimeissues.html' title='Current Downtime/issues'/><author><name>Adrian</name><uri>http://www.blogger.com/profile/17496219706861321916</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='33' height='25' src='http://bp2.blogger.com/_37wobiQ3zUs/SAKidE1C46I/AAAAAAAAAAM/F065c-5eNR8/S220/2004-12-17-adrian.jpg'/></author><thr:total>2</thr:total></entry></feed>
