Monday, July 13, 2009

Caching Windows Updates

There are two issues with caching windows updates in squid/lusca:

* the requests for data themselves are all range requests, which means the content is never cached in Squid/Lusca;
* the responses contain validation information (eg ETags) but the object is -always- returned regardless of whether the validators match or not.

This feels a lot like Google Maps who did the same thing with revalidation. Grr.

I'm not sure why Microsoft (and Google!) did this with their web services. I'll see if I can find someone inside Microsoft who can answer questions about the Windows Update related stuff to see if it is intentional (and document why) or whether it is an oversight which they would be interested in fixing.

In any case, I'm going to fix it for the handful of commercial supported customers which I have here.

5 comments:

  1. fixing this goes inside of Lusca or some external url_rewriter as i did for my windows updates !???

    Much Regards

    ReplyDelete
  2. It's an external script which sits in the background and (re)-fetches updates into Lusca as needed. Lusca itself is quite capable of caching that kind of content.

    ReplyDelete
  3. hello! iam a developer of www.thundercache.org - we create a url_rewriter to fix this problem like windows update, youtube, etc.

    I want to write a patch to Lusca, adding a functionality to create acls to fix this problem, without url_rewrite. What do you think? how i can make contact with you?

    ReplyDelete
  4. Hi Adrian,

    We've done the same for some of our commericial clients using a url_rewrite and a completely seperate daemon to manage the content.

    Works like a charm - in the end it appeared simpler to manage this outside of the squid/lusca code than to fight 206 PARTIALS ;-)

    I'd be very interested to hear feedback from you Microsoft questions.

    ReplyDelete
  5. Rodrigo: just email me by googling for "adrian chadd" and emailing that address :)

    Or post to lusca-users@googlegroups.com and begin a discussion.

    ReplyDelete