portaudit broken again (maybe fetch(1) problem, too)
in [FreeBSD]
|
Prev: NYC LOCAL: Saturday 23 August 2008 HarvestWorks: Dan St.Clair: Introduction to Supercollider
Next: Installing on Dell GX110
From: Christoph Weber-Fahr on 21 Aug 2008 17:19 Hello, Henri Hennebert wrote: > Christoph Weber-Fahr wrote: >> Henri Hennebert wrote: >>> Christoph Weber-Fahr wrote: >>>> web1# [~] portaudit -Fad >>>> auditfile.tbz 100% of 49 kB 930 kBps >>>> portaudit: Database too old. >>>> Old database restored. > Obviously, the proxy don't querry the last modification time of the file > and deliver a local copy. Yep, looks like that. In my case it is a squid 2.6.STABLE19, wich is chained through another squid. Apparently it had a broken copy, since it always delivered it with the most recent date. Fortunatley it was my squid, so I could just reset its cache. Now it behaves correctly, consistently delivering the current file witha correct mtime. But fetch seems to be guilty, too. See this transcript (I added -vv in the fetch command in portaudit.conf): > portaudit -Fad > scheme: [http] > user: [] > password: [] > host: [www.FreeBSD.org] > port: [0] > document: [/ports/auditfile.tbz] > scheme: [http] > user: [] > password: [] > host: [127.0.0.1] > port: [8000] > document: [/] > ---> 127.0.0.1:8000 > looking up 127.0.0.1 > connecting to 127.0.0.1:8000 That one is ssh-portforwarded to the proxy: > requesting http://www.FreeBSD.org/ports/auditfile.tbz >>>> GET http://www.FreeBSD.org/ports/auditfile.tbz HTTP/1.1 >>>> Host: www.FreeBSD.org >>>> User-Agent: fetch libfetch/2.0 >>>> Connection: close >>>> > <<< HTTP/1.0 200 OK > <<< Content-Type: application/x-bzip-compressed-tar > <<< Accept-Ranges: bytes > <<< ETag: "-1789851444" > <<< Last-Modified: Tue, 24 Jun 2008 12:40:02 GMT > last modified: [2008-06-24 12:40:02] Here, apparently, fetch could have learnt the right time - but didn't. Yeah, its badly outdated - no idea why squid doesn't attempt an update of that file. > <<< Content-Length: 50554 > content length: [50554] > <<< Date: Tue, 24 Jun 2008 12:43:21 GMT > <<< Server: httpd/1.4.x LaHonda > <<< X-Cache: MISS from proxy4 > <<< Age: 200385 > <<< X-Cache: HIT from mybrokenproxy.xx.xx > <<< Via: 1.0 proxy4:8000 (squid), 1.0 mybrokenproxy.xx.xx:8000 (squid/2.6.STABLE19) > <<< Proxy-Connection: close > <<< > offset 0, length -1, size -1, clength 50554 > local size / mtime: 50554 / 1219349222 > remote size / mtime: 50554 / 1214311202 See the mtime value here. For reasons unknown to me fetch decides to set a different local mtime. > auditfile.tbz 100% of 49 kB 308 kBps > portaudit: Database too old. > Old database restored. > portaudit: Download failed. And I just reproduced that after the cleaned cache. Apparently with chained proxies squid can't update the thing properly and keeps an old copy around. Looks Fishy. Maybe someone with more http and caching know how can add details here? Regards Christoph Weber-Fahr
From: Henri Hennebert on 23 Aug 2008 07:05
Christoph Weber-Fahr wrote: > Hello, > > Henri Hennebert wrote: >> Christoph Weber-Fahr wrote: >>> Henri Hennebert wrote: >>>> Christoph Weber-Fahr wrote: >>>>> web1# [~] portaudit -Fad >>>>> auditfile.tbz 100% of 49 kB 930 >>>>> kBps >>>>> portaudit: Database too old. >>>>> Old database restored. > >> Obviously, the proxy don't querry the last modification time of the >> file and deliver a local copy. > > Yep, looks like that. In my case it is a squid 2.6.STABLE19, > wich is chained through another squid. > > Apparently it had a broken copy, since it always delivered it > with the most recent date. > > Fortunatley it was my squid, so I could just reset its cache. Now it > behaves correctly, consistently delivering the current file witha > correct mtime. > > But fetch seems to be guilty, too. See this transcript (I added -vv > in the fetch command in portaudit.conf): > >> portaudit -Fad >> scheme: [http] >> user: [] >> password: [] >> host: [www.FreeBSD.org] >> port: [0] >> document: [/ports/auditfile.tbz] >> scheme: [http] >> user: [] >> password: [] >> host: [127.0.0.1] >> port: [8000] >> document: [/] >> ---> 127.0.0.1:8000 >> looking up 127.0.0.1 >> connecting to 127.0.0.1:8000 > > That one is ssh-portforwarded to the proxy: > >> requesting http://www.FreeBSD.org/ports/auditfile.tbz >>>>> GET http://www.FreeBSD.org/ports/auditfile.tbz HTTP/1.1 >>>>> Host: www.FreeBSD.org >>>>> User-Agent: fetch libfetch/2.0 >>>>> Connection: close >>>>> >> <<< HTTP/1.0 200 OK >> <<< Content-Type: application/x-bzip-compressed-tar >> <<< Accept-Ranges: bytes >> <<< ETag: "-1789851444" >> <<< Last-Modified: Tue, 24 Jun 2008 12:40:02 GMT >> last modified: [2008-06-24 12:40:02] > > Here, apparently, fetch could have learnt the right time - but didn't. > Yeah, its badly outdated - no idea why squid doesn't attempt an update > of that file. > >> <<< Content-Length: 50554 >> content length: [50554] >> <<< Date: Tue, 24 Jun 2008 12:43:21 GMT >> <<< Server: httpd/1.4.x LaHonda >> <<< X-Cache: MISS from proxy4 >> <<< Age: 200385 >> <<< X-Cache: HIT from mybrokenproxy.xx.xx >> <<< Via: 1.0 proxy4:8000 (squid), 1.0 mybrokenproxy.xx.xx:8000 >> (squid/2.6.STABLE19) >> <<< Proxy-Connection: close >> <<< >> offset 0, length -1, size -1, clength 50554 >> local size / mtime: 50554 / 1219349222 >> remote size / mtime: 50554 / 1214311202 > > See the mtime value here. For reasons unknown to me fetch decides to set a > different local mtime. I test and find that the file already exist locally and fetch simply display it's mtime (which is `date -u -r 1219349222` = Thu Aug 21 20:07:02 UTC 2008) and then replace it with the stale copy comming from the proxy. You can reproduce it like this: cd /tmp fetch -v -v http://www.freebsd.org/ports/auditfile.tbz ls -l auditfile.tbz touch auditfile.tbz fetch -v -v http://www.freebsd.org/ports/auditfile.tbz ls -l auditfile.tbz >> auditfile.tbz 100% of 49 kB 308 kBps >> portaudit: Database too old. >> Old database restored. >> portaudit: Download failed. > > And I just reproduced that after the cleaned cache. Apparently with chained > proxies squid can't update the thing properly and keeps an old copy around. try to purge the entry with: squidclient -u admin_user -w admin_password -m purge http://www.FreeBSD.org/ports/auditfile.tbz on mybrokenproxy.xx.xx Henri > > Looks Fishy. > Maybe someone with more http and caching know how can add details here? > > > Regards > > Christoph Weber-Fahr |