From: gopher-project-bounces+rachael=telefisk.org@lists.alioth.debian.org Date: Sat Dec 5 23:36:16 2009 Subject: Re: [gopher] New Gopher Proxy > > > > In a google search for "gopher proxy" I happened to stumble across this: > > > > http://gopher-proxy.stephenmorley.org/. > > > > > > This is great. > > > Now if we can just get some of the http-based gopher sites to start > > > listing these proxies, they might get picked up by the search engines. > > > > ... as long as the search engines don't start indexing them *through* the > > proxies. I've had services get DOSed by bot traffic, and I've had to block > > proxies that did (I'm not naming names; that was handled under the table). > > It seems to me that this would be a good thing, and that > responsibility for limiting bots on sensitive sites would be the > domain of the site operator, not the proxy operator. I completely disagree. Proxies shouldn't be indexable. Just like open mail proxies foster abuse on mail servers, improperly managed gopher and HTTP proxies foster the same abuse on the servers they target, inadvertently or otherwise. More to the point, bots that don't understand what gopher is will simply crawl over and over, amplifying traffic. This is why I have blocked misconfigured proxies in the past, and will do in the future if I detect them. For example, the Twitpher script got DOSed multiple times by a proxy I won't mention due to HTTP spiders running over and over it, requiring me blocking the proxy to restore service (leading to a nasty exchange with the proxy owner) as the proxy exceeded Twitter's anonymous rate limit, which was not fixed until I requested special dispensation from Twitter to make more requests. If those had been legitimate users, that would have been one thing, but these were mindless, automatic bot queries. That's a waste of resources and cycles. -- ------------------------------------ personal: http://www.cameronkaiser.com/ -- Cameron Kaiser * Floodgap Systems * www.floodgap.com * ckaiser@floodgap.com -- And if I claim to be a wise man/it surely means that I don't know. -- Kansas _______________________________________________ Gopher-Project mailing list Gopher-Project@lists.alioth.debian.org http://lists.alioth.debian.org/mailman/listinfo/gopher-project Thread start (DIR) [gopher] New Gopher Proxy (DIR) Followup: Re: [gopher] New Gopher Proxy (DIR) Followup: Re: [gopher] New Gopher Proxy