From: "Gopher-Project" <gopher-project-bounces+rachael=telefisk.org@lists.alioth.debian.org> Date: Fri Sep 13 15:56:59 2013 Subject: Re: [gopher] Joining in: I'm the maintainer/host of Gopher Proxy Evert, nice to see you here. Thanks for everything on meulie.net; it's a good treasure trove. I see Veronica-2 is busily reindexing it as we speak. > Hmm, blocking all spidering will keep search engines from indexing > Gopher-servers, some of which contain a treasure of documents from times > gone by. > > How about I only let Google in? (Does anyone use any other search engine > anyway?) This seems like a reasonable compromise to me personally, though for the record the Floodgap proxy has <meta> tags up to tell all robots not to index, even Google, and IPs or user agents that demonstrate spidering activity wind up in a blacklist when I discover them (my custom log-analysis software highlights unusually high levels of activity). But, as long as some enforcement were run on the other end to deal with bots that ignore robots.txt, speaking with my gopher server operator hat on I'd be fine with letting Google still continue to index through. > All credit goes to Stephen Morley. I merely took over his creation after > he decided to discontinue hosting. I'm glad to see you took it over. Stephen and I didn't get along so well on proxy policy, as you may have seen from previous messages, but he did a very good job designing it (much better than my strictly utilitarian effort). Any decision "the proxy cartel" comes to should also involve the pongonova proxy (Brian Koontz) since his is probably #3 in the traffic hierarchy. -- ------------------------------------ personal: http://www.cameronkaiser.com/ -- Cameron Kaiser * Floodgap Systems * www.floodgap.com * ckaiser@floodgap.com -- The early bird may get the worm, but the second mouse gets the cheese. ----- _______________________________________________ Gopher-Project mailing list Gopher-Project@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/gopher-project Thread start (DIR) [gopher] Joining in: I'm the maintainer/host of Gopher Proxy (DIR) Followup: Re: [gopher] Joining in: I'm the maintainer/host of Gopher Proxy