GopherCon '93: Internet Gopher Workshop and Internet Gopher Conference April 12-13, 1993 Minneapolis, Minnesota Prentiss Riddle riddle@rice.edu CopherCon '93 consisted of two events: an invitation-only workshop for about 75 participants focusing on technical details mostly related to the Gopher+ development effort, and a larger conference of some 250 participants covering a broader range of topics including CWIS and library applications of Internet Gopher. Both events were jointly sponsored by the University of Minnesota and by CICnet. Day 1: the Workshop GOPHER HISTORY: After greetings from our hosts, things got rolling with reports on the past, present and future of Gopher from the UMinn GopherTeam. Farhad Anklesaria began by recounting the history of Gopher development: UMinn's first attempt to design a CWIS resulted in a classic design-by-committee monstrosity, to which the Gopher Team responded by proposing a very simple protocol which they believed they could actually implement. The proposal was not well received by the CWIS committee, but after an attempt to out-source computing services at UMinn failed, its opponents retreated and the Gopher Team was permitted to proceed. (This was enlightening not only as an example of the effect of political history on technology, but also because it may explain something about the Gopher Team's mistrust of design by committee, about which more later.) The talk also illustrated how much ground Gopher has covered: from in-house demo to a very popular technology and an RFC (#1436) on the IETF standards track in just two short years. GOPHER PRESENT AND FUTURE: Mark McCahill gave an overview of the features being developed in Gopher+, the upwardly-compatible extension to the protocol (also known as "a gopher with a bag on the side to hold more"): -- Item attributes: meta-information about Gopher objects. -- Multiple views of objects. -- ASK blocks: a method of passing information from the user to the server. While originally conceived as a forms fillout mechanism, ASK blocks are now expected to provide much more, including a way to make structured queries against a database. -- "AdmitOne", a home-brewed authentication mechanism based on single-use tickets. McCahill also mentioned some future directions for Gopher development: -- Support for URNs and URLs ("Unified Resource Names" and "Unified Resource Locators"), a pair of standards in the IETF process for identifying Internet resources. -- A Gopher->Z39.50 gateway. -- The idea of combining ASK blocks with an authentication mechanism to create read-write Gopherspace. (While I find the prospect intriguing, I wonder about the wisdom of jerry-rigging it this way rather than designing the necessary protocol extensions to support it explicitly.) -- Support for Whois++ (an IETF effort intended as a simpler alternative to X.500). The other members of the Gopher development team reported on their work: -- Paul Lindner, author of the Unix server and curses client, continues to enhance the base Gopher versions while working on Gopher+. The latest version of the base Gopher client now offers a download menu for dialup users; a better CSO interface; faster I/O; and snazzy dialogue boxes. The base Gopher server now uses a config file to eliminate compile-time defaults; user-configurable filename extensions; better numbering support ("Numb=" is fixed); and improved IP-based security. The Gopher+ server and client are coming along: they already implement forms via ASK blocks (the resulting information is processed by being handed off to perl scripts); alternate views and languages; extended information about items; and AdmitOne authentication. Upcoming enhancements for the Gopher+ client include specification of preferred data types and languages to take advantages of alternate views and a pretty, integrated pager, while Paul's plans for the Gopher+ server include automated mirroring of data and the integration of a relational database. -- Bob Alberti reported on fixes to the Gopher-FTP gateway. -- Dan Torrey spoke of his work on the PC client: he had programmed himself into a corner with the previous version, so he threw it out and started over. The result is PC Gopher III, which is multi-threaded to allow for menu accessibility even in the middle of a retrieve and appears much faster to the user. Future plans include a new memory management scheme to allow for Gopher+ and files over 360K; support for other TCP/IP stacks; internal support for 3270 sessions; and improver "helper" applications (image viewers etc.). -- Dave Johnson reported on his Mac client work. He has made various speed and user interface enhancements to the base Gopher client, and also has written a Gopher+ client which supports item attributes, alternate views, ASK blocks and AdmitOne authentication. VERONICA: Fred Barrie and Steve Foster of the University of Nevada at Reno led a session on Veronica. Veronica is making great strides in the direction of reliability and responsiveness by dividing the work: at the time of the conference there were two Veronica servers up (at UNR and CNIDR), with three more expected soon. Recent improvements to Veronica software include faster server code, boolean support (AND and OR, but not NOT or parens), and better data harvesting and cleaning. Some Veronica statistics: UNR has been receiving 150,000 queries per month; it takes 30 hours to prowl "known Gopherspace", meaning the 400+ servers in the UMN master list and their children (amounting to 1200 servers all told); Veronica detects over 10 million Gopher items, which it reduces after eliminating redundancies to about 1.5 million items; Veronica's database now takes up 350 Mb for raw data and 40 Mb for the index; up to 10% of the items in Gopherspace turn over in 1-2 weeks. There is a lot to be gained by making Veronica use Gopher+ attributes, such as abstracts, keywords, or a controlled-vocabulary classification scheme. Not only will Gopher+ allow for better searching, but it should help with one of the most common user requests, namely additional context surrounding the items returned from a Veronica search. Problems still to be solved: scaling (although this may not be a problem -- Barrie and Foster think that 12 servers should be sufficient through the end of the year); how to let people control their own data harvesting (ideas: a cache at the top of the Gopher tree, perhaps in a ".veronica" file, and/or coordination through regional networks); lack of permission to publish information gathered by Veronica ("we haven't been sued yet"); "Where am I?" questions for users; topic-oriented Veronica servers; meta-searches (like a "monster mindex"). GOPHER+ DIRECTIONS: This session began with a "lively exchange of views" between the Gopher development team and members of the Gopher community who felt that Gopher would benefit by paying more attention to standards which either already exist or are in parallel development. To overdramatize the split, one could call it an argument between "design by committee" and "design over a couple of weekends". Among the points of the critics: Gopher should try to be more expandable and more easily integrated with other protocols (we should avoid the need for gateways if possible); Gopher should support MIME types, which is distinct from supporting *a* MIME type; Gopher has confused the encoding type (e.g. compressed) with the content type (e.g. text), while MIME keeps these distinct. The standards which were most often discussed as being of potential benefit to Gopher were MIME, WWW, WAIS (and Z39.50), URLs and URNs. Paul Lindner gave a short but provocative talk on a proposed "geographic" view type, "directory/GVIEW", which would present a menu as an image with "hot spots". His original idea was for "geographic" menus like maps of bus routes or Gopher sites, but it could also be used to create Gopher menus which would resemble VNS pages or WWW hypertext documents. (I am struck by the tendency of the these technologies to start resembling one another.) ACROBAT DEMO: Gary Cosimini of Adobe gave a talk on a very interesting forthcoming product called Acrobat which is intended to move Adobe from being a printer company to being a document language company. Evidently Adobe was surprised to see PostScript become a means of transmitting documents from place to place, ridden as it is with printer dependencies. Acrobat is Adobe's answer: a program which converts between PostScript and a PostScript-like document language called PDF ("Portable Document Format"). It is Adobe's bid to come up with a standard so ubiquitous that they are calling it "electronic paper". In addition to increased portability, PDF has several features desirable in a document exchange format: it retains the structure of the document, allows direct access to any page in the document, allows for embedded special fonts and replaces missing fonts on the fly, includes efficient compression techniques, and is extensible to include multimedia and programmatic objects in the future. PDF will be an open format and Adobe plans to help develop "save as" modules for common document preparation tools. Adobe plans to offer Windows and Mac viewers for PDF this summer and DOS and Unix viewers soon thereafter. If it catches on this would be a good candidate for a Gopher data type and could solve many document incompatibility problems. ATTRIBUTE REGISTRY: This was dominated by another exchange between the Gopher development team and the proponents of compatibility with other standards. The main proposal was that rather than Gopher developing its own attribute registry, Gopher should use the IANA ("Internet Assigned Numbers Authority") registration process, which according to some only takes 3 to 4 weeks. There was also further discussion of MIME. Result: the UMinn GopherTeam agreed to look harder at MIME. SOFTWARE LICENSING: The much-awaited confrontation between Shih-Pau Yen of UMinn and the Gopher community at large was heated, as expected. Mr. Yen presented a draft of a three-tiered licensing scheme: separate categories of licenses for educational institutions, small businesses and large businesses. Mr. Yen pointed out that all UMinn software released prior to GopherCon would be considered freeware and not restricted by the license, hence the license covered effectively only future Gopher+ development; furthermore, the licenses restricted only Gopher servers, not clients. Under the proposal, educational institutions would be exempt from licensing fees and in fact need not even sign the license. Rates for businesses would vary according to the size of the business and the use to which it put its servers (whether it charged for information served out); rates would go as high for large businesses as the greater of $5000 per server or 7.5% of fees charged for Gopher services. Among the criticisms of Mr. Yen's proposal: By charging fees exclusively for servers, he would penalize organizations which wish to put information into Gopherspace to the benefit of us all; it might make more sense to charge for clients. The license agreements defined fees per year and made no guarantees that fees won't go up in subsequent years, raising fears of price hikes once the market was hooked on Gopher. The language of the license agreements defined the size of a business by the gross incomes of all entities owning a share of or partially owned by the business, which would mean that one wealthy investor owning 1% of an otherwise small business could kick it up into the large business bracket. Finally, there were the familiar complaints that the UMinn code wasn't written entirely by UMinn, that it has or will soon have freeware competitors, and that it doesn't come with the sort of support structure one might expect from commercial software. Mr. Yen seemed determined to proceed with licensing fees of some kind, and it was unclear to me how much he would modify his proposal in the light of the comments at GopherCon. Day 2: the Conference The second day began with a recap of the Gopher update from day 1. REPORTS FROM MONDAY'S BREAKOUT SESSIONS: Using Other Technologies -- MIME, Kerberos, Z39.50, etc (Jane Smith, CNIDR): Repeated the plea that the Gopher developers take a harder look at other standards and strive for interoperability. Cataloging E-journals and Resources (Billy Barron, UTDallas): Spoke about the gains to be made from improved Veronica, including a controlled vocabulary for searching and use of Veronica data for research on the most common terms currently used in Gopherspace titles and Veronica searches. Experiments in the past few months suggest that Gopher classification efforts based on schemes like the Library of Congress or Dewey Decimal system are less successful than ad hoc "used bookstore" schemes. Security, Authentication and Signatures (Tamara Iversen, Northwestern): Session focused on a hole in the design of the AdmitOne authentication scheme discovered by Lee Brintle of UIowa; suggested a modular authentication scheme selectable by each Gopher site. Gopher Front-Ends to Databases (Paul Lindner): A number of efforts have taken place to integrate both relational and flat-file databases with Gopher. Two basic methods are common: a "shadow" system, in which reports are generated from the database and moved into Gopherspace, and on-line access to an interactive query engine. Paul plans to work toward integrated Gopher support of one flat-file db (DBase is the most popular) and one generic SQL db. GOPHER+ ATTRIBUTE REGISTRY: The Gopher development team agreed to consider using the IANA registration process in the future. For now, new attribute types are to be registered by sending mail to the gopher-news list and reaching some sort of consensus in the ensuing discussion. NEW GOPHER USES AND TOOLS: unless otherwise specified, any tools mentioned here are said to be available in the archives on boombox.micro.umn.edu. Jughead (Rhett "Jonzy" Jones, UUtah): "Jonzy's Universal Gopher Hierarchy Excavation And Display", this is like a local veronica -- it allows for search by title of items in a local Gopher hierarchy. It supports boolean searches (AND, OR and NOT). A Gopher tree of some 45000 items takes about 30 minutes to walk and 10 minutes to index. Procmail (Marie-Christine Mahe, Yale): a mail-to-whatever gateway which Marie-Christine has used as a way to put things into Gopherspace. WELL Gopher (Paul Holbrook of CICnet): an interesting collection of original text resources from a well-known Internet watering hole. Highlights include works by Bruce Sterling and Stewart Brand. Gopher to nkosi.well.sf.ca.us port 70. Ac (Eric Morgan, NCSU Libraries): an interactive tool for the systematic acquisition of electronic journals. OS/2 Gopher client and server (David Singer, IBM Almaden Research Center): the client is already on boombox, the server is expected soon. Gmail/gmailcal (Prentiss Riddle, Rice University): a mail-to-gopherspace gateway with an accompanying events calendar manager. Grabdata (Mahesh Subramaniam, Notre Dame): a tool to gather data from users' AFS directories, perform various validation checks, and move it into Gopherspace. Relational database of course schedules (Ed Moy, UCal): A gateway between Gopher and a relational database containing a course schedule. Works by selecting a set of courses and successively limiting it by such categories as day, department, starting time, ending time, room, instructor, etc. MOOGopher (Erik Ostrom, Gustavus Adolphus College, and Larry Masinter, Xerox PARC): An object-oriented MUD (multi-user dungeon/dimension) with an integrated Gopher client. Objects in the MUD can contain Gopher items, and users can move about Gopherspace analogously to exploring a MUD. To see this in action, telnet to Jay's House MOO at theory.cs.mankato.msus.edu port 1709. Overview of MSU tools and applications (Rich Wiggins and Dennis Boone): Lots of nifty things at MSU, including sound archives (historic voices and 1992 Presidential debates), tools to generate a Gopher "road map" and analyze Gopher logs, an "online photo gallery" of campus sites, and a "guest book" for recording user comments. The coveted stuffed gopher was won by Erik Ostrom and Larry Masinter for MOOGopher. SERVER ADMINISTRATION: The Server Administration breakout session was in some ways a continuation of the tool-off, but it got beyond tools to cover Gopher administration and design philosophy as well. Joel Cooper spoke more about how Notre Dame copies data from individuals' AFS directories into Gopherspace. The procedure includes much error-checking and requires files to include a template containing information about author, expiration, etc. Dennis Boone spoke about the design rules for the MSU CWIS, which may bear repeating here: -- Ample "About" files. -- No "hunt the wumpus" games -- hierarchy is preferable to a maze. -- Lots of navigational tools (e.g., a title indexer) -- The root directory should include help and be understandable -- Directories should not exceed a screenful in length -- Identity of data providers should be attached to all documents -- Reliability: 24 by 7 support for Gopher services -- Lowest common denominator: assume screens <= 72 columns wide and <= 16 rows high -- No dead ends, no sparse directories -- Local-use information is most important -- Break these rules when necessary Paul Lindner spoke about the trials and tribulations of getting a CWIS off the ground at UMinn. Although he and the rest of the Gopher development team are celebrities in Gopherspace, he's "just another joe" at his home institution. Some interesting tidbits: the principal servers at UMinn are a mirrored pair of Macintosh IIci's running AUX; they carry out some 50000 transactions a day. UMinn is interested in ways to provide redundant servers, including the use of DNS route shuffling. The UMinn method of CWIS design emphasizes decentralization using small, cheap hardware, typically Mac servers. They see decentralization as a proper response both to the fact that "machines go down" and the need to put power in the hands of the people with the data. I spoke about our experiences at Rice University with RiceInfo and some of the formal steps we've taken (the "RiceInfo Management Proposal", CWIS committees, Data Maintainer's Agreements, etc.). I also noted that, like MSU, we see reliability issues as encouraging reliably maintained central services over a potential chaos of less reliably maintained small servers. ORGANIZING SUBJECT-MATTER BASED GOPHERS: A lively breakout session which made me wish I could have attended the other library- and resource-oriented sessions earlier in the conference. Dan Jacobson of Johns Hopkins spoke about his biology Gopher server; he believes in the need for control to be in the hands of specialists in a field, not librarians. He also spoke of the need for context: guess what happens when a biologist does a veronica search on the word "mouse"? :-) Mike McGervin of Stanford spoke of his work building a Gopher server of medical informatics sources based on ten years' experience with the SUMEX-AIM ftp archives. Librarian Nancy John of the University of Illinois mentioned an Educom paper by Howard Strauss entitled "Anarchy in the CWIS". She spoke of the need to go beyond old methods developed for print resources (including the LC and Dewey classification schemes). One promising direction is the use of Veronica with Gopher+ attributes (Academic Discipline, Audience, Format (newspaper or novel), etc.) to provide needed context for searches. (I piped up from the audience with a plea for feedback mechanisms from users and a community organized around each subject area, rather than depending on Gophermeisters discovering resources in isolation. I'd like to see a set of discipline-specific newsgroups or mailing lists devoted to the discovery of resources. I also talked a bit about "linkmerge", my ad-hoc scheme to merge subject-tree directories from other sites. Much to my surprise, people seem to like it, even the librarians who I expected to be horrified.) CLOSING: The conference ended with a general question and answer session. Highlights: Someone suggested a common conference at which Gopher, WAIS, WWW, etc. developers could meet (a good candidate to host such a conference would be someplace like CNIDR). Paul Lindner agreed to put ".cap" support back into the Unix Gopher server as an #ifdef option. The Gopher development team talked again about farming out documentation to outside volunteers. My conclusions? There's lots happening in Gopherspace. It was very exciting to see all these talented people carrying out all of these great ideas. Gopher's explosive growth is continuing with no end in sight. At the same time, I was disappointed that Gopher+ has not matured more than it has -- there are still nagging details about the protocol, the feasibility of small clients, and the licensing of UMinn-developed software which could make Gopher+ less successful at fulfilling its goals than basic Gopher has been. I had hoped that these issues would be put behind us at the conference, but it is now clear that we will continue to struggle with them for some time. -- Prentiss Riddle ("aprendiz de todo, maestro de nada") riddle@rice.edu -- Unix Systems Programmer, Office of Networking and Computing Systems -- Opinions expressed are not necessarily those of my employer.