The TCP/IP Guide Protocols (FTP, TFTP, Electronic Mail, USENET, HTTP/WWW, Gopher) TCP/IP Interactive and Remote Application Protocols Gopher Protocol (Gopher) - - - - - - - - - - - - - Let's suppose that I told you I was going to describe a TCP/IP application layer protocol designed for the specific purpose of distributed document search and retrieval. This protocol uses a client/server model of operation, where servers provide links to related resources such as files or programs that users access with client software that displays options for the user to select. You might think that I was talking about the World Wide Web, and for good reason. However, in this case I am actually talking about one of the Web's predecessors: the Gopher Protocol. A good place to start our discussion of this protocol is with its name, which is well-chosen for a number of reasons. The Gopher Protocol was developed at the University of Minnesota, whose sports teams are called the "Golden Gophers" (Minnesota is known as the "Gopher State"). This is the direct origin of the name, but it is also appropriate because the rodent that shares it is known for burrowing, just as the protocol is designed to "burrow" through the Internet. And of course, the term "gopher" also applies to a person who performs errands, such as retrieving documents (they "go fer" this, and "go fer" that). J The Gopher Protocol was developed in the late 1980s to provide a mechanism for organizing documents for easy access by students and faculty at the university. The core principle that guided the development of the system was simplicity. Gopher is designed on the basis of a small number of core principles, and uses a very straight-forward mechanism for passing information between client and server devices. It is described in RFC 1436, published in March 1993. Information Storage on Gopher Servers Information accessible by Gopher is stored as files on Gopher servers. It is organized in a hierarchical manner similar to the file system tree of a computer such as a Windows PC or UNIX workstation. Just as a file system consists of a top-level directory (or folder) that contains files and subdirectories (subfolders), Gopher servers present information as a top-level directory that contains resources such as files, and/or subdirectories containing additional resources. Resources on different servers can be linked together using by having them mentioned in each others' resource hierarchies. It is also possible for "virtual" resources to be created that act as if they were files, such as programs that allow Gopher servers to be searched. Gopher Client/Server Operation Typical use of Gopher begins with a user on a client machine creating a TCP connection to a Gopher server using well-known TCP port number 70. After the connection is established, the server waits for the client to request a particular resource by sending the server a piece of text called a selector string. Often, when a user first accesses a server, he or she does not know what resource to request, so a null (empty) selector string is sent. This causes the server to send back to the client a list of the resources available at the top (root) directory of the server's file system tree. Directory Listing Structure A directory list sent by the server consists of a set of lines, each of which describes one available resource in that directory. Each line contains the following elements, each of which is separated by a "" character: * Type Character and Resource Name: The first character of the line tells the client software what sort of resource the line represents. The most common type characters are "0" for a file, "1" for a subdirectory and "7" for a search service. The rest of the characters up to the first "" contain the name of the resource to be presented to the user. * Selector String: The string of text to be sent to the server to retrieve this resource. * Server Name: The name of the server where the resource is located. * Server Port Number: The port number to be used for accessing this resource's server; normally 70. Each line ends with a "" character sequence consistent with the Telnet Network Virtual Terminal (NVT) specification. Upon sending the directory listing (or any other response) the connection between the client and server is closed. Menus and User Resource Selection After receiving this sort of directory list, the Gopher client software will display a menu to the user containing all the resource names the server provided. The user then selects his or her desired item from the menu, and the client retrieves it by making a connection to the appropriate server and port number and sending the selector string of that resource. If this itself represents a subdirectory, a new directory listing for that subdirectory will be sent by the server; if some other type of resource, it will be accessed according to the requirements of the resource type. For example, suppose this line were sent from the server to the client: 0Gopher Introductionintrogopher.someserver.org70 This would be presented to the user as the file called "Gopher Introduction" in a menu containing other options. If the user chose it, the client would initiate a connection to the gopher server "gopher.someserver.org" at port 70, and then send the selector string "intro" to that server to retrieve the document. Important Differences Between Gopher and The Web As I hinted at the start of this discussion, both Gopher and the World Wide Web are intended for the same basic purpose: providing access to repositories of information, with links between related documents and resources. However, they take a very different approach to how that information is accessed, especially in two key areas: user interface and resource linking. Gopher's presentation to the user is entirely oriented around its hierarchical file system. As a result, Gopher is inherently menu-based, and the user interface usually based on a simple text presentation of those menus. In contrast, information on World Wide Web servers can be organized in any manner, and presented to the user in whatever form or fashion the owner of the server desires. The Web is much more "free form", and there is no need to use a directory structure unless that is advantageous in some way. Linking in the World Wide Web is done directly between documents, most often using Hypertext Markup Language (HTML) tags. When someone writing document A mentions something relevant to document B, he or she puts a link to B right into document A itself. Gopher, on the other hand, is not designed to use links in this way. Instead, linking is intended to be done using the directory tree we described earlier. Gopher's Role in the Modern Internet There are some people who believe that Gopher is technically superior to the Web in a number of respects. They consider it cleaner to have linking be done by servers, rather than having links embedded in documents. An argument can also be made that the text orientation of Gopher is efficient, better able to ensure compatibility between platforms, and also more suited to special needs situations such as low-bandwidth links and access by those with visual impairment. Some Gopher enthusiasts thus consider it to be a "purer" hypertext system than the World Wide Web. However, history shows us that despite Gopher predating the World Wide Web, the Web overtook it in popularity in only a few short years. Today, the Web is the 900 pound gorilla of the Internet, while most people have never even heard of Gopher. What happened? I believe the main reason why Gopher lost out to the Web is that the Web is far more flexible. Gopher's use of text hyperlinks and server directory structures may be efficient, but it is limiting. In contrast, the Web allows information to be presented in a wide variety of ways. The open, unstructured nature of the Web made it an ideal vehicle for the creativity of information providers and application developers. In the mid-1990s, the Web was also perfectly poised to support the transition of computing from text to graphics, and Gopher really was not. Simply put, you can do more with the Web than you can with Gopher, and more people care about functionality and breadth of options than straight efficiency. Once the Web started to gain momentum, it very quickly snowballed, as I discuss in the topic on the Web's history. It took only a couple of years before Web use was well-entrenched, and Gopher was unable to compete. For its part, the University of Minnesota likely hastened Gopher's demise with its controversial decision to charge licensing fees to companies that wanted to use Gopher for commercial purposes. I do not believe there was anything nefarious about this: the university was on a limited budget and wanted companies that could afford it, to pay a small fee to support development of Gopher software. However, computing history has shown time and time again that there is no faster way to kill a protocol or standard than to try to charge licensing or royalty fees for it, no matter what the reason. By the late 1990s, Gopher was well on its way to obsolescence. As use of the protocol dwindled, many organizations could no longer justify the cost of continuing to run Gopher servers. Even the University of Minnesota itself eventually shut down its own Gopher servers due to low utilization. The final "nail in the coffin" for Gopher occurred in 2002, when a security vulnerability related to Gopher was discovered in Internet Explorer, and Microsoft chose to simply remove Gopher support from the product rather than fix the problem. Today, Gopher is still around, but is a niche protocol used only by a relatively small group of enthusiasts and a handful of organizations that have a past history of using it. Key Concept: The Gopher Protocol is a distributed document search and retrieval protocol that was developed at the University of Minnesota in the late 1980s. Resources are stored on Gopher servers, which organize information using a hierarchical directory structure. Gopher clients access servers to retrieve directory listings of available resources, which are presented to the user as a menu from which an item may be selected for retrieval. Gopher's chief advantage is simplicity and ease of use, but it lacks flexibility in presentation and the ability to effectively present graphics and multimedia. For this reason, despite Gopher predating the World Wide Web, the Web has almost entirely replaced it, and Gopher is now a niche protocol. - : - The TCP/IP Guide (http://www.TCPIPGuide.com) Version 2.0 - Version Date: June 7, 2004 Copyright 2001-2004 Charles M. Kozierok. All Rights Reserved. .