Re: Solderpunk and a new protocol[0] 2019-06-19 Having been less engaged in pubnix and gopher for around two weeks following the birth of my daughter, I checked in today with the intention of reading solderpunk's 3 phlog series on a protocol between gopher and the web. I have talked with sodlerpunk and others about this in the past and was then and remain excited about the possibility. I am excited for a few reasong: 1. It is always exciting to be around for the birth of something and not know what will become of it, but have high hopes 2. I LOVE the gopher community, but gopher is not without its shortcomings 3. It fits in with my view that the internet on the whole can still be anything we want it to be, but we must steer it there and not just be fed things from large companies. Change CAN come from users, and I do not believe you have to be a programer to take part. As such, I highly recommend reading solderpunk's posts and writing your own response! Now to the response in general: I. Headers - Request Headers - I cannot think of many use cases for request headers and generally agree with solderpunk's sentiments on the issue. I can see cases where making a language request could be useful. For example, I know that slugmax phlogs in english AND french. Since I do not speak French I have not viewed their French phlogs and do not know if it is duplicate content in another language or not. If it is duplicate content then a request header from a client could be useful and make it so slugmax did not need to keep separate phlogs in their feeds. Alternatively, servers could implement server-side translation of text/map documents (if this were to happen I think the server should also isnert a line at the top stating that translation has occured and give the source language so that a user may request it if need be). I think that kind of use case would really be neat. Of course, it would require those running servers to buy in to such an idea and is really not a protocol level question for the most part. - Response Headers - I very very much agree with the notion that adding open ended headers is a broken idea that just sets up abuse. I think that the proposal to have headers set up as: Is a worthy setup. The status codes will make proper issue handling actually possible... something I have struggled to be able to communicate and figure out in Bombadillo over gopher. The MIMEtype approach makes perfect sense to me for the most part. There is some consideration given in regard to having a text/map type or just having everything be one type that serves both text and links. I think I fall into the separation of interests camp. The idea to have one text format and just scan for links sounds appealing at first. With a lot of content on gopher, where things are relatively short, it makes a good amount of sense. However, what happens when a very large text document is served? I found on gopher at one point a copy of "The Fellowship of the Ring" as a plain text file. Under a system in which this could contain links or could not a client would have to parse the whole document before serving it to the user. While it should not take a crazy long time to do, it is extra processing that is needless. Having a text/plain and a text/map differentiates the files sufficiently and would let a client know not to try parsing the text/plain file. I do not know a lot about how a server/client determines MIME types. Is it infered by file types (ie. *.html)? If so, then it could make sense to, by convention, user *.map or some such for files that we would want parsed as maps. But again, this is not an area of expertise for me and I will need to do some research into how a server would know the mimetype of a file. Either way, my concern with the separation of plain text from map files (that may contain links) is whether or not to bother parsing the file or just serving it up. Encodings. This is making me flash back to working with Python 2.x. Dealing with encodings can be a pain. Sure enough, I checked the russian gopherhole[1] that solderpunk linked... bombadillo does not display most of the characters. As such, thank you for pointing that out to me! I think an encoding declaration defaulting to utf-8 is a great great solution. II. Links, links, links I absolutely agree that the itemtype i business is a dirty workaround that has served its purpose but should be improved upon. I generally like the idea of a file that has plain text (no need for item i) and has links on their own lines. Solderpunk suggests links go on their own lines and are set up as: I think, for me, we could likely lose the itemtype. The client will get a mimetype from the server (if the above header conversation plays out as suggested), and clients will be able to use that to prompt a download or the display text content or the like. The itemtype always weirded me out anyway. A user can put whatever they want... so it is an on your honor kind of thing. This trust has generally played out fine on gopher, but why bother? I like the idea of something like: ~! My Link@someurl.co.uk/main.map Which is the equivalent of: <@><url> My thought on the above is that using something at the beginning of the line to indicate a link simplifies searching for links (you dont need to read the whole line to know that you are at a link line) when parsing a document. I am not tied to the ~! combination. It is just a suggestion. The @ symbol could be any other indicator ( == comes to mind), and was used in place of tab for the specific reason that lots of people set their editors to replace tabs with spaces. It is annoying to toggle that on and off so using a standard textual character (or series of them) makes more sense to me. III. Misc - Directory default file - It occurs to me that if all files, in solderpunk's description, are both maps (can contain links) and content... what does a request to a directory get you (ie. a request that ends in a '/')? For gopher, it usually (though not always, it depends on the server) will grab a file called 'gophermap' within that directory, if one is present. Some servers, absent a gophermap, will provide a gophermap of the files in the directory. So what is the proposed behavior with this new protocol? Is it to be a part of the protocol at all or something left to servers to figure out and implement? The web has largely standardized on index.[htm|html|php|etc]. Would a base level file name be beneficial, such as index or gophermap? Or would some other behavior make more sense? I'm not sure the answers to these questions. - Media Types - After the above confusion over how MIME/media types work, I did a little research before finishing this. The wiki article[2] on the subject was quite helpful. It seems like the media type structures already in place would be very useful and provide great flexibility for clients. For example 'image/png'. A client can easily be set for anything in the image tree being a download item rather than display. This is particularly useful for terminal based clients. However, a graphical client could decide to display the image to the user (as its own document, not inline with text). I think this flexibility will prove to be quite nice. As a result, I definitely have come round to gneral support of the suggested response header from solderpunk. The one remaining question I have then is would a new media type be created for the text format under discussion (plain text plus links)? - Search/Interaction - The subject of search is an interesting one. Gopher's type 7 is an odd but solid enough way of handling things. I feel like there could be a solution via a new mediatype. Have the server send back a mime that lets the client know that an information/query request is being made. I would think that the server response could include any arbitrary text, to be used as a prompt, and the client would know to let the user enter text after displaying the response body. The prompt input would then be sent back to the same server address (possibly by the same method gopher uses... <url><TAB><user query>) and get a response back with the mediatype appropriate to that reponse (rather than the search mediatype). This would enable things like weather reports[3] or wikipedia[4] to still function well. IV. Closing thoughts I am pretty excited about this collective project that has been under discussion for some time. I think it would find users well and I think that its simplicity overall would allow it to get worked into existing gopher clients. Wouldn't it be wonderful to just not have to worry what protocol you were using? Produce your content on the one that makes sense to you and let the world view it as they see fit. :) It was not thoroughly discussed in sodlerpunk's writeup, but I do know that they are big supporters of adding tls as a core part of the protocol. I think that is a good call. [0] gopher://zaibatsu.circumlunar.space:70/0/~solderpunk/phlog/protocol-pondering-intensifies.txt [1] gopher://gopher.pclovers.ru:70/1/rus.koi8 [2] gopher://gopherpedia.com:70/0/Media type [3] gopher://gopher.floodgap.com:70/7/groundhog/us/zipcode [4] gopher://gopherpedia.com:70/7/lookup * I was having trouble using gopherpedia's search while getting the above media type link. Upon looking into it further, it seems someone has donked up their root gophermap the link points to 0.0.0.0:70. Hopefully someone over there realizes it and gets it fixed up. You can still use the link I provide, it just cannot be accessed from their main page.