[gopher] Mime under Nine

[Denis Bernard]

             Mimed File under Gopher Item Type 9

  Damien Carol made a proposal to use mimed file under the Item 0 to
carry all kind of binary files that could be base64 encoded; so, to be
assimilated to an ascii file. This proposal gave the possibility to
know which type of file is as reading the first blocks of that file
containing the label MIME. Furthermore, one could notice that using
"multipart", it is possible to know if that file is shortened or not
during the transmission according the boundary is displayed or not.
More, adding a checksum after this first part could verify the
integrity of the said file.

  Some persons said the item type "0" doesn't apply to this kind of
mimed file, even ascii encoded, due to the need to have a base64
decoder.

  From my side, I agree to the opinion that item type 0 must not be
used for mimed file.

  I saw, last days, many discussions about which item type to apply to
mimed file, PDF files, video files and so on. We all agree the fact
that item type is coded using only one ascii printable character. This
is the standard and we can't change it for instance. We could imagine
a renovation of the Gopher protocol where item type will be coded
using several ascii character. But, every year, it would be necessary
to write a new edition of the Gopher protocol!

  To day, there is much more types of text files or application files
than 20 years ago. Just considering the Unicode coded text files, you
have several flavors of UTF. For UTF-16, you have at the beginning of
the file a BOM (byte mark order) to tell if it is little-endian or
not.
So, many text files or application files hold a magic number at their
beginning. Just have a look at Wikipedia to have an idea of how many
magic numbers are in use to day:

http://en.wikipedia.org/wiki/Magic_number_%28programming%29

  An other indication of how many kinds of file are living to day is
the Iana page for mime media types at:

http://www.iana.org/assignments/media-types/index.html

  For people wishing a "p" Gopher item type standing for PDF files:
what about  presentation files (MS-PowerPoint), vCAL files, spread
sheet files, data base files or hundred flavors of text writer files?
There is so much file formats to day that applying a specific item
type is absolutely impossible with a unique printable ascii character!

  The idea of Damien of the client that reads the first blocks of an
unknown file is already an old implementation in the Unix world. The
Posix standard (now, SUSv4) provides the "file" command that make
several tests to find the type of a file, among of then by reading the
first blocks.
  On my Gentoo Linux, I have the package "sys-apps/file-5.09". Looking
at the source code, there is under the directory "Magdir", a huge list
of files giving magic numbers and their mime type. More interesting,
unlike the Posix version, it is possible to use the "--mime" option to
get the mime type!

  So, the idea of Damien to test the type of an unknown file is
possible for a client running under a Unix system. It could be done
via an external command or linking the program with "libmagic"
(#include <magic.h>). But, in practice, what happens?

  Supposing you get a menu Gopher having 10 unknowns files to test:
the client must connect again 10 times just to read the beginning of
that files. That seems heavy both for the client and the server. So,
is it hopeless? For me yes, assuming we consider only the primitive
Gopher protocol. Now, looking at the enhanced version of Gopher:
Gopher+ . There is an interesting gopher hole running both standards:

gopher://gopher.quux.org

 I suppose you have a Linux system and I suppose you have Netcat or a
Telnet client: run first this command in a console:

nc quux.org 70
or
telnet quux.org 70

then, press the "/" key and press "enter"

  You get the same menu than using the URL "gopher://quux.org/0/"
with any client.

  And now, again by "nc (or Telnet) quux.org 70", but:

press the "TAB" key, press the "$" key and press "enter"

  You get now a very different menu, because you are now using Gopher+
protocol:

(...)
+INFO: 0What's New      /whatsnew.txt   gopher.quux.org 70      +
+ADMIN:
 Admin: John Goerzen <jgoerzen at complete.org>
 Mod-Date: Tue Apr  2 21:16:57 2002 <20020402211657>
+VIEWS:
 text/plain: <1k>
(...)


  And what do you see there? A list of blocks with headers beginning
with “+” symbol, like INFO, ADMIN and VIEWS. The first one, “+INFO”,
provides the line of the gophermap that you know. The second one,
“+ADMIN”, provides several informations about name of administrator
and the date of last modification of the file. The third and last one,
“+VIEWS”, give the Mime type that we need to lunch the program
(external or internal) needed to process this file; and “<1k>” means
the approximate size of the file.

  So, the idea to introduce the Mime type in the Gopher protocol is
not recent! Please, read the Gopher+ description at:

gopher://gophernicus.org/0/doc/gopher/gopher+.txt

  If the idea of Damien of using a mimed file is not convenient from
client side, the idea of providing to the client the mime type needed
to apply the proper process for a given file is already described and
implemented to day: Gopher+ protocol! Implementation of Gopher+ in a
modern graphical client is up to you. There is no need to reinvent a
new Gopher protocol! 

  And what about my title "Mime under Nine"? My proposal is to
consider any file that do not clearly belong to "0","4","5","6","g"
and "I" item type to be under "9" item type.

-- Denis