Dilip 27 Nov 2020 ======================================================================== Shovel A collection of python scripts used to dig out this gopher hole. Written by Dilip . This system of python scripts ingest a variant of the markdown script and parses it into the raw text you see here. Syntax Guide. ============= Markdown syntax: -------------------------------------------------------------------------- | # Title | -------------------------------------------------------------------------- Sets the title for the page. For example, see Shovel at the top of this page. Must be the only thing in that line. If a date is available, it is added to the title. This is processed by the script md_parser. -------------------------------------------------------------------------- | ## Header | -------------------------------------------------------------------------- Formats as a section header. For example, see "Synatax Guide" at the top of this page. Must be the only thing in that line. This is processed by the script md_parser. -------------------------------------------------------------------------- | ### Header | -------------------------------------------------------------------------- Formats as a subsection header. See the details of the scripts below for an example. Must be the only thing in that line. This is processed by the script md_parser. -------------------------------------------------------------------------- | [Link Text](Link) | -------------------------------------------------------------------------- Adds a link as a footnote with the provided text. A footnote number will replace the link where it was first encountered. If the same link is encountered again in the page, the same footnote number will be used. This is turned into %%FOOTNOTELINK by the md_parser to be later dealt with by the annotate script and the linker script. ``` Code block. ``` The triple backtick (the sign below the ~ key, to the left of 1 on most keyboards will produce text as code. This was used to show the markdown syntax above for example. This is performed by the md_parser script. ------------------------------------------------------------------------------------------------ | - Bullet Lists | | - These are not numbered | | - Bullet points with the same number of left spaces will be indented | | together. | | - A tab is treated as 2 spaces for you barbarians who mix tabs and spaces. | | - Bullets must begin with a "-" and a space (a minus sign on most | | keyboards). | | | | 1. Numbered Lists | | 2. The script is fairly dumb. If you skip a number, the script will not | | correct it for you. | | 1. Numbered lists must begin with a number and a period. | | 2. Roman numarals, alphabets, hexadecimal / binary / imaginary / | | complex / quaternion / octonion numbers and anything else that is not a | | number according to a kindergartener will not work. | | 3. As with bullet points, points with the same number of | | left spaces are indented together. | ------------------------------------------------------------------------------------------------ * Bullet Lists * These are not numbered * Bullet points with the same number of left spaces will be indented together. * A tab is treated as 2 spaces for you barbarians who mix tabs and spaces. * Bullets must begin with a "-" and a space (a minus sign on most keyboards). 1. Numbered Lists 2. The script is fairly dumb. If you skip a number, the script will not correct it for you. 1. Numbered lists must begin with a number and a period. 2. Roman numarals, alphabets, hexadecimal / binary / imaginary / complex / quaternion / octonion numbers and anything else that is not a number according to a kindergartener will not work. 3. As with bullet points, points with the same number of left spaces are indented together. Extended syntax: ================ -------------------------------------------------------------------------- | %%TITLE{Title Text} | | %%H1{Header 1 text} | | %%H2{Header 2 text} | -------------------------------------------------------------------------- The markdown parser will emit these. They can also be emitted manually. The layout script will process these and format them in it's output. The %%TITLE is also used to identify links to this file. If multiple %%TITLE lines are found, only the first one will be used. The md_parser script records the %%TITLE as it is emitted. -------------------------------------------------------------------------- | %%BULLET_LIST{indent}{Text} | | %%NUMBERED_LIST{indent}{Text} | -------------------------------------------------------------------------- The markdown parser will emit these. They can also be emitted manually. The layout script will process these and format them in it's output. -------------------------------------------------------------------------- | %%DATE{MM-DD-YYYY} | -------------------------------------------------------------------------- If found in a file, all scripts will use this date with this file. This is supported by the utility script, all other scripts might use this. This will be cleared by the cleanup script. -------------------------------------------------------------------------- | %%QUOTE{Quoted text}{Author} | -------------------------------------------------------------------------- Formats the quoted text and attributes it to the author. The Author is optional. This must be on a line by itself. Here is an example: I am going to Nebraska for two weeks. What am I going to do? - Random girl sitting next to me in the park while I wrote this. This is managed by the layout script. -------------------------------------------------------------------------- | %%START_QUOTE{Title} | | %%END_QUOTE{Author} | -------------------------------------------------------------------------- Allows quoted text to span multiple lines. Whitespaces will be respected (if possible). Great for poems and the like. Start and End must be the only things on their lines. This is managed by the layout script. When using this syntax, let the script align the whitespace. Shake Prayer Open to them your hand to the shore, Watch them walk into the sea. Press upon them all they need, see them yearn for all they want. Gift to them the calm pool of words, watch them draw the sword. Bless upon them the satiation of peace, see them strave for war. Grant them darkness and they will lust for light. Deliver to them death and hear them beg for life. Beget life and they will murder your kin. Be as they are and they see you different. Show wisdom and you are a fool. The shore gives way to the sea. And the sea, my friends, Does not dream of you. - Steven Erikson in Reaper's Gale, chapter 21 -------------------------------------------------------------------------- | %%DISABLE_MD to disable markdown. | | %%ENABLE_MD to re-enable markdown. | -------------------------------------------------------------------------- This will disable markdown processing. For example, this was used to show how backticks are used format code blocks. This is cleared by the cleanup script. -------------------------------------------------------------------------- | %%FOOTNOTE{Footnote text} | | %%FOOTNOTELINK{Footnote text}{link} | -------------------------------------------------------------------------- Adds a footnote [1]. Footnotes can be on a line along with other text. This is emitted by the annotate script, it can also be added manually. This is managed by the annotate script. -------------------------------------------------------------------------- | %%CATEGORY{category} | -------------------------------------------------------------------------- Request to be identified under a category. This is processed by the metadata script. If the paginator script finds one of the %%CHILDREN category in a page, a link will be added to this page. -------------------------------------------------------------------------- | %%CHILDREN{category} | | %%CHILDREN_WITH_NEWEST_FIRST{category} | | %%CHILDREN_WITH_OLDEST_FIRST{category} | -------------------------------------------------------------------------- When the paginator finds a %%CHILDREN tag on a .page file, it will scan through all the children that have marked the current category and generate links to them. -------------------------------------------------------------------------- | %%ROOT | -------------------------------------------------------------------------- Mark the file containing this as the top level file in the gopher space. -------------------------------------------------------------------------- | %%DISABLE_LAYOUT | | %%ENABLE_LAYOUT | -------------------------------------------------------------------------- These are mostly emitted by other scripts in the collection to tell the layout script not to disturb something. They can also be manually added - the layout script doesn't care where this came from. -------------------------------------------------------------------------- | %%LINK{Text}{Link} | -------------------------------------------------------------------------- The linker script translates these into links. -------------------------------------------------------------------------- | %%NO_HEADER | | %%NO_FOOTER | -------------------------------------------------------------------------- The layout script is responsible for adding the header and a footer to each page. This tag tells the layout script to not perform this step. The tags are cleared by the metadata script. -------------------------------------------------------------------------- | %%INSERT{textfile.txt} | | %%BANNER{banner.txt} | -------------------------------------------------------------------------- The insert script looks for the specified file and pastes its content over to the current file. %%BANNER is like %%INSERT except that it is surrounded by %%DISABLE_LAYOUT and %%ENABLE_LAYOUT so that the layout script doesn't change the inserted text or ascii art. -------------------------------------------------------------------------- | \%% | -------------------------------------------------------------------------- For the rare occasion where want to have %% in your text. Like in the previous sentence for example. Modules and extending them. =========================== NewlineKiller ------------- This is the first script that is run. It attempts to remove as many new lines as possible using simple rules: 1. It respects blank lines added in the original Markdown source. 2. It forces a newline when the next line begins with a markdown header, a code block or a %% symbol (a good indicator that the next line begins with a keyword). 3. It copies over anything inside a multiline quote or a code block as is. 4. It does not modify anything between %%DISABLE_LAYOUT and %%ENABLE_LAYOUT The primary purpose of this script is to simplify parsing for all other scripts. The primary concern without this script is a %%QUOTE{ .... } that has several newlines in it - all scripts parse the source markdown line by line, multiple line breaks will make the code for matching the curly braces too complicated. Metadata -------- This is the second script that is run. If it finds any of these in the file, it will remove them in it's output after recording the relavant metadata. -------------------------------------------------------------------------- | %%DATE{MM-DD-YYYY} | | %%CATEGORY{category} | | %%ROOT | | %%NO_HEADER | | %%NO_FOOTER | | %%CATEGORY{category} | -------------------------------------------------------------------------- This script only records these keywords, other scripts will do something with these. MdParser -------- This is the first script that is run. It parses the following markdown syntaxes: -------------------------------------------------------------------------- | # Title | | ## Header 1 | | ### Header 2 | | [Link Text](link) | | - Bullet list | | 1. Numbered List | -------------------------------------------------------------------------- and ``` code block ``` The MdParser script is the third script that is run. It ignores everything between %%DISABLE_MD and %%ENABLE_MD. For all other markdown syntaxes that it supports (see list above) and translates them into other keywords that is understood by the layout or the linker scripts. Specifically, it emits these keywords: -------------------------------------------------------------------------- | %%TITLE{Title Text} | | %%H1{Heading 1} | | %%H2{Heading 2} | | %%FOOTNOTELINK{text}{link} | | %%BULLET_LIST{indent}{Text} | | %%NUMBERED_LIST{indent}{Text} | -------------------------------------------------------------------------- Paginator --------- The paginator script looks for one of these keywords and expands them with entries with matching categories. Indexer ------- The indexer script is currently a pass-through. It was intended to generate Indexes like those found at the back of a textbook, but I am not sure if I really want that right now. This document will be updated if I decide to go ahead and implement it. Annotate ======== The annotate script is responsible for annotating a file with the footnotes. It numbers footnotes and links separately. It adds a Notes section and a Links section at the bottom of the file if any footnotes or links are found respectively. Change Log. =========== 2020, Nov 27: Initial Version. Notes: ------ 1. This is an example