FMark

FSharp markdown implementation with additional functionality.

Namely:

• Macros
• Includes
• Spreadsheet functionality
• Table of Contents generation
• Citations and footnotes

See example.fmark for usage examples.

How to use

FMark Plugin Visual Studio Code

Functions:

• Open Preview (ctrl+shift+M)

Open a live preview of the HTML that will be generated by FMark.

• Create HTML File

Create an HTML file in the same directory as the .fmark file, with .html instead of .fmark.

Command Line Interface

USAGE: FMark [--help] [--output <path>] [--loglevel <debug|info|warning|error|fatal>] [--format <html|markdown>] [--test [<sequential>]] [<path>]

INPUT:
    <path>                specify input file path.

OPTIONS:
    --input, -i <path> CLI flag for input (not necessary, default argument is input.)
    --output, -o <path> specify output file path.
    --loglevel, -l <debug|info|warning|error|fatal> set the log level  ('0:DEBUG', '1:INFO', '2:WARNING', '3:ERROR' ,'4:FATAL').
    --format, -f <html|markdown> specify format, by default: html.
    --test, -t [<sequential>] run CI tests.
    --help display this list of options.

Note: Markdown generation is incomplete and should only be used for property based testing.

Modules

Overall Flowchart:

            ┌─────────────────────┐                      ┌───────────────┐
Source ───> │ Lex and Preprocessor│ ───> Token list ───> │ TOCite Parse  │ ──> Token list with identifiers ┐
            └─────────────────────┘           │          └───────────────┘                                 │
                                    │         │                                                            │
                                    │         └──────────────────────────────> Header+Footer list ────>────┤
                                    │                                                                      │
                                    │          ┌──────────────┐                                            │
                                    └────────> │ Markalc Parse│────────── Table ────────────>──────────────│
                                               └──────────────┘                                            │
                                                                                                           │
                                   ┌─────────┐                            ┌─────────────┐                  │
             Final Document  <──── │ HTMLGen │ <──── ParsedObj list <──── │ Main Parser │ <────────────────┘
                                   └─────────┘                            └─────────────┘

1. Lexer and Preprocessor
2. TOCite: Preparsing headers and footers
3. Markalc: Spreadsheet functionality
4. Main Parser

Vanilla Markdown

Implemented

• HTML passthrough
• Lists
• Links (relative links within file don't work in VSCode, use Create HTML File and preview that.)
• Images (images don't work in VSCode, use Create HTML File and preview that.)
• Headers
• Styling (bold, italic, bold and italic, strikethrough)
• Paragraphs
• Quotes
• Code blocks
• Tables

Design decisions

Lists

• A list ends with two endlines, i.e., \r\n\r\n
• List items are separated by one endline
• List type is determined by the first item.
• NUMBER ; DOT will be ordered list
• - or * will be unordered list
• If the first item does not start with NUMDER;DOT;Space| "- " | "* ", the list type will be default to unordered list
• Ordered list start number is determined by the first item
• The subsequent counting is +1 for each list item
• Two spaces as one level of indentation.
• A tab, \t character will be interpreted as four spaces, therefore two levels of indentation.

Header

• All headers must be preceded by two endlines, with an exception made for the first line in a file.

\n\n# header\n is a header, otherwise unless the header is the first line in a file it won't be processed as a header.

Styling

• When using underscores for styling, there must be a space (or endline on the right) on either side in order for it to be recognised.
• Asterisks for styling work as usual.

e.g.

[ ]_em_[ ], [ ]__a strong__[ ], or [ ]_em_$, [ ]__a strong__$

Paragraphs

A paragraph is some characters that does not match: * CodeBlock * Header * ContentTable (Table of Contents object) * List * Quote * Table * Reference

It terminates with two endlines. Any elements mentioned above cannot exist in a paragraph. Otherwise, they will become normal text, and will not be rendered as expected.

Building

To build fmark, dotnet and fable have to be installed first. To build the javascript as well, yarn or npm are needed.

Using the Build script

# build only javascript
build -b js

# build and test every single module
build -b testall

# build js and cli
build -b all

Markdown extensions

Macros

Supported Constructs

These are the supported constructs in the preprocessor.

Supported Features

These are the features that are currently supported by the preprocessor.

Usage

To use the preprocessor and the lexer, a string or a list of strings can be used, depending on if there are multiple lines or not. For a single string, the following can be used.

For string, the preprocess and lex functions.

[<EntryPoint>]
let main =
    let inputString = (* Read the string *)

    inputString
    |> preprocess
    |> lex
    ...

For a list of strings, one can use the preprocessList and lexList functions.

[<EntryPoint>]
let main =
    let inputStringList = (* Read the string list *)

    inputStringList
    |> preprocessList
    |> lexList
    ...

Example

In markdown using the preprocessor, one can then write the following:

Text before macro
{% macro Hello(arg1; arg2)
This is text inside the macro, with semicolons;
{% macro local(arg1; arg2)
This is the second macro
%}
Now back in the first macro.
{{ local(arg1; arg2) }}
%}
Outside both macros
Should be printed as not in scope: {{ local(arg1; arg2) }}

{{ Hello(arg1; arg2) }}

which then evaluates to

Text before macro
Outside both macros
Should be printed as not in scope: {{ local(arg1; arg2) }}


This is text inside the macro, with semicolons;
Now back in the first macro.

This is the second macro 

More complicated macros can also be created by writing html in the macros. Due to the html passthrough in the lexer, the html will be copied over literally to the output html.

Lexer

The lexer supports HTML pass through, which can be used to display raw html in markdown. Using this with macros can give very useful and interesting functions.

Interface to the Parser

The interface to the parser was done using the following Token type, which the parser takes in and can parse.

type Token =
    | CODEBLOCK of string * Language
    | LITERAL of string
    | WHITESPACE of size: int
    | NUMBER of string
    | HASH | PIPE | EQUAL | MINUS | PLUS | ASTERISK | DOT
    | DASTERISK | TASTERISK | UNDERSCORE | DUNDERSCORE | TUNDERSCORE | TILDE | DTILDE
    | TTILDE | LSBRA | RSBRA | LBRA | RBRA | BSLASH | SLASH | LABRA | RABRA | LCBRA
    | RCBRA | BACKTICK | TBACKTICK | EXCLAMATION | ENDLINE | COLON | CARET | PERCENT

Features

Supports escaping of all the special characters defined in Types. This is done by adding a \ in front of the character that should be escaped.

Tokens that match multiple characters can also be escaped by just putting a \ before it. For example, *** can be escaped by writing \***.

Extensibility

It can easily be extended by adding the type of the token to Token above. Then the string has to be linked to the token by adding it as a tuple of type string * Token to a list called charList in the Lexer.

Spreadsheet functionality

Spreadsheet functions will evaluate in-place, if they are incorrectly formatted then Markalc will leave the cell unchanged as if it were normal text inside.

To delineate an expression, start the cell with the = operator, e.g.

Supported

Arithmetic

Cell reference behaviour

Functions

Supported functions:

Functions support Range Cell References: [0,0]:[2,0] will evaluate to [0,0],[1,0],[2,0]. This can be used with other arguments, e.g. SUM{[0,0]:[2,0],1,3+2,[0,0]}

Unsupported

Table of Contents

Table of Contents can be build anywhere with %%TOC. Relative linking is supported, clicking an item from the contents table will jump to the respective header when viewed in html.

Multiple features can be stacked with , like so:

%%TOC depth=3, excludes=[Appendix;Acknowledgement]

Citations

FMark supports simple footers and styled references.

• If multiple citations with the same reference ID is found, the first citation will be used inline.
• One [^fmark] inline, and two [^fmark], some author. The first will be used.

• Reference to non-exist citation will result in "[Reference: %refID not found!]" in the HTML generated.

• The list is sorted, with footnotes in order of their numerical IDs first, References in order of apperance after. Thus the order each explanatory text comes in the fmark file does not matter.

Simple Footers

Simple footers have numerical IDs.

this is inline[^1], and so on.

[^1], this is the text to explain such line.
      this line as well.
But not this line.

The body will show up with a superscripted link:

this is inline¹ , and so on.

But not this line.

The footer texts will be collected at the end of the document.

1: this is the text to explain such line. this line as well.

Styled References

Styled references have alphabetic IDs.

Supported rendering styles:

Supported data fields

Pick a style with %%RefStyle, or it will default to Harvard.

%%RefStyle = Harvard

Then follow field1=data1, field2=data2, ... to use references.

Spaces around equal sign are allowed, so f = d, f= d, f =d or f=d all have the same effect.

This is a citation[^Mark]. This is another[^FMark] one.

[^Mark], type= Book, author= Mark Smith, title= Not a real book, year= 2018
[^FMark], type= Website, author= FMark Smith, title= Not a real website, year= 2017 url= www.example.com/website access= 2018-3-4

With Harvard, it will look like this:

This is a citation(Smith, 2018). This is another(Smith, 2017) one.

At the end of the document:

Smith, M. (2018) Not a real book.

Smith, F. (2017) Not a real website. Available from: www.example.com/website [Accessed 4th March 2018].

if Chicago style is chosen:

This is a citation(Smith 2018). This is another(Smith 2017) one.

At the end of the document:

Mark Smith. 2018. Not a real book.

FMark Smith. 2017. "Not a real website." Accessed March 3, 2018. https://www.example.com/website

Latex maths rendering

Fmark includes support for Latex maths rendering.

Surround an equation written in Latex with $$ in order to use this feature.

Example: $$2+2=4-1=3$$