[Templates v2] Add a metadata syntax to templates

[movermeyer]

As described in #534, we want to be able to template the default filepath of a template. Further, this templated filepath metadata should live within the template files themselves.

This requires specifying a syntax for use in template files where this metadata can be written, without being included in the files that are created from the template.

Create this syntax, and use it to allow for the specification of a default filepath for a template.
You should be able to template the filepath using variables just like any other content in the template.

Therefore, the process for creating a template file is:

1. Select the template to use
2. Resolve all the values for all the variables
3. Parse the contents of the metadata section to determine the default filepath
4. Strip the metadata section from the template
5. Copy the resulting template as the contents of the new note

Example

Using a "second front-matter" mock syntax (Note: this may not be the syntax we want, but is used for demonstration):

<!-- The below front-matter block is for foam-specific template settings -->
<!-- It is removed when the user creates a new note using this template -->
---
<!-- The default filepath to use when using this template -->
<!-- Relative paths are relative to the workspace, absolute paths are absolute -->
<!-- Note that you can include VSCode snippet variables to template the path -->
filepath: `journal/${CURRENT_YEAR}-${CURRENT_MONTH}-${CURRENT_DATE}_${FOAM_TITLE_SLUG}.md`
---

<!-- The actual contents of the template begin after the `---` thematic break immediately below this line-->
---
---
created: ${CURRENT_YEAR}-${CURRENT_MONTH}-${CURRENT_DATE}T${CURRENT_HOUR}:${CURRENT_MINUTE}:${CURRENT_SECOND}
tags: []
---

# ${FOAM_TITLE}

[jevakallio]

Hey @movermeyer! @riccardoferretti just gave me a walkthrough of the spec, thank you so much for working on this!

The spec all makes sense to me. I was wondering, what is the purpose of the --- thematic break separator between the "metadata frontmatted" and the "document frontmatter" blocks? If that is required for the markdown parser to read the document, that's fine. If it's not necessary, I'd perhaps opt out of it and simply have two frontmatter blocks run back-to-back?

---
filepath: "..."
---

---
tags: []
---

# Heading

Great work on the specification, I like the direction and the way it unifies the handling of different features!

[jevakallio]

Further, a suggestion:

Since we're already adding template metadata, I wonder if we could also add fields such as name and description, that would be displayed in VS Code command palette after running the "Create New Note From Template" command, instead of the template file name?

[movermeyer]

The spec all makes sense to me. I was wondering, what is the purpose of the --- thematic break separator between the "metadata frontmatted" and the "document frontmatter" blocks? If that is required for the markdown parser to read the document, that's fine. If it's not necessary, I'd perhaps opt out of it and simply have two frontmatter blocks run back-to-back?

@jevakallio We can do whatever makes sense. The syntax shown there is purely hypothetical to illustrate the point.

I wonder if we could also add fields such as name and description, that would be displayed in VS Code command palette after running the "Create New Note From Template" command, instead of the template file name?

Neat. I think I like that.

[movermeyer]

Here are some initial ideas for the syntax.
These are very much my personal opinions and draft ideas. Nothing is decided.
More thinking is required.
Input is welcome.

---

Ideals

In descending order of importance:

• Ideal 1: The metadata should be easy to read / understand
• Ideal 2: The metadata should be easy to remove so that it doesn't appear in the resulting note
  • The metadata MUST NOT exist in the resulting note
• Ideal 3: The metadata should be obviously Foam-related
• Ideal 4: The metadata should look nice
• Ideal 5: The metadata should not mess up other Markdown parsers

A note on Markdown frontmatter

There is no formal specification of frontmatter and it seems that CommonMark isn't interested in specifying it either.
It doesn't have to be YAML...some people use JSON, CoffeeScript, Javascript, TOML, and more
That said, it seems that YAML is the most common format.

Idea 1: Use a second frontmatter block

Put the foam-specific metadata into its own frontmatter block at the start of the file.
Remove it when the user creates a new note using this template.

---
foam_template:
  filepath: 'journal/${CURRENT_YEAR}-${CURRENT_MONTH}-${CURRENT_DATE}-${FOAM_TITLE}.md'
---

---
tags:
  - movies
  - top 10
  - guy_ritchie
---

Q: If the file only has one frontmatter block, how do you tell if it is Foam related or not?

A: That is why the attributes are namespaced under a foam_template attribute.
If the frontmatter contains the foam_template attribute, then we assume that it is a Foam-specific frontmatter block.

Benefits

• Simple to remove: Delete everything between the --- delimiters

Detriments

• IMO it doesn't look great
• Markdown parsers expect there to only be a single frontmatter block (breaks Ideal 5)

Idea 2: Use attributes in a single frontmatter block

Place any Foam-specific metadata in the same frontmatter block as everything else. If there is not frontmatter in the template, but there are Foam-specific metadata, then a frontmatter block is added specifically to hold the Foam-specific metadata.

The metadata are namespaced under a foam_template attribute that is parsed and removed during note creation

---
tags:
  - movies
  - top 10
  - guy_ritchie
foam_template:
  filepath: 'journal/${CURRENT_YEAR}-${CURRENT_MONTH}-${CURRENT_DATE}-${FOAM_TITLE}.md'
---

In cases where the frontmatter only contains foam_template attribute, then the entire frontmatter block is removed when the template is used to create the note.

Benefits

• Won't mess up other parsers. It's just a regular YAML frontmatter.
• IMO, it looks nicer

Detriments

• You can't nicely remove the foam_template attributes without messing up the rest of the frontmatter.
  • IMO, this is a deal breaker 😢

Programmatic modification of YAML

YAML is notoriously difficult to programmatically manipulate. Removing foam_template without disrupting the rest of the YAML is not really feasible due to YAML's complexity.

The naive algorithm for doing this:

parsed = YAML.parse(frontmatter)
parsed.delete("foam_template")
new_frontmatter = YAML.dump(parsed)

Will produce dramatically different frontmatter than what was parsed.

For example:

---
tags: # Add tags to increase SEO.
  - movies
  - top 10
  - guy_ritchie
metadata: &info
  title: The Gentlemen
  year: 2019
more_metadata: *info # Required by our IMDB parser
foam_template:
  filepath: 'journal/${CURRENT_YEAR}-${CURRENT_MONTH}-${CURRENT_DATE}-${FOAM_TITLE}.md'
---

Might become:

---
metadata: &1
  title: The Gentlemen
  year: 2019
tags: [movies, top 10, guy_ritchie]
more_metadata: *1
---

Note that they comments, list style, key order, and anchor/alias names were all lost in this roundtrip.

This is because, by the YAML spec, parsers are required not to give any consideration to these stylistic elements when serializing.
Therefore, the YAML serializer doesn't have all the metadata needed to be able to recreate the original YAML with minimal changes.

Trust me; I've spent far too much of my career wrangling with YAML parsers. 😢

Idea 3: Something else?

INI, JSON, or some other format???

[riccardoferretti]

I like approach n. 2 for all the reasons you mentioned.
In terms of cleaning up the metadata, what if we apply a simple algorithm along the lines of:

1. look for the line in YAML with the regex ^foam_template:$ (or, maybe better, even just ^foam:$)
2. if found delete all the following lines that start with a whitespace

That should:

1. be standard YAML
2. be easy to implement
3. allow to keep format of original YAML properties in generated note

thoughts?

[movermeyer]

@riccardoferretti I've implemented a first draft of this in #655.

It allows the metadata to be added to an existing YAML frontmatter block, but only using a very limited syntax.
If users cannot use this limited syntax for any reason, they can add the metadata as a second YAML frontmatter block at the start of the template.

There's still more to do in that PR, but it shows the direction I'm headed so far.

[movermeyer]

Closing this now, since we have the ability to add metadata.
I opened a follow up issue here.