Replace recommonmark with myst (docs) #1021

luizirber · 2020-06-12T23:03:04Z

MyST is a new markdown parser that supports features from ReStructureText, especially Sphinx directives and roles. This might be enough to replace the last .rst document (the index), for example, while also allowing more customization lost on the rst -> md transition.

TODO

Convert index.rst to a markdown file
~~[ ] Convert api.rst to a markdown file~~ pending on Support use of autodoc with myst executablebooks/MyST-Parser#163

Checklist

Is it mergeable?
make test Did it pass the tests?
make coverage Is the new code covered?
Did it change the command-line interface? Only additions are allowed
without a major version increment. Changing file formats also requires a
major version number increment.
Was a spellchecker run on the source code and documentation after
changes were made?

codecov · 2020-06-12T23:07:23Z

Codecov Report

Merging #1021 into master will increase coverage by 8.75%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #1021      +/-   ##
==========================================
+ Coverage   83.30%   92.05%   +8.75%     
==========================================
  Files          97       72      -25     
  Lines        8749     5453    -3296     
==========================================
- Hits         7288     5020    -2268     
+ Misses       1461      433    -1028

Flag	Coverage Δ
#rusttests	`?`

Impacted Files	Coverage Δ
sourmash/_compat.py	`52.17% <0.00%> (-47.83%)`	⬇️
sourmash/signature.py	`88.42% <0.00%> (-2.32%)`	⬇️
sourmash/lca/lca_utils.py	`93.75% <0.00%> (-2.09%)`	⬇️
sourmash/lca/command_index.py	`89.83% <0.00%> (-0.54%)`	⬇️
sourmash/sbt.py	`87.32% <0.00%> (-0.28%)`	⬇️
src/core/tests/signature.rs
src/core/src/index/sbt/mod.rs
src/core/src/ffi/minhash.rs
src/core/src/cmd.rs
src/core/tests/test.rs
... and 21 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bc2b168...ada128e. Read the comment docs.

ctb · 2020-06-13T14:37:15Z

question! does it flag missing local references for you, or are you just being extra careful in your review?

luizirber · 2020-06-13T16:14:27Z

question! does it flag missing local references for you, or are you just being extra careful in your review?

A bit of both. I replaced full URLs (pointing to sourmash.readthedocs.org), which allows finding internal references (and where they are linking to anchors that don't exist anymore). I also removed the recommendation to install pre-releases of sourmash 2.0 (!!!), and fixed some notebooks that had multiple <h1>-level headings (sphinx doesn't like that).

There are still many warnings about documents not properly linked in toctrees, I think it's useful to fix them and then use sphinx-build -W (turn warnings into errors) to avoid having any warnings pop up again.

luizirber · 2020-06-13T16:25:07Z

heh, the myst parser doesn't support Python 2.7, so this broke Travis =P

luizirber · 2020-06-13T20:47:14Z

Ready for review and merge @ctb @olgabot

ctb · 2020-06-14T15:03:10Z

I'm not immediately sure why this is happening, but make doc no longer builds the docs on this branch, despite its presence in the Makefile.

ctb · 2020-06-14T15:03:48Z

oh, I see the diff... but that doesn't work for me :)

ctb · 2020-06-14T15:15:46Z

I built it locally!

Questions:

Do we want to add -W in this PR, or make an issue and do it in another PR? It looks like it works for me on a clean checkout.
Has this been built successfully on RTD?

I haven't reviewed the content changes yet, will get into that next.

ctb

Looks great, overall! A few minor questions and comments.

ctb · 2020-06-14T15:18:14Z

doc/developer.md

@@ -205,3 +205,14 @@ For the Rust core library we use `rMAJOR.MINOR.PATH`
 (note it starts with `r`, and not `v`).
 The Rust version is not automated,
 and must be bumped in `src/core/Cargo.toml`.
+


Note that this TOC ends up at the bottom of the file. Is this intentional (and if so, why 😂)

I collected orphan links into a toc, but opted to put it in the bottom to avoid disrupting the current docs. But it might be more useful to put on the top...

doc/other-languages.md

doc/release-notes/releases.md

olgabot

This is awesome! My only comment is small and nitpicky. Some places have docs plural and some have doc singular. For clarity, is it possible to have only one spelling?

ctb · 2020-06-16T14:14:09Z

@olgabot re

Some places have docs plural and some have doc singular. For clarity, is it possible to have only one spelling?

I did some grepping in the markdown and can't find places where doc, singular, was used -- could you point us at a specific example? I'm on board with picking one just want to make sure I understand :)

olgabot

Turns out all the mentions of docs were in the Travis and tox files! The documentation folder is called doc singular, so maybe that's the way to go? Maybe less of a deal than I thought initially, but it tripped me up on my first read.

olgabot · 2020-06-17T16:57:04Z

.travis.yml

+    - <<: *test
+      python: 3.7
+      end:
+        - TOXENV=docs


docs plural

olgabot · 2020-06-17T16:57:08Z

Makefile

@@ -23,7 +23,7 @@ test: all
 	$(PYTHON) -m pytest
 	cargo test

-doc: .PHONY
+doc: build


doc singular

olgabot · 2020-06-17T16:58:04Z

tox.ini

@@ -1,13 +1,12 @@
 [tox]
-envlist=py27,py35,py36,py37,py38
+envlist=py27,py35,py36,py37,py38,docs


docs plural

olgabot · 2020-06-17T16:58:10Z

tox.ini

  storage
+
+[testenv:docs]


docs plural

ctb · 2020-06-24T16:21:47Z

Issues that need to be resolved for a merge, IMO -

make doc doesn't work
how can we refer to local static HTML files? Replace recommonmark with myst (docs) #1021 (comment)
how about local CSV files? Replace recommonmark with myst (docs) #1021 (comment)

I might work on these in a separate PR (into this one).

ctb · 2020-06-27T14:40:05Z

make doc fixed in #1053; the key problem showing up with make -d was,

  Successfully remade target file `build'.
 Finished prerequisites of target file `doc'.
 Prerequisite `build' is older than target `doc'.

so basically we just need to force the cd doc && make html command. One way to do that is to add .PHONY as a prereq.

Fixes #2604 This PR fixes links to binder tutorials that apparently have been broken since 2020 😱 , when the docs were switched over to myst-parser in #1021. To be fair, it's the kind of subtle bug that would leave users scratching their head going "maybe it's me?" - the links simply went back to the tutorial page... not sure why they weren't flagged by sphinx or myst-parser as being bad!? This PR also updates environment.yml to properly build: * includes pip * specifies minimum python version * specifies minimum sourmash version ref #2503 for why specifying the minimum sourmash and/or python versions can be important! And, finally, I ran and fixed all the notebooks... Note: I confirmed that this branch launches properly in binder. The URL is: https://mybinder.org/v2/gh/sourmash-bio/sourmash/fix/tutorial-nb --------- Co-authored-by: ccbaumler <63077899+ccbaumler@users.noreply.github.com>

replace recommonmark with myst

382c630

convert index

efb4ae7

luizirber force-pushed the doc_myst branch from 109670c to cf445fa Compare June 13, 2020 01:56

fix links with html extensions

de2ec19

luizirber force-pushed the doc_myst branch from cf445fa to de2ec19 Compare June 13, 2020 02:05

fix headings for sphinx

66336ea

fix tests on py27, add docs check in travis

4365b74

luizirber force-pushed the doc_myst branch from c77ec26 to 4365b74 Compare June 13, 2020 17:17

olgabot mentioned this pull request Jun 13, 2020

Consider using MyST for Sphinx-compatible Markdown documentation czbiohub-sf/orpheum#79

Open

add docs to toctrees

e525861

luizirber changed the title ~~[WIP] Replace recommonmark with myst (docs)~~ Replace recommonmark with myst (docs) Jun 13, 2020

ctb requested changes Jun 14, 2020

View reviewed changes

luizirber mentioned this pull request Jun 15, 2020

How to refer to a static resource? executablebooks/MyST-Parser#167

Open

warnings are errors in sphinx

79e2b9f

luizirber force-pushed the doc_myst branch from 462bc38 to 79e2b9f Compare June 15, 2020 22:15

olgabot approved these changes Jun 15, 2020

View reviewed changes

olgabot reviewed Jun 17, 2020

View reviewed changes

Merge branch 'master' into doc_myst

50a859d

Merge branch 'master' into doc_myst

809955e

ctb mentioned this pull request Jun 27, 2020

improve handling of non-markup files in docs #1051

Open

This was referenced Jun 27, 2020

should we point to release notes in docs? #1052

Closed

Update #1021 with a Makefile fix. #1053

Merged

fix make doc (#1053)

ada128e

luizirber merged commit b5c3f2e into master Jun 27, 2020

luizirber deleted the doc_myst branch June 27, 2020 16:54

ctb mentioned this pull request Jul 13, 2020

Draft release notes for sourmash v3.4.0 #1100

Closed

luizirber mentioned this pull request Oct 30, 2020

[WIP] SBT scaffold #1201

Closed

10 tasks

ctb mentioned this pull request Jun 2, 2023

MRG: fix tutorial notebook links #2633

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Replace recommonmark with myst (docs) #1021

Replace recommonmark with myst (docs) #1021

luizirber commented Jun 12, 2020 •

edited by ctb

Loading

codecov bot commented Jun 12, 2020 •

edited

Loading

ctb commented Jun 13, 2020

luizirber commented Jun 13, 2020

luizirber commented Jun 13, 2020

luizirber commented Jun 13, 2020

ctb commented Jun 14, 2020

ctb commented Jun 14, 2020

ctb commented Jun 14, 2020

ctb left a comment

ctb Jun 14, 2020

luizirber Jun 15, 2020

olgabot left a comment

ctb commented Jun 16, 2020

olgabot left a comment •

edited

Loading

olgabot Jun 17, 2020

olgabot Jun 17, 2020

olgabot Jun 17, 2020

olgabot Jun 17, 2020

ctb commented Jun 24, 2020 •

edited

Loading

ctb commented Jun 27, 2020

		storage

		[testenv:docs]

Replace recommonmark with myst (docs) #1021

Replace recommonmark with myst (docs) #1021

Conversation

luizirber commented Jun 12, 2020 • edited by ctb Loading

TODO

Checklist

codecov bot commented Jun 12, 2020 • edited Loading

Codecov Report

ctb commented Jun 13, 2020

luizirber commented Jun 13, 2020

luizirber commented Jun 13, 2020

luizirber commented Jun 13, 2020

ctb commented Jun 14, 2020

ctb commented Jun 14, 2020

ctb commented Jun 14, 2020

ctb left a comment

Choose a reason for hiding this comment

ctb Jun 14, 2020

Choose a reason for hiding this comment

luizirber Jun 15, 2020

Choose a reason for hiding this comment

olgabot left a comment

Choose a reason for hiding this comment

ctb commented Jun 16, 2020

olgabot left a comment • edited Loading

Choose a reason for hiding this comment

olgabot Jun 17, 2020

Choose a reason for hiding this comment

olgabot Jun 17, 2020

Choose a reason for hiding this comment

olgabot Jun 17, 2020

Choose a reason for hiding this comment

olgabot Jun 17, 2020

Choose a reason for hiding this comment

ctb commented Jun 24, 2020 • edited Loading

ctb commented Jun 27, 2020

luizirber commented Jun 12, 2020 •

edited by ctb

Loading

codecov bot commented Jun 12, 2020 •

edited

Loading

olgabot left a comment •

edited

Loading

ctb commented Jun 24, 2020 •

edited

Loading