Directions and Updates

For version 3.2 we have a number of items on a candidate TODO list.

In addition to the list that's in the source code, we have some more fundamental changes.

1. Use Jinja for inserting position comments into tangled chunks.
2. Implement cross-references. There are four choices, and we need a way to pick one of the choices.
3. Implement @h for "tangle-only" code; code not included in the woven document.
4. Implement multiple begin_code() templates. @d -console vs. @d -codeblock kind of thing for LaTeX.

We note that handling PlantUML diagrams gracefully is a matter of expanding the toolchain to do this. It has no impact on PyWebTool. This has been done in the internal documentation and seems to work out nicely.

Jinja for Tangled Line Numbers

Currently, there are comment-start and comment-end options in an @o command, but this is inconsistent with the Jinja templates used for weaving. A better choice is -position '# {{chunk.position}}'. This means -position '/* {{chunk.position}} */' (or whatever's appropriate for the tangled language) can be used.

Note that this permits injecting other PyWebTool details into the tangled output. The net efffect is simular to including an implicit @( @) expression in the block of code without forcing the author to do this:

@o some_file.py @{
    @<a block@>
@}

@d a block @{
    # @(chunk.position@)
    the code
@}

This would work. Explicit markers have two problems: they push the comment injection into each block of code, and they make the comment marker visible in the woven output.

The idea is that the tangling operation would reference this template to inject one extra line of code at the prevailing indent.

Cross-References

There are four ways to denote cross-reference information.

• Nothing.
• The immediate reference.
• The two variants on full path references:
  • top-down → Named (1) / → Sub-Named (2) / → Sub-Sub-Named (3)
  • bottom-up → Sub-Sub-Named (3) ∈ → Sub-Named (2) ∈ → Named (1)

One of these must be part of the code_end() macro. For example, a code_xref() macro would be part of code_end(). This means picking the desired code_xref() macro definition at the command line. Select among the four defined alternatives. --xref immediate would create

{%- macro code_xref() %}{{immediate(chunk)}}{% endmacro -%}

The --xref nothing would skip the definition which would silently do nothing in Jinja.

Add Tangle-Only Output

See Tangle-only Output for details. This creates "hidden" files where parts of them are exposed but the final @o file.py construction is not woven. This is handy for books where an author has three things:

• The main text with ordinary @d name @{code@} sections that are woven into the book.
• A "code from the book" output file, with the @h examples.py @{ @<name@>, etc. @} not actually woven into the main text.
• A "test cases for the book's code" output file, with a @h test_examples.py @{ def test_ex1(): etc. @} also not woven into the main text.

This is slightly different from the original use case of all the code being present in the documentation.

Multiple Template Choices

A chunk of code can include a template selector. --style=console, --style=bash, --style=python, etc. This can be tied to pygments styling directly. Or It can lead to a :language: line in an RST weaver template. Or something a bit more complex in a LaTeX weaver template.

This option alters both begin_code() and end_code() macros. Note that all of the alternative styles must cooperate with all of the xref options, by using the code_xref() macro.

[Completed] Add PlantUML Support

See Add plantuml support for RST for details.

This is a two-step operation.

1. Decide on docutils/plantuml integration: either a separate project, or use Sphinx.
2. Import that into this project and rewrite the documents to include the PlantUML in-line, instead of in a separate c4_diagrams.uml file.

Using Sphinx has numerous advantages over "bare" docutils.

[Completed] Use JINJA Templates

To step away from the complex string.Template() complexity, we can switch to Jinja2. This actually simplifies a number of things.

This lets us (eventually) move the configuration of templates out of the code entirely.

First, however, we can then restructure a great deal of the Weaving to use Jinja {%- for -%}...{%- endfor -%} blocks. This should the weaver, and make it easier to change an RST look-and-feel, or other output-specific markup.

Defining Markup In The .w File

One idea is to create some kind of "use this around code blocks" and "use this around code references" definitions. These would eliminate the markup-specific subclasses of Tangle and Weave. It would eliminate markup detection, also.

These are "meta-macros" that provide consistent markup application without the need for repetition.

When authoring in HTML, for example, we might include the following definitions.

@d _code @{<a name="_chunk_id"></a><pre>\n{{c.body}}\n</pre>@}

And

@d _reference @{<a href="_chunk_id">{{c.name}}</a>@}

The chunk names, _code and _reference are pre-defined. They're the specific markup to use.

The names c.body and c.name are attributes of chunks. Attributes include the chunk's internal ID and the chunk's name, and the body of the chunk.

The @f, @m and @r, similarly, would benefit from some kind of markup definition in the source file, freeing the pywebtool from knowing anything about the target markup language.

Eliminating the Markup Entirely

Using something like the Pandoc (or RST) Abstract Syntax Tree can eliminate markup. The idea here is to create the AST and then emit specific markup from the AST using Pandoc. See Issue #4 for the genesis of this idea.

This may not be practical, however. The source WEB file would still need to have some inline markup that's parsed to create the Pandoc AST.