Skip to content

Commit

Permalink
Experimental nav optimization for simple cases (mmistakes#992)
Browse files Browse the repository at this point in the history 
* Optimize simple navigation cases

Fix inefficiency reported in feedback on v0.4.0.rc2 (see discussion mmistakes#958).

This PR:

* essentially reverts `_includes/nav.html` to v0.4.0.rc1
* preserves the ARIA labels added by mmistakes#950
* adds a test to optimize builds of sites that rely on `title` fields to order pages.

Building the `endoflife.date` site (130 pages) now takes only about 7 seconds.

Building the `machinetranslate.org` site ( 350 pages) takes about 7 minutes. (Without the added test, it takes just over 5 minutes: the condition of the test is merely to compare the size of two arrays, but that is apparently enough to prevent Jekyll from applying some optimization).

A warning is added to the docs about the need for numbers to be in quotes when used as title values.

* Update navigation-structure.md

A clarification is added to the docs about the need for numbers to be in quotes when used as title values.

* Simplify the control and data flow

- Defer concatenation of `string_order_pages` with `title_order_pages` until needed.
- Replace tests on size with tests for `empty`.
- Rename variables accordingly.

* Fix child nav order

This PR started from the navigation in RC1. Some cosmetic improvements had been made in RC2. This commit adds some of those changes to this PR.

It also fixes a bug (revealed by a new regression test) due to a reference to `node.child_nav_order` instead of `child.child_nav_order`, which prevented reversal of the order in children of children. Presumably a top-level reversal should apply only to direct children, and not to grandchildren. The latter interpretation would be very confusing in a deep multi-level hierarchy.

* Allow pages with numeric titles

An omitted `nav_order` value should default to the `title` value, regardless of its type. Jekyll 3 gives build errors when numbers and strings are sorted together. This commit drops the assumption that `title` values are always strings – a 404 page naturally has a numeric title. It updates the docs page accordingly.

The extra code does not affect the build time for the `endoflife.date` site (7 seconds). For the `machinetranslate` site, changing the title of the 404 page to a number increases the build time from 7 minutes to 9 minutes – the `nav_order` numbers on that site are program-generated in the range 1..1000, which might be atypical.

This commit has not yet been checked using the regression tests.

The gemspec used for testing specifies `spec.add_runtime_dependency "jekyll", "~> 3.8.5"`, and `Gemfile.lock` shows `jekyll (3.8.7)`.

* Update nav.html

Add comment about an optimization that will be possible in Jekyll 4.

* Update nav.html

- Update the comment about optimization possibility.
- TEMPORARILY add Jekyll 3 code for conditionally optimizing.

* Update nav.html

Minor improvements and cosmetic changes.

* Major revision

This update is based on extensive experimentation and profiling with alternative versions of the Liquid code used to build the main navigation panel.

Due to the fragility of Jekyll's optimizations, combining alternative approaches with conditionals turned out to be too expensive: merely adding a condition to check whether some array of pages is empty can add about 20% to the build time!

The current code avoids sorting pages on `nav_order` and `title` fields together. The standard way of doing that in Jekyll is to use the `group_by` filter; but extracting the sorted pages from the groups turned out to be too inefficient (as seen in RC1), as was generating links directly from the groups (in RC2).

Making all pages with `nav_order` values come before all those ordered by their `title` values is not ideal (it doesn't support tweaking the relative order of two pages in a list of pages ordered by their titles) but it appears to be necessary for efficient builds on large sites.

This version has not yet been fully tested for regression, but otherwise seems to give the expected navigation on the endoflife.date and machinetranslate websites. (I'm unable to install the Python-based how2data repository on my laptop, due to package version issues on Apple silicon).

Co-authored-by: Peter Mosses <18308236+pdmosses@users.noreply.github.com>
  • Loading branch information
@mattxwang @pdmosses
mattxwang and pdmosses committed Oct 10, 2022
1 parent 48f7295 commit b4b951f
Show file tree
Hide file tree
Showing 2 changed files with 161 additions and 113 deletions.
262 changes: 154 additions & 108 deletions _includes/nav.html
View file
Original file line number Diff line number Diff line change
@@ -1,73 +1,140 @@
{%- comment -%}
Pages with no `title` are implicitly excluded from the navigation.
The `nav_order` values of pages affect the order in which they are shown in
the navigation panel and in the automatically generated tables of contents.
Sibling pages with the same `nav_order` value may be shown in any order.
Sibling pages with no `nav_order` value are shown after all pages that have
explicit `nav_order` values, ordered by their `title` values.

The values of `title` and `nav_order` can be numbers or strings.
Jekyll gives build failures when sorting on mixtures of different types,
so numbers and strings need to be sorted separately.
The `nav_order` and `title` values can be numbers or strings. To avoid build
failures, we sort numbers and strings separately. We sort numbers by their
values, and strings lexicographically. The case-sensitivity of string sorting
is determined by the configuration setting of `nav_sort`. Pages with no `title`
value are excluded from the navigation.

Note: Numbers used as `title` or `nav_order` values should not be in quotes,
unless you intend them to be lexicographically ordered. Numbers are written
without spaces or thousands-separators. Negative numbers are preceded by `-`.
Floats are written with the integral and fractional parts separated by `.`.
(Bounds on the magnitude and precision are presumably the same as in Liquid.)
{%- endcomment -%}

Here, numbers are sorted by their values, and come before all strings.
An omitted `nav_order` value is equivalent to the page's `title` value
(except that a numerical `title` value is treated as a string).
{%- assign title_pages = include.pages
| where_exp: "item", "item.title != nil" -%}

The case-sensitivity of string sorting is determined by `site.nav_sort`.
{%- comment -%}
A page with `nav_exclude: true` does not appear in the main navigation.
If it has a `parent`, it may appear in the parent's table of contents.
If it specifies `has_children: true`, it should appear in the breadcrumbs
of the child pages, but its order in relation to other pages is irrelevant.
Pages that never appear can be removed from the pages that need to be sorted.
This optimisation can be significant on a site with many pages.

In Jekyll 4, the pages to be sorted can be filtered by:

{%- assign title_pages = title_pages
| where_exp: "item", "item.nav_exclude != true or item.parent != nil" -%}

That filter is not allowed in Jekyll 3. The following iterative code gives the
same effect, but it is activated only when it will filter more than 50% of the
pages.
{%- endcomment -%}

{%- assign titled_pages = include.pages
| where_exp: "item", "item.title != nil" -%}
{%- assign unsorted_pages = title_pages
| where_exp: "item", "item.parent == nil"
| where_exp: "item", "item.nav_exclude == true"-%}
{%- assign title_pages_size = title_pages.size -%}
{%- assign unsorted_pages_percent = unsorted_pages.size
| times: 100 | divided_by: title_pages_size -%}
{%- if unsorted_pages_percent > 50 -%}
{%- assign sorted_pages = "" | split: "" -%}
{%- for item in title_pages -%}
{%- if item.nav_exclude != true or item.parent -%}
{%- assign sorted_pages = sorted_pages | push: item -%}
{%- endif -%}
{%- endfor -%}
{%- assign title_pages = sorted_pages -%}
{%- endif -%}

{%- assign string_ordered_pages = titled_pages
{%- assign nav_order_pages = title_pages
| where_exp: "item", "item.nav_order != nil" -%}
{%- assign title_order_pages = title_pages
| where_exp: "item", "item.nav_order == nil" -%}
{%- assign nav_ordered_pages = titled_pages
| where_exp: "item", "item.nav_order != nil" -%}

{%- comment -%}
Add the nav-ordered pages to the number-ordered pages or the string-ordered pages,
depending on their `nav_order` value.
Divide the arrays of `nav_order_pages` and `title_order_pages` according to
the type of value.

The first character of the `jsonify` result is `"` only for strings.
The first character of the result of `jsonify` is `"` only for strings.
Grouping by a single character also ensures the number of groups is small.
{%- endcomment -%}

{%- assign nav_ordered_groups = nav_ordered_pages
{%- assign nav_number_pages = "" | split: "" -%}
{%- assign nav_string_pages = "" | split: "" -%}
{%- assign nav_order_groups = nav_order_pages
| group_by_exp: "item", "item.nav_order | jsonify | slice: 0" -%}
{%- for group in nav_order_groups -%}
{%- if group.name == '"' -%}
{%- assign nav_string_pages = group.items -%}
{%- else -%}
{%- assign nav_number_pages = nav_number_pages | concat: group.items -%}
{%- endif -%}
{%- endfor -%}

{%- unless nav_number_pages == empty -%}
{%- assign nav_number_pages = nav_number_pages | sort: "nav_order" -%}
{%- endunless -%}

{%- unless nav_string_pages == empty -%}
{%- if site.nav_sort == 'case_insensitive' -%}
{%- assign nav_string_pages = nav_string_pages | sort_natural: "nav_order" -%}
{%- else -%}
{%- assign nav_string_pages = nav_string_pages | sort: "nav_order" -%}
{%- endif -%}
{%- endunless -%}

{%- assign number_ordered_pages = "" | split: "" -%}
{%- for group in nav_ordered_groups -%}
{%- assign title_number_pages = "" | split: "" -%}
{%- assign title_string_pages = "" | split: "" -%}
{%- assign title_order_groups = title_order_pages
| group_by_exp: "item", "item.title | jsonify | slice: 0" -%}
{%- for group in title_order_groups -%}
{%- if group.name == '"' -%}
{%- assign string_ordered_pages = string_ordered_pages | concat: group.items -%}
{%- assign title_string_pages = group.items -%}
{%- else -%}
{%- assign number_ordered_pages = number_ordered_pages | concat: group.items -%}
{%- assign title_number_pages = title_number_pages | concat: group.items -%}
{%- endif -%}
{%- endfor -%}

{%- assign sorted_number_ordered_groups = number_ordered_pages
| sort: "nav_order" | group_by: "nav_order" -%}
{%- unless title_number_pages == empty -%}
{%- assign title_number_pages = title_number_pages | sort: "title" -%}
{%- endunless -%}

{%- comment -%}
Group the string-ordered pages by `nav_order`, if non-nil, and otherwise `title`
(but appending the empty string to a numeric title to convert it to a string).

Then sort the groups according to the site setting for case (in)sensitivity.
{%- endcomment -%}
{%- unless title_string_pages == empty -%}
{%- if site.nav_sort == 'case_insensitive' -%}
{%- assign title_string_pages = title_string_pages | sort_natural: "title" -%}
{%- else -%}
{%- assign title_string_pages = title_string_pages | sort: "title" -%}
{%- endif -%}
{%- endunless -%}

{%- assign string_ordered_groups = string_ordered_pages
| group_by_exp:"item", "item.nav_order | default: item.title | append: '' " -%}
{%- assign pages_list = nav_number_pages | concat: nav_string_pages
| concat: title_number_pages | concat: title_string_pages -%}

{%- if site.nav_sort == 'case_insensitive' -%}
{%- assign sorted_string_ordered_groups = string_ordered_groups
| sort_natural: "name" -%}
{%- else -%}
{%- assign sorted_string_ordered_groups = string_ordered_groups
| sort:"name" -%}
{%- endif -%}
{%- assign first_level_pages = pages_list
| where_exp: "item", "item.parent == nil" -%}
{%- assign second_level_pages = pages_list
| where_exp: "item", "item.parent != nil"
| where_exp: "item", "item.grand_parent == nil" -%}
{%- assign third_level_pages = pages_list
| where_exp: "item", "item.grand_parent != nil" -%}

{%- assign groups_list = sorted_number_ordered_groups
| concat: sorted_string_ordered_groups -%}
{%- comment -%}
The order of sibling pages in `pages_list` determines the order of display of
links to them in lists of navigation links and in auto-generated TOCs.
{%- endcomment -%}

<ul class="nav-list">
{%- for node_group in groups_list -%}
{%- for node in node_group.items -%}
{%- if node.parent == nil -%}
{%- unless node.nav_exclude -%}
{%- for node in first_level_pages -%}
{%- unless node.nav_exclude -%}
<li class="nav-list-item{% if page.collection == include.key and page.url == node.url or page.parent == node.title or page.grand_parent == node.title %} active{% endif %}">
{%- if node.has_children -%}
<a href="#" class="nav-list-expander" aria-label="toggle links in {{ node.title }} category">
Expand All @@ -76,13 +143,8 @@
{%- endif -%}
<a href="{{ node.url | relative_url }}" class="nav-list-link{% if page.url == node.url %} active{% endif %}">{{ node.title }}</a>
{%- if node.has_children -%}
{%- assign children_list = "" | split: "" -%}
{%- for parent_group in groups_list -%}
{%- assign children_list = children_list
| concat: parent_group.items
| where: "parent", node.title
| where_exp:"item", "item.grand_parent == nil" -%}
{%- endfor -%}
{%- assign children_list = second_level_pages
| where: "parent", node.title -%}
{%- if node.child_nav_order == 'desc' -%}
{%- assign children_list = children_list | reverse -%}
{%- endif -%}
Expand All @@ -97,21 +159,17 @@
{%- endif -%}
<a href="{{ child.url | relative_url }}" class="nav-list-link{% if page.url == child.url %} active{% endif %}">{{ child.title }}</a>
{%- if child.has_children -%}
{%- assign grandchildren_list = "" | split: "" -%}
{%- for grandparent_group in groups_list -%}
{%- assign grandchildren_list = grandchildren_list
| concat: grandparent_group.items
| where: "parent", child.title
| where: "grand_parent", node.title -%}
{%- endfor -%}
{%- if node.child_nav_order == 'desc' -%}
{%- assign grandchildren_list = grandchildren_list | reverse -%}
{%- assign grand_children_list = third_level_pages
| where: "parent", child.title
| where: "grand_parent", node.title -%}
{%- if child.child_nav_order == 'desc' -%}
{%- assign grand_children_list = grand_children_list | reverse -%}
{%- endif -%}
<ul class="nav-list">
{%- for grandchild in grandchildren_list -%}
{%- unless grandchild.nav_exclude -%}
<li class="nav-list-item {% if page.url == grandchild.url %} active{% endif %}">
<a href="{{ grandchild.url | relative_url }}" class="nav-list-link{% if page.url == grandchild.url %} active{% endif %}">{{ grandchild.title }}</a>
{%- for grand_child in grand_children_list -%}
{%- unless grand_child.nav_exclude -%}
<li class="nav-list-item {% if page.url == grand_child.url %} active{% endif %}">
<a href="{{ grand_child.url | relative_url }}" class="nav-list-link{% if page.url == grand_child.url %} active{% endif %}">{{ grand_child.title }}</a>
</li>
{%- endunless -%}
{%- endfor -%}
Expand All @@ -123,65 +181,53 @@
</ul>
{%- endif -%}
</li>
{%- endunless -%}
{%- endif -%}
{%- endfor -%}
{%- endfor -%}
{%- assign nav_external_links = site.nav_external_links -%}
{%- for node in nav_external_links -%}
<li class="nav-list-item external">
<a href="{{ node.url | absolute_url }}" class="nav-list-link external">
{{ node.title }}
{% unless node.hide_icon %}<svg viewBox="0 0 24 24" aria-labelledby="svg-external-link-title"><use xlink:href="#svg-external-link"></use></svg>{% endunless %}
</a>
</li>
{%- endfor -%}
{%- endunless -%}
{%- endfor -%}
{%- assign nav_external_links = site.nav_external_links -%}
{%- for node in nav_external_links -%}
<li class="nav-list-item external">
<a href="{{ node.url | absolute_url }}" class="nav-list-link external">
{{ node.title }}
{% unless node.hide_icon %}<svg viewBox="0 0 24 24" aria-labelledby="svg-external-link-title"><use xlink:href="#svg-external-link"></use></svg>{% endunless %}
</a>
</li>
{%- endfor -%}
</ul>

{%- if page.collection == include.key -%}
{%- comment -%}
`page.collection` is the name of the Jekyll collection that contains the page,
if any, and otherwise nil. Similarly for `include.key`.

If the current page is in the collection (if any) whose navigation is currently
being generated, the following code sets `first_level_url` to the URL used in
the page's top-level breadcrumb (if any), and `second_level_url` to that used
in the page's second-level breadcrumb (if any).

For pages with children, the code also sets `toc_list` to the list of child pages.
{%- endcomment -%}

{%- for node_group in groups_list -%}
{%- for node in node_group.items -%}
{%- if node.parent == nil -%}
{%- if page.grand_parent == node.title
or page.parent == node.title
and page.grand_parent == nil -%}
{%- if page.collection == include.key -%}
{%- for node in first_level_pages -%}
{%- if page.grand_parent == node.title or page.parent == node.title and page.grand_parent == nil -%}
{%- assign first_level_url = node.url | relative_url -%}
{%- endif -%}
{%- if node.has_children -%}
{%- assign children_list = "" | split: "" -%}
{%- for parent_group in groups_list -%}
{%- assign children_list = children_list | concat:
parent_group.items | where: "parent", node.title -%}
{%- endfor -%}
{%- if node.child_nav_order == 'desc' -%}
{%- assign children_list = children_list | reverse -%}
{%- endif -%}
{%- assign children_list = second_level_pages | where: "parent", node.title -%}
{%- for child in children_list -%}
{%- if child.has_children -%}
{%- if page.url == child.url
or page.parent == child.title
and page.grand_parent == child.parent -%}
{%- if page.url == child.url or page.parent == child.title and page.grand_parent == child.parent -%}
{%- assign second_level_url = child.url | relative_url -%}
{%- endif -%}
{%- endif -%}
{%- endfor -%}
{%- endif -%}
{%- endif -%}
{%- endfor -%}
{%- endfor -%}

{% if page.has_children == true and page.has_toc != false %}
{%- assign toc_list = "" | split: "" -%}
{%- for parent_group in groups_list -%}
{%- assign toc_list = toc_list
| concat: parent_group.items
| where: "parent", page.title
| where: "grand_parent", page.parent -%}
{%- endfor -%}
{%- if node.child_nav_order == 'desc' -%}
{%- if page.has_children == true and page.has_toc != false -%}
{%- assign toc_list = pages_list
| where: "parent", page.title
| where: "grand_parent", page.parent -%}
{%- if page.child_nav_order == "desc" -%}
{%- assign toc_list = toc_list | reverse -%}
{%- endif -%}
{%- endif -%}

{%- endif -%}
12 changes: 7 additions & 5 deletions docs/navigation-structure.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -44,11 +44,10 @@ nav_order: 4

The parameter values determine the order of the top-level pages, and of child pages with the same parent. You can reuse the same parameter values (e.g., integers starting from 1) for the child pages of different parents.

The parameter values can be numbers (integers, floats) and/or strings. When you omit `nav_order` parameters, they default to the titles of the pages, which are ordered alphabetically. Pages with numerical `nav_order` parameters always come before those with strings or default `nav_order` parameters. If you want to make the page order independent of the page titles, you can set explicit `nav_order` parameters on all pages.
The parameter values can be numbers (integers, floats) and/or strings. Pages with numerical `nav_order` parameters always come before those with string `nav_order` parameters. When you omit `nav_order` parameters, they default to the titles of the pages. If you want to make the page order independent of the page titles, you can set explicit `nav_order` parameters on all pages. All pages with explicit `nav_order` parameters
come before all pages ordered by their `title` values.

By default, all Capital letters come before all lowercase letters; you can add `nav_sort: case_insensitive` in the configuration file to ignore the case. Enclosing strings in quotation marks is optional.

> _Note for users of previous versions:_ `nav_sort: case_insensitive` previously affected the ordering of numerical `nav_order` parameters: e.g., `10` came before `2`. Also, all pages with explicit `nav_order` parameters previously came before all pages with default parameters. Both were potentially confusing, and they have now been eliminated.
By default, all Capital letters come before all lowercase letters; you can add `nav_sort: case_insensitive` in the configuration file to ignore the case. Enclosing strings in (single or double) quotation marks is optional. Numeric values are not enclosed in quotation marks, e.g., `42`, `-1.0`; numbers in quotation marks are lexicographically ordered, so `"10"` comes before `"2"`, for example.

---

Expand All @@ -70,7 +69,7 @@ nav_exclude: true

The `nav_exclude` parameter does not affect the [auto-generating list of child pages](#auto-generating-table-of-contents), which you can use to access pages excluded from the main navigation.

Pages with no `title` are automatically excluded from the navigation.
Pages with no `title` are automatically excluded from the main navigation.

---

Expand Down Expand Up @@ -229,6 +228,9 @@ This would create the following navigation structure:
+-- ..
```

{: .note }
Currently, the navigation structure is limited to 3 levels: grandchild pages cannot themselves have child pages.

---

## Auxiliary Links
Expand Down

0 comments on commit b4b951f

Please sign in to comment.