feat(core): Implement paragraph shaping (parshape)

[Omikhleia]

These are the changes I made to the linebreak algorithm as mentioned in #1263

I am suggesting them for inclusion in the standard SILE. They are pretty limited, @simoncozens had already prepared all the way to them. They are a good basis to start playing with, I hope.

• I renounced to use a fixed pre-computed array as TeX does, a function (ideally "memoizing" already computed lines) is much easier and we have all the power of Lua to make fancy things.
• On the way, I cleaned up a bit the hangafter/hangindent stuff I had earlier done, it should be easier to maintain and understand this way.
• The small change to the typesetter also makes things easier IMHO, as the right/left skips are handled differently in TeX and SILE.
• A test is included, using a fixed pre-computed circle (that's better for a non-regression test!)

As for an actual package using this feature and computing things dynamically... Honestly, I am not sure my current colophon package should make it into the standard SILE now... I mean, it works quite nicely for my needs, but it is quite fragile with several wild assumptions. I don't plan something better for now, but I committed it to my own package repo - So you may look at the PDF in the "examples" folder (for the Woah! effect, I hope 😄 ) and to the actual documentation in the "docs" package (to see why and where it could be so fragile that it might be a mess to maintain in the official distribution), and make your own opinion. And beyond the inherent flaws of the approach, it's also fairly specialized (with "ornaments" I selected, etc.), and I might also add more shapes eventually (triangles/trapezes obviously come to mind), if I ever need them... at some uncertain point... But this just express my concerns and I am open for discussion.

[Omikhleia]

Fixed a failing test. Forgot we add other ways to break lines (notably the break-firstfit) that do not use the Knuth linebreaking.

[Omikhleia]

Still failing on Lua 5.1 and LuaJit on a simple comparison. Hmm I am at loss here, and don't have Lua 5.1 on this machine.

[Omikhleia]

Still failing on Lua 5.1 and LuaJit on a simple comparison. Hmm I am at loss here, and don't have Lua 5.1 on this machine.

Doh, fixed and rebased. Sorry for the noise. Something with bidi.sil still fails on luajit-openresty. Tricky.

[Omikhleia]

Following #1268 I realize I should check what happens to headers/folios/footnotes when this shaping is active when the page break occurs (and the current solution for hangindent/hangafter fix in footnotes and folios should also probably be addressed, it's kind of ad hoc). Kind of minor (I don't think users would normally use the parshaping accross pages, but heh).

[ctrlcctrlv]

Neat feature, but I think the code needs work before it enters SILE core.

Also, was any thought given as to how this works in TTB mode?

[Omikhleia]

Also, was any thought given as to how this works in TTB mode?

Nope. I don't know how to check RTL, TTB, BTB with scripts I can't even read. The line-breaking algorithm is agnostic to the writing direction, so I would expect it to somehow work, but I have no real idea what it implies.

[alerque]

Thanks for the contribution, this sounds great. Sorry I had been traveling and haven't been very on top of reviews, but I'll get this checked out eventually.

[alerque]

The example of the circle was really cool, but that's what I think it should be: an example. I'm going to move it to the example gallery and drastically simplify the test case to something where the numbers can be debugged easier by hand. Basically just enough to prove that the shape table is being used but not much more.

[alerque]

I'm happy to make these changes ... and probably will right now at least as a POC, but I'm open to feedback on the changes I'm proposing here.

[alerque]

I'm working on porting the circle example to something more suited to be adjustable and show off the caching, but my ideas for changing the parShape API are implemented above. Feedback welcome, I don't claim to have hit this nail on the head.

[alerque]

Here is an example using the modified parshape API drawing a paragraph in a circle using a center point and radius:

\begin[papersize=a6]{document}
\language[main=en]
\hyphenator:add-exceptions[lang=en]{iso-pe-ri-me-tric}% No idea where that weird word should really break.
\nofolios
\begin{script}

SILE.settings.set("linebreak.tolerance", 200)
SILE.settings.set("document.baselineskip", "1.6em")
SILE.settings.set("document.parindent", 0)

-- The math here finds intersection points for a line crossing a circle, making a bunch of assumptions about
-- the line always being horizontal. Only touching doesn't count, only a full intersection is a win for this.
-- http://csharphelper.com/blog/2014/09/determine-where-a-line-intersects-a-circle-in-c/
local sliceofcircle = function (c, r, h)
  local cx, cy, y = c, r, h
  local B = 2 * -cx
  local C = cx * cx + (y - cy) * (y - cy) - r * r
  local det = B * B - 4 * C
  if det <= 0 then
    return false
  else
    local enter = (-B - math.sqrt(det)) / 2
    local exit = (-B + math.sqrt(det)) / 2
    return enter, exit
  end
end

SILE.registerCommand("incircle", function (options, content)
  local center = SU.cast("measurement", options.center or "50%fw")
  local radius = SU.cast("measurement", options.radius or "50%fw")
  local oldParShape = SILE.linebreak.parShape
  SILE.typesetter:leaveHmode()
  SILE.settings.set("linebreak.parShape", true)
  SILE.linebreak.parShape = function (self, line)
    local c = SU.cast("number", center)
    local r = SU.cast("number", radius)
    local h = SILE.measurement((line-1).."bs"):tonumber()+3
    local enter, exit = sliceofcircle(c, r, h)
    if not enter then return 0, self.hsize, 0 end
    local width = SILE.measurement(exit - enter)
    return enter, width, nil
  end
  SILE.process(content)
  SILE.typesetter:leaveHmode();
  SILE.linebreak.parShape = oldParShape
  SILE.linebreak.parShapeCacheClear()
  SILE.settings.set("linebreak.parShape", false)
end, "Format paragraphs in a fixed circle")

\end{script}
% Galileo, 1638:
\begin[radius=30%fw]{incircle}
The area of a circle is a mean proportional between any two regular and similar
polygons of which one circumscribes it and the other is isoperimetric with it.
In addition, the area of the circle is less than that of any circumscribed
polygon and greater than that of any isoperimetric polygon. And further, of
these circumscribed polygons, the one that has the greater number of sides has
a smaller area than the one that has a lesser number; but, on the other hand,
the isoperimetric polygon that has the greater number of sides is the larger.
\end{incircle}
\end{document}

[Omikhleia]

Here is an example using the modified parshape API drawing a paragraph in a circle using a center point and radius:
...
SILE.linebreak.parShape = oldParShape
SILE.linebreak.parShapeCacheClear()

The explicit cache cleaning sounds real weird. Also, the example could set the settings temporarily so that the default parshaping stops being called afterwards.

[Omikhleia]

Also this:

SILE.settings.set("linebreak.tolerance", 200)
SILE.settings.set("document.baselineskip", "1.6em")
SILE.settings.set("document.parindent", 0)

Could be done temporarily in "incircle". The example is not very useful otherwise - one would not want to have this globally in a normal use case. (If one does so, we'll get question why plenty of other things starts working weird - a non-stretchable baselineskip gives pretty awful results on pages with footnotes etc.)

[alerque]

The explicit cache cleaning sounds real weird.

I grant the interface perhaps could use refining, but I couldn't think of any way to get the benefit of cached results but also allow the use of new shapes after the first one without some sort of cache flush mechanism. The SILE.linebreak object is almost completely stateless.

Also, the example could set the settings temporarily so that the default parshaping stops being called afterwards.

True, before that example actually hits the gallery issues like that should be addressed and it could be commented etc. I just needed a POC that was a little more like somebody might use it rather than the test file which did the minimal work possible to hit all the possible interfaces.

Could be done temporarily in "incircle". The example is not very useful otherwise - one would not want to have this globally in a normal use case.

Of course.

(If one does so, we'll get question why plenty of other things starts working weird - a non-stretchable baselineskip gives pretty awful results on pages with footnotes etc.)

Sure. The \incircle function could take care of using the current baselineskip but temporarily dropping any strech/shrink. That sounds like a useful addition before the example goes in the gallery. But that is for a PR on the website repo...

Lets keep this issue more about the API and whether the right things are exposed vs. handled.

[Omikhleia]

The explicit cache cleaning sounds real weird.

I grant the interface perhaps could use refining, but I couldn't think of any way to get the benefit of cached results but also allow the use of new shapes after the first one without some sort of cache flush mechanism. The SILE.linebreak object is almost completely stateless.

At the end of the loop in lineBreak:doBreak() , the paragraph has its final breaks all computed (which lineBreak:postLineBreak() just builds into a result list), so that's perhaps where your cache could be cleared. Next time the algorithm is invoked (for a subsequent paragraph), the inputs (hsize and sideways) could be different so the cache would need to be invalidated. True, in many cases they'd be the same (e.g. in a sequence of identical paragraphs), but it cannot be guaranteed (the next paragraph submitted to lineBreak:doBreak() by the typesetter could have a different size).

[alerque]

Also, was any thought given as to how this works in TTB mode?

@ctrlcctrlv Yes. We're just twiddling standard parameters already used used "linewise" by the typesetter. The typesetter already handles flipping them around based on the writing and advance directions.