New section type that doesn't minimise

[ftyers]

At the moment we add regexes in sections. Minimising regexes takes a long time. So perhaps we could have a special type="regex" section that does not minimise, it would speed up compilation of regex-heavy dictionaries.

This will likely break binary compatibility.

[mr-martian]

or it could union with some other section after that section has been minimized to avoid having to create a new section in the binary.

[ftyers]

@mr-martian that sounds a bit more complicated. Also, it would be cool to be able to give weights to sections, but I'll open another issue for that.

[mr-martian]

Upon poking around a bit, I've determined that this would not break the binary format, since section types are just encoded as strings and lt-proc already handles multiple sections of the same type. Have lt-comp relabel type="regex" to type="standard" would result in complete backwards compatibility, or lt-proc can just recognize section names ending in @regex and treat them like @standard.

Either way, this should probably be accompanied by a way to mark <pardef>s as non-minimizing for the same reason. regex="yes", perhaps.

[TinoDidriksen]

This should be optional. For development it should be fast to compile and test, but for distribution it should heavily optimize to the smallest/fastest output binary.

[mr-martian]

Also, it occurs to me that this is tricky because lt-comp minimizes each pardef separately in addition to each section.

[unhammer]

But this is about speed – is minimising each pardef on its own slow? (Last time I checked, the section minimisation at the end was the slow step.)

[mr-martian]

Another alternative is that 0493630 added the ability to compile dictionaries in several pieces, which should alleviate the burden of frequently recompiling the regex sections.

[mr-martian]

In fact, we could have globally shared regex sections, as proposed in apertium/apertium#161

[unhammer]

minimisation has gotten quite a bit faster lately. but there's a related pr at #165