Split "cell catalog" between "main grid" and subpatterns

[ISSOtm]

Note

Help is wanted because I do not have the motivation to implement this; I am however willing to provide guidance to anyone having that motivation. Feel free to contact me.

Context / Motivation

Currently, the biggest problem with using fO over hD is the dreaded Error: The song has NNN unique cells, the max is 256! error.

In the interest of size, fO collects cells (note+instr+FX) into a "catalog", and stores cells as indices into this catalog; in the interest of size and speed (again), those indices are single bytes, which inherently forbid more than 256 entries in the "catalog".

Implementation

A way to relax the limitation is to split the catalog in two¹: one for the "main grid", and one for subpatterns. Then the limitation only applies to each one individually, which should be productive considering that typical cells for patterns and subpatterns are quite different.

As far as fO itself is concerned, this is really simple: duplicate the pointer to .cellCatalog, and change the variable being loaded from in TickSubpattern.

Issues

Song export

First off, teNOR currently assumes that it can compute the "overlap" optimisation by comparing the cells themselves; this wouldn't become true anymore, because nothing guarantees a given cell would have the same ID across both catalogs.

Therefore, cells should instead be compared via some kind of IDs, of three kinds: shared, main only, and subpattern only. Then, the shared cells would get the same indices across both catalogs (so the first few, for simplicity).

Output size

There is also a size problem: there is already somewhat considerable padding within a cell catalog², and splitting the catalog in two like that would multiply the padding by 2.5!

This could be mitigated by fusing the two catalogs like they currently are if that's possible; then both labels are identical, and only a single header byte is wasted—but that increases the implementation's complexity significantly.

Footnotes

1. Other options have been considered for splitting, such as one catalog per channel; but these are very impractical to implement, at least with fO in its current form. ↩

2. The size of the catalog is 512 + nb_cells bytes, which can be a lot compared to the theoretical minimum size of 3 × nb_cells. ↩

[sttng]

I'd suggest 2 catalogues: One for Pulse 1 and Pulse 2. Second one for Wave and Noise.
Maybe its easier to implement ? Also in my experience, the Noise channel is special in the way that (for many of my songs) it has more "unique" cells.

[ISSOtm]

Unfortunately, under the current architecture, it is not feasible to implement a per-channel catalog without a significant performance penalty.

[ISSOtm]

Something that would help, especially in subpattern-heavy songs: instead of storing subpattern cells the same way as hUGEDriver (putting the "jump" target in the instrument field + upper note bit), “outline” the jump target right next to the row ID (index into the catalog).

In theory, this makes subpatterns slightly larger; but, in practice, subpattern rows can hardly be dedup'd because they tend to contain unique jump targets (as a fix for SuperDisk/hUGETracker#172). This increase should thus be countered by an improvement of the “catalog”'s efficiency—which would also help raise the limit encountered by this issue.

Also worth noting is that thus, the subpatterns become stored in a different format than the main patterns, and thus two “row ID pools” become necessary.