[makeotfexe] Feature files causes huge memory consumption

[khaledhosny]

While porting feaLib tests to makeotfexe, I came accross a seemingly innocent file that causes makeotfexe to consume huge amount of memory (not sure if it is leaking memory or not), it basically consumed all available memory. I tried to debug it, but it is rather difficult when it causes the OS to start killing applications to free memory.

The file (from fonttools/fonttools#1307).

[cjchapman]

@khaledhosny I'll look into it. Thanks.

[cjchapman]

@khaledhosny Looks like it's stuck in an infinite loop here:

afdko/c/makeotf/makeotf_lib/source/hotconv/feat.c

Lines 4221 to 4224 in ddb273a

	for (i = 0; i < h->DFLTLkps.cnt; i++) {
	State *st = &h->DFLTLkps.array[i];
	callLkp(st);
	}

...because something in the callLkp() call stack keeps causing h->DFLTLkps.cnt to increment.

[cjchapman]

...this, specifically:

afdko/c/makeotf/makeotf_lib/source/hotconv/feat.c

Lines 3029 to 3032 in ddb273a

	if (accumDFLTLkps) {
	/* Save for possible inclusion later in lang-specific stuff */
	*dnaNEXT(h->DFLTLkps) = h->curr;
	}

[cjchapman]

@khaledhosny I have to leave now due to family commitments. If you want to take it from here, great, if not, I'll plan on picking this back up on Monday.

[khaledhosny]

I’m not having much luck with debugging. This is the smallest file to trigger the infinite loop.

languagesystem DFLT dflt;
lookup b {
    sub b by B;
} b;
lookup c {
    sub c by C;
} c;
feature liga {
    lookup b;
    script DFLT;
        lookup c;
        language dflt;
            lookup c;
} liga;

[cjchapman]

@khaledhosny OK, no worries, I'll plan on working on this when I get back to the office on Monday.