Design Strategies Questions for the Frescobaldi Approach

(Urs Liska)

When Peter Bjuhr started an attempt to tackle MusicXML export from LilyPond through Frescobaldi, I reviewed the original [Requirements collection] (https://github.com/openlilylib/ly2xml/wiki/Requirements-and-development-ideas). IMO one basic issue to consider is that there isn't a direct way from a LilyPond input file to a printed PDF or a MusicXML file. There are many steps that LilyPond performs during parsing and preprocessing, e.g.

• substituting variables,
• transposing,
• unfolding repeats,
• applying music functions
• modify the music through engravers that listen to events.

Just think of the extreme case of a whole algorithmic composition stuffed into Scheme code in LilyPond input files.
Trying to get all this by interpreting the input files (with a Python script) would actually mean reimplementing everything LilyPond's parser does already. And keeping the script in sync with LilyPond's parser development.
I really think what is needed is a shared approach, letting LilyPond itself do as much work as possible and use external tools for managing, merging and finishing everything.

In this document I will try to lay out my thoughts on a potentially promising design strategy. It is full of open questions but I hope it may serve as a more concrete starting point than the other document.

Relationship between LilyPond and Python processing

From the collected hints and opinions from the other document I find [this discussion] (https://github.com/openlilylib/ly2xml/wiki/Requirements-and-development-ideas) on the LilyPond issue tracker particularly interesting. Apart from the quoted comment 15 I would like to point out number 10:

The only problem is that the lilypond structure is not ideally suited for full MusicXML 2.0 export: All other graphical output formats (eps, ps, pdf, png, svg) simply export some graphical objects with a fixed position on a page, so at that stage the musical information is no longer available, so MusicXML export has to hook in earlier.

The pure musical structure can probably be easily extracted from the music stream (e.g. by an engraver, listening to all kinds of events), but at that stage the page layout has not been done, so the great layout of lilypond scores could not be exported directly.

On the other hand, the final graphical objects don't have any link to the music object that caused them, so one would also have to add such a link to the grobs.

This means that trying to create a MusicXML document preserving LilyPond's layout decisions is much more complex than a simple transformation.

The MusicXML export would then work in two stages in a lilypond run:

1. In an engraver create the xml tree for the pure musical content
2. at the same time, also listen to created graphical objects and add a
   pointer to the corresponding xml node
3. After the layout is done, a MusicXML backend goes through the graphical
   object and exports all positioning information (most of which is new in
   MusicXML 2.0) to their linked xml nodes.

Another important aspect was mentioned by David Kastrup: The music stream does not contain all the content of a score. What he explicitly mentioned was things like titles etc. but we'd have of course to dig deeper into that issue.

Even when we don't try to include layout information I think these two aspects give valuable hints how one could start. I would suggest something along these lines:

1. When compiling an input file with LilyPond let an engraver listen to anything it can catch and produce an intermediate output file from this information.
   This output file can already be XML or another format in case that proves more straightforward.
   The engraver should tag each element of this file with the position in the input files so one can link this later in order to 'merge' information from different paths.
2. The new script should parse the result file of the previous step and try to add as much information as it can get from parsing the input files by itself, merging it and creating a correct MusicXML file. To perform this it makes use of the links to the input file positions.

I don't know if that's possible, but ideally step 1) would be done without actually doing the layout of the score, just as MIDI output doesn't do that. This would dramatically reduce processing time.

Where to hook into LilyPond process

The following is more or less guesswork because I don't know enough about how LilyPond actually processes a file, how C++, Scheme and included LilyPond code interact and which information is present and processed at which point.

If it is really an engraver that listens to all musical events and that would be able to 'print' them in an intermediate file then one probably can simply write such an engraver and add it to the score context.
I think it would be best if such an engraver could work as straightforward as possible: Simply take the information of the musical event, 'enrich' it with contextual information and print it out as literally as possible (e.g. simply copying any names).

For a future project trying to also translate layout information this engraver would additionally have to pass some information along to the stages when the grobs are actually laid out. E.g. tag the created grob with the id of the generated xml node (and perhaps with the position in the input file) so the layout information generated later can be linked to the musical information.

How to integrate in Frescobaldi

How to integrate in LilyPond later