Add a markup language for describing Toga app layouts

[freakboy3742]

What is the problem or limitation you are having?

Toga apps are currently defined by constructing a layout in Python code. This works well, but there is a group of users who have requested the ability to define their GUI layout with a markup language, rather than code.

Although this risks becoming Yet Another Yet Another Markup Language, there would be some potential benefits - most notably, allowing for separation between GUI definition and business logic, which could be useful if/when a GUI app designer is in use.

Describe the solution you'd like

It should be possible to define app layout in terms of a non-Python markup language.

The exact format for the markup language will need to be the subject of the design process - XML, JSON, YAML, or even a DSL in Python. It might even be worth defining the markup language in abstract terms so that any markup language could be mapped to the internal format with the appropriate parser.

Whatever the final markup format, it should be possible to:

• Define a structure of nodes, children and leaf nodes in a widget tree
• Assign IDs to the nodes on the tree
• Associate styles with each node of the tree
• Associate handlers that are defined in the Python code (either as bare functions, or as properties/methods on the app) with nodes on the tree.

The final API would end up being something like:

class MyApp(toga.App):
    def startup(self):
        self.build_layout("path/to/file.toga")

or, avoiding the need for a startup method at all:

class MyApp(toga.App):
    ...

app = MyApp(layout="path/to/file.toga")

Describe alternatives you've considered

• Don't do this at all. We don't need YA-YAML.
• Don't define another markup language - write parsers for existing markup languages (like QML, KML, or GLADE). It's unclear the extent to which this will be possible, given the way those languages are tightly bound to their underlying GUI frameworks.

Additional context

The good news is that Toga is especially well suited to a markup-based construction approach. The content of a widget is a widget that has children that have children; that's essentially identical to what HTML defines.

It's also possible to build this completely external to Toga's core. Although build_layout() has been defined as a member method on toga.App in the example above, it could be just as easily defined as a standalone method that takes app as an argument.

[Kuna42]

An idea, for a markup type:

• Glade:
  A graphical GUI building tool.

Sadly it is not maintained anymore, but this would be better, because this don't require any specific syntax knowledge for the designer.

[HalfWhitt]

This is very much an uninvestigated first thought, so take it with a grain of salt, but from what little I've done with Kivy in the past, their markup language felt quite usable and clean. The fact that it uses colons and indentation for nesting makes it feel particularly natural to pair with a Python project.

I don't know to what extent it would be possible to use it, or something modeled on it, but it could be a good starting place.

[Heus-Sueh]

QML and Blueprint have very similar and clean layouts:

QML:

import QtQuick

Rectangle {
     id: page
     width: 320; height: 480
     color: "lightgray"

     Text {
         id: helloText
         text: "Hello world!"
         y: 30
         anchors.horizontalCenter: page.horizontalCenter
         font.pointSize: 24; font.bold: true
     }
}

Blueprint:

using Gtk 4.0;

template MyAppWindow : Gtk.ApplicationWindow {
   title: _("My App Title");

   [titlebar]
   HeaderBar header_bar {}

   Label {
     styles ["heading"]
     label: _("Hello, world!");
   }
}

but QML seems a little better to me

[freakboy3742]

Clarifying two details that seem obvious to me, but for the sake of clarity:

Firstly, a "fully compliant" Toga markup language needs to support all of Toga's features, preferably without requiring any special handling or maintenance. If the API for a widget is extended to add a new parameter, ideally the parser would automatically acquire handling for that new attribute without requiring the developer to explicitly add that support. For that reason, QML, KML, or Blueprint may not be appropriate because they don't have features that Toga provides, or they embed naming conventions that don't match Toga's (e.g., naming a window "Gtk.ApplicationWindow" doesn't make much sense in a Toga context, and neither does "y=30" from the QML example).

Secondly - with that said, it's entirely plausible that one could write a QML (or any other markup) parser that can use those definitions, without being the "official" markup. If the problem statement is framed as "Toga should provide a mechanism by which a layout can be specified declaratively", and probably provides a built-in mechanism to do that parsing, it's entirely plausible that others may want to write a mapping from QML/KML/Blueprint to Toga; Toga should probably provide a way for those declarative backends to plug into any mechanisms that Toga provides.

[Heus-Sueh]

Here is an example highly inspired by QML:

import toga
from toga.style import Pack
from toga.style.pack import CENTER, RIGHT, LEFT, COLUMN, ROW, BOTTOM

MainWindow {
    size: (800, 500)
    resizable: True
    show: True

    Box {
        id: main_box,
        style: Pack(
            padding: 10,
            direction=COLUMN,
            alignment="center",
            flex=1
        )
        children: [box1, box2]
        widgets {
            Label {
                id: helloText,
                text: "Hello world!",
                style: Pack(
                    padding: 10,
                    alignment: "center",
                    font.size: 24,
                    font.style: "oblique"
                )
            }
            Button {
                id: button_test,
                text: "button",
                on_press: my_function
                style: Pack(
                    padding: 10,
                    alignment: "center",
                ) 
            }
        }
    } 
}

the "widgets" parameter there would be the widgets that would be added to that box

[freakboy3742]

Yes, that is a markup language. Yes, it even shares some superficial similarities with QML.

However, it won't load into any QML designer (or, if it does, it will be ignoring all the Toga-specific features).

On top of that, it's not a syntax that can utilise any existing parser - it's not XML, JSON, YAML, TOML, or any other common format - and there's no (to the best of my knowledge) standalone QML parser, so we'd need to write our own parser.

What would be the benefit of adopting QML syntax... when the resulting document can't be loaded by any QML tooling, and would require extensive effort to write a custom parser?

[Heus-Sueh]

Yes, that is a markup language. Yes, it even shares some superficial similarities with QML.

However, it won't load into any QML designer (or, if it does, it will be ignoring all the Toga-specific features).

On top of that, it's not a syntax that can utilise any existing parser - it's not XML, JSON, YAML, TOML, or any other common format - and there's no (to the best of my knowledge) standalone QML parser, so we'd need to write our own parser.

What would be the benefit of adopting QML syntax... when the resulting document can't be loaded by any QML tooling, and would require extensive effort to write a custom parser?

Sorry, now I understand your point, writing a parser may require maintenance as new widgets are added and this may not be desirable as the core team is already busy with other things

[buddha314]

Design and portability are a high priority for me. Perhaps we can do something like HTML, where the divs and defined with classes. That way new developers would instantly know how to navigate the layout.

[freakboy3742]

Look - the markup language is the least complex part of the puzzle. Here's something I knocked out in 20 minutes:

import toga
from xml.etree import ElementTree as ET

def build_from_xml(app, content):
    parser = ET.XMLPullParser(["start", "end"])
    parser.feed(content)
    current = []
    for event, element in parser.read_events():
        if event == "start":
            if element.tag == "item":
                items = current[-1][2].setdefault("items", [])
                items.append(element.text)
            elif element.tag == "App":
                pass
            else:
                if element.text and element.text.strip():
                    args = [element.text]
                else:
                    args = []
                kwargs = {}
                for key, value in element.attrib.items():
                    if key.startswith("on_"):
                        if value.startswith("."):
                            kwargs[key] = getattr(app, value[1:])
                    elif key == "style":
                        style = {}
                        for item in value.split(";"):
                            a, v = item.split(":")
                            style[a.strip()] = v.strip()
                        kwargs["style"] = Pack(**style)
                    else:
                        pass
                        # convert any other attribute from XML to kwargs
                current.append((element.tag, args, kwargs))
        elif event == "end":
            if element.tag == "item":
                pass
            elif element.tag == "App":
                pass
            else:
                tag, args, kwargs = current.pop()
                klass = getattr(toga, element.tag)

                if element.tag == "MainWindow":
                    content = kwargs.pop("children")[0]
                    instance = klass(*args, **kwargs)
                    instance.content = content
                    app.main_window = instance
                else:
                    instance = klass(*args, **kwargs)
                    items = current[-1][2].setdefault("children", [])
                    items.append(instance)

Then, in your startup method, call:

    def startup(self):
        build_from_xml(
            self,
            """
                <App>
                    <MainWindow>
                        <Box style="direction: column">
                            <Button on_press=".press_handler">Hello World</Button>
                            <Selection>
                                <item>Alpha</item>
                                <item>Beta</item>
                                <item>Gamma</item>
                            </Selection>
                        </Box>
                    </MainWindow>
                </App>
            """
        )

This parser:

• Converts any <item> children of a tag into an items argument to the tag they're a part of
• Converts all other <Tag> tags into a toga.Tag instances
• Interprets the body content of the tag as the first argument to the tag constructor
• Converts any on_* attribute into a handler. If the value of the attribute starts with ., it's assumed to be an attribute of the app.

This is a long way from being complete - it has no error handling, there's a lot of work needed on the style parser, and there's lots of other attributes that need to be handled. But this is also 20 minutes work. My point is: If someone actually cares about this problem, it's not hard to do.

[tritium21]

To add my 2 cents to the matter, I am partial to not defining a Toga Markup Language at all, but instead designing a system by which toga can lay itself out from a dictionary of 'primitive' python types (string, int, float, list, and dict mostly).

Part of the appeal of using a markup language is the ability to treat layout as data, which means you can more easily build tooling around it, more easily hand it off to a designer, and make the code easier to reason about. A drawback of "Toga Markup Language" is that its... another markup language. It scratches the itch of the desire to make the layout a data operation, not a coding operation, but it does so by making us remember yet another language.

If you can do your layout with self.apply_layout(json.loads(...)), that json string can come from anywhere, from any tool. You wouldn't have to use json, you could use TOML for layout, or YAML, or if there really is a strong desire for a true "Toga Markup Language", that can be implemented by a third party trivially.

[freakboy3742]

To add my 2 cents to the matter, I am partial to not defining a Toga Markup Language at all, but instead designing a system by which toga can lay itself out from a dictionary of 'primitive' python types (string, int, float, list, and dict mostly).

Maybe we could come up with a domain specific language to make this syntax a little easier to read... maybe call it... Python? :-)

Seriously - a "system by which Toga can lay itself out from ... primitives" is what we have right now. Adding "dictionaries" to that sentence is no different to using a markup language, except that you've chosen Python primitives as your markup language.

Part of the appeal of using a markup language is the ability to treat layout as data, which means you can more easily build tooling around it, more easily hand it off to a designer, and make the code easier to reason about. A drawback of "Toga Markup Language" is that its... another markup language. It scratches the itch of the desire to make the layout a data operation, not a coding operation, but it does so by making us remember yet another language.

Sure - and I've said things to this effect a couple of times in this thread. The problem is that for a designer with no coding experience, Python is also Yet Another Markup Language - and consideration of this needs to be part of the evaluation process.

[freakboy3742]

A general note to anyone contemplating contributing to this thread

This thread has become the absolute definition of a bikeshed discussion. Unless you're making a serious implementation proposal (by which, I mean you've got a working PR and you're looking for feedback as a precursor to adding the markup language to Toga's repo), may I humbly suggest that you're not actually making a meaningful contribution to the discussion by postulating a hypothetical markup syntax.

The request for a markup syntax has been made many times. The use case is reasonably well understood. The options are also reasonably well understood. We have now firmly reached the "put up or shut up" phase.

[tritium21]

With that said... close this issue as wont-fix, and accept no PRs on it. for this feature to be of any value, it needs top level buy that there clearly isn't at this time.

[freakboy3742]

With that said... close this issue as wont-fix, and accept no PRs on it. for this feature to be of any value, it needs top level buy that there clearly isn't at this time.

What led you to that conclusion? I've said fairly explicitly that this is a feature we do want to add. There is top level buy in.

It isn't a high personal priority at present - but that doesn't mean we're not interested in the idea. If someone else was to to tackle the problem, and is able to produce a PR that works and addresses the design concerns raised in the discussion so far, I'll review it.

What I'm not interested in are endless "hey! We could use QML"-style comments that don't actually get us any closer to an actual implementation.

[tritium21]

Who is going to implement a feature when the discussions on the shape of the feature is shut down with "put up or shut up". You are extremely clearly signaling that you don't want it. Which is fine. Don't accept it.

[freakboy3742]

Allow me to be explicit. We do want this feature. If someone wants to discuss the specific details of a specific proposal, I'll gladly engage in that discussion.

However, that discussion will almost certainly happen on a pull request, because the easiest way to prove you're serious is by producing at least a prototype implementation of the idea that you're suggesting. The other way would be a full formal specification for a markup language - although, honestly, an implementation will likely be easier to write.