RichTextBlock data type

[bertrand-s]

Enhancement Request

Use Case:

• Display formatted text (bold, italic, paragraphs, tables, hyperlinks...) in chat messages or desktop notifications
• Edit rich content in a text editor prior to sending it into a chat or a notification system
• Support "custom tags" to embed context data in the text document (e.g. references to FDC3 instruments, to one or multiple contacts, to an RFQ, etc.)
• Support document attachment
• Support the possibility to introduce "action buttons" (or links) that could trigger local commands or FDC3 intents (note: this case is not covered in this proposal but it can be easily added later on)

At first sight we could think of Markdown as a good candidate for formatted text. However Markdown is not easily extensible and the custom tags, action buttons or attachment use cases would be hard to support. Besides markdown comes in different flavours and all libraries are not always compatible.

As such, the proposal is to use a simple JSON format that can be easily transformed into Markdow (or into HTML) and that can be then displayed in many different environments.

Other constraints:

• the format should be easy to produce - i.e. an application should not require any specific library to generate a formatted message
• the format should be easy to parse - i.e. no complex parser should be required (like for Markdown or HTML)
• the format should be easy to display - e.g. through a native implemntation or through an easy transformation into HTML
• the format should be adaptable to Rich Text Editors - as we may want to give users the possibility to edit (or re-edit) a pre-formatted message (e.g. when the message is received by the chat application and before its submission in the chat room)

Proposal

The proposal is to define a JSON-based structure composed of 2 element types

• block elements, that would force a carriage return and take the maximum available width - like an HTML div element (e.g. paragraphs, blockquotes, bullet list, tables...)
• inline elements, that would be displayed as inline blocks embedded in text elements - like an HTML span element (e.g. bold or italic sections, hyperlinks, inline tags...)

To keep things simple, block containers should either contain children of type block or children of type inline elements

As an example:

• Paragraphs should only contain inline elements
• Table div should only contain block elements
• The RichTextBlock root element should only contain block elements

The goal of the proposal below is to introduce the basic structural elements. The attributes associated to each elements are deliberately kept simple to focus on the core structure. Further improvements (e.g. new attributes or new elements) could be then added through future proposals.

/**
 * Rich Text Block type
 */
export interface RichTextBlock {
    kind: "richtextblock";
    content: RtBlockElt[];

    // the following props could be added to support multiple languages:
    // lang: string; // language used by the content value (default lang)
    // translatedContent: {lang:string; content:RtBlockElt[];}[]
}

// --------------------------------- Blocks  
type RtBlockElt = RtParagraph | RtHeading | RtBulletList | RtOrderedList | RtBlockQuote | RtTable | RtAttachments;
// could be be extended with 
// - RtImageBlock : to support images
// - RtCodeBlock : to highlight some content as code (cf. HTML)

interface RtParagraph {
    kind: "p";
    content: RtInlineElt[];
}

interface RtHeading {
    kind: "h";
    level: 1 | 2 | 3;
    content: RtInlineElt[];
}

interface RtOrderedList {
    kind: "ol";
    content: RtListItem[];
}

interface RtBulletList {
    kind: "ul";
    content: RtListItem[];
}

interface RtListItem {
    kind: "li";
    content: RtBlockElt[]; // first element must be a paragraph
}

interface RtBlockQuote {
    kind: "blockquote";
    readlony: boolean; // whether the content should be editable in the RTE
    content: RtBlockElt[];
}

interface RtTable {
    kind: "table";
    content: RtTableRow[];
}

interface RtTableRow {
    kind: "tr";
    content: RtTableDiv[];
}

interface RtTableDiv {
    kind: "td";
    content: RtBlockElt[];
}

interface RtAttachments {
    kind: "attachments";
    content: RtData[];
}

interface RtDataTag {
    kind: "data";
    dataType: string; // e.g. "finos.rfq"  - custom type to know which renderer should be used
    displayType?: string; // e.g. "finos.instrument" or "finos.contact" or custom value supported by the chat system
    data: any; // the data content
    name: string; // string to display
    title?: string; // tooltip
}

// --------------------------------- Inline content
type RtInlineElt = string | RtStrongText | RtEmText | RtTextLink | RtDataTag;
// could be completed with
// - RtCode: to support inline 'code' style (i.e. monospace font + grayed background - cf. github markdown)

interface RtStrongText {
    kind: "strong"; // aka. bold
    content: RtInlineElt[];
}

interface RtEmText {
    kind: "em"; // aka. italic
    content: RtInlineElt[];
}

interface RtTextLink {
    kind: "link"; // aka. HTML anchor (a)
    href: string;
    content: RtInlineElt[];
}

// Custom tag examples:
interface RtInstrumentTag extends RtDataTag {
    tagType: "fdc3.instrument";
    data: Instrument; // cf. 'fdc3.instrument'
}

interface RtChatUserMention extends RtDataTag {
    tagType: "fdc3.chat.mention";
    data: { [applicationName: string]: { userId: string; userName: string; } }
}

This structure is very similar to HTML and can be easily transformed into documents loaded into rich text editors like proseMirror for instance

Apart from the usual elements, it contains 2 "special" content nodes:

• RtAttachments which would allow to convey structured data in a block element (e.g. to share data through a chat message). The application rendering the text block could decide to render those data as attachment icons or as 'rich content' (through a custom renderer supported by the target application and associated to the attachment data type)
• RtTag that would allow to convey similar structured data in inline elements. One possible use case would be for Symphony to support hash tags, cash tags or mention tags.

Note: The previous type structure may seem quite verbose, but this can be easily hidden to the developer through helper functions (that could be provided by the fdc3 lib):

// Rich text block with table and paragraphs
let rtb1 = rtb([
    p("table example:"),
    table([
        tr([
            td("1.1"),
            td("1.2")
        ]),
        tr([
            td([p("2.1")]),
            td([
                p("2.2"),
                p("++")
            ]),
        ]),
    ])
]);

// example with strong and em elements
let rtb2 = rtb([
    p(["hello", strong(["brave", em(" new "), "world!"])])
]);

[bertrand-s]

Hi @rikoe - another proposal (in case you are looking for topics for next Thursday meeting 😁 )

[kriswest]

A few notes from meeting:

• Context Data & Intents Discussion Group, 17 Feb 2022 #596

Notes:

• Would be great to see some JSON examples of a context container a rich text block.
• This is a fairly extensive addition to the standard (but one with several obvious use cases), it might be useful to see several firms start to use it before its permanently added to the standard
  • In order to achieve that uptake it would either need to be hosted elsewhere OR
  • FDC3 could adopt an experimental features policy, as proposed in:
    • Support for Experimental Features #533
      • PR: 533 experimental features #549
  • A reference implementation for conversion to/from HTML or markdown would (eventually) be really useful to help achieve uptake.
    • rendering for data tags would likely need to be customized for each use, hence being able to supply a function to render those would be useful

[openfin-johans]

Hi all,

more of a technical issue perhaps - but is FDC3 really equipped to create and maintain a new "rich text" standard? Is there no existing standard that - even if not perfect - would be more practical to adopt?

That said, I think the key thing now, is to formulate how you should send “body” or “message” type data (ie unstructured) in fdc3 context objects. Suggesting to standardize on eg “plain text”, “xml”, “adaptive card”, “symphony message ml”, etc - similar to the recent efforts for eg dates and currency? That way applications will know what the type is and can decide if they can handle it or not. This would allow for a new rich text format to be tested and put into use by interested parties as a custom type before debating a new standard.

Best regards,

Johan

[nemery-flextrade]

That is a valid concern. I think we might get away with it as the proposal seems like it covers a lot of formatting/content types. I remember we discussed the concern that Markdown, while common, is difficult to extend, but what of the extensibility of adaptive card (I'm afraid I'm not overly familiar with it)?

[pauldyson]

I have very mixed feelings on this.

I can absolutely see where the advantages of a custom approach may come in as the use case of passing rich text with embedded context from one system to another is very powerful.

On the other I'm dead against our devs spending time writing custom generators and/or parsers when we have already have a lot of stuff that generates HTML/XML for use in emails, websites, internal portals, file-based and API-based integrations.

ISTM that HTML could do the job even if it is a bit of overkill. Use of data attributes could meet the need to embed contexts and use of inputs or smart parsing of links could be used for 'action buttons'. Regardless, most of our use cases involve at least the potential to pass links that it would be nice not to render in full https://blah.blah/blah horror.

I think if you're going to encourage adoption of a JSON-based standard you'll need not just to define the standard but also supply a fast and robust Javascript 'reference implementation' of both a generator and a parser/convertor.

Otherwise I think you need to consider using what's already there, even if you define an acceptable subset and defined outcomes for what happens should people stray outside of that defined subset.

[nemery-flextrade]

Use of data attributes could meet the need to embed contexts and use of inputs or smart parsing of links could be used for 'action buttons'

I can see data attributes meeting the need for context but I think we need to be careful about the usage of links as targets for action buttons - I think most if not all of the major 'desktop' providers offer adaptors for non-web applications (.NET/Java usually).

On the other I'm dead against our devs spending time writing custom generators and/or parsers when we have already have a lot of stuff that generates HTML/XML for use in emails, websites, internal portals, file-based and API-based integrations.

Thinking about it now, I'm not so worried about the generation of the JSON but more the rendering on the other side. Initially I thought this wouldn't be such a big deal but now I'm wondering how much overhead would have to go in to writing/maintaining 'native' renderers for those who don't use a cross-platform framework - it could quite easily snowball.

That said, I think we do need to decide on a small selection of standards for this formatting - too many and we risk fragmenting the user/application base. While we could implement some sort of plain text fall-back, that still (in my mind) runs the risk of some applications being relegated to second class citizen status because they use format 'X', which, while permitted under the standard is vastly less popular than 'Y' leading to 'X' having to fall-back to plain text more often.

[bertrand-s]

Thanks for your feedback.
I must say I agree with most of what you said - i.e. I would rather use an existing standard instead of proposing something new. As a recap, here are the different options today:

• mardown: main issues are that it is not easily extensible and that it requires rather complex (and heavy) libraries to get parsed. There are also high risks to generate wrong markdown (I have seen that many times). And finally there are also several markdown variants in the market.
• HTML: main issues is that is too rich as it targets pixel-perfect rendering. It is also difficult to extend (you cannot easily attach a JSON structure to html elements), doesn't support theming by default, requires heavy renderers / parser and last but not least is not really compatible with wysiwyg editors (unless we choose an HTML subset, but then the work is equivalent to the RichTextBlock proposal)
• adaptive card - this one would be closer to our needs, but IMO it is also too rich (i.e. there are many options that we don't want or cannot easily support - at least for Symphony). It also requires a rather heavy rendering library and there is no wysiwyg editor for it. Besides it is not supported by a standardisation body and can evolve in different directions without notice.

This is why I think the RichTextBlock approach is still the best option, even though it seems like re-inventing the wheel. That said I agree that we probably need to complement the proposal with

• a list of helper functions to help generating the JSON structure (easy - cf. below)
• helper functions to easily generate HTML or markdown from RTB (if applications want to use an HTML or markdown renderers - that's rather easy as well as RTB is somehow like an Abstract Syntax Tree)
• an open-source wysiwyg web editor. This is more complicated as it would require more work, but we somehow plan to do it at Symphony anyway - so then it's just a matter of deciding whether we agree to open-source it or not.

@kriswest to come back on your comment, here is how RTB would be generated (i.e. through helper functions and not 'by hand' - btw. this approach holds true if we were to generate HTML instead of RTB)

// RTB generation with TypeScript helper functions 
// -> type validation and auto-completion
let tableSample = rtb([
    p("table example:"),
    table([
        tr([
            td("1.1"),
            td("1.2")
        ]),
        tr([
            td([p("2.1")]),
            td([
                p("2.2"),
                p("++")
            ]),
        ]),
     ])
])

// HTML equivalent
let tableHTML = "<p>table example:</p><table class=\"rtb\"><tr><td><p>1.1</p></td><td><p>1.2</p></td></tr><tr><td><p>2.1</p></td><td><p>2.2</p><p>++</p></td></tr></table>";

// RTB - produced by the helper function
let tableRTB = {"kind":"richtextblock","content":[
    {"kind":"p","content":[
         "table example:"]
    },{"kind":"table","content":[
        {"kind":"tr","content":[
            {"kind":"td","content":[{"kind":"p","content":["1.1"]}]},
            {"kind":"td","content":[{"kind":"p","content":["1.2"]}]}
        ]},{"kind":"tr","content":[
            {"kind":"td","content":[{"kind":"p","content":["2.1"]}]},
            {"kind":"td","content":[{"kind":"p","content":["2.2"]},{"kind":"p","content":["++"]}]}
        ]}
    ]}
]}

[nemery-flextrade]

@bertrand-s w/r/t WYSIWYG editor for adaptive card: https://adaptivecards.io/designer/

If we intend to provide a utility library for handling translation to HTML / Markdown I'm much happier - though I feel obliged to mention that these should probably be available for at least C# and Java as well as JS/TS as again, many of the desktops offer adaptors for those languages. If you are able to provide a JS/TS implementation I would be happy to port.

A WYSIWYG would be nice if you do decide to open source it - while it's not high priority for developers it would be great to have something that more business oriented folks could use to design content.

[bertrand-s]

Well, the link you shared is not really a WYSIWYG editor, but a specialised JSON editor (they call it a card payload editor).
What we need is an editor that end users can use to edit actual content, this is why I consider that there is no real WYSIWYG for adaptive cards (my feeling is that their syntax is actually too complex for that as it wasn't design with this concern in mind).
The reason behind the editor requirement is that we want to give end-users the possibility to adapt/complement a message when we receive a chat intent

[kriswest]

We heard at:

• Context Data & Intents Discussion group - 9 June 2022 #742
  That this proposal is being re-thought. Removing from the 2.1 milestone for now. @bertrand-s @pierreneu please close the issue if NOT moving ahead in some form.

[kriswest]

@bertrand-s @pierreneu @Yannick-Malins will this issue be revived one day, or have you gone a different direction? (If so please close it)

[Yannick-Malins]

hi @kriswest could you close this? i don't have the rights

cheers