[Lens] Formula transition enhancements

[MichaelMarcialis]

The problem

Lens has recently seen the addition of a new "Formula" feature, allowing uses to write their own custom formula in situations where our "Quick functions" do not meet the user need. Users are able to freely move between these "Quick functions" and "Formula" using a tab-based interface, however the way transitions are handled between the tabs is a little rough around the edges.

When transitioning from the "Quick functions" tab to the "Formula" tab, we convert the last selected quick function into the equivalent formula automatically for the user (this is good). When transitioning from the "Formula" tab to the "Quick functions" tab, the visualization continues to use the custom formula, the "Select a function" field is in a null state (no quick function selected), and a warning message is presented to the user telling them that their formula is still being applied (this is bad). They are told they must select a quick function in order to overwrite their formula, destroying the formula in the process (this is also bad).

Ideally, we shouldn't be 1) scaring users with such a warning message, 2) unnecessarily overwriting any custom formula work, and 3) demanding that users take additional actions in order to quickly visualize both quick functions and formula results when transitioning between tabs.

Possible solution

My first instinct is to say that when transitioning from the "Formula" tab to the "Quick functions" tab, we should never allow for "Select a function" to be in a null state. A quick function should always be selected.

In this case, assuming the user has made a custom formula and then traverses from the "Formula" tab back to the "Quick functions" tab, I think we should revert to the last selected quick function prior to the user switching to make a custom formula.

We'd also need to make sure that whatever changes/customizations the user made in the "Formula" tab are not lost, so that if the user made a mistake and wishes to go back to "Formula", their custom formula remains.

This direction avoids needing to present the user with a warning message, prevents overwriting any custom formula they've created, and requires no extra action to visualize results when transitioning between tabs.

This enhancement may be related to: #75204

Example

1. User starts in "Quick functions" tab, with "Median" function selected.
2. User changes to "Formula" tab, with the median formula automatically populated in the editor.
3. User customizes formula.
4. User goes back to "Quick functions" tab, where "Median" function gets reapplied (as it was the last one selected).
5. If user goes back to "Formula" tab, their previous formula customizations remain in the editor.

[elasticmachine]

Pinging @elastic/kibana-app (Team:KibanaApp)

[flash1293]

@MichaelMarcialis I was looking into this and there is one scenario I'm wondering about:
What if there was no preselected quick function (e.g. when the visualization is part of a dashboard and the user just opened it to edit or when they were coming from TSVB?) Should we treat this as if the user would have clicked a new empty dimension slot? It would look like this:

Clicking on the formula tab would still allow switching back the existing formula.

[MichaelMarcialis]

@MichaelMarcialis I was looking into this and there is one scenario I'm wondering about: What if there was no preselected quick function (e.g. when the visualization is part of a dashboard and the user just opened it to edit or when they were coming from TSVB?) Should we treat this as if the user would have clicked a new empty dimension slot? It would look like this:

Clicking on the formula tab would still allow switching back the existing formula.

@flash1293: I'm not sure I'm understanding. Wouldn't this be exactly how the transition between formula and quick functions works today, just without the callout notifying users that the formula is still in effect?

Or are you saying that no quick function should ever be pre-selected (whether transitioning from the formula tab or creating a new dimension)? I imagine that would be problematic from a defaults standpoint, as it would cause a visualization not to be rendered until the user has manually configured the required dimensions, correct?

[flash1293]

Wouldn't this be exactly how the transition between formula and quick functions works today, just without the callout notifying users that the formula is still in effect

Yes, my question is about keeping this behavior if there is no "last selected quick function", but selecting it if we know about it.

[MichaelMarcialis]

Yes, my question is about keeping this behavior if there is no "last selected quick function", but selecting it if we know about it.

Are you saying in situations where there is no selected quick function (such as when a user manually adds a dimension to a visualization layer, prior to a field being selected) that we continue to show it in its unselected state when transitioning from formula to quick functions? If so, yes, I believe that makes sense.

However, if a field has indeed been selected, I fear that allowing the quick functions to exist in a state where no function is selected creates more problems than it solves (both in the transition from formula to quick functions and the transition from static value to quick functions). Personally, I'd prefer a quick function always be selected if the field is known (either by recalling the last selected quick function or selecting the most likely function as a sensible default).

[flash1293]

Are you saying in situations where there is no selected quick function that we continue to show it in its unselected state when transitioning from formula to quick functions? If so, yes, I believe that makes sense.

Yes, exactly

However, if a field has indeed been selected, I fear that allowing the quick functions to exist in a state where no function is selected creates more problems than it solves. Personally, I'd prefer a quick function always be selected if the field is known.

Agreed.

I think we are on the same page here, I will try to implement this (there are some code structure issues with this, this is why we split it out back then) and open a PR where we can iterate in case it's necessary

[dej611]

I had a quick look at this issue and tried to understand how things will change with the proposed changes.
I'll try to summarize the various scenarios in here:

Nr	Scenario	Variant	Current behaviour	Proposed changes	How to reproduce old behaviour
1	Edits an existing dimension (quick fn) opening the flyout	--	Show Quick function panel	--	--
2	Edits an existing dimension (formula) opening the flyout	--	Show Formula panel	--	--
3	Changes from Quick function to Formula	--	Show fn translated into Formula	--	--
4	Changes from: Quick function ➡️ Formula (no changes) ➡️ Quick function	Function selected initially	Show Quick function with nothing selected + ⚠️ (Formula preserved temporarily)	♻️ Show Quick function selected at beginning (Formula preserved *)	--
		Function not selected initially		♻️ Show Quick function panel in unselected state (Formula preserved *)	--
5	User changes from Formula => Quick function	--	Show Quick function with nothing selected + ⚠️ (Formula preserved temporarily)	♻️ Show Quick function panel in unselected state (Formula preserved *)	--
6	User changes Formula ➡️ Quick function (no edits) ➡️ Formula	--	Show existing formula	--	--
7	User changes Formula ➡️ Quick function and edits few things ➡️ Formula	--	New Quick function gets translated into formula	Previous Formula is restored (But Quick function config should be preserved *, right?)	User edits manually Formula to new version?
8	User lands on Quick function ➡️ Formula ➡️ Closes panel ➡️ Reopens dimension panel as Formula ➡️ Quick functions	Function selected initially	Show Quick function with nothing selected + ⚠️ (Formula preserved temporarily)	♻️ Show Quick function selected at beginning (Formula preserved *)	--
		Function not selected initially		♻️ Show Quick function panel in unselected state (Formula preserved *)
9	User lands Formula ➡️ Quick function ➡️ Closes panel ➡️ Reopens dimension panel as Quick Function ➡️ Formula	Formula defined initially AND User edits when in Quick function	Show fn (latest) translated into Formula	♻️ Restore Formula defined initially	User rewrites the new Formula manually
		Formula defined initially AND no edits when in Quick function		♻️ Restore Formula defined initially	--
		Formula not defined initially AND User edits when in Quick function		♻️ New Quick function gets translated into formula	--
		Formula not defined initially AND no edits when in Quick function		♻️ Empty Formula shown	--
10	User clicks to manually add a dimension field on a visualization layer. ➡️ They are shown the configuration flyout in default "Quick function" mode with no function selected. ➡️ They switch to "Formula" mode (without having selected a field). ➡️ They compose a custom formula. ➡️ They attempt to return to "Quick function" mode.	--	Show Quick function with nothing selected + ⚠️ (Formula preserved temporarily)	♻️ Show Quick function panel in unselected state (Formula preserved *)	--

Notes:

• Preseved * ➡️ inferring from the scenarios above I think a quick function or a Formula are preserved as long as the editor visualization session last. Opening or closing the flyout should not impact that. Everything is cleared once the user leaves the editor.

[MichaelMarcialis]

Did I get scenarios right @MichaelMarcialis ?

Thanks for putting this matrix together, @dej611! Adding my questions/comments for you below:

Scenario 5

Are you asking what happens given the following scenario?

1. User clicks to manually add a dimension field on a visualization layer.
2. They are shown the configuration flyout in default "Quick function" mode with no function selected.
3. They switch to "Formula" mode (without having selected a field).
4. They compose a custom formula.
5. They attempt to return to "Quick function" mode.

If so, I believe this is referring to the scenario @flash1293 and I discussed above, and that we should give the "Quick function" mode that default, unselected state (which I imagine would then also treat the visualization in the workspace as if that dimension has not been populated yet). There would be no callout indicating that a formula is still being applied, as it wouldn't be. Let me know if I've misunderstood though.

Scenario 7

If it's been detected that a user has customized their own formula, then I believe navigating from "Formula" to "Quick function" and then back to "Formula" mode should continue to retain that custom formula (and no longer auto-translate the previously selected quick function to formula). Doing so ensures that the user won't inadvertently lose their work.

Scenario 8

The result here depends on whether a quick function was selected in the first step:

• If no quick function had been selected in the first step, then my expected final result would be that the user is shown the quick functions in an unselected state (i.e. same result as scenario 5).
• If a quick function was selected in the first step, then my expected final result would be to show that the last selected quick function is active (i.e. same result as scenario 4).

Scenario 9

Again, the result here depends on whether a quick function was selected in the first step AND whether a custom formula was authored in the second step:

• If no quick function had been selected in the first step and no custom formula was authored in the second step (meaning the formula editor is empty), then my expected final result would be to show the formula editor in an empty state.
• If a quick function was selected in the first step and no custom formula was authored in the second step (meaning the quick function was auto-translated into the formula editor), then my expected final result would be to show the formula editor populated with the auto-translated formula (i.e. same result as scenario 3).
• If a custom formula was authored in the second step, then my expected final result would be to show that custom formula as active (i.e. same result as scenario 6).

Let me know if this makes sense or if you have any additional questions.

[dej611]

Are you asking what happens given the following scenario?

1. User clicks to manually add a dimension field on a visualization layer.
2. They are shown the configuration flyout in default "Quick function" mode with no function selected.
3. They switch to "Formula" mode (without having selected a field).
4. They compose a custom formula.
5. They attempt to return to "Quick function" mode.

If so, I believe this is referring to the scenario @flash1293 and I discussed above, and that we should give the "Quick function" mode that default, unselected state (which I imagine would then also treat the visualization in the workspace as if that dimension has not been populated yet). There would be no callout indicating that a formula is still being applied, as it wouldn't be. Let me know if I've misunderstood though.

I haven't covered this specific scenario in the matrix, so I'd call it Scenario 10 (will update the matrix), but yes Scenario 5 will fall into this as well.

I've updated the matrix table to cover all scenarios now with all their variants.

[MichaelMarcialis]

I've updated the matrix table to cover all scenarios now with all their variants.

Thanks, @dej611. The updates to the matrix appear correct to me.

[markov00]

Closing this because it's not planned to be resolved in the foreseeable future. It will be tracked in our Icebox and will be re-opened if our priorities change. Feel free to re-open if you think it should be melted sooner.