Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Propagate muti-cell paste event to Dash #261

Merged
merged 20 commits into from
Jan 31, 2024
Merged

Conversation

emilhe
Copy link
Contributor

@emilhe emilhe commented Dec 29, 2023

Currently, the cellValueChanged events are propagated to Dash one-by-one. This becomes a problem in cases, where events are fires faster than Dash can handle them, e.g. when pasting multiple cells into the grid. For these cases, only the first event triggers a Dash callback, with all other changes lost.

This PR changes the event collection mechanism to include all events fired within a specific time window (with the length currently defined by CELL_VALUE_CHANGED_DEBOUNCE_MS = 1). By making the window large enough that all events in e.g. a multi cell paste operation can be processed (initial tests indicate that 1 ms is sufficient), all changes can be propagted to Dash. Hence the callback will now receieve a list of change events rather than a single change event.

Here is a small example that demonstrates the usage,

import pandas as pd
import dash_ag_grid as dag
from dash import Dash, Input, Output, html, callback

df = pd.read_csv(
    "https://raw.githubusercontent.com/plotly/datasets/master/ag-grid/olympic-winners.csv"
)

app = Dash(__name__)
app.layout = html.Div(
    [
        dag.AgGrid(
            id="grid",
            rowData=df.to_dict("records"),
            columnDefs=[{"field": i} for i in df.columns],
            defaultColDef={"resizable": True, "sortable": True, "filter": True, "editable": True},
            dashGridOptions={
                "enableRangeSelection": True,
            },
            enableEnterpriseModules=True,
            columnSize="sizeToFit",
        ),
        html.Div(id="dummy"),
    ],
)


@callback(
    Output("dummy", "children"),
    Input("grid", "cellValueChanged")
)
def update(cell_changed):
    return str(cell_changed)


if __name__ == "__main__":
    app.run(debug=True)

Copy-pasting (the whole) row 0 to row 4 will yield the following output with the main branch,

{'rowIndex': 4, 'rowId': '4', 'data': {'athlete': 'Michael Phelps', 'age': '23', 'country': 'United States', 'year': '2008', 'date': '24/08/2008', 'sport': 'Swimming', 'gold': '8', 'silver': '0', 'bronze': '0', 'total': '8'}, 'oldValue': 6, 'value': '8', 'colId': 'total', 'timestamp': 1703846436733}

With the updates propose in the PR, the output will be,

[{'rowIndex': 4, 'rowId': '4', 'data': {'athlete': 'Michael Phelps', 'age': '23', 'country': 'United States', 'year': '2008', 'date': '24/08/2008', 'sport': 'Swimming', 'gold': '8', 'silver': '0', 'bronze': '0', 'total': '8'}, 'oldValue': 'Aleksey Nemov', 'value': 'Michael Phelps', 'colId': 'athlete', 'timestamp': 1703846303490}, {'rowIndex': 4, 'rowId': '4', 'data': {'athlete': 'Michael Phelps', 'age': '23', 'country': 'United States', 'year': '2008', 'date': '24/08/2008', 'sport': 'Swimming', 'gold': '8', 'silver': '0', 'bronze': '0', 'total': '8'}, 'oldValue': 24, 'value': '23', 'colId': 'age', 'timestamp': 1703846303491}, {'rowIndex': 4, 'rowId': '4', 'data': {'athlete': 'Michael Phelps', 'age': '23', 'country': 'United States', 'year': '2008', 'date': '24/08/2008', 'sport': 'Swimming', 'gold': '8', 'silver': '0', 'bronze': '0', 'total': '8'}, 'oldValue': 'Russia', 'value': 'United States', 'colId': 'country', 'timestamp': 1703846303491}, {'rowIndex': 4, 'rowId': '4', 'data': {'athlete': 'Michael Phelps', 'age': '23', 'country': 'United States', 'year': '2008', 'date': '24/08/2008', 'sport': 'Swimming', 'gold': '8', 'silver': '0', 'bronze': '0', 'total': '8'}, 'oldValue': 2000, 'value': '2008', 'colId': 'year', 'timestamp': 1703846303491}, {'rowIndex': 4, 'rowId': '4', 'data': {'athlete': 'Michael Phelps', 'age': '23', 'country': 'United States', 'year': '2008', 'date': '24/08/2008', 'sport': 'Swimming', 'gold': '8', 'silver': '0', 'bronze': '0', 'total': '8'}, 'oldValue': '1/10/2000', 'value': '24/08/2008', 'colId': 'date', 'timestamp': 1703846303491}, {'rowIndex': 4, 'rowId': '4', 'data': {'athlete': 'Michael Phelps', 'age': '23', 'country': 'United States', 'year': '2008', 'date': '24/08/2008', 'sport': 'Swimming', 'gold': '8', 'silver': '0', 'bronze': '0', 'total': '8'}, 'oldValue': 'Gymnastics', 'value': 'Swimming', 'colId': 'sport', 'timestamp': 1703846303491}, {'rowIndex': 4, 'rowId': '4', 'data': {'athlete': 'Michael Phelps', 'age': '23', 'country': 'United States', 'year': '2008', 'date': '24/08/2008', 'sport': 'Swimming', 'gold': '8', 'silver': '0', 'bronze': '0', 'total': '8'}, 'oldValue': 2, 'value': '8', 'colId': 'gold', 'timestamp': 1703846303491}, {'rowIndex': 4, 'rowId': '4', 'data': {'athlete': 'Michael Phelps', 'age': '23', 'country': 'United States', 'year': '2008', 'date': '24/08/2008', 'sport': 'Swimming', 'gold': '8', 'silver': '0', 'bronze': '0', 'total': '8'}, 'oldValue': 1, 'value': '0', 'colId': 'silver', 'timestamp': 1703846303493}, {'rowIndex': 4, 'rowId': '4', 'data': {'athlete': 'Michael Phelps', 'age': '23', 'country': 'United States', 'year': '2008', 'date': '24/08/2008', 'sport': 'Swimming', 'gold': '8', 'silver': '0', 'bronze': '0', 'total': '8'}, 'oldValue': 3, 'value': '0', 'colId': 'bronze', 'timestamp': 1703846303493}, {'rowIndex': 4, 'rowId': '4', 'data': {'athlete': 'Michael Phelps', 'age': '23', 'country': 'United States', 'year': '2008', 'date': '24/08/2008', 'sport': 'Swimming', 'gold': '8', 'silver': '0', 'bronze': '0', 'total': '8'}, 'oldValue': 6, 'value': '8', 'colId': 'total', 'timestamp': 1703846303493}]

Currently, the time window size is hardcoded, but it could also be changed to be a property of the grid so that users get more fine-grained control. Please let me know, if this approach is preferred or not.

@BSd3v
Copy link
Collaborator

BSd3v commented Dec 29, 2023

Have you tested out what this does with row editing and changing all the values?

@emilhe
Copy link
Contributor Author

emilhe commented Dec 29, 2023

@BSd3v what do you mean by "all the values"? I have tried changing multiple values by (1) deleting multiple values and (2) pasting multiple values into the grid. In both cases you get an entry in the list per cell. Are there other ways to change multiple cell values at once?

@BSd3v
Copy link
Collaborator

BSd3v commented Dec 29, 2023

Yes, with fullRowEditing.

There is a test that uses this functionality, it adds a class to the cell based upon if it was edited.

Increase it to editing more than one column before moving on and see if this catches the event properly.

@emilhe
Copy link
Contributor Author

emilhe commented Dec 29, 2023

@BSd3v do you have a link to a test/example that demonstrates/uses this functionality (in Dash)?

@BSd3v
Copy link
Collaborator

BSd3v commented Dec 29, 2023

Here it is:

tests/test_row_data_sync.py

I thought it was associated with an editing test. But it's just rows data syncing.

Just need to see if your fixes capture multiples from this event if more cells are edited before exiting the row editing.

@emilhe
Copy link
Contributor Author

emilhe commented Dec 29, 2023

@BSd3v thanks! For fullRowEditing, only a single event is captured at a time. I think that makes sense; event though the whole row is being edited, only a single cell is edited at the time, so there should be one event per callback.

@BSd3v
Copy link
Collaborator

BSd3v commented Dec 29, 2023

Even when you have edited multiple?

@emilhe
Copy link
Contributor Author

emilhe commented Dec 29, 2023

@BSd3v yes. The event 'bundling' mechanism is time-based, so it shouldn't matter that you are in this particular mode.

@BSd3v
Copy link
Collaborator

BSd3v commented Dec 29, 2023

Here, run this, I get three when I edit 3 cells and then move down to the next row:

import dash_ag_grid as dag
from dash import *
import pandas as pd


app = Dash(__name__)

df = pd.read_csv(
    "https://raw.githubusercontent.com/plotly/datasets/master/ag-grid/olympic-winners.csv"
)

app.layout = html.Div(
    [
        dag.AgGrid(
            columnDefs=[{'field': x} for x in df.columns],
            rowData=df.to_dict("records"),
            columnSize="sizeToFit",
            defaultColDef={"editable": True},
            id="grid",
            getRowId="params.data.nation",
            dashGridOptions={'editType': 'fullRow'}
        ),
        html.Div(id="log")
    ]
)

app.clientside_callback(
    """function countEvents(changes) {
        console.log("FIRE");
        return changes? changes.length : 0;
    }""",
    Output("log", "children"),
    Input("grid", "cellValueChanged"),
    prevent_initial_call=True,
)

app.run(debug=True, port=5000)

This is what I was wondering about.

Here it is when I just pass the stringified object back to the output:

image

@BSd3v
Copy link
Collaborator

BSd3v commented Dec 29, 2023

I'm merging into v31 bump, its possible that this is able to handle correctly in that version with React 18.

@emilhe
Copy link
Contributor Author

emilhe commented Dec 29, 2023

@BSd3v I dont really understand what the issue is with React 18 - but if it's solved, that would be great 😃

@BSd3v
Copy link
Collaborator

BSd3v commented Dec 29, 2023

@emilhe, me neither.

You can run app's in React 18 like this:

external_scripts=[
               'https://cdnjs.cloudflare.com/ajax/libs/react/18.2.0/umd/react.production.min.js',
               'https://cdnjs.cloudflare.com/ajax/libs/react-dom/18.2.0/umd/react-dom.production.min.js'
           ]

@BSd3v
Copy link
Collaborator

BSd3v commented Dec 29, 2023

Looks like in React18, the state isnt even updated when it triggers the second time

@emilhe
Copy link
Contributor Author

emilhe commented Dec 29, 2023

@BSd3v Thanks! I'll try that out. Do you know how to clear multiple cells at once from the test (like when you select using click + shift and click delete)?

@BSd3v
Copy link
Collaborator

BSd3v commented Dec 29, 2023

You can only do it with range selection, you cant do it without enterprise. 😛

@BSd3v
Copy link
Collaborator

BSd3v commented Dec 29, 2023

Eh, you can do this:

import dash

dash._dash_renderer._set_react_version("18.2.0")

To run in v18.

I have a version working now, it just updates the variable instead of the state.

BSd3v and others added 2 commits December 29, 2023 12:04
@emilhe
Copy link
Contributor Author

emilhe commented Dec 29, 2023

Thanks for the update! I have adjusted/cleanup up the code a bit. I also think that I understand why the initial version had problems with React 18; version 18 introduces some performance optimizations, which may introduce race conditions, which could explain the behavior we saw.

@BSd3v
Copy link
Collaborator

BSd3v commented Dec 29, 2023

Thanks for the update! I have adjusted/cleanup up the code a bit. I also think that I understand why the initial version had problems with React 18; version 18 introduces some performance optimizations, which may introduce race conditions, which could explain the behavior we saw.

I was thinking that was along the lines of it, haha. Much more performant means things that were in order could not be, haha.

@emilhe
Copy link
Contributor Author

emilhe commented Dec 29, 2023

@BSd3v about the fullRowEditing case. You are correct - the events are all fired at once when the row editing finishes. Hence, with the current implementation, all but a single edit event is lost. With the new implementation, all of the events are collected. So this PR solves the edit-events-lost issue for that case also.

CHANGELOG.md Outdated
Comment on lines 9 to 10
### Fixed
- [#262](https://github.com/plotly/dash-ag-grid/issues/262) Fixed all-but-one `cellValueChanged` events suppressed for multi-cell edits
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rather than ### Fixed this should be a ### Changed with a very prominent note that it's a breaking change. So it's good that we're about to release a new major version for AG Grid v31 or this kind of change would not be allowed 😅 But it makes sense, I don't see any non-breaking way to support these events.

@BSd3v this makes me wonder, are there any other props that might have similar problems? If so now is the time to convert them into arrays too!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If backwards compatibility is a main concern, we could

  • Set the type of cellValueChanged to union of single/list of events
  • Add a new prop that represents CELL_VALUE_CHANGED_DEBOUNCE_MS , which defaults to -1, in which case we fallback to the old behavior, thus returning a single element. Setting the value it to 0 (or higher) will enable the new behavior, thus returning a list

My initial implementation was actually along these lines, but @BSd3v suggested to target a simpler implementation (the one in this PR) in favor of backwards compatibility since a major release is in the making.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, I agree the simpler API is more important than backward compatibility, given the upcoming major release and the fact it would be annoying to use the prop if sometimes it’s a list and sometimes not. Just make sure the changelog makes the breaking change clear.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So far, I haven't found anything that operates this way.

The team has also been working on the v31 updates for the docs, and those types of things haven't popped up that I am aware of.

The one thing I don't know about is the grid's state (new feature of the grid).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have updated the comment, indicating that this is a change. Should I note explicitly that the change is breaking (I am not sure how breaking changes are documented in this project - I didn't see any mentioned of breaking changes previously in the changelog)?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would be the first since switching to version 2 from 1.

@alexcjohnson
Copy link
Collaborator

@emilhe this looks great, thanks! I just had 2 small comments, then will let @BSd3v merge as he sees fit in conjunction with the v31 work.

Copy link
Collaborator

@BSd3v BSd3v left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing really to add, just those two things.

CHANGELOG.md Outdated
Comment on lines 9 to 10
### Fixed
- [#262](https://github.com/plotly/dash-ag-grid/issues/262) Fixed all-but-one `cellValueChanged` events suppressed for multi-cell edits
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So far, I haven't found anything that operates this way.

The team has also been working on the v31 updates for the docs, and those types of things haven't popped up that I am aware of.

The one thing I don't know about is the grid's state (new feature of the grid).

src/lib/fragments/AgGrid.react.js Outdated Show resolved Hide resolved
@BSd3v BSd3v linked an issue Jan 16, 2024 that may be closed by this pull request
Copy link
Collaborator

@BSd3v BSd3v left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💃 after the slight modification to the changelog

CHANGELOG.md Outdated Show resolved Hide resolved
Co-authored-by: BSd3v <82055130+BSd3v@users.noreply.github.com>
Copy link
Collaborator

@BSd3v BSd3v left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💃

@BSd3v BSd3v merged commit d3f7726 into plotly:main Jan 31, 2024
3 checks passed
@olejorgenb
Copy link

Will two or more edits not originating from paste events, multi delete or fullRowEditing (or similar) realistically ever be batched?

@emer-bestseller
Copy link

What other cases are you thinking about?

@olejorgenb
Copy link

olejorgenb commented Apr 16, 2024

I wonder if there's cases where I can be sure the list will only contain one change, since the logic is simpler to implement (or rather - I have existing logic that is a bit hairy to refactor). In my case don't think it's possible to paste content for instance.

Will a user be able to trigger grouped changes by changing multiple cells manually fast?

@BSd3v
Copy link
Collaborator

BSd3v commented Apr 16, 2024

No, the user will not be able to trigger the events that quickly. The debounce is set at 1 ms. Maybe a script could do it, but then you have other issues. ;)

This is really only helpful for things like paste and fullRowEditing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

On multi-cell edits, only a single cellValueChanged event is emitted
5 participants