Import failed on duplicated key when importing existing chart that its dataset was changed

[SharonCastel]

Every time we import a dashboard with chart that its dataset was changed, we're Getting the error:
psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint "uq_slices_uuid"

How to reproduce the bug

1. export dashboard from source env and import it to target env
2. On the source env:
   1. Go to dashboard
   2. Edit the chart
   3. change its dataset to another
   4. save (overwrite) the chart and the dashboard
   5. export the dashboard
3. import the dashboard(2.5) on the target env with overwrite option.

Expected results

The import will overwrite the existing chart on the target env and update it to the new dataset (which by the way exists already on the target env)

Actual results

The import failed with the error:
psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint "uq_slices_uuid"

Screenshots

Traceback (most recent call last): File "/app/superset/cli/importexport.py", line 163, in import_dashboards ImportDashboardsCommand(contents, overwrite=True).run() File "/app/superset/dashboards/commands/importers/dispatcher.py", line 68, in run raise exc File "/app/superset/dashboards/commands/importers/dispatcher.py", line 57, in run command.run() File "/app/superset/commands/importers/v1/__init__.py", line 72, in run raise self.import_error() from ex superset.dashboards.commands.exceptions.DashboardImportError: Import dashboard failed for an unknown reason 2022-12-22 10:57:29,174:ERROR:superset.cli.importexport:There was an error when importing the dashboards(s), please check the exception traceback in the log Traceback (most recent call last): File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context self.dialect.do_execute( File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute cursor.execute(statement, parameters) psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint "uq_slices_uuid" DETAIL: Key (uuid)=(8eab9982-aaaa-450c-bbbb-1761d8c207f6) already exists.

Environment

• browser type and version: chrome, version 108.0.5359.124
• superset version: Superset 2.0.0
• python version: Python 3.8.12
• any feature flags active:
  FEATURE_FLAGS = {
  "DYNAMIC_PLUGINS": True,
  "ENABLE_TEMPLATE_PROCESSING": True,
  "VERSIONED_EXPORT": True,
  "DASHBOARD_RBAC": True,
  "DASHBOARD_CROSS_FILTERS": True
  }

Checklist

Make sure to follow these steps before submitting your issue - thank you!

• [ V ] I have checked the superset logs for python stacktraces and included it here as text if there are any.
• [ V ] I have reproduced the issue with at least the latest released version of superset.
• [ V ] I have checked the issue tracker for the same issue and I haven't found one similar.

Additional context

When I delete the chart from the target env and reimport it succeeded (unless there is another chart with the same change)
In the code I see there is a select from tables before the insert into slices which probably makes that issue.

[adimyth]

Were you able to find a solution to this?

[Zhichi57]

The problem is a query to the database that determines the existence of the chart.
It contains the condition
WHERE slices.datasource_id = %(datasource_id_1)s AND slices.uuid = %(uuid_1)s
Because of this, the existence of the diagram is determined incorrectly and is not overwritten.
The query execute in a file
https://github.com/apache/superset/blob/master/superset/models/helpers.py#L300

[SharonCastel]

Yes you right, it can query by the uuid only I believe. Do you know if it fixed on version 2.0.2?

[ShaliniIruvuru]

It is working fine with owners permission. If Dashboard dont have Owners(None) then it is asking for create new chart. other wise it is working fine.
Screenshots:-

[SharonCastel]

Hi All,
I checked it again with owners on the new version 2.1.0 and unfortunately the issue persists :( when the dataset on the import file is different than the one on the target chart the import of the dashboard failed from unknown reason:

This is critical feature in order to have life cycle for superset dashboards.

[ShaliniIruvuru]

Hi SharonCastel,

I have also checked it again with owners on the new version 2.1.0 .I didn’t get any error,it’s working fine.

Loom video:-
https://www.loom.com/share/2e9f5d32fae44af592f0c0924b5905c5

[SharonCastel]

Hi @ShaliniIruvuru,
The edit is working well, but the problem is when you change the data set to another.
As I wrote before: when the dataset on the import file is different than the one on the target chart the import of the dashboard failed from unknown reason.
So instead of "edit dataset" please check the "swap dataset".

[ShaliniIruvuru]

Hi @SharonCastel ,Checked in swap dataset also.I didn't get any error.Just i followed which you added that steps in how to reproduce the bug.
Loom video:-
https://www.loom.com/share/3c0d421d3dd5419e8ed437f6aeb107e6

[SharonCastel]

I watched the video but didnt understand which file did you import.. unfortunately if its the same file as you just exported so it wont reproduce the issue..

[ShaliniIruvuru]

Yes @SharonCastel I used for importing changed dataset file.

Can you please provide detailed information along with steps.It is more helpful if you provide video.

[SharonCastel]

How to reproduce the bug
export the dashboard - save it to the last step
Go to dashboard
Edit the chart
change its dataset to another - swap dataset
save (overwrite) the chart and the dashboard
import the file from the first step with overwrite option.

[ShaliniIruvuru]

Hi @SharonCastel ,I am not getting any error.I followed what ever you provided the steps and also providing the steps which I followed...Can you please go through
1.Export the dashboard(Chosen Sales Dashboard.Here selected one chart having cleaned sales data).

2.Go to sales dashboard.

3.Edit the chart for Total revenue

4.Do Swap dataset (Here change the dataset to Birth names)

5.Save (Overwrite) the chart and Dashboard.

6.Import the dashboard

[danikv]

any updates on that issue ? I can confirm that it's still happening on the main branch.

[SharonCastel]

Export the dashboard(Cho

I believe you imported it on the same environment so the local id is the same and the issue wont bug you, but if you export one and import it to another environment it fails with duplicate error on the id which is the sequence of the db different on each env.

[jfrag1]

#24821 was merged which should resolve this issue; it's currently tagged for the 2.1.2 and 3.0 releases, so it should be included in both of those.

[SharonCastel]

@jfrag1 you are the best!!! thank you so much for the fix!