[dashboard] Development Environment

[hughhhh]

As a user, I would like to do the following when it comes to iterating and making changes to a dashboard.

• have a separate dashboard (dev dashboard) that I can only see, and when I'm ready easily push the dev dashboard into to production dashboard.
• In my dev dashboard I would like the datasource to be different. For instance in my dev environment I will be making changes to the underlying sql/datasource. I don't want this to effect production with these changes to the datasource.

[hughhhh]

Potential SIP in the making:
https://paper.dropbox.com/doc/Dashboard-Development-Environment-for-Superset--AVcP3vzVlz5vqPJo3e7b2RaGAg-QzUooQYObZB9HlMDINcxY

[vylc]

potential temp workaround - related to #4002

[hughhhh]

#4002 is definitely in the right direction for this workflow
The only difference is that a given dashboard creator would need to follow this flow instead:

1. Create dashboard [unpublished]
2. Iterate on the dashboard [unpublished]
3. Publish dashboard [published]
   If you want to make changes to this published dashboard.
4. Make a copy of this publish dashboard [unpublished]
5. Iterate/Develop on dashboard [unpublished]
6. Publish dashboard with more change [published]
7. Switch out reference for this dashboard (slug_name, id)
8. Deprecate/Delete old dashboard

Pitfalls:
With this flow depending on how the user has shares the dashboard it could be difficult to handle #4 in the If you want to make changes to this published dashboard section. If the user, is sending people urls with slug_name i.e. superset.net/superset/dashboard/${slug_name} we could build functionality into the dashboard view to switch out the names once some publishes the dashboard.

With id's, superset.net/superset/dashboard/${id} there would need to be work on switching this reference in the metastore db.

Thanks @vylc for the reference

[mistercrunch]

Related: we spoke in the past about having some sort of non-numeric id (a uuid) on all models as an alternate key from the id sequence we have now. That alternate key would be used for things like import/export across environments.

Another thought is having support for proper versioning for dashboards and charts. I'm thinking something much like reversion (the Django reusable app). This basically inserts new entries as you update, and allows you to roll back to previous versions. That may be overkill for now, but may be an option. It overloads the model quite a bit, think about the different version of each chart, and the dashboard version being a collection of charts at a point-in-time. It's a lot for users to handle and has to be designed well to be intuitive.

[cwormsl2-zz]

Just want to jump in here, to clarify the ask.

Say I build a dashboard with a bunch of charts and features, and I release the initial version and people begin to use it. Now I want to continue to iterate on it and add new features and charts without disrupting the prod version people can see, until all testing and QA is done.

And there are two types of changes I can see being made. One is if you add a new chart, or feature to the dashboard, and make no changes to the datasource. The second is if you need to update the data source, like add new columns, switch from a personal schema table to a prod table etc.

I think the above flow Hugh described would work for the first use case, but not the second. Currently when you copy the dashboard it uses the same data source in the charts in both the original and the copied version. So any changes you make to the data source with affect both dashboards. It would be great when you copy the dashboard to be able to easily switch out the data source for all charts in the copied version so you can make changes. And then once happy with the changes be able to push that to the prod version of the dashboard.

[mistercrunch]

I think datasources currently have a unique constraint on "database + schema + table_name", which prevents people from creating many diverging datasources that point to the same physical table.

For reasons mentioned here and other concerns around diverging needs from different owners around the same datasource, I think we should remove that constraint.

[hughhhh]

@cwormsl2 just realized you could also just use staging to test datasource updates.

1. dump production -> staging
2. update staging datasource
3. see how the dashboard transforms.

when you are ready to migrate to prod just update the datasource.SQL field in production

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[toransahu]

+1