Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: import/export dashboards via cli #5991

Merged

Conversation

arpit-agarwal
Copy link
Contributor

@arpit-agarwal arpit-agarwal commented Sep 27, 2018

This PR is initial implementation to close #5821

@mistercrunch
Please validate correctness of implementation

I will add documentation and test in couple of days.

@arpit-agarwal
Copy link
Contributor Author

arpit-agarwal commented Sep 28, 2018

@mistercrunch I am not sure why cypress test is failing as there is no change related to that.
On my dev machine it is working fine.
Please validate if any recent merge to master may causing this.
@kristw This is related recent commit on master. can you help me in fixing cypress build here

@kristw
Copy link
Contributor

kristw commented Sep 28, 2018 via email

@arpit-agarwal
Copy link
Contributor Author

arpit-agarwal commented Sep 29, 2018

Thanks for update @kristw. This is open for review. I will commit testcase and docstring docs in couple of days.

@arpit-agarwal
Copy link
Contributor Author

arpit-agarwal commented Oct 1, 2018

@mistercrunch @kristw
I have added test for export cases. Import I have to revert as it was causing integrity error.
I can mock the actual db import by mocking framework but decide against it as it is not used anywhere.

The build is failing for unrelated test on async query celery. Can you help me fix that

test_run_async_query (tests.celery_tests.CeleryTestCase) ... 2018-10-01 11:35:56,304:INFO:root:The 'superset worker' command is deprecated. Please use the 'celery worker' command instead.
FAIL

@mistercrunch I saw this job also failed for same test case couple of days ago.

Arpit Agarwal added 5 commits October 1, 2018 20:39
@arpit-agarwal arpit-agarwal force-pushed the feat/dashboard_import_export_cli branch from 30a8d04 to f5922f5 Compare October 1, 2018 15:13
@codecov-io
Copy link

codecov-io commented Oct 1, 2018

Codecov Report

Merging #5991 into master will increase coverage by 0.01%.
The diff coverage is 43.1%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #5991      +/-   ##
==========================================
+ Coverage   63.44%   63.45%   +0.01%     
==========================================
  Files         443      444       +1     
  Lines       23753    23803      +50     
  Branches     2638     2638              
==========================================
+ Hits        15070    15105      +35     
- Misses       8670     8685      +15     
  Partials       13       13
Impacted Files Coverage Δ
superset/views/core.py 74.11% <0%> (+0.37%) ⬆️
superset/cli.py 50% <23.33%> (-3.61%) ⬇️
superset/dashboard_import_export_util.py 66.66% <66.66%> (ø)
superset/connectors/sqla/models.py 81.13% <0%> (+0.75%) ⬆️
superset/db_engine_specs.py 55.7% <0%> (+0.87%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 414a4bf...a6545cc. Read the comment docs.

@mistercrunch mistercrunch merged commit 7388294 into apache:master Oct 1, 2018
@@ -547,6 +550,34 @@ def test_import_druid_override_identical(self):
self.assert_datasource_equals(
copy_datasource, self.get_datasource(imported_id))

def test_export_dashboards_util(self):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mistercrunch @arpit-agarwal I've noticed this test failing the py36-postgres tests when a pr has not made any changes to this code. For example https://travis-ci.org/apache/incubator-superset/jobs/436740957 Can you take a look?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mistercrunch @arpit-agarwal this is blocking merging all PRs into master

Copy link
Contributor

@williaster williaster Oct 4, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mistercrunch @arpit-agarwal this flaky test is still failing most builds for py36-postgres therefore blocking merges. Can you either PTAL or we will revert this PR if it's not fixed by noon today PDT?

cc @john-bodley @kristw @michellethomas @graceguo-supercat

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd say revert.

Copy link

@graceguo-supercat graceguo-supercat Oct 4, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i made a test branch which reverted this whole PR, and it passed all CI tests. So i created #6035 to revert this. Thank you!

Copy link
Contributor Author

@arpit-agarwal arpit-agarwal Oct 5, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank guys.
@graceguo-supercat @williaster @mistercrunch
However I don't understand how this PR can halt postgres.

I had seen stalled postgres even before this PR was merged. Look this job for a instance that was commit before this PR merged.

I also see build not passing after reverting the PR. see

We may need to find the first stalled build to see the root cause. I am out in a pycon India i will grok the logs once back

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@arpit-agarwal Thank you very much for investigation. After I reverted this PR, what we observed is, most PRs passed all CI tests, includes postgres one. But master branch still failed at postgres test. Right now i am totally confused what exactly happened :(

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As suggested we can look at the content of PR #5693 to fund the route cause.

I do see some changes in test in that PR related to test.
Any recent configruration changes on tarvis or postgress version or lsome python lib?

graceguo-supercat pushed a commit to graceguo-supercat/superset that referenced this pull request Oct 4, 2018
@graceguo-supercat graceguo-supercat mentioned this pull request Oct 4, 2018
graceguo-supercat pushed a commit that referenced this pull request Oct 4, 2018
@graceguo-supercat
Copy link

this PR is now reverted. see #6035.

@arpit-agarwal
Copy link
Contributor Author

arpit-agarwal commented Oct 5, 2018

I quickly looked at logs it seems this is the first failure point introduced by PR #5693

@arpit-agarwal
Copy link
Contributor Author

@graceguo-supercat

I see master build is passing now. did we made some changes in configuration to fix it?

If all is going good can we raise The pR again to merge this feature.

If we are skeptical about test we can have a separate PR for test?

@graceguo-supercat
Copy link

Yes i made some change from our travis setting. Sorry I reverted your PR and now think your PR should be good 👍. Do you want to create a new PR? i will approve it and merge.

@arpit-agarwal
Copy link
Contributor Author

arpit-agarwal commented Oct 9, 2018

Yes. I will create a new PR shortly

@arpit-agarwal
Copy link
Contributor Author

@graceguo-supercat I have raised #6061 with these changes.

It will be great if we can document the travis changes for future reference.

betodealmeida pushed a commit to lyft/incubator-superset that referenced this pull request Oct 12, 2018
* feat: import/export dashboards via cli

* style: fixed lint error

* test: added test for import and export util

* test: removing import test as it is causing integrity issues

Import is a wrapper around exist functionality so we can go ahead without a test or mock the actual db operation using https://docs.python.org/3/library/unittest.mock.html

And validate the wrapper operations only.

* test: remove test data file

* test: removed usage of reserved keyword id
betodealmeida pushed a commit to lyft/incubator-superset that referenced this pull request Oct 12, 2018
@mistercrunch mistercrunch added 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 0.28.0 labels Feb 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 0.28.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Allow import and export dashboards and charts by CLI
7 participants