Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DO info #1311

Merged
merged 37 commits into from
Dec 12, 2019
Merged

DO info #1311

merged 37 commits into from
Dec 12, 2019

Conversation

simon-contreras-deel
Copy link
Contributor

@simon-contreras-deel simon-contreras-deel commented Dec 3, 2019

Fix:

Note: it needs a new version of carto-python: CartoDB/carto-python#155

Acceptance:

  • Step 1: using user without any dataset or geography subscribed:

  • Step 2:

    • subscribe dataset and geo works
    • Download private dataset works
    • Download private geography works
    • Enrich private dataset
    • enrich with project quotas to 0 should fail
  • Step 3:

    • Download using file_path parameter

@simon-contreras-deel simon-contreras-deel changed the base branch from develop to release/1.0b7 December 4, 2019 10:06
@simon-contreras-deel simon-contreras-deel requested review from Jesus89 and removed request for Jesus89 December 10, 2019 10:15
@Jesus89 Jesus89 changed the base branch from release/1.0b7 to develop December 10, 2019 10:23
@simon-contreras-deel simon-contreras-deel mentioned this pull request Dec 10, 2019
8 tasks
fix do open data and first subscription
@simon-contreras-deel simon-contreras-deel marked this pull request as ready for review December 10, 2019 14:15
@alrocar
Copy link
Contributor

alrocar commented Dec 10, 2019

Not sure why tests are failing, @oleurud should I start reviewing or do you prefer having 🍏 tests?

@simon-contreras-deel
Copy link
Contributor Author

Although it is falling in travis, we can do the review and the acceptance from a local version.

Of course, before merging, we need to realease carto-python 1.8.2 and use it in setup.py

Copy link
Contributor

@alrocar alrocar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some suggestions, nothing huge. Let me know what do you think, but I guess we can start with acceptance.

@@ -497,3 +498,6 @@ def _get_summary_data(self):
else:
log.info('Summary information is not available')
return None

def __str__(self):
return "<Dataset.get('{}')>".format(self._get_print_id())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why overriding this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is a @arredond request. When an error happens, we are showing a very huge error message due to the big string created for the Dataset and basically it is difficult to understand

@@ -147,9 +150,21 @@ def _get_credentials(self, credentials=None):

return _credentials

def _get_remote_full_table_name(self, user_project, user_dataset, public_project):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's weird having this method and the _download one in Entity. I'd suggest to refactor to a DownloadableEntity class and if we want to keep using inheritance, then make it to extend Entity and Dataset and Geography inherit from that DownloadableEntity.

If we are not going to do it now, please, let's create a separate issue so it can be prioritized at some point.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_get_remote_full_table_name is useful in any case because we are basically getting this info in different places but IMO the entity is responsible for that. I will do the same for Variable in the future if everybody agrees.

About _download method is a private one and it is called from Catalog and Geography entities. For me, a DownloadableEntity makes sense but don't you think we already have a lot of classes in the catalog?


try:
file_path = bq_client.download_to_file(_WORKING_PROJECT, user_dataset, view)
file_path = bq_client.download_to_file(project, dataset, table, file_path)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are still some warns below these lines, could we use a log.info instead?

@@ -147,9 +150,21 @@ def _get_credentials(self, credentials=None):

return _credentials

def _get_remote_full_table_name(self, user_project, user_dataset, public_project):
project, dataset, table = self.id.split('.')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see we do the split thing in several places to take the project, dataset and table_name. I'd wrap that into a utils method or something.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is a bit weird, because you must know the order when doing the split. Probably, these entities (at least Dataset, Geography, and Variable) should have project, dataset and table_name as properties to avoid doing that. I would tackle it in a different PR

@alrocar alrocar self-assigned this Dec 11, 2019
Copy link
Member

@Jesus89 Jesus89 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Let's merge this to develop to allow integrating #1281, and we continue testing develop branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants