Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🐛 fix overriding table's short name #1981

Merged
merged 1 commit into from
Nov 27, 2023
Merged

Conversation

Marigold
Copy link
Collaborator

@Marigold Marigold commented Nov 27, 2023

The problem is in these lines

# Read economy data.
tb_economy = pr.read_csv(z.open(DATA_PATH_ECONOMY), metadata=metadata, origin=origin)
tb_economy.metadata.short_name = "emissions_weighted_carbon_price_economy"

# Read coverage data.
tb_coverage = pr.read_csv(z.open(DATA_PATH_COVERAGE), metadata=metadata, origin=origin)
tb_coverage.metadata.short_name = "emissions_weighted_carbon_price_coverage"

metadata refers to the same object and the second tb_coverage.metadata.short_name updates the short name of the first table, too. Function create_dataset, which accepts two tables, then overwrites the tables.

I've added a uniqueness check for table short names and also fixed this by copying metadata object inside pd.read_csv function.

(Found this bug in our full rebuild.)

Copy link
Contributor

@pabloarosado pabloarosado left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! The only thing I don't understand is why this happened now, and not when the step was created, less than two months ago.

@Marigold
Copy link
Collaborator Author

That's a good question. I don't know either. It could be related to the plog changes (I were doing a lot of changes to tables.py) or it could be that the table somehow stayed cached.

@Marigold Marigold merged commit f0f8a4a into master Nov 27, 2023
5 checks passed
@Marigold Marigold deleted the fix-table-overwrite branch November 27, 2023 08:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants