fix(datasets): do not double encode the data as json when saving an A… #301
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Thanks @FlorianGD Is this a bug or it has been always like this? I suspect this is introduced in #180. Is it only affecting |
|
This looks like a good fix to me. Thanks @FlorianGD |
noklam
approved these changes
Aug 10, 2023
SajidAlamQB
approved these changes
Aug 10, 2023
…PIDataSet Signed-off-by: Florian Gaudin-Delrieu <fgaudindelrieu@idmog.com> Signed-off-by: Florian Gaudin-Delrieu <florian.gaudindelrieu@gmail.com>
Signed-off-by: Florian Gaudin-Delrieu <florian.gaudindelrieu@gmail.com>
FlorianGD
force-pushed
the
fix/api-dataset-save
branch
from
a027c9b
to
a364aa1
Compare
lrcouto
pushed a commit
that referenced
this pull request
Aug 15, 2023
#301) * fix(datasets): do not double encode the data as json when saving an APIDataSet Signed-off-by: Florian Gaudin-Delrieu <fgaudindelrieu@idmog.com> Signed-off-by: Florian Gaudin-Delrieu <florian.gaudindelrieu@gmail.com> * chore(lint): make pyling happy Signed-off-by: Florian Gaudin-Delrieu <florian.gaudindelrieu@gmail.com> --------- Signed-off-by: Florian Gaudin-Delrieu <fgaudindelrieu@idmog.com> Signed-off-by: Florian Gaudin-Delrieu <florian.gaudindelrieu@gmail.com> Co-authored-by: Nok Lam Chan <nok.lam.chan@quantumblack.com> Signed-off-by: lrcouto <laurarccouto@gmail.com>
lrcouto
added a commit
that referenced
this pull request
Aug 24, 2023
* Add CONTRIBUTING.md file to the Kedro-Plugins repository Signed-off-by: lrcouto <laurarccouto@gmail.com> * fix(datasets): Correct pyproject.toml syntax for optional dependencies (#302) * Fix pyproject.toml syntax for optional dependencies Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com> * refacor out the base dependencies Signed-off-by: Nok <nok.lam.chan@quantumblack.com> * add comments Signed-off-by: Nok <nok.lam.chan@quantumblack.com> * format pyproject.toml Signed-off-by: Nok <nok.lam.chan@quantumblack.com> * Reorder pandas dependencies Signed-off-by: Nok <nok.lam.chan@quantumblack.com> * reorder spark dependencies Signed-off-by: Nok <nok.lam.chan@quantumblack.com> * remove polars-base and delta-base Signed-off-by: Nok <nok.lam.chan@quantumblack.com> --------- Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com> Signed-off-by: Nok <nok.lam.chan@quantumblack.com> Co-authored-by: Nok <nok.lam.chan@quantumblack.com> Signed-off-by: lrcouto <laurarccouto@gmail.com> * Bring CONTRIBUTING.md file in line with the one on the main Kedro repo Signed-off-by: lrcouto <laurarccouto@gmail.com> * Bring CONTRIBUTING.md file in line with the one on the main Kedro repo Signed-off-by: L. R. Couto <laurarccouto@gmail.com> Signed-off-by: lrcouto <laurarccouto@gmail.com> * docs: Fix broken link to datasets docs in README.md (#304) fix broken link to datasets docs Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> Signed-off-by: lrcouto <laurarccouto@gmail.com> * ci: Add docs rtd check on `kedro-datasets` (#299) * Try adding docs rtd check on kedro datasets Signed-off-by: Merel Theisen <merel.theisen@quantumblack.com> * Add Read the Docs configuration for kedro-datasets Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> Signed-off-by: lrcouto <laurarccouto@gmail.com> * fix(datasets): do not double encode the data as json when saving an A… (#301) * fix(datasets): do not double encode the data as json when saving an APIDataSet Signed-off-by: Florian Gaudin-Delrieu <fgaudindelrieu@idmog.com> Signed-off-by: Florian Gaudin-Delrieu <florian.gaudindelrieu@gmail.com> * chore(lint): make pyling happy Signed-off-by: Florian Gaudin-Delrieu <florian.gaudindelrieu@gmail.com> --------- Signed-off-by: Florian Gaudin-Delrieu <fgaudindelrieu@idmog.com> Signed-off-by: Florian Gaudin-Delrieu <florian.gaudindelrieu@gmail.com> Co-authored-by: Nok Lam Chan <nok.lam.chan@quantumblack.com> Signed-off-by: lrcouto <laurarccouto@gmail.com> * build(datasets): release 1.5.2 (#305) * release 1.5.2 minor release to fix the kedro-datasets optional dependencies issue * Update RELEASE.md * Trim whitespace Signed-off-by: Nok <nok.lam.chan@quantumblack.com> --------- Signed-off-by: Nok <nok.lam.chan@quantumblack.com> Signed-off-by: lrcouto <laurarccouto@gmail.com> * Add plugin contribution details to CONTRIBUTING.md Signed-off-by: lrcouto <laurarccouto@gmail.com> * Update CONTRIBUTING.md Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> * Update CONTRIBUTING.md Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> * Apply minor changes on CONTRIBUTING.md Signed-off-by: lrcouto <laurarccouto@gmail.com> * Push code of conduct text block to the top on CONTRIBUTING.md Signed-off-by: L. R. Couto <laurarccouto@gmail.com> * Fix trailing whitespace lint error Signed-off-by: L. R. Couto <laurarccouto@gmail.com> --------- Signed-off-by: lrcouto <laurarccouto@gmail.com> Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com> Signed-off-by: Nok <nok.lam.chan@quantumblack.com> Signed-off-by: L. R. Couto <laurarccouto@gmail.com> Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> Signed-off-by: Florian Gaudin-Delrieu <fgaudindelrieu@idmog.com> Signed-off-by: Florian Gaudin-Delrieu <florian.gaudindelrieu@gmail.com> Co-authored-by: Dmitry Sorokin <40151847+DimedS@users.noreply.github.com> Co-authored-by: Nok <nok.lam.chan@quantumblack.com> Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> Co-authored-by: Merel Theisen <49397448+merelcht@users.noreply.github.com> Co-authored-by: Florian Gaudin-Delrieu <9217921+FlorianGD@users.noreply.github.com> Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
PtrBld
pushed a commit
to PtrBld/kedro-plugins
that referenced
this pull request
Aug 27, 2023
kedro-org#301) * fix(datasets): do not double encode the data as json when saving an APIDataSet Signed-off-by: Florian Gaudin-Delrieu <fgaudindelrieu@idmog.com> Signed-off-by: Florian Gaudin-Delrieu <florian.gaudindelrieu@gmail.com> * chore(lint): make pyling happy Signed-off-by: Florian Gaudin-Delrieu <florian.gaudindelrieu@gmail.com> --------- Signed-off-by: Florian Gaudin-Delrieu <fgaudindelrieu@idmog.com> Signed-off-by: Florian Gaudin-Delrieu <florian.gaudindelrieu@gmail.com> Co-authored-by: Nok Lam Chan <nok.lam.chan@quantumblack.com>
PtrBld
pushed a commit
to PtrBld/kedro-plugins
that referenced
this pull request
Aug 27, 2023
…-org#294) * Add CONTRIBUTING.md file to the Kedro-Plugins repository Signed-off-by: lrcouto <laurarccouto@gmail.com> * fix(datasets): Correct pyproject.toml syntax for optional dependencies (kedro-org#302) * Fix pyproject.toml syntax for optional dependencies Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com> * refacor out the base dependencies Signed-off-by: Nok <nok.lam.chan@quantumblack.com> * add comments Signed-off-by: Nok <nok.lam.chan@quantumblack.com> * format pyproject.toml Signed-off-by: Nok <nok.lam.chan@quantumblack.com> * Reorder pandas dependencies Signed-off-by: Nok <nok.lam.chan@quantumblack.com> * reorder spark dependencies Signed-off-by: Nok <nok.lam.chan@quantumblack.com> * remove polars-base and delta-base Signed-off-by: Nok <nok.lam.chan@quantumblack.com> --------- Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com> Signed-off-by: Nok <nok.lam.chan@quantumblack.com> Co-authored-by: Nok <nok.lam.chan@quantumblack.com> Signed-off-by: lrcouto <laurarccouto@gmail.com> * Bring CONTRIBUTING.md file in line with the one on the main Kedro repo Signed-off-by: lrcouto <laurarccouto@gmail.com> * Bring CONTRIBUTING.md file in line with the one on the main Kedro repo Signed-off-by: L. R. Couto <laurarccouto@gmail.com> Signed-off-by: lrcouto <laurarccouto@gmail.com> * docs: Fix broken link to datasets docs in README.md (kedro-org#304) fix broken link to datasets docs Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> Signed-off-by: lrcouto <laurarccouto@gmail.com> * ci: Add docs rtd check on `kedro-datasets` (kedro-org#299) * Try adding docs rtd check on kedro datasets Signed-off-by: Merel Theisen <merel.theisen@quantumblack.com> * Add Read the Docs configuration for kedro-datasets Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> Signed-off-by: lrcouto <laurarccouto@gmail.com> * fix(datasets): do not double encode the data as json when saving an A… (kedro-org#301) * fix(datasets): do not double encode the data as json when saving an APIDataSet Signed-off-by: Florian Gaudin-Delrieu <fgaudindelrieu@idmog.com> Signed-off-by: Florian Gaudin-Delrieu <florian.gaudindelrieu@gmail.com> * chore(lint): make pyling happy Signed-off-by: Florian Gaudin-Delrieu <florian.gaudindelrieu@gmail.com> --------- Signed-off-by: Florian Gaudin-Delrieu <fgaudindelrieu@idmog.com> Signed-off-by: Florian Gaudin-Delrieu <florian.gaudindelrieu@gmail.com> Co-authored-by: Nok Lam Chan <nok.lam.chan@quantumblack.com> Signed-off-by: lrcouto <laurarccouto@gmail.com> * build(datasets): release 1.5.2 (kedro-org#305) * release 1.5.2 minor release to fix the kedro-datasets optional dependencies issue * Update RELEASE.md * Trim whitespace Signed-off-by: Nok <nok.lam.chan@quantumblack.com> --------- Signed-off-by: Nok <nok.lam.chan@quantumblack.com> Signed-off-by: lrcouto <laurarccouto@gmail.com> * Add plugin contribution details to CONTRIBUTING.md Signed-off-by: lrcouto <laurarccouto@gmail.com> * Update CONTRIBUTING.md Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> * Update CONTRIBUTING.md Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> * Apply minor changes on CONTRIBUTING.md Signed-off-by: lrcouto <laurarccouto@gmail.com> * Push code of conduct text block to the top on CONTRIBUTING.md Signed-off-by: L. R. Couto <laurarccouto@gmail.com> * Fix trailing whitespace lint error Signed-off-by: L. R. Couto <laurarccouto@gmail.com> --------- Signed-off-by: lrcouto <laurarccouto@gmail.com> Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com> Signed-off-by: Nok <nok.lam.chan@quantumblack.com> Signed-off-by: L. R. Couto <laurarccouto@gmail.com> Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> Signed-off-by: Florian Gaudin-Delrieu <fgaudindelrieu@idmog.com> Signed-off-by: Florian Gaudin-Delrieu <florian.gaudindelrieu@gmail.com> Co-authored-by: Dmitry Sorokin <40151847+DimedS@users.noreply.github.com> Co-authored-by: Nok <nok.lam.chan@quantumblack.com> Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> Co-authored-by: Merel Theisen <49397448+merelcht@users.noreply.github.com> Co-authored-by: Florian Gaudin-Delrieu <9217921+FlorianGD@users.noreply.github.com> Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
APIDataSet
Description
When saving an
APIDataSet, the data was doubly encoded as json (making it a string instead of a dict for example), once by us and once by therequestslibrary. This PR fixes that.Development notes
The
jsonparameter ofrequests.requestis now a python object instead of a string. The tests have been updated to catch the problem.Checklist
RELEASE.mdfile