Release 4.0 append output for save_results

[afinit]

Describe your changes

I expect changes, but this seems like a good start for discussion.

• adds error handling for an illegal outformat param
• adds append param that works for both json and csv. defaults to True
• changes the code to use with to open csv file

Before this PR, neither json nor csv files would append data. The json path reads in the existing data, but throws away the observations and only keeps the metadata.

The part that needs some discussion:
With this PR, existing json and csv data are treated differently. When json data is appended, we overwrite any observations that have the same sample name as our new data. When csv data is appended, we are strictly appending without any consideration of the data that is already in the file.

I don't like this difference in behavior

Options for fixing the differences

There are probably other options, but these are the 2 ideas I came up with to resolve the current difference in behavior

Option 1

Read existing CSV data with pandas and remove any rows with the same sample name as the new data

Option 2

Read existing data for both json and csv. Modify conflicting sample names to have an appended _### where we autoincrement that number for new data. This would look something like:

# existing data:
sample,trait,value,label
default,string,string,none
default_001,boolean,1,none

# new data:
sample,trait,value,label
default,string,string2,none
default,boolean,0,none

# data written:
sample,trait,value,label
default,string,string,none
default_001,boolean,1,none
default_002,string,string2,none
default_002,boolean,0,none

Type of update

new feature

Associated issues

Addresses #886

For the reviewer

See this page for instructions on how to review the pull request.

• PR functionality reviewed in a Jupyter Notebook
• All tests pass
• Test coverage remains 100%
• Documentation tested
• New documentation pages added to plantcv/mkdocs.yml
• Changes to function input/output signatures added to updating.md
• Code reviewed
• PR approved

[codecov]

Codecov Report

Merging #1225 (4b2d1b9) into release-4.0 (381cf6f) will not change coverage.
Report is 2 commits behind head on release-4.0.
The diff coverage is 100.00%.

@@              Coverage Diff              @@
##           release-4.0     #1225   +/-   ##
=============================================
  Coverage       100.00%   100.00%           
=============================================
  Files              157       157           
  Lines             6759      6767    +8     
=============================================
+ Hits              6759      6767    +8     

Flag	Coverage Δ
unittests	100.00% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed	Coverage Δ
plantcv/plantcv/classes.py	100.00% <100.00%> (ø)

[nfahlgren]

@all-contributors please add @afinit for code, doc, test.

[allcontributors]

@nfahlgren

I've put up a pull request to add @afinit! 🎉

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 100.00%. Comparing base (381cf6f) to head (4b2d1b9).
Report is 218 commits behind head on release-4.0.

Additional details and impacted files

@@              Coverage Diff              @@
##           release-4.0     #1225   +/-   ##
=============================================
  Coverage       100.00%   100.00%           
=============================================
  Files              157       157           
  Lines             6759      6767    +8     
=============================================
+ Hits              6759      6767    +8     

Flag	Coverage Δ
unittests	100.00% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
plantcv/plantcv/classes.py	100.00% <100.00%> (ø)