CHANGELOG

All notable changes to dvas will be documented in this file.

The format is inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).


## [vX.Y.Z]
### Added:
### Fixed:
### Changed:
### Deprecated:
### Removed:
### Security:

## [v1.1.0]
### Added:
 - [fpavogt, 17.08.2023] Enable the ingestion of manual flag files by the uaii2022 recipe
 - [fpavogt, 24.07.2023] Add DOI & pypi info to docs following first public release
### Fixed:
 - [fpavogt, 07.09.2023] Fix #295
 - [fpavogt, 11.08.2023] Fixes #288 [GDP SHC info is now extracted by dvas]
### Changed:
 - [fpavogt, 11.08.2023] Request python>=3.11 and pandas>=2.0
 - [fpavogt, 24.07.2023] Change Github Issues to use the new format
### Deprecated:
### Removed:
### Security:

## [v1.0.0]
### Added:
 - [fpavogt, 24.07.2023] Add new pypi CI action for automated release
 - [fpavogt, 19.05.2023] Ingest SHC info into the profile metadata
 - [fpavogt, 16.06.2023] Update docs for v1.0.0

## [v0.9.5.dev0]
### Added:
 - [fpavogt, 20.04.2023] Ingest and carry lat/lon data for GDPs and CWS (no uncertainties)
 - [fpavogt, 18.04.2023] Introduce and deploy "wvec", a new variable for the Wind vector calculations
### Fixed:
 - [fpavogt, 14.04.2023] Fix a bug associated to pandas 2.0.0 in fancy_bitwise_or()
### Changed:
 - [fpavogt, 14.04.2023] Parameter names are now proper abbreviations (fixes #282)
 - [fpavogt, 14.03.2023] Units of wdir are now typset properly in LaTeX as $^{\circ}$
 - [fpavogt, 13.03.2023] Units of RH are now %RH

## [v0.9.4.dev0]
### Added:
 - [fpavogt, 23.02.2023] Add first_timestep(.uncertainty) to the CWS netCDFs
 - [fpavogt, 22.02.2023] Add new "comment" field to dvas prms (fix #277 wdir info)
### Fixed:
 - [fpavogt, 22.02.2023] Fix #280 (optimize DB queries to avoid superfluous conditions)
 - [fpavogt, 22.02.2023] Fix #278 and #275
 - [fpavogt, 22.02.2023] Fix #276 (correct UTLS, add MUS, rm troposphere)
 - [fpavogt, 22.02.2023] Fix #277 (gph AMLS)
 - [fpavogt, 08.02.2023] Adjust pylint action for pylint 2.16
### Changed:
 - [fpavogt, 08.02.2023] Break-up dot products to keep memory use low

## [v0.9.3.dev0]
### Fixed:
 - [fpavogt, 08.02.2023] Adjust pylint action for pylint 2.16
### Changed:
 - [fpavogt, 08.02.2023] Break-up dot products to keep memory use low

## [v0.9.2.dev0]
### Fixed:
 - [fpavogt, 31.01.2023] Ignore NaNs when unwrapping angles (fix #273)

## [v0.9.1.dev0]
### Added:
 - [fpavogt, 26.01.2023] Add option to store profile data on disk instead of the db
 - [fpavogt, 23.01.2023] Add choice of mid to sync from val
### Fixed:
 - [fpavogt, 17.01.2023] Left align ylabels accross suplots (fix #272)

## [v0.9.0.dev0]
### Added:
 - [fpavogt, 2023-01-12] Add synop cloud code to GDP and CWS metadata
 - [fpavogt, 2023-01-06] Add clean TOD option to biglambda recipe step (fixes #249)
 - [fpavogt, 2023-01-05] Sync via global match can be restricted to a specific range of values (fix #269)
 - [fpavogt, 2022-11-18] Add/connect UTLS, PBL, and Free Tropopause region flags
 - [fpavogt, 2022-11-10] Add a WAS_INVALID flag to keep track of fixed GDP points, e.g. via #205
### Fixed:
 - [fpavogt, 2022-11-22] Fix #266 by keeping track of KS alpha level in metadata of CWS
 - [fpavogt, 2022-11-14] Include fid in plots and filenames (fix #261)
### Changed:
 - [fpavogt, 2023-01-11] Refactor the corr. coeffs. routine to improve performances
 - [fpavogt, 2022-11-10] Cleanup filename of KS plots
### Removed:
 - [fpavogt, 2021-01-05] Removed 'ucr' in favor of 'ucu' (fixes #268)

## [v0.8.1.dev0]
### Fixed:
 - [fpavogt, 2022-11-09] Fix #262 - handle angular wrapping in resample
 - [fpavogt, 2022-11-09] Fix Runtime warning caused by uc_tot=0 cases in gdp_vs_cws plots
 - [fpavogt, 2022-11-08] Fix #235 and #236 (wdir wrap-around)

## [v0.8.0.dev0]
### Added:
 - [fpavogt, 2022-10-28] Add netCDF export recipe step
### Fixed:
 - [fpavogt, 2022-10-31] Fix #253 (no more Int64 flags) and #259 (resampled points now inherit the anchor flags)
 - [fpavogt, 2022-10-31] Fix #252 and #256
 - [fpavogt, 2022-10-28] Fix #254
### Changed:
 - [fpavogt, 2022-10-28] Clean-up TAGS and FLAGS, incl. rename of "raw" to "original" (fix #255)

## [v0.7.1]
### Added:
 - [fpavogt, 2022-09-13] Add new INVALID flag for GDP points with NaN uncertainties (see #244)
 - [fpavogt, 2022-08-19] Add kn2ms conversion function
 - [fpavogt, 2022-08-05] Add support for the M20, iMS100, ATMS-3710, and WxR-301D sondes
 - [fpavogt, 2022-07-13] Set resampled points to NaN if they are >=1s away from an original point (fix #233)
 - [fpavogt, 2022-07-12] Add support for the PS01B3M sonde
 - [fpavogt, 2022-07-12] Modify GetreldtExpr() to optionally round time stamps, and add dedicated test
 - [fpavogt, 2022-07-12] Add support for the RS41-SG, the ATMS-3710, and the HT03G-1U sondes
 - [fpavogt, 2022-07-07] Add support for the iMet-54-AA sonde
 - [fpavogt, 2022-07-06] Add support for the DFM-17 sonde
 - [fpavogt, 2022-07-01] Add subplot with relative contributions of GDPs to CWS in gdp-vs-cws (fix #226)
### Fixed:
 - [fpavogt, 2022-10-07] Correct bug in RS92 GDP config file, cleanup demo script
 - [fpavogt, 2022-09-29] Fix #242
 - [fpavogt, 2022-08-23] Fix #241
 - [fpavogt, 2022-08-22] Fix #240
 - [fpavogt, 2022-08-05] Fix #238
 - [fpavogt, 2022-07-15] Fix #232 - wdir curves no longer jump from 0 to 360 in plots
 - [fpavogt, 2022-07-12] Deal with the floating point errors associated to pd.dt.total_seconds()
 - [fpavogt, 2022-07-12] Fix typo in model mid field
### Changed:
 - [fpavogt, 2022-08-19] Add new metadata fields for first, launch, and burst points (fix #231)
 - [fpavogt, 2022-07-15] Changed InfoManager sorting rule to include mid (fix #234)
 - [fpavogt, 2022-07-12] Check that time stamps are strictly increasing in the resampling strategy
 - [fpavogt, 2022-07-01] Boost version to 0.7.0.dev0, after move to MeteoSwiss organization

## [v0.6.0.dev1]
### Added:
 - [fpavogt, 2022-06-29] Cleanup the tropopause function, to support the WMO, MCH and GRUAN algorithm
 - [fpavogt, 2022-06-07] Basic recipe step can now issue eid-edt files for the UAII plot preview software
 - [fpavogt, 2022-06-03] Add new recipe step to flag zones of interest, compute the tropopause, etc ...
 - [fpavogt, 2022-06-03] If no burst point is specified, flag any points beyond the max altitude as "descent"
 - [fpavogt, 2022-05-25] Add participant_preview plot and associated recipe step
 - [fpavogt, 2022-05-19] Allow DB to store timezone info with datetime fields
 - [fpavogt, 2022-05-17] Add ability to skip recipe steps beyond a certain point
 - [fpavogt, 2022-05-11] Add colors to on-screen logging
 - [fpavogt, 2022-05-05] Fix #209 by returning full covariance matrices and plotting them
### Fixed:
 - [fpavogt, 2022-06-30] Fix #169 - remove pampy dependancy
 - [fpavogt, 2022-06-30] Fix #222 - prf.has_flg() now returns bool instead of int64
 - [fpavogt, 2022-06-30] Fix #223 - add --debug option to dvas_run_recipe entry point
 - [fpavogt, 2022-05-17] Fix wrong convention in get_shifts_from_alt
 - [fpavogt, 2022-04-26] Fix #205 with a dedicated fix in the uaii2022.basic.cleanup recipe step
 - [fpavogt, 2022-04-14] Fix #208 by improving dvas plots
### Changed:
 - [fpavogt, 2022-07-01] Make 3.10 the minimum required Python version
 - [fpavogt, 2022-07-01] Remove pytest-datafiles dependancy
 - [fpavogt, 2022-06-10] Change day_night tag to timeofday, and make it "optional" by relying on the RS41 GDP to apply if to all
 - [fpavogt, 2022-06-08] Refactor hardcoded variable names
 - [fpavogt, 2022-05-30] Add official support for Python 3.10
 - [fpavogt, 2022-05-25] Cleaned-up config files
 - [fpavogt, 2022-05-25] Changed names of Parameter config file and associated db table
 - [fpavogt, 2022-05-17] The flight overview plot is now using the index i for x (rather than time), to show the synced profiles
### Deprecated:
### Removed:
### Security:

## [v0.5.0] (2022-04-14)
### Added:
 - [fpavogt, 2022-04-13] New CI action to check the code version is incremented for new releases
 - [fpavogt, 2022-04-01] Connect `from_step_id` to recipe HL entry-point, incl. automatic disabling of DB reset
 - [fpavogt, 2022-03-31] Fix-up assembly of Delta Profiles, and implement dedicated dtas plots
 - [fpavogt, 2022-03-29] Add the `reset_db` recipe parameter to allow not systematically reseting it
 - [fpavogt, 2022-03-21] Add new uc_budget diagnostic plot to dvas
 - [fpavogt, 2022-03-16] Enable the cropping of descent data in the "cleanup" recipe step
 - [fpavogt, 2022-03-15] Propagate flags through process_chunk(), weighted_means(), and delta()
 - [fpavogt, 2022-03-08] Flag descent data in basic cleanup step. Add new 'descent' flag
 - [fpavogt, 2022-03-03] Enable the resampling of profiles with a new cleanup recipe step
 - [fpavogt, 2022-03-01] Add prf_summary recipe (fix #177)
 - [modolol, 2022-02-28] Global DB extract
 - [fpavogt, 2022-02-25] Add support for iMS-100 GDP beta 2
 - [fpavogt, 2021-11-30] Add new "force-all-valid" strategy to combine GDPs
 - [fpavogt, 2021-11-26] Add ability to feed lambda fcts to csv_skiprows
 - [modolol, 2021-11-03] Add `mid` in DB search options
 - [fpavogt, 2021-07-02] Add `has_tag()` convenience method to `Profile` and `MultiProfile` classes
### Fixed:
 - [fpavogt, 2022-04-13] Fix #84
 - [fpavogt, 2022-03-18] Fix #163
 - [fpavogt, 2022-02-25] Fix #179
 - [fpavogt, 2022-02-21] Fix #192
 - [fpavogt, 2021-12-02] Fix issue #181
 - [GonzagueRomanens, 2021-11-26] Fix issue #187
 - [fpavogt, 2021-11-19] Fix issue #178
 - [fpavogt, 2021-11-03] Fix issue #172
 - [fpavogt, 2021-09-16] Fix issue #166 and #165
 - [modolol, 2021-07-12] Fix issue #185
 - [modolol, 2021-07-12] Fix issue #160
### Changed:
 - [fpavogt, 2022-04-05] Adjust paths input structure of the dvas recipes, improve recipe tag handling
 - [fpavogt, 2022-03-18] Modified & improved gdp_vs_cws plots into a dedicated recipe step
 - [fpavogt, 2022-03-01] Simplified the logging by removing the specific names
 - [fpavogt, 2022-03-01] Add dev option to setup.py
 - [modolol, 2022-02-28] Remove mandatory tag creation in config
 - [fpavogt, 2022-02-25] Remove all correlations between distinct GDP models
 - [fpavogt, 2021-11-26] Update RS41 GDPs to v1
 - [modolol, 2021-11-24] Modify 'header' rules in origdata
 - [fpavogt, 2021-11-22] Various logging improvements
 - [fpavogt, 2021-10-18] Clean-up default csv na values in origdata

## [v0.4.1] (2021-07-16)
### Added:
 - [fpavogt, 2021-07-14] Add `csv_encoding` parameter to config files, and set it to 'altin_1' for RS-41
 - [fpavogt, 2021-07-02] Add `has_tag()` convenience method to `Profile` and `MultiProfile` classes
### Fixed:
 - [modolol, 2021-07-12] Fix issue #160
### Deprecated:
 - [fpavogt, 2021-07-14] Make Python 3.9.0 and pandas 1.3.0 minimum required versions for dvas

## [v0.4.0] (2021-06-30)
### Added:
 - [fpavogt, 2021-06-23] Add `get_validities()` routine to identify which levels to combine in CWS
 - [fpavogt, 2021-06-16] Add `dvas_optimize` entry point to find ideal CHUNK_SIZE value
 - [fpavogt, 2021-05-25] Add recipe_config.yml and associated initialization function for recipes
 - [fpavogt, 2021-05-18] Add pytest fixture to test LaTeX plots using the command line argument "--latex"
 - [modolol, 2021-05-11] Add interpreter to mix raw parameter in config file
 - [fpavogt, 2021-04-19] Add config info for variables other than T, and for the M10 raw file
 - [modolol, 2021-04-13] Add DB exploring tool
### Fixed:
 - [modolol, 2021-06-10] Fix missing correction + bugs in PR #147
 - [modolol, 2021-06-08] Cleanup towards v04 (config module)
### Changed:
 - [fpavogt, 2021-06-30] Ensure that the default db config files are only found in`dvas_demo`
 - [fpavogt, 2021-06-23] `has_flg()` now returns 0 if flags are <NA>
 - [fpavogt, 2021-06-18] Rename `corcoefs()` as `coeffs()`, and move it to new `dvas.tools.gdps.correlations` module
 - [fpavogt, 2021-05-18] Improve looks of plots, and split-up "orig_data_config.yml"
 - [fpavogt, 2021-05-18] Change parameter names in tests, for consistency with main code

## [v0.3.0] (2021-04-12)
### Added:
 - [fpavogt, 2021-03-15] Add extract() class method to MultiProfile
 - [fpavogt, 2021-03-15] Make MultiProfile iterable using __getitem__() (see #114)
 - [modolol, 2021-03-11] Import flags from CSV files
 - [fpavogt, 2021-03-10] Add resampling strategy for RS and GDP Profiles
 - [fpavogt, 2021-03-03] Add new gdps sub-module in dvas.tools
### Fixed:
 - [fpavogt, 2021-04-01] Fixed sync routines following #129, now with dedicated tests
 - [fpavogt, 2021-03-31] Bring the ks_test() routine back from the dead
 - [fpavogt, 2021-03-31] Profile.__getattr__() now always returns a pd.Series, even for an index (#130)
 - [fpavogt, 2021-03-05] Fix test_utils.py tests
 - [modolol, 2020-02-25] Fix issue #113
 - [modolol, 2020-02-17] Fix issue #96 (enhancement toward v0.3)
### Changed:
 - [fpavogt, 2021-03-31] MultiProfile.get_prms() now returns a DataFrame instead of a list (#129)
 - [fpavogt, 2021-03-29] Use masked arrays when dealing with Jacobians and correlation matrices to correctly handle NaNs
 - [fpavogt, 2021-03-03] Re-design GDP routine to use bigger Profile chunks to speed things up
 - [modolol, 2020-02-25] DB access methods
 - [modolol, 2020-02-23] Change evt_dt -> edt
 - [modolol, 2020-02-23] Change evt_id -> eid, rig_id -> rid
 - [modolol, 2020-02-22] Change InstrType -> Model. Change mdl_id in tag -> mid from Model table
 - [modolol, 2020-02-22] Add data source to InfoManager

## [v0.2.0] (2021-02-16)
### Added:
 - [fpavogt, 2021-02-01] Add dvas_recipes package, to host the demo and high-level UAII routines
 - [modolol, 2020-01-29] Add instrument, tag and parameter informations in MultiProfiles
 - [modolol, 2020-12-29] Add metadata management in config and DB.
 - [fpavogt, 2020-12-17] Add a rebase strategy and fix #83.
### Fixed:
 - [modolol, 2020-02-11] Disconnect global db from each tests
 - [fpavogt, 2021-02-02] Fix #102 & #87. Lift cap on pandas version.
 - [modolol, 2020-01-29] Fix #94
 - [fpavogt, 2020-12-21] Fix #60
 - [fpavogt, 2020-12-15] Signature-changing "deepcopy" decorator now leads to correct decorated signatures (see #70).
### Changed:
 - [fpavogt, 2021-02-05] Additional restructuring of dvas_recipes, to give users propers packages.
 - [fpavogt, 2021-02-05] Renamed dvasError to DvasError.
 - [modolol, 2020-01-29] Standardize table field names.
 - [modolol, 2021-01-14] Removes instrument from the mandatory configuration (and all dependencies) + add product id to instrument
 - [modolol, 2020-12-18] Modify logger
 - [modolol, 2020-12-18] Add dvas.dvas facade
 - [modolol, 2020-12-16] Move data.data.update_db to new module data.io
 - [modolol, 2020-12-16] Modify config metadata interpreter syntax
 - [modolol, 2020-12-15] Modify config generator interpreter syntax
### Security:
 - [fpavogt, 2020-12-21] Add suggestion for contributors to sign their commits with GPG keys inside CONTRIBUTING.md

## [v0.1.1] (2020-12-14)
### Fixed:
 - [fpavogt, 2020-12-14] Fix auto-publications of docs + minor code tweaks

## [v0.1.0] (2020-12-14)
### Added:
 - [GonzagueRomanens, 2020-12-19] add SampleDatasetMultipayload
 - [fpavogt & modolol, 2020-12-07] Cleanup towards v0.1
 - [fpavogt & modolol, 2020-12-03] Winter cleanup towards v0.1
 - [fpavogt, 2020-11-05] Refactor ProfileManager class into Profile, grouping 'alt', 'val' , and 'flag' in a pandas DataFrame
 - [modolol, 2020-11-02] Tropopause calculation function
 - [fpavogt, 2020-07-23] Add GRUAN submodule, with GDP correlation rules
 - [modolol, 2020-07-16] Add dummy examples
 - [modolol, 2020-07-03] .dev_utils directory in package
 - [fpavogt, 2020-06-30] Add docs CI & a basic netCDF4 test
 - [modolol, 2020-06-12] Add tags table in DB and config items generator
 - [fpavogt, 2020-06-10] Create a plot submodule, including dedicated style sheets
 - [modolol, 2020-05-25] Add logging possibilities
 - [modolol, 2020-05-18] Add pytest-env, pytest-datafiles in pip install CI_pytest.yml
 - [fpavogt, 2020-05-14] Github Action to check if this CHANGELOG gets modified :)
 - [fpavogt, 2020-05-14] Add README, CODE_OF_CONDUCT, CONTRIBUTING files
 - [fpavogt, 2020-05-07] Github Actions to validate syntax using pylint and run pytests
 - [fpavogt, 2020-05-07] CHANGELOG file based on https://keepachangelog.com
 - [fpavogt, 2020-05-07] Sphinx documentation with primary pages and todo list

### Changed:
 - [modolol, 2020-11-02] Data classes types
 - [modolol, 2020-07-16] load data search syntax
 - [modolol, 2020-07-14] Use design pattern chain of responsibility for update_db
 - [fpavogt, 2020-07-02] Set GPL v3.0 license
 - [modolol, 2020-06-23] Remove instr_id. Use SN instead (adaptation for GPD integration)
 - [modolol, 2020-06-22] Licence and copyright files headers
 - [modolol, 2020-06-18] Modify TypedProperty to use pampy PyPI package
 - [modolol, 2020-05-18] Adapted setup.py and CI_pylinter.yml

### Fixed:
 - [modolol, 2020-06-17] Fix bug in data loading (dvas.database.database)
 - [modolol, 2020-06-17] Remove automatic make dir in module import and global path assignment
 - [fpavogt, 2020-06-12] Automatically load the `base` style sheet for plots & new tests
 - [modolol, 2020-06-12] Set os.chmod(777) -> skip for Windows and check for R/W for Mac and Linux
 - [modolol, 2020-05-25] Adapt for passing multi OS_CI
 - [modolol, 2020-05-18] Code cleaning (bis)
 - [modolol, 2020-05-15] Code cleaning