Refactor parsing and formatting #122
Conversation
…o Performance's "DIVIDEND" type
… include taxes, fees, saveback correctly 1. Removes card_failed_transaction type due to it not being relevant for PP 2. Adds INCOMING_TRANSFER_DELEGATION and from issue pytr-org#116 (card_successful_oct, TAX_REFUND, benefits_saveback_execution) to event.py 3. Introduces enums and dataclasses in event.py to a. structure and facilitate type matching b. reduce Event object's memory size c. differentiate between pp_types and types that require further csv formating. 4. Introduces hacky tax and fee parsers in event.py 5. Introduces event_formatter.py to generate multiple csv lines from certain events (saveback, taxed income, transactions with fees) Requires extensive testing (functionality + python version)
What remains: 1. Check if still compatible with python 3.8 2. Add Fees field in csv file to avoid having to generate a new Fee-type line for each fee occurence 3. Improve parsers with more logs
…nclude tax value and no further info (KapESt, Soli, Kirche)
Adds a cli argument to dl_docs and export_transactions to chronologically sort the exported csv transactions Removes unused imports Various small modifications
…nd-alone taxes and fees events
|
Apologies for the small post-review change: I had the late realization that PP transactions have embedded taxes and fees fields. I therefore added taxes and fees columns to the csv file, which voids the need to generate stand-alone tax and fee events and trims event_formatter.py in the process. |
|
Sorry, I was first too euphoric and then tested it a little bit more. Before your last commit, taxes and fees were positive and thus gave a wrong total sum. Also I realized, that card_failed_transaction are not handled at all. These only need to be added, if and only if the transaction is of type card_failed_transaction and of status= "executed". |
…ction events with executed status to REMOVALs 3. Renames UnprocessedEventType to ConditionalEventType for more clarity
|
Notwithstanding PP's indifference to the sign of fees and taxes, sticking to the negative sign convention for both makes more sense indeed. |
Refactors the parsing of all_events.json and the formatting into all_transactions.csv to capture all pp events generated by tr events, especially taxes, fees, and savebacks.
In detail:
1. Removes card_failed_transaction type due to it not being relevant for PP2. Adds INCOMING_TRANSFER_DELEGATION and events from #116 (INTEREST_PAYOUT, card_successful_oct, TAX_REFUND, benefits_saveback_execution) to event.py
3. Introduces enums and dataclasses in event.py to a. structure and facilitate type matching b. differentiate between pp_types and types that require further csv formatting
4. Introduces tax and fee parsers in event.py
5. Introduces event_formatter.py to generate multiple csv lines from certain events (saveback,
taxed income, transactions with fees). Fees and taxes are added as csv columns to avoid stand-alone transactions.6. Adds a cli option argument to dl_docs and export_transactions to chronologically sort the exported csv transactions
Additionally:
The all_events.json available to me parses into an accurate all_transactions.csv; however, I highly encourage everyone to test the pr on their own event logs.
I did not observe breaking changes when testing with the docker image of python 3.8