Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[PAGOPA-1564] feat: added update_cup_iban.py script to update cup iban in ecConfigTable #47

Merged
merged 3 commits into from
Mar 25, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 21 additions & 8 deletions scripts/README.md
Original file line number Diff line number Diff line change
@@ -1,25 +1,25 @@
### Prerequisites
# Prerequisites
- [azure-data-tables](https://docs.microsoft.com/en-us/python/api/overview/azure/data-tables-readme?view=azure-python)

`pip3 install -r requirements.txt`

### Scripts
#### fix_ecconfig.py
# Scripts
## fix_ecconfig.py
The script iterates on `ecconfig` table items and removes whitespaces from `Iban` column

##### How-to
###### DEV
### How-to
#### DEV
python3 fix_ecconfig.py --env=dev --account-key=<azure-access-key>

###### UAT
#### UAT

python3 fix_ecconfig.py --env=uat --account-key=<azure-access-key>

###### PROD
#### PROD

python3 fix_ecconfig.py --env=prod --account-key=<azure-access-key>

###### LOCAL
#### LOCAL

python3 fix_ecconfig.py --env=local

Expand All @@ -35,3 +35,16 @@ python3 buildEcConfigTable.py --ec ./inputData/export_enti.csv \
--env d \
--key xxx
```
## update_cup_iban.py
Iterates over `ecConfigTable` and for each Creditor Institution makes a call to ApiConfig iban enhanced api to retrieve
the `CUP` iban or the last inserted one.
### How to Use

```
python3 update_cup_iban.py \
--subkey <APIM SUBKEY TO CALL APICONFIG> \
--apim-url <APIM BASE URL ES. "https://api.platform.pagopa.it/apiconfig/auth/api/v1/"> \
--tables-primary-storage-account-key <TABLE STORAGE PRIMARY KEY> \
--tables-storage-endpoint-suffix <TABLE STORAGE ENDPOINT SUFFIX ES. "core.windows.net"> \
--tables-storage-account-name <TABLE STORAGE ACCOUNT NAME> \
```
81 changes: 81 additions & 0 deletions scripts/update_cup_iban.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
import argparse
import requests
from requests.exceptions import HTTPError
from azure.data.tables import TableClient
from azure.data.tables import UpdateMode


class UpdateIban(object):
def __init__(self):
print("[INFO][__init__] reading args parameters")
parser = argparse.ArgumentParser()
parser.add_argument('--subkey', required=True, help='subkey to invoke apim endpoint')
parser.add_argument('--apim-url', required=True, help='apim URL')
parser.add_argument('--tables-primary-storage-account-key', required=True, help='')
parser.add_argument('--tables-storage-endpoint-suffix', required=True, help='')
parser.add_argument('--tables-storage-account-name', required=True, help='')
args = parser.parse_args()
print("[INFO][__init__] parameters correctly oarsed")

print("[INFO][__init__] getting parameters values")
access_key = str(args.tables_primary_storage_account_key)
endpoint_suffix = str(args.tables_storage_endpoint_suffix)
account_name = str(args.tables_storage_account_name)
self.connection_string = f"DefaultEndpointsProtocol=https;AccountName={account_name};AccountKey={access_key};EndpointSuffix={endpoint_suffix}"
self.table_base = "pagopapcanoneunicosaecconfigtable"
self.subkey = str(args.subkey)
self.apim_url = str(args.apim_url)
print("[INFO][__init__] parameters correctly retrieved")

def update_entities(self):

print("[INFO][update_entities] connecting to ecconfigtable")
try:
with TableClient.from_connection_string(self.connection_string, table_name=self.table_base) as table:

print("[INFO][update_entities] iterating over ecconfigtable")
entities = list(table.list_entities())
for i, entity in enumerate(entities):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this script should be run after init of pagopapcanoneunicosaecconfigtable ?
what is the first step before python3 update_cup_iban.py ...

i.e. this script will update the ibans but which one initially creates the entities?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The first script to launch is buildEcConfigTable.py but we must take into account that it requires to be reviewed as it is currently based on data downloaded from PdA that has been dismissed, and at the moment we don't have the complementary service exposed by SelfCare/Area Riservata.
Anyway I'm going to better explicit this situation in readme file

id_ci = entity['RowKey']
cup_iban = self.get_iban_by_ci(id_ci)
if cup_iban == entity['Iban']:
entity['Iban'] = cup_iban
insert_entity = table.upsert_entity(mode=UpdateMode.REPLACE, entity=entity)
print(f"[INFO][update_entities] updated entity: {insert_entity}")
else:
print(f"[INFO][update_entities] iban {entity['Iban']} is already updated with CUP iban {cup_iban}")

except Exception as e:
print(f"[ERROR][update_entities] HTTP error occurred: {e}")

def get_iban_by_ci(self, id_ci):

print(f"[INFO][get_iban_by_ci] getting CUP iban for {id_ci} EC")
try:
api_url = f"{self.apim_url}/creditorinstitutions/{id_ci}/ibans/enhanced?label=0201138TS"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add comment before this line with somethig like that

2 PROVINCIA / CITTA' METROPOLITANA	1	SERVIZI PROVINCIALI / CITTA' METROPOLITANA	
Tributi e diritti gestiti direttamente dall'Ente Provinciale.	
138	Canone Unico Patrimoniale - CORPORATE	
TS	(SOLUZIONE CENTRALE GESTITA DA PAGOPA) 
La Legge di Bilancio 2020 (Legge n. 160/2019) ha previsto, a decorrere dal 1° gennaio 2021, l’istituzione  di un “canone unico patrimoniale” 
che unifica in un solo prelievo TOSAP, COSAP, l’imposta comunale sulla PUBBLICITÀ e AFFISIONI, ed altre imposte locali.
Questa soluzione è gestita centralmente da Pagopa S.p.a.	
36	9/0201138TS/	05/03/2021	01/01/2080	Attività produttive e commercio + Altro	

to give label 0201138TS context

headers = {'Ocp-Apim-Subscription-Key': self.subkey}

print(f"[INFO][get_iban_by_ci] calling api: {api_url}")
response = requests.get(api_url, headers=headers)
print(f"[INFO][get_iban_by_ci] response code: {response.status_code}")
response.raise_for_status()
json_response = response.json()
iban_enhanced = json_response['ibans_enhanced']
cup_iban = ""
if len(iban_enhanced) > 0:
cup_iban = iban_enhanced[0]['iban']
else:
print(f"[ERROR][get_iban_by_ci] no iban found associated to EC {id_ci}")

print(f"[INFO][get_iban_by_ci] found cup iban {cup_iban} for EC {id_ci}")
return cup_iban

except HTTPError as http_err:
print(f"[ERROR][get_iban_by_ci] HTTP error occurred: {http_err}")
except Exception as err:
print(f"[INFO][get_iban_by_ci] generic error occurred: {err}")


if __name__ == "__main__":
update_iban = UpdateIban()
update_iban.update_entities()