Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IMF: XMLParseError: KeyError: 'TIME_FORMAT' #239

Open
zymon opened this issue Mar 7, 2023 · 0 comments
Open

IMF: XMLParseError: KeyError: 'TIME_FORMAT' #239

zymon opened this issue Mar 7, 2023 · 0 comments

Comments

@zymon
Copy link

zymon commented Mar 7, 2023

import pandasdmx as sdmx

imf = sdmx.Request("IMF")
df=imf.dataflow("1PI")

Output:
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): sdmxcentral.imf.org:443
DEBUG:urllib3.connectionpool:https://sdmxcentral.imf.org:443 "GET /ws/public/sdmxapi/rest/dataflow/IMF/1PI/latest?references=all HTTP/1.1" 200 None

--- SS without DSD ---
{1: False}

--- <class 'pandasdmx.message.StructureMessage'> ---
{2: <pandasdmx.StructureMessage>

id: 'IREF967191' prepared: '2023-03-07T13:45:10+00:00' receiver: sender: source: test: False}

--- <class 'pandasdmx.model.AgencyScheme'> ---
{'AGENCIES': <AgencyScheme SDMX:AGENCIES(1.0) (16 items): SDMX Agency Scheme>}

--- <class 'pandasdmx.model.DataStructureDefinition'> ---
{'ECOFIN_DSD': <DataStructureDefinition IMF:ECOFIN_DSD(1.0): ECOFIN Data Structure Definition>}

--- <class 'pandasdmx.model.Agency'> ---
{'IMF': , 'SDMX': }

--- <class 'pandasdmx.model.DataflowDefinition'> ---
{'1PI': <DataflowDefinition IMF:1PI(1.0): Coordinated Portfolio Investment Survey (CPIS) for Collection>}

--- <class 'pandasdmx.model.CategoryScheme'> ---
{'CS_SDMX_CENTRAL': <CategoryScheme IMF:CS_SDMX_CENTRAL(1.0) (1 items)>}

--- <class 'pandasdmx.model.Categorisation'> ---
{'b5979306-0fc2-8166-4370-fbff3bdd8189': <Categorisation IMF:b5979306-0fc2-8166-4370-fbff3bdd8189(1.0): Category 'IMF:CS_SDMX_CENTRAL(1.0).4' linking to Dataflow 'IMF:1PI(1.0)'>}

--- <class 'pandasdmx.model.Codelist'> ---
{'CL_DATADOMAIN': <Codelist IMF:CL_DATADOMAIN(1.0) (64 items): Data Domain>, 'CL_INDICATOR': <Codelist IMF:CL_INDICATOR(1.0) (99856 items): Economic Indicator>, 'CL_REF_AREA': <Codelist IMF:CL_REF_AREA(1.0) (709 items): Reference Area>, 'CL_AGE': <Codelist SDMX:CL_AGE(1.0) (5 items): Age>, 'CL_CONF_STATUS': <Codelist SDMX:CL_CONF_STATUS(1.0) (5 items): Confidentiality Status>, 'CL_DECIMALS': <Codelist SDMX:CL_DECIMALS(1.0) (16 items): Decimals>, 354018: <Codelist SDMX:CL_FREQ(1.0) (8 items): Frequency>, 354019: <Codelist SDMX:CL_FREQ(2.0) (9 items): Frequency>, 354091: <Codelist IMF:CL_OBS_STATUS(1.0) (19 items): Observation Status>, 354092: <Codelist SDMX:CL_OBS_STATUS(1.1) (17 items): Observation Status>, 'CL_OCCUPATION': <Codelist SDMX:CL_OCCUPATION(1.0) (619 items): Occupation>, 'CL_SEX': <Codelist SDMX:CL_SEX(2.0) (2 items): Sex>, 'CL_TIME_FORMAT': <Codelist SDMX:CL_TIME_FORMAT(1.0) (21 items): Time Format>, 356688: <Codelist IMF:CL_UNIT_MULT(1.0) (11 items): Unit Multiplier>, 356689: <Codelist SDMX:CL_UNIT_MULT(1.0) (9 items): Unit Multiplier>, 357446: <Codelist IMF:CL_OBS_STATUS(1.0) (0 items)>, 357447: <Codelist SDMX:CL_OBS_STATUS(1.1) (0 items)>, 357789: <Codelist IMF:CL_UNIT_MULT(1.0) (0 items)>, 357790: <Codelist SDMX:CL_UNIT_MULT(1.0) (0 items)>, 357841: <Codelist SDMX:CL_FREQ(2.0) (0 items)>, 357842: <Codelist SDMX:CL_FREQ(1.0) (0 items)>, 'CL_UNIT_MULT': <Codelist IMF:CL_UNIT_MULT(1.0) (0 items)>, 'CL_OBS_STATUS': <Codelist IMF:CL_OBS_STATUS(1.0) (0 items)>}

--- <class 'pandasdmx.model.ConceptScheme'> ---
{'ECOFIN_CONCEPTS': <ConceptScheme IMF:ECOFIN_CONCEPTS(1.0) (8 items): Concepts for EcoFin DSD>, 'CROSS_DOMAIN_CONCEPTS': <ConceptScheme SDMX:CROSS_DOMAIN_CONCEPTS(1.0) (135 items): SDMX Cross Domain Concept Scheme>}

--- Name ---
{357890: ('en', '1PI_CONSTRAINT')}

--- <class 'pandasdmx.reader.sdmxml.Reference'> ---
{'1PI': <pandasdmx.reader.sdmxml.Reference object at 0x0000018C12FA2E90>}

--- <class 'pandasdmx.model.MemberSelection'> ---
{357893: <MemberSelection DATA_DOMAIN in {'1PI'}>, 358179: <MemberSelection INDICATOR in {'IAP_EUR', 'IAPDOF_S_XDC', 'IAPEOFM_XDC', 'IAPDOFI_L_EUR', 'IAPONH_EUR', 'IAPDSF_XDC', 'IAPUS_EUR', 'IAPDJY_S_XDC', 'IAPDA_S_USD', 'IAPDUS_XDC', 'IAPEUS_XDC', 'IAPONH_USD', 'IAPDA_S_EUR', 'IAPB_EUR', 'IAPDOFO_S_EUR', 'IAPDG_S_USD', 'IAPEOFM_USD', 'IAPDSF_L_XDC', 'IAPDOC_L_USD', 'ILPD_S_EUR', 'IAPDBP_L_XDC', 'IAPDONH_S_EUR', 'IAPOF_XDC', 'IAPDBP_EUR', 'IAPD_L_XDC', 'IAPDOC_L_EUR', 'IAPEOC_USD', 'IAPDOFO_EUR', 'IAPDUS_S_EUR', 'IAPON_EUR', 'IAPBP_USD', 'IAPDEU_L_USD', 'IAPDEU_S_USD', 'IAPDONO_L_USD', 'IAPDOF_XDC', 'IAPDONH_S_USD', 'IAPEEU_EUR', 'IAPDOC_L_XDC', 'IAPDUS_S_USD', 'IAPD_EUR', 'IAPDOF_S_USD', 'IAPONO_EUR', 'IAPDONH_L_XDC', 'IAPDJY_S_EUR', 'IAPDUS_S_XDC', 'IAPJY_EUR', 'IAPDON_S_XDC', 'IAPDBK_S_XDC', 'IAPEJY_EUR', 'IAPDUS_L_EUR', 'IAPD_S_EUR', 'ILPE_EUR', 'IAPEONN_EUR', 'IAPE_XDC', 'IAPG_EUR', 'IAPDBK_USD', 'IAPDBK_EUR', 'IAPDSF_S_XDC', 'IAPEONO_XDC', 'IAPG_XDC', 'IAPDBK_S_USD', 'IAPDONO_XDC', 'IAPDON_USD', 'IAPDBK_L_EUR', 'IAPEB_EUR', 'IAPDG_L_XDC', 'IAPDBK_XDC', 'IAPEUS_USD', 'IAPDONH_USD', 'IAPONO_XDC', 'IAPDG_L_EUR', 'IAPDONN_S_EUR', 'IAPDON_EUR', 'IAPD_S_USD', 'IAPDA_L_USD', 'IAPEOFO_XDC', 'IAPDJY_L_USD', 'IAPDJY_L_XDC', 'IAPA_USD', 'IAPDONN_S_XDC', 'IAPEOFO_USD', 'IAPDOFI_L_XDC', 'IAPDOFM_L_EUR', 'IAPUS_XDC', 'ILPD_S_USD', 'IAPEBP_XDC', 'IAP_USD', 'IAPDEU_USD', 'IAPEONO_EUR', 'IAPDON_XDC', 'IAPDJY_USD', 'IAPDBK_L_XDC', 'IAPEOFI_USD', 'IAPESF_XDC', 'IAPD_S_XDC', 'IAPDONN_S_USD', 'IAPEOFI_XDC', 'IAPONO_USD', 'IAPEBP_USD', 'IAPEG_XDC', 'IAPJY_USD', 'IAPEU_USD', 'IAPDEU_L_XDC', 'IAPDOF_S_EUR', 'IAPEONH_XDC', 'IAPDOFM_XDC', 'IAPDOFI_S_USD', 'IAPDONN_L_USD', 'IAPDEU_L_EUR', 'ILPD_USD', 'ILPD_EUR', 'IAPDOFM_USD', 'IAPD_XDC', 'IAPEON_XDC', 'IAPDONO_S_XDC', 'IAPEA_XDC', 'IAPOC_USD', 'IAPE_EUR', 'IAPDUS_L_USD', 'IAPDBK_S_EUR', 'IAPDONN_USD', 'ILPD_L_EUR', 'IAPDONN_EUR', 'IAPDSF_S_EUR', 'IAPONN_XDC', 'IAPDON_L_XDC', 'IAPE_USD', 'IAPDONH_L_EUR', 'ILP_XDC', 'IAPSF_USD', 'IAPDG_XDC', 'IAPOF_USD', 'IAPEON_EUR', 'IAPDG_S_EUR', 'IAPESF_USD', 'IAPDA_USD', 'IAPEEU_XDC', 'IAPDBP_S_EUR', 'IAPDEU_XDC', 'ILPD_XDC', 'IAPA_EUR', 'IAPDBP_S_USD', 'ILPD_S_XDC', 'IAPD_L_USD', 'IAPSF_XDC', 'IAPDUS_USD', 'ILP_USD', 'IAPJY_XDC', 'IAPDJY_L_EUR', 'IAPDOC_S_USD', 'IAPBP_XDC', 'IAPDA_EUR', 'IAPDOFI_EUR', 'IAPDJY_EUR', 'IAPOFI_XDC', 'IAPESF_EUR', 'IAPEUS_EUR', 'IAPEON_USD', 'IAPOFM_XDC', 'IAPDJY_S_USD', 'IAPDOFM_S_EUR', 'IAPEOFM_EUR', 'IAPDOFI_S_XDC', 'IAPDG_S_XDC', 'IAPDOFO_S_USD', 'IAPDONH_XDC', 'IAPDONO_L_EUR', 'IAPD_L_EUR', 'IAPDONN_XDC', 'IAPDONO_L_XDC', 'IAPEU_EUR', 'IAPDBP_USD', 'ILPE_XDC', 'IAPDOFM_EUR', 'IAPONH_XDC', 'IAPDOFO_USD', 'IAPDONH_L_USD', 'IAPDOF_EUR', 'IAPDONH_S_XDC', 'IAPDEU_S_XDC', 'IAPSF_EUR', 'IAPONN_EUR', 'IAPG_USD', 'IAPEG_USD', 'IAPDBP_L_USD', 'IAPOFM_USD', 'IAPOFI_USD', 'IAPDG_USD', 'IAPDOFO_S_XDC', 'IAPEBP_EUR', 'IAPDSF_S_USD', 'ILPE_USD', 'IAPOFO_EUR', 'IAPDEU_EUR', 'IAPEOF_EUR', 'IAPEU_XDC', 'IAPDOFO_XDC', 'IAPDONO_USD', 'IAPDG_L_USD', 'IAPEONH_EUR', 'IAPDONO_S_EUR', 'IAPOC_EUR', 'IAPDOC_XDC', 'IAPDBK_L_USD', 'IAPDJY_XDC', 'IAPDONO_EUR', 'IAPDOC_S_EUR', 'IAPEB_USD', 'IAPDON_S_USD', 'IAPDA_S_XDC', 'IAPDOFM_L_XDC', 'IAPDOF_L_EUR', 'IAP_XDC', 'IAPDA_L_XDC', 'IAPDON_L_USD', 'IAPEOFI_EUR', 'IAPDOC_EUR', 'IAPEEU_USD', 'IAPDONN_L_XDC', 'IAPDOFI_USD', 'IAPEOFO_EUR', 'IAPUS_USD', 'IAPOFO_XDC', 'IAPEONH_USD', 'IAPDOFO_L_XDC', 'IAPEOC_XDC', 'IAPDOF_L_XDC', 'IAPB_USD', 'IAPDG_EUR', 'IAPBP_EUR', 'IAPDOFI_L_USD', 'IAPEOF_XDC', 'IAPDA_L_EUR', 'IAPDEU_S_EUR', 'IAPDA_XDC', 'IAPEONN_XDC', 'IAPDONH_EUR', 'IAPDOC_USD', 'IAPDBP_XDC', 'IAPB_XDC', 'IAPEG_EUR', 'IAPDONN_L_EUR', 'IAPDOF_USD', 'IAPEA_USD', 'IAPA_XDC', 'IAPEJY_USD', 'IAPEA_EUR', 'IAPDOFM_L_USD', 'IAPON_USD', 'IAPDOFO_L_EUR', 'IAPDUS_L_XDC', 'IAPEOC_EUR', 'IAPONN_USD', 'IAPDSF_EUR', 'IAPOC_XDC', 'ILPD_L_XDC', 'IAPOFO_USD', 'IAPDOFI_S_EUR', 'IAPDBP_S_XDC', 'IAPD_USD', 'IAPDSF_USD', 'IAPDOFI_XDC', 'IAPDOFO_L_USD', 'IAPOF_EUR', 'IAPOFM_EUR', 'IAPDONO_S_USD', 'IAPDON_L_EUR', 'IAPDOFM_S_XDC', 'IAPEONN_USD', 'IAPDOFM_S_USD', 'IAPDOC_S_XDC', 'IAPDSF_L_EUR', 'IAPON_XDC', 'IAPOFI_EUR', 'IAPDSF_L_USD', 'IAPEONO_USD', 'IAPDBP_L_EUR', 'ILP_EUR', 'IAPDOF_L_USD', 'IAPEJY_XDC', 'IAPDON_S_EUR', 'IAPDUS_EUR', 'IAPEOF_USD', 'ILPD_L_USD', 'IAPEB_XDC'}>}

--- Value ---
{358180: 'PT1M', 358181: 'P6M', 358182: 'P3M', 358183: 'P1M', 358184: 'P1Y', 358185: 'P7D', 358186: 'P1D'}

<com:KeyValue xmlns:com="http://www.sdmx.org/resources/sdmxml/schemas/v2_1/common" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:mes="http://www.sdmx.org/resources/sdmxml/schemas/v2_1/message" xmlns:str="http://www.sdmx.org/resources/sdmxml/schemas/v2_1/structure" id="TIME_FORMAT">
com:Value/
com:Value/
com:Value/
com:Value/
com:Value/
com:Value/
com:Value/
</com:KeyValue>

Traceback (most recent call last):
File "r:\SharedLocal\Workplace\2023\03\pandasSdmx_test.venv\Lib\site-packages\pandasdmx\reader\sdmxml.py", line 299, in read_message
result = func(self, element)
^^^^^^^^^^^^^^^^^^^
File "r:\SharedLocal\Workplace\2023\03\pandasSdmx_test.venv\Lib\site-packages\pandasdmx\reader\sdmxml.py", line 1172, in _ms
arg["values_for"] = cl.get(elem.attrib["id"])
^^^^^^^^^^^^^^^^^^^^^^^^^
File "r:\SharedLocal\Workplace\2023\03\pandasSdmx_test.venv\Lib\site-packages\pandasdmx\model.py", line 883, in get
raise KeyError(id)
KeyError: 'TIME_FORMAT'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "", line 1, in
File "r:\SharedLocal\Workplace\2023\03\pandasSdmx_test.venv\Lib\site-packages\pandasdmx\api.py", line 514, in get
msg = reader.read_message(response_content, dsd=kwargs.get("dsd", None))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "r:\SharedLocal\Workplace\2023\03\pandasSdmx_test.venv\Lib\site-packages\pandasdmx\reader\sdmxml.py", line 316, in read_message
raise XMLParseError from exc
pandasdmx.exceptions.XMLParseError: KeyError: 'TIME_FORMAT'

This was first discovered in sdmx1: khaeru#102

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant