Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

XMLSchemaConverter not able to convert the text data to xml #316

Closed
priyatamnano opened this issue Jul 1, 2022 · 20 comments
Closed

XMLSchemaConverter not able to convert the text data to xml #316

priyatamnano opened this issue Jul 1, 2022 · 20 comments
Labels
question Further information is requested

Comments

@priyatamnano
Copy link

priyatamnano commented Jul 1, 2022 •
edited by brunato
Loading

this converter is able to convert the below xml data to dict data

<ns0:LoanTerm Units="Months" Type="TotalTerm" PaymentType="PrincipleAndInterest">360</ns0:LoanTerm>
'ns0:LoanTerm': [{'@Units': 'Months', '@Type': 'TotalTerm', '@PaymentType': 'PrincipleAndInterest', '$': '360'}]

But not able to convert back to xml when am using from_json

error:
xmlschema.validators.exceptions.XMLSchemaChildrenValidationError: failed validating <Element '{urn:ais:schema:banking:loans:sunloan:SubmitSLNApplication}ApplicationBatch' at 0x10faaabd0> with XsdGroup(model='sequence', occurs=[1, 1]):

Reason: The content of element '{urn:ais:schema:banking:loans:sunloan:SubmitSLNApplication}ApplicationBatch' is not complete. Tag 'Identifier' expected.
@brunato
Copy link
Member

brunato commented Jul 5, 2022

Probably related with the #315, if the content type is mixed. I will resolve them together, using some test cases as start.

Thank you

@priyatamnano
Copy link
Author

any update ? am getting other issues when am sending soap xml to json ,it is giving errors

elif isinstance(obj, (converter.dict, dict)) and len(obj) == 1:
for key in obj:
match = re.search(r'[{\w]', key)
this one come len(obj) == 1: as 2 instead of one as it contains 2 keys, is there any fix for this ?

<soapenv:Body xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/" xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:wmb="urn:suncorp-com-au:wmb" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd">
soapenv:Envelope
soapenv:Header
wmb:ServiceNamex</wmb:ServiceName>
wmb:Actionx</wmb:Action>
</soapenv:Header>
soapenv:Body
<ns0:ApplicationBatch xmlns:ns0="ux" ProductionData="No">
@brunato

@brunato
Copy link
Member

brunato commented Jul 6, 2022 •
edited
Loading

I'm going to complete a set of tests with a fix for #315. About the default converter (XMLSchemaConverter, but also with UnorderedConverter, that is a variation of the default converter), you have to provide a cdata_prefix (usually '#').

In any case with mixed content the order of children cannot be preserved normally (cdata and child elements with the same name). For a warranty on preserving structure and order the only choice is to use the JsonML convention.

@priyatamnano
Copy link
Author

priyatamnano commented Jul 6, 2022 via email

@priyatamnano
Copy link
Author

priyatamnano commented Jul 6, 2022 via email

brunato added a commit that referenced this issue Jul 6, 2022
@brunato
Fix BadgerFishConverter decode of mixed content (issue #315)
e9c138d 
  - Check also for #316 (is a misconfiguration, not a bug)
  - Fix etree_elements_assert_equal() testing helper
@brunato
Copy link
Member

brunato commented Jul 6, 2022

I'm sorry, but Is difficult to follow issue if you not provide a clear explanation of the case, with XML data and at least the relevant part of the XSD schema.

Also please avoid replying by email, use web form and format data content instead.

Thank you

@priyatamnano
Copy link
Author

priyatamnano commented Jul 6, 2022 •
edited by brunato
Loading 

xs = xmlschema.XMLSchema(
                self.xsd_file, converter=xmlschema.JsonMLConverter(dict_class=dict)
            )
            xs.import_schema(
                namespace=self.location, location=self.location, build=True
            )
 self.data = xs.to_dict(self.xml_file, unordered=True)          

xs = xmlschema.XMLSchema(
            self.xsd_file,
            converter=xmlschema.JsonMLConverter(dict_type=dict),
        )
        xs.import_schema(
            namespace=self.location, location=self.location, build=True
        )
        json_data = json.dumps(self.data)
        xml_data = xmlschema.from_json(
            json_data,
            xs,
            unordered=True,
            converter=xmlschema.JsonMLConverter(dict_type=dict),
            preserve_root=True,
            namespaces={
                'soapenv': 'http://schemas.xmlsoap.org/soap/envelope/',

            },

@brunato
Copy link
Member

brunato commented Jul 6, 2022

Tried your lines of code, retrieving soapenv schema from https://schemas.xmlsoap.org/soap/envelope/.

You miss to provide the path argument, eg.:

xml_data = xmlschema.from_json(json_data, xs, path='soapenv:Body', unordered=True, converter=xmlschema.JsonMLConverter(dict_type=dict), preserve_root=True, namespaces={ 'soapenv': 'http://schemas.xmlsoap.org/soap/envelope/'})

xml_data is identical to xml source data (nikola_valid.xml). The file nikola_valid.xml contains an invalid character &, that has to be replaced by &amp; entity.

I check if it is possible to retrieve the right encoding element with a full search into defined global elements.

@priyatamnano
Copy link
Author

yes, have changed the encoding.
but after that also am getting this error
raise XMLSchemaTypeError(msg.format(type(obj)))
xmlschema.exceptions.XMLSchemaTypeError: The first argument must be a sequence, <class 'str'> provided

79 if not isinstance(obj, MutableSequence):
80 import pdb;pdb.set_trace()
81 -> msg = "The first argument must be a sequence, {} provided"
82 raise XMLSchemaTypeError(msg.format(type(obj)))
83 elif not obj:
84 raise XMLSchemaValueError("The first argument is an empty sequence")
85
86 data_len = len(obj)
(Pdb) obj
'wmb:ServiceName'
(Pdb) isinstance(obj, MutableSequence)
False
(Pdb) MutableSequence
<class 'collections.abc.MutableSequence'>
(Pdb)

@priyatamnano
Copy link
Author

Aslo want to keep namespace tag as and

but the xml_from_json converting it like below . How to keep it is as original format ?
</ns2:ApplicationBatch>
</ns0:Body>
</ns0:Envelope>
</ns0:Body>
</ns0:Body>

@brunato brunato added the question Further information is requested label Jul 7, 2022
@brunato
Copy link
Member

brunato commented Jul 7, 2022

Provide a full namespace map (not only 'soapenv' ...) when you call decode/to_dict.

Best regards

@brunato brunato closed this as completed Jul 7, 2022
@priyatamnano
Copy link
Author

priyatamnano commented Jul 7, 2022 •
edited
Loading

Have given that, but it is becoming ns2 instead of ns0

<soapenv:Body xmlns:ns2="urn:ais:schema:banking:loans:sunloan:SubmitSLNApplication" xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:wmb="urn:suncorp-com-au:wmb">
soapenv:Envelope
soapenv:Header
wmb:ServiceNameLOAN</wmb:ServiceName>
wmb:ActionSUBMITAPPLICATION</wmb:Action>
</soapenv:Header>
soapenv:Body
<ns2:ApplicationBatch NumberOfApplications="1" ProductionData="No">
<ns2:Identifier UniqueID="1036921-SUN" Type="BrokerAssigned" />
<ns2:RevisionNumber LIXIVersion="1.4" LenderVersion="1.0" />
ns2:Submission

@brunato any idea ?
also this one ns2="urn:ais:schema:banking:loans:sunloan:SubmitSLNApplication" should be in ApllicationBatch instaed of soapbody

@priyatamnano
Copy link
Author

json_data = json.dumps(self.data)
xml_data = xmlschema.from_json(
json_data,
xs,
unordered=True,
converter=NikolaBadgerFishConverter(dict_type=OrderedDict),
preserve_root=True,
validator="strict",
path='soapenv:Body',

            namespaces={
                'soapenv': 'http://schemas.xmlsoap.org/soap/envelope/',
                'soapenc': "http://schemas.xmlsoap.org/soap/encoding/",
                'wmb': "urn:suncorp-com-au:wmb",
                'xsi': "http://www.w3.org/2001/XMLSchema-instance",
                'wsse': "http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd",
                'ns0': "urn:ais:schema:banking:loans:sunloan:SubmitSLNApplication"

            },
        )
    except urllib.error.URLError:
        message = f"No such file or directory {self.xsd_file}"
        raise FileNotFoundError(message)
    except Exception as error:
        logger.error(error)
        raise XmlToDictConversionError(
            f"Not able to convert dict data to xml string : {error}"
        )

    return xmlschema.etree_tostring(xml_data, {
                'soapenv': 'http://schemas.xmlsoap.org/soap/envelope/',
                'soapenc': "http://schemas.xmlsoap.org/soap/encoding/",
                'wmb': "urn:suncorp-com-au:wmb",
                'xsi': "http://www.w3.org/2001/XMLSchema-instance",
                'wsse': "http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd",
                'ns0': "urn:ais:schema:banking:loans:sunloan:SubmitSLNApplication"
            }, xml_declaration=True)

@brunato
Copy link
Member

brunato commented Jul 7, 2022

You cannot use 'ns0', 'ns1' ... prefixes in the map because they are reserved (by ElementTree)

@priyatamnano
Copy link
Author

priyatamnano commented Jul 7, 2022 •
edited
Loading

Then how can i achieve this ?
i want to get same xml as my input xml without changing anything ? any idea ?

<soapenv:Body xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/" xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:wmb="urn:suncorp-com-au:wmb" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd">
<ns0:ApplicationBatch xmlns:ns0="urn:ais:schema:banking:loans:loan:SubmitSLNApplication" NumberOfApplications="1" ProductionData="No">
<ns0:RevisionNumber LIXIVersion="1.4" LenderVersion="1.0"/>
ns0:Submission

@priyatamnano
Copy link
Author

priyatamnano commented Jul 7, 2022 •
edited
Loading

i think this is the issue why xmlns keeping ns0': 'urn:ais:schema:banking:loans:sunloan:SubmitSLNApplication'}, instead of it should present in 'ns0:ApplicationBatch ' dict ?

lx.to_dict()['@xmlns']
{'soapenc': 'http://schemas.xmlsoap.org/soap/encoding/', 'soapenv': 'http://schemas.xmlsoap.org/soap/envelope/', 'wmb': 'urn:suncorp-com-au:wmb', 'xsi': 'http://www.w3.org/2001/XMLSchema-instance', 'wsse': 'http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd', 'ns0': 'urn:ais:schema:banking:loans:loan:SubmitSLNApplication'}

@brunato

@priyatamnano
Copy link
Author

after removing from that it is giving correct result, do you know how can put that inside applicationbatch at the time dict creation ?

@brunato

@priyatamnano
Copy link
Author

any idea @brunato

@priyatamnano
Copy link
Author

priyatamnano commented Jul 7, 2022 •
edited
Loading

This ns0': 'urn:ais:schema:banking:loans:sunloan:SubmitSLNApplication' is coming as @xlmns attribute ,instead of

{'ns0:ApplicationBatch': [{'@NumberOfApplications': '1', '@productiondata': 'No', 'ns0:Identifier': [{'@uniqueid': '1036921-SUN', '@type': 'BrokerAssigned'}],

{'@xmlns': {'soapenc': 'http://schemas.xmlsoap.org/soap/encoding/', 'soapenv': 'http://schemas.xmlsoap.org/soap/envelope/', 'wmb': 'urn:-com-au:wmb', 'xsi': 'http://www.w3.org/2001/XMLSchema-instance', 'wsse': 'http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd', 'ns0': 'urn:ais:schema:banking:loans:oan:SubmitSLNApplication'}, 'soapenv:Body': {'soapenv:Envelope': [{'soapenv:Header': {'wmb:ServiceName': [{'$': 'LOAN'}], 'wmb:Action': [{'$': 'SUBMITAPPLICATION'}]}, 'soapenv:Body': {'ns0:ApplicationBatch': [{'@NumberOfApplications': '1', '@productiondata': 'No', 'ns0:Identifier': [{'@uniqueid': '1036921-SUN', '@type': 'BrokerAssigned'}],

@priyatamnano
Copy link
Author

priyatamnano commented Oct 11, 2022 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
question Further information is requested
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@brunato @priyatamnano